Public Alphas, closed Betas, Early Access — nowadays it feels as though video games are being designed and built before our eyes. With our help. Our feedback. What does that mean for the video games we play and the people who make them? We spoke to the developers behind the Evolve and Battlefield Hardline Betas to find out.
Once upon a time video games were built, tested briefly in-house and sold to the general public. That was it. That was the process.
Today, that is no longer the case. Obviously. Now things are a little more complicated.
Making a video game that works on all possible levels? A product that bring the work of dozens if not hundreds of different people together? A product played online by millions of different players, in different countries, with different internet service providers, using different modems, different consoles — different everything? Not as easy as you might think.
But that’s what Betas are for.
And Alphas. And in-house testing. And focus groups. If you ask Jon Bloch — a producer at Turtle Rock Studios – you’ll need to make use of all resources at your disposal if you want to release something as sizeable as Turtle Rock Studios latest game Evolve. And once you’ve taken stock of all those resources, collected all that data? You’re gonna have another problem on your hands: filtering the noise from what’s truly valuable.
Managing that volume, making sense of it all? It’s a massively difficult task. A full time job.
“Actually it’s more than a full time job,” laughs John. “It’s multiple full time jobs.”
Fast forward. Another video game. Another beta. Visceral Games: the team working on Battlefield Hardline. They’re currently in the same boat, at a different point in the journey. Just three weeks ago Visceral was in the midst of a beta, managing the same situation Turtle Rock Studios once managed. This is what it means to develop video games in the present tense. It means giving your consumer base the chance to provide feedback instantly and early.
We’re living in the age of the alpha. And the Beta. And the day-one patch. That’s simply how things are done nowadays.
For a team on the verge of a major launch, Visceral is more comfortable than you might think. Considering this is the team’s first pop at the Battlefield franchise, you’d expect nerves, jitters. No. The opposite. The Visceral team radiates confidence, for two main reasons. The first: a high level of confidence in their work. Second: they’ve gone through this process before.
E3 2014: EA announces Battlefield Hardline. Then, almost in the same breath, EA announces a public beta. That sort of one-two punch is rare. Terrifying if you’re a developer. Imagine that level of stress: Visceral was making its debut on one of the most beloved games series on the planet. Not only did they have the jitters of the announcement to deal with, they had that added pressure: within minutes a tremendous amount of people would be playing the video game they’d dedicated years of their life to building. This is not a normal situation. This is what they call a baptism of fire.
But Jeff Zaring, a Lead Multiplayer Designer at Visceral, wouldn’t have it any other way. It’s the reason for his confidence. Right this second, as he awaits the launch of the Battlefield Hardline Beta, he’s safe in the knowledge that he’s been here before. The mindframe is different. The first Beta, back at E3, did enough to convince the team they had something good, says Jeff. They learned much from that experiment. They’ve implemented the necessary changes. He expects this second Beta to convince the world they have something great.
And therein lies the major reasons for having a Beta in the first place: the chance to test a game’s strengths with Joe Public. As tests go, it’s an invaluable one.
“We had a huge list of things to change after the first beta,” explains Jeff.
Those changes included something as severe as reworking an entire section of Hardline’s new game mode, Heist.
“We were getting a lot of feedback from our users on that one,” he says.
Locations were changed, the pacing was changed, the team added a whole new ‘defend’ loop. User feedback was invaluable, he says, when it came transforming the game from ‘good’ to ‘great’.
“When we put out the beta we realised that we had something good on our hands and if we took the time we could make something great,” adds Scott Probst, a Senior Producer on Hardline. “That was across the board on the game and the feedback we were receiving. We started the conversation in the company: this thing has limitless potential, what can we do?”
Turtle Rock ‘Studios has a theory about this kind of thing. That theory is simple: get your video game in the hands of players as quick as humanly possible. That’s the most efficient way to confirm whether or not something works.
An addendum to that process: measuring people’s raw responses, taking their feedback, trying to make sense of all that raw data, and then implementing the necessary changes.
It’s a daunting task with multiple different dimensions.
But for Turtle Rock Studios, it’s a process that begins as soon as their game is playable, whether that game is Left 4 Dead or their latest title Evolve. It’s a multi-faceted task: it begins with internal playtesting, with the team themselves, later the team might bring in focus groups to test certain aspects of the game. Later journalists might get hands-on, then there’s the Alphas and the Betas.
All these processes are just a way for Turtle Rock Studios to acquire an almost immeasurable amount of data on how their video game is being played.
“That’s really important to us,” says John.
Data. A hell of a lot of data. How are players moving, where are players moving to? Why? What weapons are they using, what weapons are they not using? That kind of stuff.
And more stuff. Different stuff. So much ‘stuff’. The sheer amount of telemetry data being measured at Turtle Rock Studios is terrifying. An alpha or a beta is all about recording that kind of data on a large scale, but it’s also about taking that data and comparing it to actual vocalised reactions from the user base.
Does feedback match the data? Is there are a disparity? Learning the correct lessons from a beta is a juggling act, sometimes the player’s perception of what is ‘broken’ or ‘unbalanced’ is actually subverted by the data Turtle Rock Studios receives.
“We use the telemetary data as a backup to the raw reaction,” explains Rock. “You can have balance tweaks and people start screaming, ‘that ability’s OP now! You gotta nerf that thing’. Then you look at the telemetry data and that tells you something completely different.
“Maybe it wasn’t the ability itself that was underpowered, maybe it was something else that was overpowered. Or maybe there was something else that was causing that ability to feel overpowered. Maybe that’s the thing that needs to be tweaked. It’s all about using that data and the reactions together.”
The customer isn’t always right. Not always.
“A lot of it comes down to what we’re trying to accomplish. What’s the goal of that gametype, that game mode. We have to ask ourselves does this serve our goals? Do the goals have to change? Dealing with all the feedback, weighing it, it’s a pretty big task.”
Hardline’s initial Beta was the baptism of fire. It was the initial offering. ‘Here’s what we’ve been working on for all this time, hope you like it!’ Visceral and Turtle Rock Studios will tell you the same thing: you don’t really know if you have something until you toss it headlong into the wild.
And that goes double for a game like Battlefield Hardline: a game that tinkers with a well established set of mechanics, a game that resets your expectations of what its predecessors were capable of. But now that Visceral knows it has ‘something’, it has the confidence, that permission almost. In that respect Hardline’s second beta is less about making massive changes and more about validation. The overarching theme: ‘we listened to your feedback, we’ve made the appropriate changes, how ya like me now?’
In short: the second beta was about the little things
“Things like progression, balance of weapons and game modes,” says Scott. “We were keeping a close eye on that so when the game launches people are having a great time.”
And let’s not forget, in the wake of debacles like the Sim City launch of the Halo: The Master Chief Collection, there’s nothing more valuable than a last minute check: Visceral is keen to make sure its game actually works on a technical level.
“From an engineering perspective this is about making sure all the back-end functionality players expect to work actually works,” explains Scott. “Server rotations, maps rotations — all that stuff. We have engineers working around the clock to make sure we have a stable launch, so that when we launch everyone can hop in and play.”
Turtle Rock Studios were openly criticised for launching a Beta after Evolve had gone gold. Critics were quick to attack the culture of day-one patches and the practice of shipping largely unfinished games to the general public. But making sure that back-end is fully functional – testing its capabilities – is paramount. On launch day hundreds of thousands of people are going to be playing your video game. Your servers have to be prepared for that level of strain. Consider it a warm-up.
“Just because the game went gold,” explains Bloch, “that doesn’t mean we all just go out partying. There’s a lot to do.”
The cynical among us might suggest that a Beta, so late in the game, is essentially marketing. A last gasp chance to sell an online focused video game to the public.
And interestingly Bloch doesn’t necessarily disagree. In some respects Evolve’s final beta, so close to release, was like an old-school playable demo.
“It’s kind of interesting because as much as we want to use it for feedback, there are a lot of people that are going to treat it like a demo, because the code is pretty much final. It’s near final product and people are going to look at it in a certain way.
“We have to treat it like a demo even though our intention is to learn from it.”
The process of making a video game has changed. Clearly.
Alphas, Betas, early access. More and more consumers are being asked to contribute to video games while they are being made. We watch them develop, we watch them evolve. To a certain extent we’ve become part of that process.
In other words: game development, on a grand scale, has become an open conversation.
“I think it’s a huge opportunity because it’s no longer a situation where developers are saying ‘here’s an experience, go play it,’” says Scott.
If you ask Jon Bloch, this kind of process has served Turtle Rock Studios well. He can’t imagine ever working on a game without that kind of feedback.
“It’s kinda the Turtle Rock way,” he explains.
“Everyone that is gonna buy the game is giving us feedback before it’s even done. It’s like, what do you want us to make? We’re making something we think is awesome, but what do you want?
“I don’t see us changing that methodology any time soon.”
Comments
16 responses to “The Reasons Why Video Games Have Beta Testing”
Problem is that while people like Turtle Rock maybe believe betas are required, their games are still stinking piles of technical issues regardless upon release. Still getting a load of problems when playing Evolve.
Last night I was rolling a match as Goliath, got every one incapacitated/dead with only the support making a B line under stealth; I manage to catch him with a lucky flame breath attack, but as I go to finish the job off I clip through geometry and proceed to fall out of the map and die. I still occasionally fall through hunter spawn, I still get routine quick white flashes on maps without weather effects and I still get CTD or disconnects at random periods during a game.
Odd, unlocked almost every character now and I’ve not had a single issue
Beta testing or Advertising? With how little some games (Particularly EA’s) change from Beta to retail… It’s anyone’s guess.
That’s been my experience with EA’s ‘beta testing’ program. all the bugs i carefully logged were never fixed.
I still think of Battlefield: Hardline as Ferguson Police Did Nothing Wrong: The Game. It’s spectacularly tasteless.
This is generally not something that’s gathered during the beta process.
I’ve been surprised by the lack of mainstream outrage, but maybe 3 weeks is too early. Anyway the campaign looks pretty campy 😛
It’s a shoot ‘em up cops and robbers game.
Hopefully people are smart enough to tell the difference between high-fantasy action games and day-to-day police work.
People manage to play normal Battlefield while they’re a remote flick away from real news stories about dead soldiers from an assortment of conflicts around the world. I don’t think it’ll be an issue.
I always found it pretty funny how much gear they could take directly from BF4 and put into a police setting.
Seems to me companies are cheating and the people who pay for early access are the beta testers….
Which means, you should be getting paid, not having to pay.
It blows my mind that they’ve figured out a way to make customers pay for unfinished games. Anyone who pays for early access is a stupid as the publishers are greedy. ( small indy game sI understand but not places like EA )
Great story, Mark! Good to hear straight from the developers on these types of things. Personally I am all for Alphas/Betas if it means a better game on launch. Battlefield Hardline was unique for me in that they managed to change my opinion of the game over the course of the 2 betas. After the first beta I was not interested in the game at all. After the 2nd beta — with the changes they made and the inclusion of ‘Hotwire’ as a playable mode — I am now completely interested in the game and will be buying it day 1.
Could have sworn that in the old days this step just involved paying for play testers.
Now they’re just exploiting the fan base to get that work done for free. Ah well, gotta cut corners where you can with how expensive AAA is these days.
This doesn’t really bother me that much as open betas give you a good taste of a game before it launches so you know if you should spend your hard earned cash.
I got my fill in about 3 hours of the Titanfall beta and didn’t bother purchasing. I played almost 30 hours of the Hardline beta and cant wait for it to release. (Obviously i’m still not going to pre order because I don’t hate myself.)
Ironically was not interested in Evolve in the slightest (didn’t even DL the beta) but became interested after seeing the GG review the other night (though not at full price) but Hardline – I was excited about that last year only to be kind of underwhelmed by the Beta a few weeks back – after playing BF4 it kind of felt watered down somehow. I feel like somewhere there is a great heist game waiting to happen, it’s not Hardline, I couldn’t get that feeling with Payday either – maybe once GTA V’s Heists come out they’ll scratch that itch!
The reason: X-rebirth or stillbirth as its commonly known.
Evolve is well worth the money. Enjoying it a lot. You need to think a lot and play to your class advantages. Pretty balanced though I still suck as the monster.
I have said it a few times before here but Betas should always be free (IE a Demo) – its a joke if a company is making you pay for a game that may be broken as all hell unless you have tested it out first.