Quality Assured: What It’s Really Like To Play Games For A Living

Quality Assured: What It’s Really Like To Play Games For A Living

There’s an old commercial for Westwood College that’s become something of a running joke in the video game world. Two young men sit at a couch, hammering away at PlayStation controllers. A woman walks in. “Hey guys, finish testing that game yet?” she asks. “I’ve got another one I need designed.”

“We just finished level three and need to tighten up the graphics a little bit,” says one of the men. Then he turns to his friend, smiling like he just won the lottery. “Hey, I can’t believe we got jobs doing this.”

“I know,” the other guy says. “And my mum said I would never get anywhere with these games.”

For a very long time, that’s how people have imagined the life of a video game tester, not as 9-to-5 job but as the fantasy of teenagers everywhere. Who wouldn’t want to sit on a comfy couch and play games all day, taking the occasional break to tighten up the graphics on level three?

Reality betrays those delusions. Video game quality assurance (QA), as testing is called, is often perceived as “playing games for a living” but it might better be described as breaking them. It’s a low-paying, occasionally rewarding, often frustrating job that has both more and less to do with the quality of today’s games than you might expect.

A professional QA tester doesn’t just sit by the television, crack a Mountain Dew, and saunter through level 5 of the latest shooter; he or she spends 14 straight hours running into different walls to see if they’re all solid. Proper video-game testing is more akin to abstract puzzle-solving than it is to getting a top score in Donkey Kong, despite what you may have seen in college commercials like Westwood’s. “It takes a very specific attitude and outlook to really be good in the QA world,” a veteran game tester told me. “It goes beyond a passion for video games, and definitely beyond the notion that you get to play video games for a living.”

Game testers are, by nature, underappreciated; they are only noticed when something goes wrong. QA veterans say the job is stressful, tedious, and often seen as a doorway to other parts of game development rather than a viable career path. Often, testers work on temporary contracts or for outsourcing companies that don’t let them communicate directly with a game’s developers. And when a game is particularly buggy or broken — as many recent releases have been — it’s customary for observers to blame QA. They are the safeguards, after all — the final wall between a programmer’s mistakes and a customer’s money. It’s in the name: Quality Assurance. They’re supposed to assure quality.

But when a big game ships broken, is QA really to blame? How could testers possibility not find some of the bugs that show up in the games we play? Why do so many servers break all the time? Just what do QA people do all day, anyway?

Over the past few months I’ve had extensive conversations with several dozen current and former QA testers — many of whom spoke anonymously in order to protect their careers — in an attempt to explore the world of video game testing and try to explain what it’s really like to play games for a living. Some said they hated working in QA; others said they couldn’t imagine doing anything else. Almost all agreed on one thing: not a lot of people understand how QA actually works.

Quality Assured: What It's Really Like To Play Games For A Living

For as long as there have been video games, there have been bugs. Some are relatively harmless and have even attained mythological status, like Pokémon‘s enigmatic Missingno. Others are seared into video game history, like Minus World, an unbeatable Super Mario Bros. level that you can only access by glitching into a wall. Many bugs have been discovered, abused, and enjoyed by the assiduous speedrunning community — how else would you beat Ocarina of Time in 17 minutes?

Those are the friendly bugs, though. Most video game glitches are irritating at best and game-breaking at worst, which is why every game goes through quality assurance, an extensive testing process implemented to ensure that everything’s working properly. The term “QA” draws from the world of products — microwaves, cars, assembly lines — and in many ways video game testing is no different. A tester’s job is to poke, prod, and play the hell out of a game until all the kinks are gone, like a factory-worker smoothing out the latest toy.

There’s no game industry standard when it comes to QA procedure — every game is different and every company has its own process — but a tester typically spends months playing builds of the same game over and over again in all sorts of different ways. The more bugs a tester finds, the higher his or her perceived value to the company. This is one hell of a challenge, of course: video games are complicated sets of interlocking systems that require careful, meticulous bug-testing, which can involve testing the same level repeatedly with slight variations — using a new character, wielding a different weapon, following an alternate pathway — and recording everything that happens.

Take Grand Theft Auto V, for example. On Rockstar’s giant open-world game, QA testers had to divide and conquer. “You would have individual testers assigned certain missions, or tasks, mini games etc.,” said one person who helped test the game. “Normally starting with the big stuff and working down. So doing story missions in order, then heists, then side missions and random characters until you moved on to testing the strip club and prostitutes.”

Sometimes, that tester said, they’d also have to devote tons of time to granular parts of the game, like when Rockstar’s designers asked a group of QA staff to test everything players could do with the game’s automated taxi service. They quickly found that taking a taxi to a new mission would trigger the mission without properly disposing of the cab, leading to some amusing moments as a taxi drove around and tried to back up during cut-scenes.

“I think working on a project like that is made a lot better by the small moments where something truly stupid happens,” the tester said. “Talking pigs randomly standing up like a person and walking away, randomly being shot out of the sky in a plane by an ambient pedestrian whose physics had fired him into space. Trevor pulling his trousers down then never animating to pull them back up and spending the rest of the entire game with his trousers around his ankles. Franklin’s dog used to instantly die if he touched water… he’d just fall into a pool and sink to the bottom like a rock as soon as his paws got wet.”

Quality Assured: What It's Really Like To Play Games For A Living

Finding bugs is just the first step — the second, significantly harder process is trying to reproduce glitches so the company’s engineers can zap them. A tester can’t just write down something like “Trevor’s pants won’t stay on” and send it to the programming team; what could engineers possibly do with that information? In order to track down, isolate, and fix a bug, coders need to know exactly how it happens, which can be a tricky riddle to solve given how many variables are in video games. Good QA testers quickly learn to keep track of every action they take — from big to small — so they can at least try to reproduce any bug they stumble upon. “I love that working QA is often like being paid to solve puzzles,” said veteran tester Rob Hodgson, who’s done the job for eight years now. “Figuring out how to reproduce that bizarre error you encountered, step by exacting step, is thrilling to the right kind of person.”

A typical day for a QA tester could vary drastically based on the project, the role, and the position. An outsourcer might have to spend 10 hours slamming into every wall in the latest Call of Duty to see what breaks (“collision testing”). An embedded QA tester might have to work with a programmer to find out why the framerate keeps dropping on the Android version of their new mobile game. The varied and monotonous nature of QA can lead to some unexpected challenges; for example, one tester who worked on the rhythm game Rock Band said the clack-clack-clack of plastic drumming could be so maddening, they had to set a rule: no instruments on Tuesdays and Thursdays.

As they play, testers have to type up bug reports, using software like JIRA to explain what happened and how it happened. Programmers — who ideally at this point are no longer working on new content and are exclusively fixing bugs — analyse the reports and respond as necessary, sometimes with questions, problems, and occasionally snarky comments.

When a console game is almost done, it has to go through certification, a process in which the publisher (like, say, EA) will ask the console manufacturer (Sony, Microsoft, or Nintendo) to check the game for game-crashing bugs. During this certification process, a second layer of QA staff called compliance testers must go through the game with their own fine combs, checking to see if everything lives up to expectations. Every console-maker has its own stringent checklist dictating everything from error messages to achievements, and if a game doesn’t make the cut, the publisher will have to fix things and try again, deadlines be damned. “Microsoft required all games to be able to access the Xbox 360 menu from literally everywhere in the game,” said one tester who worked in compliance for a major game publisher. “Sony required that you couldn’t skip the first time you see the publisher/development studio screens at the beginning of the game. Nintendo didn’t want any swearing in their games so every text input had an extensive filter we had to try and break.”

Quality Assured: What It's Really Like To Play Games For A Living

The Telltale Speedrun

“I didn’t play BioShock Infinite for at least two years after its release,” an ex-QA tester told me recently. He’d worked for 2K and spent extensive time testing out Infinite, and he wasn’t really happy with how the game turned out, noting that it didn’t live up to the original.

“The one thing that got me to play the game again was seeing speed runs of BioShock. [We’d spent] a lot of nights speed-running the game. It was interesting to see what the speed-runners used to cut levels.”

One night recently, the tester saw this video of a speedrunner breaking the game’s world record. When he got to the 11-minute mark, he started freaking out.

“I got so mad because they use a bug to get outside the level and automatically move forward,” the tester said in an email. “I SHOULD HAVE CAUGHT IT!”

Quality Assured: What It's Really Like To Play Games For A Living

For a long time the video game industry presented QA testing as a dream job: hey kids, come get paid to play games all day! But in recent years, horror stories have emerged — QA testers tell tales of monotonous work, gruelling hours, and poor treatment from companies that see them as replaceable cogs in the development machine.

There’s the money problem, for example. The job has low requirements — typical entry-level tester positions don’t need much experience or even a college degree — and there’s a ton of demand, so the pay is mediocre. A 2014 Gamasutra salary survey pegged the average QA tester’s yearly pay at $US54,833, but that only reflected full-time staff; most testers are contractors working either directly with developers or at third-party QA houses that juggle multiple publisher clients. Many of those contractors told me their salaries ranged from $US10 to $US15 an hour — that averages out to $US21-31,000 per year.

Testers also talk of feeling disrespected at the workplace. Many in QA, especially contractors, have told me they weren’t allowed to speak directly to the developers, and that their only communication came in the form of written bug reports. “It was also sort of an unspoken rule that temps shouldn’t directly contact the devs,” one tester told me. “Any communication was typically routed through the full-time QA Leads. As testers, all our interactions with dev were through comments in the bug database, which is far from an ideal form of communication. It was easy to interpret a developer’s comment/question on a bug as snarky/irritated when that wasn’t necessarily their intention.”

This isn’t the case at every studio — “When you give your testers benefits, upward mobility, job security, and respect, it attracts the right people for the job,” said Ariel Smith, a tester at the MMO studio Cryptic who told me she loves the gig — but disrespect for QA is certainly a widespread trend. Several testers told me they have had to use side entrances to enter their offices and that they weren’t allowed to mingle with the rest of the staff. Others said it’s common for other developers to screw with them in all sorts of ways; one frequent story, for example, is of the engineer who fixes a bug yet continually sends QA messages like “could not reproduce.” In a typical studio, QA is seen as the bottom of the totem pole. This is in part because of the nature of their job — a tester’s role is to show other people where they screwed up. That’s always going to bruise some egos.

“The QA only care about finding bugs; the developers only care about fixing them,” a former tester told me. “They aren’t a team and they aren’t working together. It’s almost like a game of tennis. The testers actually want the build to be broken because it ensures they have work to do. So, the two sides are kind of working against each other, which isn’t healthy for production.”

Quality Assured: What It's Really Like To Play Games For A Living

At some game companies, higher-ups give testers strict bug quotas and threaten to shorten their contracts if they don’t find enough glitches, which can result in a weird sort of tension as testers compete over who’s finding the biggest bugs first. Sometimes, QA employees will find creative ways to work more hours so they get paid more and make themselves seem more valuable to the company. “There were some testers who would hold onto bugs to make sure there’s overtime,” one tester told me. “If there was no OT scheduled for the weekend they would enter a [major] bug Friday afternoon. In some cases this would cause overtime.”

QA testers also have to deal with game development’s other systemic issues — particularly, mandatory crunch and frequent layoffs. Big developers tend to hire dozens of QA testers toward the end of big projects only to let them all go once the game ships. Instead of partying with the rest of the developers, they’re out looking for new jobs.

This all adds up to what would seem to outsiders like an unquestionably heinous gig, but the job does have bright points. A number of current and former testers have told me that despite QA’s many challenges, testing video games can be rewarding and educational in a unique way.

“I’ve enjoyed my time as a tester and I would do it again if I had to,” said Obed Navas, a former tester who worked on games like BioShock and Call of Duty. “Even though ‘QA Tester’ might not be the most glamorous title, and you run the risk of losing the passion to play at home, in the end, being able to see your name in the credits, and having specific items on you that people know you can’t find anywhere …and them asking you where you got it, being able to respond ‘I worked on it,’ it’s a great feeling and I take pride in that.”

Quality Assured: What It's Really Like To Play Games For A Living

Lunchtime Woes

Rob Hodgson, a video game tester, was working on an early build of the multiplayer game Fallen Earth during lunch one day when suddenly the servers crashed. Not long afterwards, they crashed again. Baffled, he tried to reproduce the bug, but he had no luck — neither he nor anyone else on his team could figure out what was causing such a catastrophic error.

“It made no sense,” Hodgson told me recently. “We got one of the programmers to watch the logs as it happened, and he all but shrugged and threw up his hands. It seemed like someone had run off the edge of the world, and the servers were choking, trying to track him. But no one on the [quality assurance] team was testing the world edges, and the designers were all out to lunch!”

After scratching their heads for a while, Hodgson and team asked around and eventually found out what was bringing down the servers. “Turns out that the designers going out to lunch was precisely the problem,” Hodgson said. “One of them had just been setting his horse on autorun and going out to eat. Some days it would hit a tree and get stuck. Some days it would get into the endless, flat plains beyond, and melt the servers.”

Quality Assured: What It's Really Like To Play Games For A Living

From Assassin’s Creed Unity to the PC version of Arkham Knight, today’s games seem to be shipping with more problems than ever before. It’s easy to pin the blame on QA. Sometimes, that very well may be the case — a number of veterans told me that they have worked alongside sloppy, apathetic testers who just didn’t care enough to do things right. Two different former testers told me they’d watch colleagues smoke weed during lunchtime just about every day; one said the QA staff would all wear sunglasses in a futile attempt to hide it.

But lots of testers say they’re finding most if not all the bugs that ship in today’s games. The problem is that nobody’s fixing them.

Most testers operate using a process called triage, where bugs are prioritised based on importance. Top priority are issues that make the game crash — the “showstoppers,” as they’re called. Other glitches are categorized based on how important testers think they are. Usually, producers and programmers will take the time to fix showstoppers — it’d be hard for any game to make it past certification with any of those. But small and even moderate-sized bugs often stick to games like barnacles, the victims of tight deadlines and programmers who can only do so much in the time they’re given.

“We would often find that either the risk of fixing a bug or the time it would take weren’t worth it, especially when we were doing something really uncommon or deliberately breaking the game,” said one tester who worked on The Elder Scrolls V: Skyrim, a game that’s known for being both extraordinarily massive and uncommonly buggy.

“Some of our bugs would get resolved as ‘won’t fix’ or ‘post-release.’ Post-release meant, ‘This would be nice, but it isn’t really needed right now. If it becomes a thing people are calling out on the forums or online, maybe we’ll reassess this.’ I think there’s a lot more leeway nowadays than there was maybe 20 years ago, [because] you can say, ‘OK we have this nasty bug but it can be fixed in the day-one patch.’ That’s acceptable behaviour now.”

Quality Assured: What It's Really Like To Play Games For A Living

As games get bigger and bigger, making leaps in both graphical fidelity and mechanical complexity, they’re generating more and more bugs — bugs that might not be so easy to catch or fix. Development studios can’t just change a schedule because they’re overwhelmed by glitches. Unless the publisher agrees to what could be a costly delay, that holiday release date ain’t going anywhere, no matter how broken the game might be. So during testing, several QA staffers told me, they focus just on catching the bugs that would prevent them from getting through console certification. That might mean ignoring some of a game’s other, possibly more significant problems.

“The game developers and the publishers had independent QA teams,” recalls one compliance tester who worked for a major publisher. “We would submit a bug with as much detail to recreate the condition as possible, the development team would determine if it was really a bug or not, and then they would either invalidate the bug or attempt to fix it. We would receive a revised build with ‘X’ number of bugs handled, double-check the problem was fixed, and then move on to the next thing. This aspect of the process, from a gamer’s perspective, was the most frustrating. Bugs that caused inconvenience for the player were often considered invalid because they wouldn’t affect the ability to release the game, and could be addressed later if people got upset.”

Other compliance testers say that, due to deadlines, they have had to find loopholes or do the bare minimum necessary to sneak through certification and start printing discs, promising that a day-one patch would come on release day.

“If we were getting down to the wire and multiplayer was super broken, we’d focus test efforts on single-player/offline areas instead,” a tester told me over email. “[Console-makers] would often fail games for severe issues in online functionality, but still allow us to release to manufacturing on the condition we provided a day-one title update to fix issue(s) X, Y, and/or Z. There wasn’t a way to guarantee all players would get a patch for single player/offline bugs since offline gameplay wouldn’t require an internet connection. So we’d tighten up offline content for final certification, knowing there were issues with online content, and then scramble to patch the online issues in time for launch. This was (unfortunately) a rather common process, and was usually due to a rushed & unrealistic timeline rather than dev/QA incompetence.”

Sometimes, bugs will make it into a game for reasons that seem baffling to outside observers. Writes one tester who did contract work for Nintendo of America: “The Sky Drop bug in Pokémon Black and White was something that we caught, but the game had already been released in Japan at that point so instead of fixing the issue they purposefully left it in the game to keep parity.” The tester added that despite these idiosyncrasies, Nintendo makes an explicit point of recording every single bug and asking testers to include video footage for every report they submit. “I think this contributes greatly to their reputation for having quality games,” he said.

One other compliance tester’s explanation for the increasing complexity of bug-testing might make gamers angrier than ever at publishers: it’s DLC, he says.

“Let’s say we have a game with 40 pieces of microtransactions/DLC,” the tester said in an email. “Even if some or all these 40 DLC packages are simple unlocks of on-disc content, we have to test every possible combination of these 40 DLC packages, in all game modes, using different combinations of storage media (20 DLC packs on the hard drive, 15 on a USB flash drive, 5 on a memory unit, etc), mixing up what the host has versus what clients have in different online game modes, verifying graceful handling in the event some dipshit has all his DLC on an external storage device and decides to unplug that device, with each Title Update (1.01, 1.2, etc), with different save data, etc., etc.

“It could (and did) get the point where it’s literally impossible for humans to physically test all possible combinations/situations. It’s something that would get exponentially more difficult as more variables are added — like if you can purchase DLC in-game.”

In short, this shit is tough. Multiplayer games are especially difficult for today’s QA departments to handle; even hundreds of testers can’t successfully replicate what will happen when hundreds of thousands of people are all shooting aliens on live servers. And while some testers say developers are working more closely with QA than they have in the past, nobody can change an unrealistic deadline.

“Milestone schedules nowadays are totally absurd,” one former tester told me. “What [a] lot of people probably don’t understand is that even for titles that have been in development for, say, three years, only 1.5 of those years is actually full, proper production. And of that, only nine months in QA. And of that, only three months at full QA capacity. By then, we might be at content lock. And so by the time we are able to speak out, the game is reaching Beta. This is a bit of an exaggeration for some larger titles, but reality for many I’ve worked with. When QA speak, we are heard… there’s just no time to react.”

Quality Assured: What It's Really Like To Play Games For A Living

‘No Hard Feelings’

Writes one former tester:

We were bundled away into bullpens. I was working with people that had been testing [game] for years and years, were so burnt out on it, but were still going because they had no other hope. Out of the probable hundreds of testers they go through a year, I think maybe two or three ever got hired on. They weren’t acquiring any new, valuable experience or skills during all that time. We got condescending emails saying “Congratulations! You’ve just earned eight hours of vacation time!” like we were kids getting stars for good behaviour. I don’t need that. Just put it on my paystub.

Meanwhile, new cars were showing up in the lot every couple of months. Ultra fancy, decked out cars that probably cost more than four or five testers’ annual income. They were throwing parties for themselves. When you’re grinding away and can’t afford new clothes regardless and you see that sort of thing, it takes a toll. When we launched, there was a massive round of layoffs. I think close to around 100, but I have no idea. All I know is, QA was pretty much wiped to bare bones. A few hours after they were all escorted off the campus, the launch party kicked off.

Things got bad for me in a lot of ways; a girl I was seeing at the time [left me]… I met a lot of good people that knew how terrible the situation was for us temps, and good people were leaving because of it. It frustrated everyone I worked with. I got bumped around from team to team. That last team was small with way too much work, but I learned a lot and they were really good people. When I told them I was leaving, there were no hard feelings. There was almost a hint of encouragement, like I was telling them I was going to quit drinking or something beneficial like that.

Quality Assured: What It's Really Like To Play Games For A Living

Is QA a viable long-term career? Or will it forever be a means to an end, a pathway to other, more interesting parts of game development?

While reporting for this story, I talked to around 60 QA testers, and of those, only four said they saw QA as a long-term career. Many used QA as a way to segue into producer or designer jobs at game companies; others tried it for a few months or years and then gave up, moving on to more lucrative fields. Those who do stay in QA and succeed in the role can find themselves moving into QA management, a field that’s more about handling other people than it is helping make video games. That’s unsatisfying for some folks, especially given that the stereotype of testers being young, immature, and unkempt is often true.

“As you move into the role of QA Management, you start to have experiences that nobody could ever prepare you for,” said one veteran QA lead. “Testers showing up in pajamas for work, testers sexually harassing one another, testers getting caught stealing… Even the most mundane thing like ordering in food can become a challenge. QA is a field that skews very young. Having the average age of a team be 18-19 is not uncommon, and often times this is their first real job. It’s a massive challenge to deal with young personalities and hormones.”

Several experienced game developers have told me there are few employees more valuable than a senior QA person — someone who knows how to properly write bug reports; who knows to focus on big-picture issues instead of some clipping or graphical issues; who knows how to work with other people to maximise efficiency and avoid dupes. Those people are rare — low pay and poor treatment combine to drive many QA testers away from the field before they get very far.

“Before I left, I sat down with my manager, the associate producer on the game, and asked him [for] advice,” said James, a former tester who asked me not to use his last name. “I wanted to know how to make the jump from temp QA to permanent QA in development or anywhere else. He blatantly told me, ‘Distance yourself from QA… the sooner you can get out of QA and distance yourself from ever working in it the better.’ After about another eight months, I left QA for good and gave up on working in the video game industry. I am now working at a developer of commercial software, and life is a little less fun, but much more stable. The money is also much better.”

Quality Assured: What It's Really Like To Play Games For A Living

Based On A True Story

The movie Grandma’s Boy, which came out in 2006 and stars Allen Covert as a video game tester, is something of a running joke among QA. People always ask testers: how much of Grandma’s Boy is true? The answer is usually “not a lot.”

One scene in the movie, in which the egotistical video game designer J.P. looks over a failed prototype and watches his main character’s head fall off, is often brought up as indicative of the movie’s inaccuracies:

Silly, right? Bugs don’t really happen like that. Except for when they do.

“It was in a certain Kinect game that involved obstacle courses in a TV show style presentation,” a tester for a major publisher told me. “There were quick intros of the show’s announcers at the beginning of each level where they’d provide commentary and set the scene. Once, and only once, one of the reporter’s head shifted off his neck and into empty space next to his body – then dropped straight down off of the screen. Nobody could ever reproduce it, and trust me, we tried.

“It was surreal – here I’d developed a distaste for Grandma’s Boy due to its inaccurate and somewhat trivializing take on my profession (‘Did that guy’s head just fall off? That shit doesn’t HAPPEN in real games!’), and it was being proven right in a small way. I’ll never forget it.”

Quality Assured: What It's Really Like To Play Games For A Living

Fantasies like Westwood College’s are a far cry from the real world of video game testing. In real QA, the pay is low, the work is tough, and many testers find themselves in unenviable positions, outsourcing or doing contract work for game developers who don’t even know their names. (Writes one tester: “I was once sent an email from HR asking for input on planning the company’s holiday party — I responded with some feedback and was then told that temporary employees weren’t invited.”)

Even those who found QA to be a satisfying, rewarding job say things could be better. Way better. A widespread push for change could lead not just to healthier work environments but to more experienced, efficient testers who work directly with game developers to ensure that more bugs get fixed.

“Game testing is, in general, a job of unsung and untrained heroes,” reads a report by an old QA company called ST Labs. “For some companies, a typical game testing strategy still consists of throwing sheer numbers of testers on a game as it nears release and hoping that they find the big problems. Understanding the games issues and technology and going about game testing in a systematic and thorough manner is still a new concept for most game development companies.”

That report, written by Duri Price and Ilya Pearlman, came out in 1997. Eighteen years and four console generations later, not much has changed. The good game testers are still unsung and untrained heroes, doing their best to get games fixed despite unreasonable schedules, unmotivated co-workers, and a job that’s unappreciated at best.

There’s another way of looking at it, too.

“You *will* get stuck on shitty games, you *will* get stuck with shitty people, and you *will* get stuck with a Producer/PM that thinks QA Testers are *shitty* and beneath them,” one veteran said. “You will work long hours, be underappreciated and see your amazing bug that you spent six hours working on be swept aside and labelled Known Shippable. But you will also see your name in the credits, you will also realise that you contributed to something really cool, and you will also move on to another amazing project.”

Top illustration by Jim Cooke

You can reach the author of this post at jason@kotaku.com or on Twitter at @jasonschreier.


  • Testing is definitely a thankless job. Games, even great games, are generally not fun at all until the last few milestones.

    Speaking as a grumpy developer, though, there are always two sides to a story. The two departments are often in conflict.

    I don’t have a blanket ‘don’t talk to me’ policy, but if I’m several hours into debugging something, trying to keep the state of the entire system in my head, and you pop up beside me and say ‘Hey! Your last check in broke the menu! ‘ I will probably swear at you. Email me, submit a proper bug report, roll back to a previous version and test something else. Follow procedure and be professional.

    Also, if you don’t provide steps to reproduce the bug then don’t get offended when it bounces back as ‘works for me’.

    There’s also nothing worse than a frustrated tester trying to use their position as a stepping stone.

    Do not submit a design suggestion as a bug report, unless there’s a specific category for that. Feel free to suggest that category if it doesn’t exist.

    And definitely do not add your opinion on what might be wrong in the code if you’ve never seen the code. Just give me clear, concise steps so that I can see the problem for myself.

    A really good QA person is invaluable and, sadly, extremely rare.

    • And definitely do not add your opinion on what might be wrong in the code if you’ve never seen the code. Just give me clear, concise steps so that I can see the problem for myself. I was guilty of that one a couple of times, I think. I remember once there was some numbers screwing around in the colour switcher dealie for customising characters when you jumped between particular screens, I think I had a theory that there were X and Y values from the graph being switched around that was doing it. The dev commented back that I was right though, and seemed happy enough about it, so yay 😛

      • I’m probably too grumpy about that 🙂

        I worked with someone who often gave his opinion instead of giving a full bug report. If he had submitted a full report and then said “possibly this is caused by xyz” then I think I would have been OK with it.

        That was a long time ago, though. I think I’d handle things better these days.

  • QA’s still exist? In this business model of “we’ll just patch it later” and “Haha, we’ll make people pay us to get Early Access and THEY can test our games” I thought QA’s went extinct.

    • Oh, QA still exist. Businesses still require them for two things:
      1) The ability to say that their product has been tested.
      2) The ability to point the finger of blame at someone when things go wrong.

      QA’s didn’t go extinct, people just stopped acknowledging them.
      Disclosure: I am a software tester and have done over a decade of games and software QA. This is also a poke at the software industry in general, most places I’ve worked have been pretty good with acknowledging QA.

  • As someone who has worked in the local industry in QA for more than 12 years and am now in a management position, I can say a lot of this is true though not all of it.

    The big issue I believe is that at least in the past, games for some reason largely get QA and the dev cycle in general completely wrong. They don’t follow best software development practices at all. This mentality is slowly changing however, which is good, but there’s still a long way to go I think before the development and QA of games matches that of commercial software.

    I could write a pretty lengthy post here but I won’t. I’ll just say that QA should be treated as a member of the development team (because they are) and QA testers have their own set of skills that are required to be good at the discipline. It’s much better to properly interview and hire a smaller number of skilled testers on full time than a large number of unskilled testers on contract that you ditch after the project is done. If QA was easy anyone would be able to do it but we know that’s not the case.

    QA should also be involved with development almost from the start, working alongside the developers as things are being designed and developed, designing and writing test cases as they go. Documentation needs to be kept up to date so QA actually has a source of truth to work from. Unit tests should be written with QA’s input as the code is being written. QA should assist with the design of automated tests if it’s applicable to do so. The waterfall model of “do all the QA at the end” that so many games have used in the past is a broken model and always has been, and needs to die in a fire. The earlier you get QA onto things the earlier problems are detected which means less problems and less time spent on those problems at the end.

    Publishers and developers also need to account for QA time in their schedules – not just testing time but bug fixing and verification time as well. If you’re waiting until the end of the project before pulling on most of your QA you’re going to get a metric ****tonne barrage of bugs coming your way and you should be ready for that. But as I said, that model is broken and shouldn’t be followed. You do QA properly throughout development and you don’t run into that situation in the first place.

    • So much this ^
      While I’ve not worked in the games industry I have studied Project Management and seen bad project management in action.

  • Interesting… less showstopping/crippling bugs on consoles due to certification process, while PC games are not bound by any of that. Explains the last major PC ported titles works well on 2 out of 3 platforms imo. Kind of wish there was PC Game Certification.

  • hmm from my experiences in the US, yes game QA probably not a great career move until the culture changes.
    In business systems- QA is dead.
    Ill explain- I was a tester for 15 years before I moved on.
    When I started software testing was about breaking the software so that it didnt for the customer.
    These days though, most testers arent too bright and are more ‘checkers’ in that they check the features that their consultancy was contracted to build were actually built and that they basically work. That isnt the same thing.
    The things we did 15 years ago aren’t done anymore- the profession been reduced down to the lowest common denominator: a job.
    It used to have an element of craftsmanship- hell- I was a developer and a tester. There was no differentiator. Now I have as little to do with testing after what it has become.

Show more comments

Log in to comment on this story!