Cyberpunk 2077 QA CEO Claps Back, Says People Don’t Understand Testing

Cyberpunk 2077 QA CEO Claps Back, Says People Don’t Understand Testing
Get off, you damn bug! (Screenshot: CD Projekt)

Cyberpunk 2077 will always be a cautionary tale of how not to make and release a game. It was notoriously marred by technical bugs and developmental woes and now, according to a new report from YouTuber Upper Echelon Gamers, a QA firm allegedly misled CD Projekt Red during the game’s development. But in response, the quality assurance company’s CEO, Stefan Seicarescu, has stated that this is all just a big misunderstanding, according to a VGC interview.

Upper Echelon Gamers posted a video on June 25 going over some deets he got from a whistleblower at Quantic Lab, a Romanian-based outsourcing quality assurance testing team. Quantic Lab has had a hand in ironing out bugs in some big-name games, including Desperados III, Destroy All Humans!, The Witcher 3: Wild Hunt, and Cyberpunk 2077. The whistleblower, an anonymous Quantic Lab employee who UEG believes to be authentic based on a series of documents they claim to be in possession of, including a 72-page quality assurance testing file and detailed spreadsheets tracking worker productivity, explained in a June 23 email to UEG what went down during Cyberpunk 2077‘s QA testing.

Read More: CDPR Says ‘Vast Part’ Of Fixing Cyberpunk 2077 Is Done, Focused On Other Projects

According to the source, things started getting bad for Cyberpunk 2077 QA testing around late 2019. Quantic Lab leads were apparently sent to Poland to work directly with developer CD Projekt Red. The team was supposed to consist of “veteran testers,” folks that had “extensive experience with quality assurance who understood the process and workflow,” UEG stated. However, those that showed up were allegedly “junior testers” who had less than a year or, in some instances, just six months of work in the field. According to UEG’s source, CD Projekt Red wasn’t aware of this junior tester team, instead believing they were getting veterans from Quantic Lab who had worked on The Witcher 3.

But it wasn’t just Quantic Lab’s QA department that caused hiccups in Cyberpunk 2077‘s development. Quantic Lab upper management allegedly instituted a “bug quota” policy that required each individual tester to submit no less than 10 bugs per day. The thinking was that the new policy, which inevitably overworked employees, would increase productivity and further polish the game. To accomplish this, though, testers bombarded developers with thousands of minuscule errors, from items clipping to missing textures.

According to UEG, the QA team focused too much on negligible or low-priority bugs to meet the quota. The source claimed the torrential rain of superfluous glitches drenched workers across departments. You should watch the full video.

In response to UEG’s video, Quantic Lab CEO Stefan Seicarescu lowkey told VideoGamesChronicle that the whole thing was a misunderstanding. He didn’t appear to deny or address any of the allegations specifically; instead, he clarified that the claims made in the video were full of inaccuracies about QA testing.

“The video published on social media as mentioned in your article starts with incorrect statements about Quantic Lab’s history,” Seicarescu said. “There seems to be a lack of understanding in the process of how a game is tested before its release to the market.”

Seicarescu said no global publisher leans on just a single QA team, suggesting CD Projekt might’ve recruited multiple groups to debug Cyberpunk 2077.

“Quantic Lab supports over 200 projects per year from several global leading publishers and continues to maintain a quality comes first approach to all the work we undertake,” Seicarescu said. “All our customer agreements are confidential but in general, global publishers are working with several QA outsourcing companies, not depending solely on one, in addition to internal QA resources at developer level in most cases. Each project we undertake is unique with regard [to] project requirements. Project direction is agreed and adjusted accordingly as per real time requirements with our clients. Quantic Lab always strives to work with transparency and integrity with our industry partners.”

Kotaku has reached out to CD Projekt and Quantic Lab for comment.

 

Comments

  • Company creates structure to incentive behavior.

    Humans comply, Company gets confused.

    Something about Sparrows and bugs and Rattle snake breeding springs to mind!

  • “Quantic Lab upper management allegedly instituted a “bug quota” policy that required each individual tester to submit no less than 10 bugs per day. The thinking was that the new policy, which inevitably overworked employees, would increase productivity and further polish the game. To accomplish this, though, testers bombarded developers with thousands of minuscule errors, from items clipping to missing textures.”

    As a QA professional for about 2 decades that includes experience as a Lead, I’ve never liked the “bug quota” as a KPI, for *exactly* this reason.

    If you impose this kind of thing on them, testers will deliberately hold onto bugs once they have reached their daily quota so they can log them the next day, or deliberately raise duplicates at the end of the day if they don’t have enough, or focus on raising frivolous minor bugs in low priority areas when they should be concentrating on the more critical ones that may be more difficult to reproduce first. And that’s exactly what it sounds like has happened here.

    It also depends entirely on what areas your QA staff are working in, as some aspects of the game are inherently more bug-prone than others. High level bugs might be easy to reproduce and investigate but low level engine bugs might take hours or even days to track down.

    The “quality” part of Quality Assurance does not mean “quantity” of bugs raised. That’s always been nonsensical to me. You don’t ask your programmers to write x amount of lines of code per day, or your artists to create x amount of triangles per day, after all. So you shouldn’t be asking your QA staff to raised x amount of bugs a day, either. Quality is how good the end product is, not how many bugs you raise.

    • As a QA professional of also near 2 decades – you are spot on. If they did this to a company I worked at today, I would resign almost immediately.
      (Almost, because I’d at least try to convince them otherwise)
      It shows an incredible lack of understanding of what quality is.

      They did at one place try to implement a KPI based on bug count, despite our words to the contrary, so my lead and I went nuts with it. 350 bugs each in a month. The dev team leads came back with words like “untenable”, “demoralising”. Point proven, KPI removed.

      It’s collaborative, not a damn competition.

    • I’m also a tester and have been for a stupidly long time.

      A couple of points- testers don’t add quality. If we are good at our craft we tell the team how much quality is in the product.
      If we are not good at our craft we either don’t find anything or if we do they’re things that add no value to fix.

      Each bug requires a lifecycle to be undertaken, such as triage, analysis, prioritisation, fix, deploy, re-test and (ideally) closure.
      This entire process is waste.
      It is best that the bug is simply avoided in the first place by having good requirements that are clear and comprehensive.

      The best way is to involve testers as part of the design and requirements phase.
      The worst thing you can do is bring testers in late and hope that everything will work out.

      Based on what I’ve read- cyberpunk had bad quality from the start and it was doomed.

      • Yes there is that too. QA needs to be involved from the start, not just brought in at the end. QA starts with the design process and they are as much a part of the development team as anyone else.

  • Translation:
    We do lots of things, with lots of people, that are good. We know what we’re doing, trust us. They shouldn’t just rely on us alone though.

    “Have some juniors, and they will raise you lots of bugs!” is not a good approach.
    The performance review process would have been a nightmare too.
    “I see you only raised 80 bugs this fortnight. Do you think you only deserve 80% of your pay?”

  • Upper Echelon Gamers… yikes that’s what counts as a gaming source these days?! That click bait artist?

Show more comments

Log in to comment on this story!