Cyberpunk 2077 Has Revived A Debate Over How Games Are Reviewed

Cyberpunk 2077 Has Revived A Debate Over How Games Are Reviewed

With the first round of Cyberpunk 2077 reviews — or impressions, in the majority of cases — officially out, the internet is starting to process the state of 2020’s biggest game. Along with concerns over the amount of bugs, division is emerging over the lack of clarity around the game’s console performance, the developer’s approach to transparency, and exactly what is being kept from the public before release.

A large running theme among the coverage earlier this week was that, right now, Cyberpunk 2077 is pretty buggy. Exactly how buggy varies from person to person. I ran into countless visual glitches and a quirk early on that stopped progress, with some occasional hard crashes to desktop along the way.

Others had a different experience. PCGamesN, as just one example, had more severe problems, with corrupted save games and bugged quests:

And that’s just the stuff that’s intentional. Even after a 49GB day-zero patch – the product of CD Projekt Red’s unfortunate last-minute crunch – Cyberpunk is still plagued with bugs. These include quest progression issues and save-file corruption – though fortunately it autosaves so often that I never lose too much progress – and an amazing variety of visual glitches of which floating objects are only the most noticeable. On one memorable occasion, I give a peddler of illegal snuff braindances two buckshot shells to the face, point-blank. Rather than his head exploding, he sinks gently into a squat against the wall, his mouth open in surprise. Though he’s dead, I can still talk to him and browse his wares – when I ask about business, the corpse’s head rotates to face me, jaw slack, then snaps back to staring straight ahead. It’s extraordinarily creepy.

Games of this size and scope are always going to be buggy. But there’s a difference between not being able to grab loot because they’re carrying too many bobby pins, versus not being able to move forward because the game hard crashes or randomly resets your PC (which happened to me a few times).

So Cyberpunk 2077 isn’t alone here. But what’s fuelled the fire is a decision that no console codes would be shipped out prior to release, which has brought a bigger spotlight on how games are reviewed, and precisely what can and can’t be said about a video game before its official launch.

Ghost of Tsushima
Ghost of Tsushima (Screenshot: Gabriel Esteves – Email)

Here’s the score. Whenever there is a review period for any game, there’s always some kind of agreement between the publisher, website, YouTuber, influencer, whoever the content creator on the other side happens to be. This generally covers some basics: don’t leak the damn game, don’t publish footage until day X and hour Y.

In other circumstances, publishers and developers may ask for additional conditions. Most of these are generally based around spoilers, and are generally pretty reasonable. A good example: reviewers playing Death Stranding were asked to only use official screenshots during the first major embargo, which lifted a week before launch. Once the game officially launched on November 8, press were asked to only post edited footage to the end of Chapter 3 at a maximum of 20 minutes. But before that? No direct screenshots, no direct video, and no B-roll was provided.

Another example: Ghost of Tsushima. Unlike other games, there were no preview or early embargoes for Sucker Punch’s epic. Reviewers were allowed to capture their own footage of any open world location, activity or collectible, but any quests, tales or the main story was limited to the “A New Horizon” mission, near the beginning of the game’s second act. But if there were bugs or thoughts or anything that happened after that point, reviewers were free to mention it, provided it didn’t include any spoilers. Similarly, capture was restricted to a total of 30 minutes of video, 20 screens, with a maximum of 10 GIFs — that could only be 10 seconds long.

It’s worth noting that you never hear about all of this, because the boilerplate language on these things often say you can’t even mention the restrictions.

If you’re getting early access to any big game, there’s always going to be some give and take. The whole process operates on a degree of trust. It’s also a commercial arrangement: the developer and publisher are relying on the audience of whoever follows that content creator/website/outlet/newspaper/whatever to help drive sales of the game. But it also exists as a level playing field. Embargoes keep things as even as possible, ensuring that a small Australian website or YouTube channel gets the same access as, say, Polygon or IGN.

Of course, there are instances where this doesn’t happen at all. There’s been scenarios where our US partners have been playing something not available to outlets in Australia, or instances where everyone gets access to the game, but only major US/European channels were included on an earlier round of previews, with others tied to a later, second embargo. It was prominent during the review cycle for the next-gen consoles: this website, for instance, was included in the first round of PS5 previews and review units, but missed out on the first impressions of the Xbox Series X hardware. This doesn’t include industry blacklists either, which can be global, sometimes regional and, in certain cases I’ve heard, restricted to individuals. (As in, an outlet won’t be specifically blacklisted, but a publisher may ask that a particular individual not be assigned a review of a certain game.)

I mention all of this because, for the most part, Cyberpunk 2077‘s review restrictions were actually pretty reasonable. Unlike every other game from a major publisher this year, CD Projekt didn’t impose any restrictions on user screenshots ahead of the initial review embargo. That’s usually a massive no-go for many studios: even the PC re-release of Death Stranding this year wouldn’t let users take their own screenshots, forcing everyone to publish hideously ugly letterboxed images with watermarks. (In case you’re wondering: indie games, and many BB games, typically have little to no restrictions at all, although many will often kindly ask that spoilers are avoided as a courtesy.)

There’s always debate on how long a review embargo should be. I’ve been pretty vocal about how six days was not even close to enough for Cyberpunk, but it’s not the first time that’s happened. There’s even weirder scenarios around certain publishers where a review embargo doesn’t lift until the day after its available for release locally. (This happened, for instance, with Jedi: Fallen Order.)

But still, all of this usually works in balance. Reviewers accept it not necessarily because of an implicit fear of losing future access, but because what’s asked is generally pretty fair. You don’t want to spoil things for your own audience, after all, and it’s very, very rare where an embargo actually stops you from doing your job as a reviewer.

Cyberpunk’s review terms meant images like this, captured on a GTX 1080 at a lower resolution, could be published. Normally, however, all you’d be allowed to see would be …
cyberpunk 2077 sex scenes
… this official shot. It’s clearly at a higher resolution and it’s from the same scene — when you meet Evelyn Parker at the beginning of the game — but it also might have no relation to the conditions, or hardware, people are actually playing on. A key difference here too: look at the background behind Evelyn, particularly to her left.

Where Cyberpunk has copped heavy criticism is for two things, one understandable, and the other not so much.

The first is around the lack of review coverage for console codes. This would have risen a red flag under any normal circumstances, but it was always going to be a problem given that CD Projekt specifically delayed Cyberpunk 2077 to December to allow for more performance optimisations on all platforms. Understandably, that does two things: it suggests that the older consoles were having a harder time running the game to an acceptable level, so much so that management saw fit to delay the game’s entire release.

So, obviously, people want to know how that shook out. For anyone weighing up whether to get the game now or later — especially if the experience will be tied into your access on a next-gen console or new PC upgrades — that’s a key factor. But it was also further highlighted when CD Projekt published footage of the game on Xbox One X and PS4 Pro, but not the original Xbox One or PS4.

Because of this, a couple of major reviewers publicly said they wouldn’t be posting their reviews, or said they’d declined access to the review code entirely. That’s a big deal: even though the review period was relatively short, every hour with a game as big as Cyberpunk counts. That’s especially true if making content is your daily job, because it means you miss out on the algorithm-driven views that naturally come from the entire gaming industry all talking about the one thing, all at the same time.

Jeremy Penter, creator of the ACG YouTube channel, said on Reddit that he’d struggled to get a code, despite the publisher initially reaching out. “I could not figure out why I was having so many issues with getting code when they had asked me to cover it originally. Then was discussing it and realised, I cover all the platforms and had discussed console versus PC in depth and what I wanted to do in later videos,” Penter wrote.

Australian reviewer Ralph “SkillUp” Panebianco added that he wouldn’t be publishing his review until he could show “the reality of the game with my own footage”.

The release of B-roll footage is completely standard, and again, happens with preview and review footage. Another major example: reviewers could only use official B-roll footage for their The Last of Us 2 reviews, and user-taken screenshots were forbidden as well. But none of that even came up in discussion, because a much bigger talking point was a restriction around the second half of the game, which prevented any public discussion around the switch from Ellie to Abby.

Even amongst the hardcore PC crowd, concerns have been raised over the game’s performance. A few outlets have noted that the game was supplied with Denuvo DRM. One outlet, Tom’s Hardware, published initial benchmark figures but later removed them “at the request of the publisher”. Some parts of the internet have picked up their pitchforks over that alone, questioning why DRM is being used given CD Projekt Red’s ownership of the DRM-free storefront GOG, with some even saying the official PC requirements were misleading.

cyberpunk 2077

But these complaints are lacking context. For one, it’s bizarre that Cyberpunk 2077 is being criticised for using DRM before launch when they’re not the only publisher to do so. You didn’t see, for instance, any complaints against Bethesda for using Denuvo DRM during the review period. DOOM Eternal‘s Denuvo was patched out months after release, but nobody criticised the studio for locking things down before the game was released. Devil May Cry 5 used Deunvo as well, and while nobody likes it, it also didn’t receive any of the flak that CD Projekt Red are getting now.

Given that Cyberpunk 2077 will exist on a platform that will inevitably end up being copied, torrented and patched a little more easily courtesy of how GOG operates, it seems deeply unfair that they’re copping criticism for this one. Similarly, Denuvo DRM having a performance impact isn’t new to Cyberpunk. It’s not always a big deal, but the performance hit is there. But because CD Projekt Red — and Nvidia, as well, who supplied reviewers with guides about the game’s ray tracing features and performance prior to launch — were open about Deunvo’s potential impact, it’s become a talking point. They’re effectively being criticised for their transparency here. That’s deeply regrettable given Denuvo’s being patched out for launch anyway, something other games often take weeks, even months, to do.

And by the way, there’s the other side of the console coin here too. Cyberpunk 2077 isn’t the first game to withhold code from a certain platform until launch. Consoles were excluded this time, but over the last few years — at least in Australia — local distributors and publishers have sometimes excluded access to the PC version of a game at launch, whereas PS4 or Xbox One codes would be distributed days, sometimes a week or more, beforehand.

But it also puts a spotlight on the reviews process, and what information should and shouldn’t be in the public eye. Is it fair to reveal benchmarks considering the Deunvo overhead, but also considering the game has been patched over the review period, and that multiple Nvidia drivers have been made available prior to launch — and none of this includes any potential optimisations from the day-one patch?

I’d argue that it’s not. It makes sense to mention what the experience was like, of course — if the performance wasn’t enough to reasonably play with ray tracing enabled, that’s definitely worth mentioning. And the Cyberpunk restrictions were generous enough that people could actually show the difference in still shots, something many other publishers don’t and haven’t allowed.

None of this, of course, excuses that customers have every right to see and know whether a game runs satisfactorily well on their hardware before making a purchase. But the whole episode just highlights how little we talk about how games are reviewed.

Cyberpunk 2077 has found itself in the spotlight, primarily because it’s permanently hogged it for the better part of the last few years. So it’s going to receive an extra degree of analysis and criticism, not because a blockbuster production of this size doesn’t deserve it but because other major titles have skated under the radar in a way that Cyberpunk isn’t being allowed to.

There’s always the hope that the extra focus on Cyberpunk creates an environment where publishers going forward are more open — having seen the criticism Cyberpunk received — but if my experience of the industry is any indication, I doubt much will change. That said, sometimes there’s progress: the public discourse and backlash meant publishers stopped organising “review trips” for games, resulting in fairer, less clouded coverage of those titles.

Ultimately, many of the launch issues surrounding Cyberpunk 2077 will be rectified in due course. Anyone who played the launch day, or pre-release versions of The Witcher 3 can attest to the amount of performance improvements and optimisations that game received, not to mention massive quality of life fixes with the UI and animation system. And the experience of games like No Man’s Sky, Star Wars: Battlefront 2 and even Fallout 76 are worth remembering: everything is, and always has been, a work in progress. (That said, some fixes are way more important than others.)

Cyberpunk 2077 will continue being the centre of debate, discussion and division for weeks and months to come. That’s a good thing — it opens the floor to a better, more open dialogue what about games are, what they could and should be, the way they’re marketed, and how we as a community deal with it all. But we also need to talk about the wider industry, how the review process works and how different publishers and games try to put their own stamp on what you, the consumer, ultimately end up seeing.


The Cheapest NBN 1000 Plans

Looking to bump up your internet connection and save a few bucks? Here are the cheapest plans available.

At Kotaku, we independently select and write about stuff we love and think you'll like too. We have affiliate and advertising partnerships, which means we may collect a share of sales or other compensation from the links on this page. BTW – prices are accurate and items in stock at the time of posting.

Comments


24 responses to “Cyberpunk 2077 Has Revived A Debate Over How Games Are Reviewed”

Leave a Reply

Your email address will not be published. Required fields are marked *