The recent Cyberpunk 2077 review fiasco has people asking (again) why reviews of big budget games are what they are. Let’s talk about it.
This won’t be the first time I’ve covered the topic here. About eight years ago, halfway through the Obama administration, I wrote about the strings attached to many major video game reviews — the agreements review outlets make with game publishers in order to get copies of games days or weeks before a game comes out. It’s weird and fraught, arguably necessary. Usually, in exchange for an early copy of a game, the reviewer agrees to a date before which they won’t run their review, is asked to not spoil things, and maybe fields a few other requests, reasonable or not.
Examples I shared in 2013 included Microsoft asking reviewers to not mention Halo 4’s prologue, Nintendo barring Kid Icarus: Uprising reviewers from mentioning certain weapons and powers, or, in a throwback to 2008, Konami prohibiting Metal Gear Solid 4 reviewers from saying how long the game’s cutscenes were (spoiler: really, really long).
People depend on game reviews to help understand if a new game is worth playing or buying, and the most effective means to provide game reviews is still to somehow get game makers to send copies early. That’s what leads to all of these review agreements and the need, once again, to talk about them. After all, readers deserve honest and open reviews, and part of such an approach should involve making it clear what we can cover in a review — and what we agree not to.
Nearly a decade since I last covered this, some things have changed, some have not, yet much of how this all plays out remains invisible to most people who read game reviews. That alone makes it worth exploring the topic again. There’s also a renewed question about whether these agreements do more harm than good. We agree to many of them at Kotaku, but not all, and have worked through our own ever-evolving process regarding all of this.
The Cyberpunk situation, to reactivate your memories of December 2020, involved publisher and development studio CD Projekt Red providing reviewers with PC copies of its ballyhooed new game a week prior to release. A week isn’t a lot of time, but it’s also pretty standard.
The really bad part was that it only sent download codes for PlayStation 4 and Xbox One review copies barely a day before the game came out and — this is key — after reviews of the PC copy were able to run.
Here’s what we wrote in our day-one impressions, which we intentionally refrained from calling a review due to the various constraints and console blindspots:
Kotaku got the game less than a week before embargo, and only on PC. (CDPR has not sent us code for the console version of the game, though we’ve been asking.)
Those console versions ran so poorly that CDPR later apologised for them, and the people at PlayStation, at least, removed the game from their digital store. CD Projekt also barred reviewers from showing footage they captured from the game, though screenshots were permitted.
To some onlookers, CDPR had gamed the system, and worse, reviewers let them.
CD Projekt’s approach to supplying Cyberpunk review code wasn’t exceptionally unusual. I don’t mean to exonerate it — as it allowed for a pretty bad version of a major game to slip past reviewers — but some perspective here may help, especially to establish a better process for reviews in the future.
The lead time for playing and reviewing the PC copy of Cyberpunk was average. It was tough, mind you. Playing through a game that’s dozens of hours long in a week is a challenge, as few outlets are set up to have reviewers drop everything to play one game. But we’re offered plenty of games with less lead time, and given the production timelines of games, we can’t expect publishers to send developers’ work far in advance of it reaching a shippable state.
For Cyberpunk, the review embargo lifted a few days before the game was out on any platforms. Some publishers do it that way, others won’t let reviews run until the minute the game is out in New Zealand, the first place on the planet where the date changes over to release day.
Review embargoes that only lift on release day inherently block a reviewer’s ability to show anything of the game before the public can buy a copy. In that regard, while Cyberpunk’s restriction on reviewer-captured video footage was disappointing, the company had at least agreed to let reviewers run screenshots from the PC version and let them run a couple of days before release. That approach is preferable to pre-release review agreements, such as The Last of Us Part II‘s, that limit visuals to company-provided screenshots.
It’s common for companies to not offer every version of a game. Sometimes they’ll ask which platform you want, or that may default to an outlet’s preference. If they’re blocking one version, it can be a red flag that something might be amiss or that marketing wants to focus coverage on a certain version. Sony, for example, happily sent out PS5 copies of last November’s Spider-Man: Miles Morales, but if you wanted PS4 copies, you had to ask. The game ran fine on PS4, but Sony clearly wanted to tie talk of that game with talk of its new console. Ubisoft sent code for every version of Assassin’s Creed Valhalla prior to its launch, except for PS5 code, ensuring that release-day coverage of the game’s next-gen performance would focus on how the game ran on Xbox Series X, the platform where it was marketed as a major launch game.
The refusal to send console code was the most uncommon aspect of the Cyberpunk review process, though we’ve also seen many other instances of publishers holding back review code as they wait for a final pre-release performance patch to land. That’s what appears to have happened with Cyberpunk’s console code, though the game’s pre-release patch didn’t sufficiently improve the game.
Some game makers simply don’t let reviewers play any version of a game before it’s out. This is an increasingly common approach, especially for games with significant online components, as publishers will say that their game won’t run — or can’t be fairly judged — until its servers are turned on. Because of that, we don’t get Destiny expansions any earlier than the public does. Even this year’s Call of Duty didn’t come early.
CDPR’s approach, paired with a brazen willingness to ship a console version of horrendous technical quality, exposed the pitfalls of these arrangements. While they do help ensure game developers’ work will be fairly critiqued only after it’s ready to be played, such policies are also vehicles of marketing and messaging. They are the means by which reviews of the most highly anticipated game of the year were barely able to mention that, on the platforms most people were going to play it on, it ran terribly.
Why would anyone agree to this stuff?
I can answer most authoritatively for Kotaku, of course. The answer, simply, is time. Time is the big factor, because games, you may have noticed, are very long, and so a fundamental challenge for any outlet — of any reviewer and any editorial team around them — is figuring out how to find the time to play a video game thoroughly enough to have something illuminating or helpful to say about it by the time people want to know whether to try to play it themselves.
Review outlets regularly grapple with the logistical lunacy of trying to review, on deadline, works of art that take 20, 40 or even 80 hours to experience. Reviewing is tough work in any field, but I nevertheless envy the movie reviewer who can start and finish what they’re reviewing in three hours. I’m jealous of the music reviewer who could spend a week listening to an album to just finish it, but probably doesn’t have to. I laugh at the luck of the book reviewer who may need dozens of hours to get to the last page, but who probably doesn’t need to troubleshoot why the pages in chapter eight didn’t turn correctly, nor figure out how to elegantly note in the review that the vowels on pages 59 and 213 sometimes show up as fish, though maybe that’ll be fixed in the next printing.
It simply takes a lot of time to review games, so you always want to get the game as early as possible, even if you can’t review it in time. Game companies and creators know this and take the opportunity to influence whatever you might write, some farmore egregiously than others.
So, here’s what happens: When we want to review a game, we contact a game publisher or indie developer and ask for a download code (companies rarely send discs or cartridges anymore), or sometimes they send it before we’ve asked.
An indie developer will usually reply with a code and maybe the lightest of restrictions: often just the date the review can run. These were the restrictions for the November indie Bugsnax, as provided along with code by the game’s PR agency:
REVIEWS: Video/written reviews and all deep dive coverage for Bugsnax are embargoed until MONDAY NOVEMBER 9TH AT 6AM PACIFIC, 9AM EASTERN
*** Bugsnax is a narrative game. Please avoid story spoilers.
We got that code a luxurious 12 days before reviews were permitted, and 15 days before the game came out.
A bigger-budget publisher will often include more restrictions, or at least be more specific. Ubisoft, for example, sent out review code for Assassin’s Creed Valhalla 12 days before when reviews could run, which was also release day, at least in New Zealand. Ubisoft sent along a 31-page reviewer’s guide that included suggestions about what reviewers should check out. It also included a list of 13 spoilers that the company requested reviews not cover, including aspects of the ending and the true nature of specific characters.
Most of the time, these content restrictions involve spoilers that many reviewers wouldn’t get into anyway. If they are too constraining, there’s always the option to play the review copy early but hold the review until after the restrictions are lifted. That’s never later than release day.
In my article eight years ago, I noted the Nintendo restriction on the 3DS game Super Mario 3D Land would have prevented our reviewer from even alluding to the fact that the surprise second half of the game was the best thing about it. We simply delayed running our review until the game was out so we could note that.
Holding a review back can be tough for outlets that rely on a traffic surge when a review embargo lifts. Outlets want to be in those Google searches and review round-ups, though a bigger outlet like Kotaku can afford to sit on a review and still get traffic when it does publish.
Aside from delaying a review, an outlet can just run impressions when a review embargo lifts, signalling to readers that we have thoughts about a game but aren’t ready to assess it in full. Frankly, most launch day reviews these days are similarly incomplete, since so many games change after release.
When we recognised that the Cyberpunk review situation made it nearly impossible for us to run a review we’d be happy with prior to the game’s release, we only ran an “impressions” / “review in progress,” and took our time to write and run an actual, complete review of the game on Christmas Eve, two weeks after the game came out.
Then there’s our review for The Last Of Us Part II, which we were able to run a week prior to release, same as our competitors. We chose to do that. There were lots of strings attached, hence this paragraph in our review:
If you’ve read this far into the review looking for more details about just what this game is about, there’s a catch. Even though many of the game’s plot details were unexpectedly leaked via a hack in late April, a condition of us being able to review the game a week early with a Sony-provided copy is that we can’t spoil certain things. There’s plenty in the game I wouldn’t have spoiled anyway, since it’s full of events that are meant to surprise. And I can say that, having dug into those leaks, there was plenty in the game that surprised and shocked me. But key parts of the game’s story are off-limits here — not just how the game ends, for example, but also how it begins — as are large portions vital to explaining why I felt how I did about it. People who’ve seen the leaks may know some of this — though not all — but for the purposes of this review, there’s a whole lot to talk around.
Our review was able to cover a lot, but restrictions that tight are deeply frustrating. Our experience with that helped inform our decision to hold off on the Cyberpunk review.
A sort of side note: Sometimes the restrictions on a review are as weird as they are minuscule, making the decision on how to handle things more challenging. One that sticks out in my memory, from 2016, is a proviso included from Sony along with a review code for Uncharted 4. Sony’s PR asked reviewers to not mention who lives, dies, or falls in love — ok, fine? — but, more extraordinarily, requested that reviewers avoid acknowledging, “Whether [sic] exists a supernatural element, or the nature of any supernatural elements that may be in the game.” The twist they were trying to preserve, as best I could discern, was that while previous Uncharted games included supernatural elements that some fans disliked, 4 didn’t have any. And Sony would rather we not get into that. That restriction felt excessive, too heavy-handed in shaping how reviewers discussed the game, and I regret not objecting to it. But I also don’t think it was significant enough to delay the review and deprive our readers of our take on this game for several more days.
The companies we deal with for access to advance copies vary widely in approach, and some deliver that ultimate reviewer’s cliché: a mixed bag. Sony and Nintendo can include some frustrating restrictions (oh, the time Nintendo limited video clips of a new Zelda game to 30 seconds max), but they provide game code very early, which helps our reviewers a lot. Nintendo’s short turnarounds are two weeks, and the company often has code available up to a month in advance. Atlus sends code really early, too.
Ubisoft tends to go light on content restrictions, as does EA, though the latter also make the kind of games that are tougher to spoil.
Take Two is a trip, to put it nicely, because it tends to send the most onerous of review agreements that it swears are just about keeping the contents of a pre-release game from leaking, but its dense document for, say, 2014’s Civilisation Beyond Earth, includes the following description of what “confidential information” is constrained by the agreement:
The term “Confidential Information” includes all analyses, compilations, studies or other documents prepared by the Recipient or any of the Recipient’s Representatives which contain or are based upon (in whole or in part) any information which is furnished or made available by 2K or any of its representatives, including, but not limited to, any trade secret, information, prices, technique, algorithm, computer program (source and object codes), game content and features, screen shots, game concepts and storylines, programming techniques, design, drawing, formula or test data, relating to any research project, work in process, future development, engineering, manufacturing, marketing, servicing, financing or personnel matter relating to 2K, its present or future products, sales, suppliers, clients, customers, employees, investors or business, whether in oral, written, graphic or electronic form.
Yikes! By my reading, they’re not just gagging the person with the Civ code from covering the game, they’re gagging them from talking about anything happening at 2K that their reporting might uncover. Another part of the agreement even refers to the information governed by the agreement to be anything the reporter gets “at any time before or after the date hereof.”
2K PR always gets an email from me complaining that their agreements would seem to pertain to a whole lot more than just the review copy of a game, and sometimes we figure out a workaround, as we did in that case.
That 2014 Civilisation: Beyond Earth agreement had another unpleasant part: a gag clause on acknowledging the agreement’s existence for five years. Such agreements really should be avoided, as should any that make it needlessly hard to be transparent with readers.
That’s a look behind the scenes. But what do readers want? I believe readers want timely coverage of games. They want reviews from reviewers who aren’t rushing through a game, who’ve had the time they need to experience the work and collect their thoughts, and who can write and speak freely. They want reviews that aren’t all constrained into commenting on and covering the same narrow range of publisher-approved topics.
To be able to do that, we really need to be able to get games in advance, and therefore will remain in this kind of push-and-pull with the game creators who provide access to code. It’s just the nature of it, and, if outlets are transparent, I believe it can still serve readers well.
I do hope readers want a reasonable amount of transparency about this, that they want an approach to reviews that acknowledges what we can and can’t talk about. And I hope that such an approach, long Kotaku’s goal and one more explicit as I put it into writing here, can contribute to critical coverage of games that respects game creators and players alike, while ensuring reviewers can do their job in a reasonable way.
And, hey, if the people providing games don’t see it that way, outlets can always get games after launch. We hope readers will continue to show up for that coverage with an understanding of why and how it gets to them.
Leave a Reply