We’ve changed the way we review video games here at Kotaku. That makes this a perfect time to explain our approach to reviews and how the new system will work.
Two things will stand out to those who have read last week’s Resident Evil Revelations review and this week’s review of Final Fantasy XIII-2:
- They appear to be in two entirely different formats.
- They both have the rather large word “YES” on them, as if that was…. a… score??
Hell of a system, huh?
Here’s how it works: Kotaku is staffed by talented writers whose job it is to write about a radically diverse range of games. They review role-playing games, music games, online-centric games, sports games and more, and I believe that our writers should have the flexibility to review those games in the way that best suits the game. They can review the game by writing an essay or writing bulletpoints; they can review a game with a poem or a comic strip. The format for the main part of their review will conform to whichever approach best suits the reviewer’s voice and the game they’re writing about. That’s why the two reviews linked to above look different.
Under this new system, I expect each Kotaku writer’s reviews will become more instantly recognisable and associated with that writer. And why not? Each review can only ever be the opinion of one person, so it’s all the better if regular readers can develop a sense of the reviewer’s style and taste.
But many readers — most readers, I’d wager — are short on time and don’t have the patience for creative writing. They just need to know if the game is worth playing.
For those readers, we will have one of three answers: YES / NOT YET / NO.
Every Kotaku review will include one of those three answers atop a sidebar that cuts to the chase. If you are in a hurry, the sidebar is the only thing you need to read. It will tell you who made the game (though, intentionally, not who published the game), what it’s like, when it’s out, how much we played of it and which two things the reviewer most liked and most disliked about the game. It will answer that worth-playing question and tell you, succinctly, why.
Each review sidebar will also include a collection of made-to-order back-of-box quotes, because… because they’re funny.
So, yes, we are finally scoring our reviews. For the longest time, Kotaku avoided putting a number on reviews. The editorial team was worried that people would just read the number and skip the words. They feared that the meaning of that number would be warped and that passionate readers would ascribe reasons for that number that had nothing to do with the reviewer’s feeling about the game.
In short, they worried that the number would undermine the review.
Screw the number. You really only have one question about these games, anyway: Is it worth playing? And we have those three possible answers for you.
I expect that we’ll say YES to many games. After all, we’re kind people here and we think many games are worth dabbling with in some way. You might prefer that we tell you whether the game is worth buying — or whether we’d buy it. But the problem with the first, we’ve learned, is that we don’t know how much money you have. The problem with the latter is that, as reviewers, we probably didn’t have to buy the game we reviewed in the first place… so what would we know?
We may not say NO a lot, because how bad does a game have to be to be not worth playing? And if it’s that awful, maybe it is worth playing (remember, we will explain why we’re giving an answer).
I predict we’ll use NOT YET a lot, as a way to ward people off from games that feel not worth full price (income caveats notwithstanding) or is too buggy and in need of patches or has an online component we’ve not yet been able to assess. Reviews labelled NOT YET may evolve into YESes or NOs, and we’ll notify readers of changes when they kick in.
Reviews will be timely, but we don’t want to make the Assassin’s Creed mistake. A few years ago, Mike Krahulik (Gabe) over at Penny Arcade zeroed in on a potential reason for the negativity in many reviews of the first Assassin’s Creed:
Imagine what an open ended sandbox title must look like to a reviewer especially right now. How many games do they have piling up on their desks? A game like Assassins Creed isn’t meant to be played under a deadline. You shouldn’t be trying to beat it as fast as you can so you can move on to Mass Effect or Mario Galaxy. As soon as I gave myself a deadline all of a sudden I understood all their complaints. It was like a fucking Escher painting. I had put myself in their shoes and suddenly the landscape flipped and I could see games from their perspective. In the end I wasn’t angry at them for their bad reviews. I actually just felt bad for them.
Mike was right. Sometimes, when you’re reviewing a game on deadline, you start to resent the game, because you are reviewing it on deadline.
I’m not guessing this. I know this, because it has happened to me.
If I get a copy of a new game from a publisher and have a few days to play it before the obligatory day-one review, sometimes I find myself still playing it at 2am the night before the game review needs to run. I’m fed up with the game by then, just wishing it will end so I can get some sleep and then write the review already. If I was you — if I’d bought the game and had no deadline — I’d be delighted to discover there’s a whole extra chapter or that what I thought was the final boss battle was just a fake-out leading to a bigger tussle. But, as a reviewer on deadline, I hate that kind of thing. I just want the game to be done. I’ve been there, and I don’t want to risk that these kind of feelings will contaminate our reviews.
Yes, our reviews will be timely. Yes, they will be timed to run when a game comes out or as soon as a publisher who sent an early copy allows early reviewers to collectively run their reviews — whichever comes first. But if the game is too long — if it’s just beating up me or one of my colleagues — we will delay the review. We will not risk penalising a game because we had to play it in an unnatural rush. If it’s a big game and we feel we have played enough to say something about it, we might run a NOT YET review and be crystal clear that we’re not done with the game. We would update the review later. But that will be a rarity.
I’m proud of how we’ve reviewed MMOs, and we will be expand a version of that approach to more games. For any big MMO, we would wait a month to review the game, writing diaries in advance. We did this because we accepted that MMOs evolve and are shaped by their online communities. The same is true for more and more shooters and other online-connected games. Our reviews must take this kind of evolution into account.
You can therefore expect that games that have some evolving or major online-centric component will be flagged with a notice that indicates that the review will be updated a week later. Those reviews may have a NOT YET, though if you consider a game like last year’s Assassin’s Creed: Revelations I would have YESed it for its fun singleplayer but still included a note that I’d add a fuller assessment of its multiplayer a week later. Any additions or changes made to the review will be marked clearly.
A few years ago, it would have been sacrilege to say we’d be willing to update our reviews. But if games can evolve week by week, so too can our reviews. For now, we will confine that evolution to one week, as we check in on a game a week later if the structure of the game merits that. If we think we need to check in on games beyond that, then we’ll… evolve this policy, too!
I hope you will enjoy and appreciate our approach to reviewing games. I’m not so worried about whether it is better or worse than how other outlets run reviews. I’m focused on whether our new, refined approach suits the values I have for reviews:
Reviews must be interesting.
Reviews must be honest.
Reviews must be timely.
Reviews must be easy to consume if you just need some shopping advice.
Reviews must be worth reading.
And, sure, reviews would be better if they had some ready-made back-of-box quotes in them.
![[Razor] Avatar](https://secure.gravatar.com/avatar/2af54a459dd1e0d262592d6a7321f7cb?s=40&d=https%3A%2F%2Fwww.kotaku.com.au%2Fwp-content%2Fthemes%2Fgmg%2Fassets%2Fimages%2Fkotaku%2Favatar.png&r=g){kind=avatar}
Leave a Reply