Review scores: they’re not just arbitrary and meaningless, they’re toxic to discussion, too. Grand Theft Auto V reviews came out today, and with them came a whole host of poisonous Internet comments not about the content of those reviews, but about the numbers attached to them. Arguments raged hard: is this game worth a 10? A 9.5? Holy crap, did that reviewer really give it a 9?
I’ve already written extensively about how review scores are hurting the games industry, but this is another issue. Review scores, the numbers slapped onto the ends of video game critiques like spoiled mayonnaise on a perfectly good pastrami sandwich, are problematic because they lower the level of discourse. They make us all dumber. And if we all did away with them, we’d be forced to discuss games by their strengths and weaknesses, with the sort of nuance that review scores make impossible.
Inspired by today’s reviews, let’s look at some reasons I’d love to see all review scores go away.
1. They encourage people to draw comparisons between experiences that can’t be compared.
Is Grand Theft Auto V, an open-world mayhem generator, better than Gone Home, a quiet, focused story about a girl and her family? There’s no answer to that question. They’re totally different types of games, and they should be judged as separate entities, critiqued and examined through totally different lenses.
Yet because of review scores, and because of score aggregators like Metacritic, gamers feel encouraged to stack them side by side, to judge that because one game has a better score. See this comment on GameSpot today:
Of course, any rational gamer can recognise that those two games can’t be compared, but with review scores around, can you really blame people for drawing the connection? GameSpot gave Gone Home a 9.5 and GTA V a 9.0. That’s a fact. One is a multi-hundred-million-dollar blockbuster with 38-minute-long credits while the other was made by four people in a house, but because game reviewers are quantifying quality, we’re encouraged to compare the two. One is half a point higher than the other, therefore one is half a point better than the other. How else are readers supposed to interpret that?
2. They discourage nuance and criticism.
Review scores drive us to treat game criticism like an Olympic competition. Will my favourite new product score a 10? Will it get docked points for doing something wrong? Did the developers do enough to win the gold medal?
When we try to quantify experiences, there’s no room for embracing the notion that we can love flawed games. We leave no room for nuance in criticism, because any attempt to look at a game’s triumphs and failures is met by people wondering how those features affected the score.
Look at this Polygon comment:
Or maybe it’s not a number but a video game, a fascinating blockbuster video game that is worth discussing and dissecting, and maybe we should play it and look at it and try to figure out whether it does make people uncomfortable, and whether that’s a good thing, and what it has to say. When Polygon gives GTA V a 9.5/10, that number is and will always have baggage attached to it, and readers will forever want to talk about what it did to deserve losing half a point.
3. They discourage criticism that strays from the norm.
Today, the Escapist‘s Greg Tito made the cardinal error of giving Grand Theft Auto V a score that diverged from what people wanted it to be. Instead of offering up a perfect or near-perfect 10, Tito gave the game a shocking — wait for it! — 3.5 out of 5. Three-point-five stars.
Naturally, commenters flipped out. How dare he diverge from the community’s expectations?
Imagine how people might have reacted if that review critiqued the game without giving it a score? Would they perhaps talk about that criticism? Would they discuss the contents of that review instead of arguing over whether it really should have been closer to a 4/5?
4. They contribute to weird tribalism.
In the world of gaming, people like to attach themselves to companies and groups. We have a tendency to cling to games and developers like they’re sports teams or wrestlers. We want to see them beat out the competition.
Review scores exacerbate this problem, and lead to reactions like this, via Destructoid:
And the Metacritic War continues.
OK. So you might be thinking — these reactions are insane, yes, but that’s the fault of crazy commenters who overreact to review scores, not the scores themselves.
I disagree. I think this problem starts at the roots, and I think that by distilling the essence of an experience — a subjective, personal experience that, because of the nature of video games, only we will ever have — into a number, we do a disservice to video games and to the people who play them. By saddling video games with scores or points or whatever we want to call them, and by attempting to quantify experiences, we do harm both to discussion and to the way we perceive and compare and analyse the games we play. We need to make review scores go away.
Comments
29 responses to “The Problem With Review Scores, Part V”
Heh. One of my favourite games of the year was Deadpool which scored …*checks Metacritic* oh dear.
Yet I had fun with it so whatever. If people want to let review scores dictate their own fun with a game let them I say. Only upsetting themselves.
I think review scores are a reasonable metric but the problem is people think that’s the ONLY metric. If they’re hyped for a game and it doesn’t get a 10/10, then their entire world comes crumbling down, as if they bought the wrong smartphone or drive the wrong car (HOLDEN FTW, FORD SUCKS).
I don’t understand it one bit, especially when they do this “oh GTAV only got 9.5 so Gone Home must be better” bullshit.
I had a 9/10 orange but man… apples. Even a 7/10 apple beats a 9/10 orange. You would have to get me a whole bag of 10/10 oranges before I’d feel differently about my apples, because a 10/10 apple is a whole other thing, man.
I agree, I loved Deadpool, but had slight misgivings before I played it due to the poor reviews the game received.
If I ever start a games review website I think I will give abstract symbols as review scores instead numbers.
I award this comment Ω / 10
surrealistgamer.net gave GTAV a green treble clef? LOL, I guess that means it’s worse than Gone Home which they gave a red flounder. Credibility just took a massive nosedive!
I went to that link and was so disappointed that it didn’t exist 🙁
whenever I ‘review’ something, the score i give it is whatever i’d give the personification of that game / movie / book / whatever for their birthday.
Another problem I see is reviewers using trollbait for example that Gamespot review for GTA V, all they have to do is use words like sexism and misogyny and the article will explode, it’s a pathetic way to attract sensation to an article.
…or maybe there is a lot of sexism and misogyny in GTA V?
You’ll notice they don’t talk about sexism and misogyny when reviewing most other games.
Thing is, I don’t need a reviewer to tell me to like a game or not. I can make up my own mind.
I tend to agree with the gist of the article, but Greg Tito is a bad example. His reviews always generate the same response, and I think its because its never obvious to the reader why he assigns the scores he does based on the text of the review. That to me is his failing.
That is, he makes the scoring system even worse, because it has no obvious basis.
I’ve never read one of his reviews before, just read that one, and I understand perfectly why he gave it the score he did.
I also love that he reviewed this game as art, not as a mechanical toy.
His whole review basically comes down to: technically brilliant, bad artistic choices. Which I find helpful and which confirms my impression of the game.
A score exists because people are lazy. I wish it wasn’t so but what are you gonna do when we rate everything, not just games, on a scale of 1 to 10 even subconsciously. i.e. I had a better burger at this maccas while the other maccas that I went to didn’t put enough McMayonnaise on it, therefore I would rate the former higher than the latter.
I much prefer a review with meaningful discussion about what it did well and what it did poorly but on the whole people would much rather look at a number on a scale as justification to their purchase.
People also have likely made up their mind long before a review for a game comes out. They know if they are planning to buy it or not already, they also have some preconceived faith in the game being awesome even if it drops the ball and can’t see past their ego to the faults in a game. These are the people that are in uproar over other people not believing it’s as amazing as they have convinced themselves it is, even if it is not.
tl;dr
Reviews are opinion pieces, when people see opinions that don’t align with their own, they go on tilt.
edit* for punctuation, apologies to the eyes of anyone who read it before I fixed that second paragraph
A slight disagreement on your first sentence, they also exist because people got tired of shelling out $100+ for a game that wasn’t worth the CD case it came in.
Thus, review scores are a form of early warning for others, saving us from the likes of Duke Nukem Forever.
Like any opinionated advice, reviews need to be balanced with discussion, which is where the rest of your comment is absolutely spot on.
Well put. By and large, scores distract from the review itself, and end up with commenters dissecting the score against the reviewer’s reputation, rather than focussing on the thing they’re all interested in – the game. You rarely see comments like “Wow, that score was rather low, I wonder what portion of the game warranted such a reaction?” Instead, you see “Wow, that score was rather low, the reviewer is clearly insane” or “… the reviewer clearly is overly sensitive” or “… the reviewer blah blah blah”.
Why not “… the game must cover some controversial themes that merit further investigation or discussion”? Because reviews are mainly about helping people decide whether to purchase the game or not. People can’t discuss the nuances of the game from a position of ignorance (ie. if they haven’t played it yet), so they attack the reviewer. The other problem is that the people who have played the game, and read the reviews, are looking for confirmation of their own opinion to rationalise their expense, and are still fresh from the game with eyes clouded by adrenaline and hype. When they find that a reviewer is more critical of the game than they’re prepared to be at the time, they attack the score.
I understand the need for comments sections on reviews, as it allows the community to balance the criticisms in the review against how the (generally less critical) masses reacted to the game… but can we put spam filters over those comments sections for whoever attacks the score? It may be a number, but that doesn’t make it quantitative analysis. It’s still subjective, and should be treated as such.
I feel like a numerical score is fine as a quantifier of how well a game sets out to achieve what it intends to achieve. Candy Crush Saga is a totally different game to The Last Of Us, but how good are each of them at being their own individual type of game?
You could justifiably compare, say, GTA V to Sleeping Dogs – they’re going for the same kind of thing, how good is each of them at that thing? By the same token, you might give Sleeping Dogs an 8 at what it does and Candy Crush Saga a 9.5 at what it does, but if I’m looking for an open-world kind of game then Candy Crush Saga isn’t going to be better for me.
The problem with this is that it requires defining what it is a game sets out to do, and that can be difficult. And also you still end up with a numerical score that people will take as a straight value judgement anyway, regardless of your intentions.
To be fair, the Gamespot review is somewhat interesting as Carolyn seems to be making an effort to point out issues with “misogyny” – regardless of the parody nature of those inclusions.
I can’t really find any other review that thinks that was an issue.
Polygon did too. There was another review which looked at it and thought that Rockstar did alright (can’t remember which). It is an issue to some people, is the point; especially when GTA is considered such an influence on other games.
More interesting were the comments saying that as a transgendered person, she doesn’t deserve to be a journalist and is ruining a website that children frequent. Just because she didn’t give the game a perfect score. Sad stuff
The quicker we do away with scores, the easier my job with be. As a reviewer myself, I made the decision to not use review scores in my reviews because I don’t think that a number can quantify a review. Sure, a game could be a 10 to you, but what if it’s a 7 to me? They’re so dangerous to the industry…
Yeah the same thing happens with other media, such as TV shows and movies. My (ex) wife loved Bloodrayne (one of Uwe Boll’s terrible movies) whereas I found so many inconsistencies and examples of bad acting that it was a snooze fest for me. Her rating? 7/10. My rating? 2/10.
Back to games though, Awesomenauts? I think it is rubbish. Everyone else likes it, so if I made a review on a site that also submits to metacritic it may go down 1 percent (given aggregate of 100 reviews or less).
There is too much emphasis on numbers and scores that attempt to justify, when we get down to brass tacks, whether a game should be purchased or not. I understand that sometimes you may be required to tow the company line (cough *Gamespot* cough) in order to present a glimmering review but we are not stupid. GTAV had a clear, concise date of release, a well-established engine/history and clever viral marketing, so it is natural that it is a good game. Duke Nukem Forever was in Development Hell for 10+ years, had a rushed marketing cycle and turned out to be less than perfect. Then there are the games that flip this idea on it’s head like Beyond Good and Evil, a fantastic game with great gameplay elements and humour but with very few numbers being pushed out.
I don’t have the solution, but hopefully someone else does, because scores and “good” reviews don’t always justify the purchase for everyone.
Who the hell puts mayo on pastrami anyhow?
I don’t think giving games a score is bad, I also don’t think it’s neccesary to change the practice based on an outraged minortiy.
I take reviews on much like those of movies or cars with a grain of salt. The score means very little to me, but rather it’s the content of the articles which I tend to take on board. They sometimes make an interesting point to consider whilst your watching or testing. I’ve played some great games with relatively low scores often seeing but overcoming a reviewers gripe.
Of course I throw the above out the windows when I see a positive review about the Twilight movies. Seriously Vampires don’t sparkle, fairy’s do (Que soccer mum fan rage).
100% agree. Get rid of the scores, they don’t do anybody any good.
Forums like NeoGAF can be a disgusting place within review threads. Any time a big name game (especially Sony franchises) get review scores below expectations, some of the reactions are putrid. There were death threats within the Uncharted 3 review thread a while back.
I think the 1-10 scale is simply too wide when we’re dealing with something as complex and multi-faceted as games tend to be in their design. I find it particularly absurd that anyone could presume to pinpoint their opinion with such precision as to rate a game 8.3 – as if there were some measurable sign of quality over a 8.2 game. Even increments of 0.5 give people way more choice than they really need.
Movies have the right idea with a 5-star system, it breaks the score down into the basic emotions you might have. The wider the range you give, the more you encourage people to overthink their score and the less likely they are to commit because there’s so many different ways you could assess the quality of a game. I think a 3-point system would be ideal for games: like, mixed or dislike. That way you don’t even have to think about it. You don’t have to balance in your mind the value of gameplay, art, technical proficiency, story, originality or value – you just have to say whether you like it, dislike it or can’t quite decide either way. In the end, you’ll still have a measurable percentage of how many reviewers liked a game but because of the generalised nature of the system, what you’re really getting is an indication of how likely you are to enjoy the game.
The major problem with video games at the moment is that they dont stand the test of time, so a game thats 10/10 now will be a crappy 3/10 in 5-10 years time, whilst a movie will continue to stay as good as it ever was.
In the future, no idea how far, lets just go with 3 more generations of consoles as a point of reference, games will likely have no issues what so ever with frame rates being too low or graphics not being sufficient.
Every game will have great graphics and frame rates. The reviews will be based much more on things like art style, story and gameplay rather than the issues that occur in the game itself.
If you’re buying a game because of a number, you’re doing it wrong (and have a very interesting collection of games). I realised long ago that paid, “official” reviewers are generally unreliable. There are three major problems I see with the reviews of major sites (as in, not a regular blog column by an internet personality or similar):
1) These sites derive a lot of their income and ability to access games due to the publishers and consistently “low” (aka realistic) scores are going to hurt that relationship.
2) Reviewers only have a certain amount of time to review a game, meaning they often miss out on the true fun or worth of a game because they don’t have a chance to fully explore it.
3) There are only a certain number of reviewers available in the office, meaning there are quite a few occasions when someone who doesn’t like games of a particular type, or has an unfair bias towards a type of game ends up being the reviewer.
I’ve been pretty successful at finding good games by reading what people with similar tastes and opinions to myself say. Forums and comment sections are great too because really, all you need to know is what features the game has, and what people found good and bad about it. The rest is your own decision.