Today, the gaming website Eurogamer announced plans to ditch review scores, joining outlets like Kotaku and Rock Paper Shotgun in Fighting The Good Fight against the arbitrary and meaningless quantification of video game quality.
That’s good news both for Eurogamer and for video game criticism in general. It’s also good news for anyone concerned about the disproportionate influence of Metacritic over how and when video games are made. To quote Eurogamer:
Over the years, we’ve come to believe that the influence of Metacritic on the games industry is not a healthy one (and we’re not alone in this opinion in the industry, either). This is not the fault of Metacritic itself or the people who made it, who just set out to create a useful resource for readers. It’s a problem caused by the over-importance attached to Metascores by certain sectors of the games business and audience – Metascores which are, let’s remember, averages of dozens of numerical values, ascribed more or less arbitrarily, in different systems, by a wide range of reviewers expressing a wide range of opinions. The result has been conservatism in mainstream game design and a stifling of variety in critical voices. In short: it’s meant less interesting and innovative games.
This is spot on. It might also make you wonder: just what are the problems with Metacritic? When Eurogamer talks about the “over-importance attached to Metascores,” what are they referring to? Is there any evidence that Metacritic has a negative impact on the way video games are developed and published?
Let’s take a trip way back to the strange times of 2013, when Barack Obama was president and there was a new Assassin’s Creed coming out in the fall. In April of that year, we published an investigative report called “Metacritic Matters: How Review Scores Hurt Video Games.”
In that report, we talked to a couple dozen people who have worked for game developers, publishers, and press outlets. We shined a spotlight on some of the many problems with Metacritic — how it affects developer bonuses and negotiations; how it hangs over game-makers like a scarlet letter; and how both publishers and developers try to game the system in any way they can.
Some salient highlights:
- Independent studios like Obsidian Entertainment (South Park: The Stick of Truth) and the now-defunct Airtight Games (Murdered: Soul Suspect) are frequently asked to show their Metacritic scores while meeting with publishers about potential deals to make new games.
- Metacritic scores are also tied to bonuses; Obsidian lost out on a cool $US1 million because Fallout: New Vegas was one point away from 85, according to sources.
- Publishers can and will do whatever they can to skew Metacritic scores. One developer told me of the time he hired an infamously-negative-scoring reviewer to write a mock review for his studio’s game — not because the developer wanted feedback, but because they wanted to make sure this reviewer would have to disclose himself from writing a review and impacting the Metascore accordingly.
- Some smaller websites can be influenced in more blatant ways — one writer told me that Sega guaranteed his website an exclusive review of Super Monkey Ball if the score was higher than an 8.0.
- A lot of people are unhappy with the system — and there seems to be a rapidly-growing sentiment that things need to change.
If you’re interested in learning more about why so many people see Metascores as such a big problem, go check out the full piece.
You can reach the author of this post at jason@kotaku.com or on Twitter at @jasonschreier.
Comments
26 responses to “A Look At Metacritic’s Many Problems”
The other problem with metacritic ratings is that they’re really not that indicative of how “good” a game is for consumers anyway. I much prefer Rotten Tomatoes’ model where its merely a % of positive reviews (and Steam’s internal ratings system seems to work similarly).
What will actually happen:
The end products that the audience are gravitating towards – even though they profess not to – such as microtransactions, pre-order perks, re-mastered software, questionable privacy and security practices, and just plain broken games will flourish.
You get told every day that cable television is expensive, anti-consumer and out of date but you put up with it. The same thing will happen to games, but because you are satisfied with the overall de-valuing of what you are happy to purchase, they will get away with it all the same.
What was the last game you bought and why? I will bet it had something to do with news you heard before its review came out.
Eurogamer introduced two new, inadequate scoring systems (google star ratings, but only 3-5 is used), and refuses to mark down games for being broken at launch with their day 1 reviews.
Fight the good fight, guys!
This. Buckets of shit like AC: Unity and Halo MCC are still being released completely broken. This is a far more dangerous threat to the games industry and culture than Metacritic will ever be. DLC as a business model is a far more dangerous threat than Metacritic will ever be.
I’m not sure why everyone is an expert on scoring systems. Every time I hear someone talk, it’s always simplified to the point where it’s understandable for whomever the imbecile is saying it. Reviews aren’t broken because something arbitrary like a review score (almost as arbitrary as peoples’ views on them) cannot be broken since it doesn’t “work” in the first place. Reviews will never be right and it’s damn selfish and childish to look at anyone else’s score and say “Wrong! I had glitches at launch!”
What if they said; “We didn’t.”
Whoaaaa!!! Is your world shattered now? Is your tiny little keyhole perspective all covered up? Care less about review scores, read more and stop thinking any whim you have or feeling you get is factually accurate and quantifiable fore everyone else. I could not fathom any other reason review scores would cause such dissonance in insecure people.
Damn right.
I don’t think removing scores is that helpful. Eurogamer’s new system of essential / recommended / avoid is meaningless for all those games in the middle of the spectrum. It would be nice to have a bit more of an idea of how the reviewer likes the game in an overall way. My preference would be the Hatsune Miku Project Diva f system: Perfect, Excellent, Good, Standard, So Close (or Missxtake).
I agree. I don’t think removing scores is really going to do anything positive. It’s people’s attitudes and perspectives that need to change. There is no problem with Metacritic (or other) scores. The quote from Eurogamer puts it best when they say:
The article title should be, instead, “A Look At The Many Problems With People’s Attitudes Toward Metacritic Scores”.
Their piece introducing the new system does indicate that they won’t be using the Essential/Recommended/Avoid ratings for all games, they’ll simply come with no recommendation. To quote:
http://www.eurogamer.net/articles/2015-02-10-eurogamer-has-dropped-review-scores
I see, so the ‘non-recommended’ is kind of like a fourth category? That makes more sense. That would be the equivalent of ‘average’ I guess.
As I wrote in my post below, Essential/Recommended/Avoid is basically the same as giving games a rating (out of stars, numbers, potatoes, or whatever). @spanner
The Eurogamer article says the replacement for the review score is a 140 character summary at the top of the review (and presumably in lists). The essential/recommended/avoid badges are in addition to this.
I’ve never taken any notice to Metacritics scores. If I did I’d have missed out on some great games. I do prefer the steam model with it’s ‘Mostly Positive’, ‘Mixed’, etc….status. But then I go straight to the negative reviews in Steam to see if a game has issues or doesn’t live up to what it’s says it’s selling before buying.
This. Frankly, I go straight to the negative reviews in anything, even if it’s positive.
I want to see the worst that people have to say about a game to see if there are any dealbreakers in there for me. If the worst people have to say actually sounds pretty OK to me, then fuck it. Why not?
I like that they’ll be publishing first impressions of certain online games before they launch, and the full review afterwards, along the lines of RPS (who take the same approach with Early Access games also). Abandoning the review scores should allow more flexibility in not having to deliver everything in one piece.
I like this if only because it discourages lazy journalism (here’s some scores guys, do what you make of them. peace!”) and also lazy consumerism. If you want a suggestion whether you should or not purchase a game you should read the whole review and do some critical thinking, not just jump to the scores and act accordingly.
A rose by any other name…
Recommended, Essential or Avoid is basically the same as 1, 3 or 5 stars. Or 2/6/10 out of 10, or whatever the hell other relative score you want to give it. It just uses words instead of a number. The end result is the same.
Also since unfortunately we live in 2015, the days of business driven, tight time frame releases that inevitably end in buggy games, I’m pro the idea of websites updating reviews. A buggy game at launch can be a different experience to a patched one a month later.
Some do that already which is nice. But it’s something that could help consumers if done better.
As per the point above, as if people didn’t already translate numbers into those words in their own minds. I for one however don’t particularly give a shit about reviews as I’ve played badly reviewed games and had a blast, while equally finding games that got good reviews to be far from entertaining.
My inner cynic suspects that they real reason for this is because number scores make it easier to call reviewers out on their shilling for publishers. By not including a score, they can write fluff pieces that promote games rather than critique them and call them “reviews”. Metacritic user reviews (once you weed out the haters and the paid reviews) are one of the only sources on honest criticism out there (along with Totalbiscuit). No wonder the “Gaming media” is desperately trying to discredit them; they expose the “professionals” for the lying shills they are.
Don’t assume you’re a literary expert. That’s all. Just because you disagree with something doesn’t mean it’s wrong. “Honest criticism” isn’t just criticism you agree with, y’know. It isn’t just stuff you understand. Just because you call something a fluff piece, just because you say the word “shill”; doesn’t mean you understand the basics of creative writing. The truth is that your education coupled with arrogance and anger is the problem, not EVERYONE ELSE. Nothing you say is remotely true in a generalised sense. Go and read. Don’t assume you’re somehow more intelligent than anyone writing reviews. Seriously, overstated comprehension of literary technique by readers is probably the biggest problem in gaming media. Critique writing if you’re educated enough to do so but if all you can muster is a bunch of massive generalisations that feel more psychopathic than sane, rethink how much you know.
Isn’t it weird when people list EVERYTHING as the problem, that they don’t actually feel crazy?
lolwut? What the hell are you talking about? I’m making no comment about the literary merits of the reviews, I’m commenting on their independence and their honesty. To whit, I think the professional review sites are all bent and that dropping review scores is nothing but an attempt to obscure their biases behind vague terms like “recommend” instead of a concrete figure like X/10. The review sites want to drop scores and discredit Metacritic, because scores and Metacritic user reviews expose the disconnect between their reviews and the typical user experience.
I’ve heard of this really cool support group for your beliefs. Can’t remember the name, Gamerfence maybe? Anyway everyone’s very cool and accepting and not at all paranoid, sexist and abusive.
We don’t, and you should take your own advice and seriously go and re-read that typed diarrhea you just posted. I seriously hope you are on recreational drugs and weren’t trying to come across all intelligent like.
Sigh. SO barking up the wrong tree.
Yeah, fight all you want, the fact is that if Metacritic didn’t exist, Publishers would create their own. You’ve got it back to front. Lesions are not evil because they cause cancer. Cancer causes the lesions. This is a classic case of mistaking the symptoms for the cause.
The major publishers are not beholden to art, they don’t give a shit about the ‘art’ of games, except in so far as how many millions of dollars nerds are willing to pay for that art. They can’t let all that money slosh around being wasted on creatives, it needs to be carefully corralled by businesses – big boys, who know what to do with that money and make it flow where it belongs: investor pockets.
The major publishers are not beholden to art, they are beholden to stockholders and investors. Business people, not artists. The fact that games ARE art is an enormous inconvenience to them, a factor to be reduced as much as possible by focus group testing, market research, and sales performance analysis. Art is like a lightning strike. Business does NOT like the unpredictability of that, and wants to secure predictable, safer, earlier returns.
Hence the focus on offering rewards for preorders before reviews come out, franchise syndrome, DLC, season passes, multiplayer everything, social networking integration, etc, etc.
Three years of paying a shitload of people who are not earning you any money, followed by launch sales, and a short tail. And even when the product is critically acclaimed it can be a commercial failure.
That’s a TERRIBLE proposition for investors.
Metacritic is NOT the cancer of the gaming industry, just a lesion. It is a sign of what is working deeper under the surface: investors trying to measure their returns. They want metrics for success and failure as ‘key performance indicators’ and then to hold their partner businesses (studios) to those KPIs.
You know how you predict something? You fucking measure it.
If metacritic didn’t exist, they’d create something in-house to take those measurements anyway. Positive forum posts, reddit mentions, Steam reviews, whatever.
My biggest problem with Metacritic (and most scoring systems) is the fact that games can only lose points for flaws, but gain nothing for ambition. For example, Mario Kart 8 has an 88/100 on MC, whereas something like Beyond: Two Souls only has a 70/100. This is because Mario Kart is nearly “flawless” (i.e. without flaw), but IMO is not a very good game because it is basic and highly unoriginal. Beyond is by no means perfect, but definitely tried to do something new and push the boundaries of gaming. This subtlety is lost when a game’s worth is distilled down to a single number.
Which is why I like Kotaku’s system. Instead it distils the game in to a simple list of key pros and cons, and it lets me judge if I’d like it or not. I can ignore a con like “multiplayer is weak” or “short runtime” but I can’t forgive “overly simplistic story” or “been-there-before gameplay”
i go to comments sections for my reviews….