Something Is Wrong With Video Game Reviews

Something Is Wrong With Video Game Reviews

Something Is Wrong With Video Game ReviewsA month or so ago I was interviewed by ABC2’s Good Game on the topic of video game reviews and, more specifically, review scores. You can watch their feature on the issue tonight at 8.30, but I thought I’d take the time to outline some of my own thoughts on the matter. Something seems to be wrong with video game reviews and, personally, I’d like to see them change.

Something Is Wrong With Video Game ReviewsBy most measures Super 8 is a good movie. I thoroughly enjoyed it. My wife cried at the end, while I thought the ending was a teensy bit underwhelming. My friend, who watched with us at the cinema, complained afterwards about the plot. My brother in-law wondered why every single shot in the entire movie had lens-flare – a legitimate complaint.

At the moment, on Metacritic, Super 8 is sitting at around 72. A resounding success – an apt reflection of a well made movie with a handful of flaws.

Compare that to L.A. Noire – a game I did not enjoy. My wife thought it had an interesting concept. My friend loved it and is currently on his second playthough. My brother said it was the worst game he has ever played and has since traded the game in.

At the moment L.A. Noire is sitting at 89 on Metacritic. This is not an apt reflection of opinion. Not in my experience, at least.

There is something wrong with video game reviews.

Roughly a month ago I was interviewed by the TV show Good Game about MetaCritic and the issue of reviews – why do video game reviews skew so highly when compared to music or cinema. Are game journalists pressured by publishers to give high scores to mediocre games, have I ever been excessively pestered for giving a game a low score, what have the consequences been… stuff like that.

Good Game airs tonight at 8.30pm on ABC2, so I’ll leave it to the show to answer those questions – but it occurred to me during the interview that the act of reviewing a video game, based on a number of factors – some within our control, some not – is a broken process that needs to be reworked from the ground up if they are to retain any real value to consumers. Because, in their current state, video game reviews are practically worthless.

I could spend hours going into specifics – the publisher/editorial relationship, the price of games, the expectations of readers, the language we use, the curious hivemind thinking that plagues MetaCritic – but mainly, ultimately, I think the issue with reviews is one of perspective.

We have to start treating video games as an experience. Not a product.

Something Is Wrong With Video Game ReviewsGames are inherently more complicated than any other medium. There are far more parts holding the experience together compared with any other medium you could name – mechanics, visuals, design. With that in mind I think reviewers, and gamers in general, have been more likely to discuss video games in the same way they would discuss a TV or a brand new car. A game is ‘polished’. Mechanics are ‘fundamentally sound’. Reviews are scientific almost – the discrete parts of the product are taken apart and discussed using the same language used to review a brand new toaster.

But a game is not a toaster. It doesn’t perform a function in that cold clinical way, it won’t help you make a delicious sandwich – it is a video game, it is an experience.

If L.A. Noire was a toaster – I would recommend that appliance. Thoroughly. That toaster has had plenty of money invested in its development. It is fundamentally sound. If I were to take that toaster apart it would be made up of the finest components money can buy. What an incredible toaster L.A. Noire would make.

Something Is Wrong With Video Game ReviewsL.A. Noire is not a toaster, it is an experience. It’s entertainment – it’s a video game. But when we discuss video games there is very little attempt to engage with it as such. Why aren’t we asking these questions: does it make sense? Is this engaging? Is anyone having any fun here?

Because at the end of the day, L.A. Noire is not a toaster – Super 8 is not a toaster. They’re both subjective experiences. But the difference is this: I could walk out of the cinema, hop onto Rotten Tomatoes and find a million different reviews that reflect that experience. I’d have a far more difficult time doing that with L.A. Noire.

I didn’t enjoy L.A. Noire. It’s probably unfair to use that game as an example, but look at it from this perspective: ex-Gizmodo Editor Nick Broughall thinks it’s the game of the year so far. So do plenty of my friends. My brother? He hates it with the fury of a thousand angry suns. You probably have your own perspective.

But that’s the point – the video game reviews that you and I read do not reflect that variety. They skew towards an almost clinically similar conclusion, almost universally. That makes them practically worthless to you and I as consumers.

There is something wrong with video game reviews.

And I’d like to see them change.

Comments

  • Out of curiosity Serrels, the LA Noire user review on metacritic is 7.4 – is this more in line with your expectations?

    http://www.metacritic.com/game/playstation-3/la-noire

    I’ve been giving a lot of thought to the user ratings in metacritic as of late – they seem to fit more in line with my experience of the games in question.

    How do you view the user ratings system? Do you think this ‘crowdrating’ method has more potential than game journo reviews (which often curiously have the ability to be skewed or biased)?

    • I find user reviews to be far too filled with outrage to be fair, really. They’re often either giving 10s to something because it got a a, trying to compensate, or they give 1s for the same reason.

      Now, there should be very few 1s, and probably even fewer 10s. I find the information in most critic reviews to be more helpful than the final score.

      • I’ve found some reviews are hit and miss with the information they portray. Some reviews have missed glaring problems or issues or even concepts, and I’ve found myself playing the game going ‘how did they miss this?’.

        While I agree that crowdsourced reviews are mostly a) nerd rage or b) fanboys, the end score from this amalgamation of geekery more closely reflects how I feel about a game than the combined official reviews. It’s also a handy source for finding any glaring issues, flaws or so forth with the game that may have been ‘skimmed over’ by the reviewers (for whatever reason).

        Therefore; go to official reviews for information about the experience, then go to the gamer reviews for the value placed on the experience and anything glaring(just take a tablespoon of salt with you).

        As a species we enjoy classifying things. I don’t think we’ll ever get rid of a ratings system but we need a more meaningful and broad way to demonstrate a subjective experience to people of varying tastes and interests.

        The question is how we go about this?

        • For me it depends how you view the data on metacritic. If you read the reviews, yeah, your only going to get hyperbole and opinion from the individuals writing them.

          But metacritic isnt about individual reviews, its about the ‘meta’ of those reviews – you know thus the name…. Its about getting a quantifiable value for the quality of a product from the qualitative user reviews.

          Long story short, the user scores are actually quite accurate – once you get enough user reviews. The maths works something along the lines of, skewing the averages between 1s and 10s to find a mid point where the score can ‘settle’. So you will have all these people giving different data in different forms, but once you have a broad enough sample (see enough reviews) you begin to simply get the broad trend of these reviews.

          The one falling down, is that over a longer time period (years, not months) these scores tend to decay, as newer reviews are less positive. This is mainly a factor of how well the game ‘ages’ as people buying it after release will tend to compare it to the new, rather than its ‘peers’ (stuff released at the same time).

          On the topic of differences vs film review, the obvious answer stems from GearBox’s comments Re: Duke Nukem Forever.

          Film reviewers are respected for their opinions, and present opinions as such – a qualitative perspective on something given a quantifiable value, with full understanding that it is 100% subjective. Moreover the power dynamic is different – reviewers are viewed as peers of the cinema industry and getting the respect of a reviewer with no bias is basically a requirement to be taken seriously.

          As evidence of this have a look at http://www.metacritic.com/movie/cars-2

          The user score and reviewer scores are almost identical!

          Now look at games reviewers. Games are always given higher marks than the user reviews tend to lean towards. Why the higher scores?

          Two choices, one is the review method is off, two is that reviewers are bought. Answer would be a combination.

          There is more than a bit of evidence that reviews are bought, GameSpot and Jeff Gerstmann, the whole GearBox telling reviewers to play nice go elsewhere, but that can happen in movies.

          As Serrels said, its about a review of a product where your attempting to quantify the whole way down, examine the pieces.

          Using the toaster analogy, all the parts that make up the toaster are exquisite. But Mark missed the ONE part that is the MOST important. DOES IT MAKE GOOD TOAST!

          You can get a platinum coated toaster with diamond encrusted knobs that gently serenades you as it cooks. But if it makes bad toast, whats the point?

          Mark said that he would buy the La Noir toaster, because the parts were great. But according to his opinion the toast wasnt worth it, that right there is the crux of the matter.

          So what can be done? Its all down to the reviewers, looking at how they review, what they say. Then ignoring this and giving the whole ‘gut’ feeling to the review. XD

      • This is spot-on. User reviews are far too often emotionally motivated – people either want to tell everyone how much they absolutely love or absolutely loathe it. There’s very little room for mid-range reviews. At least a critic has some kind of integrity and responsibility to a larger publishing entity.

      • Yeah, I agree.

        I generally find them to be unreliable and unrepresentative. They attract the extremes of opinion and often don’t at all reflect mainstream attitudes to the game or movie.

        • Does anybody know how metacritic calculates that user review average? Is it just a straight out average of all the scores submitted? Or do they try to filter out the worst excesses at either end e.g. exclude the top and bottom few percent and take an average of what’s left?

  • You make a fine point there, Mark. I have often thought the same of video game reviews being reviewed with greater reference to the technical aspects of a game rather than the experience of playing it. Sure, I can understand that if the technical makeup of a game hinders the experience, then it should rightly be scrutinised, but there is far too much emphasis on it at the moment.

    The opinion of the experience of playing a game, as a whole, is highly subjective. I think we can all agree on that. Therefore, if reviews did start focusing more on these aspects of a game, rather than simply the technical, it could create an even larger divide between the highest and lowest scores we see on Metacritic. Though, to be honest, I can’t help but feel that this would be a much more accurate reflection on the enjoyment of a game. Judging a game as a whole, rather than the sum of its parts.

  • Isn’t that why kotaku invented the likes/hates system, to review games based on experience rather than mechanics/sound/design.

    Maybe video game reviews need to change. But as experiences and impressions vary with every user, the best a subjective reviewer can do is score based on measurable features and their own experiences and let the community do the rest.

    • I’ve always liked the way Kotaku does game reviews, at least in the way it dispenses with a pointless numerical score.

      There’s a risk that disassembling a game into ‘Likes/Dislikes’ still weighs it up clinically without really trying to appreciate the whole experience.

  • I use review scores as a very loose guide only. I’ll play, watch, listen to and experience things I want to (and can afford to) and come to my own conclusion.

    Some things which are now considered great were once considered rubbish. Some things that are now considered rubbish might one day be considered wonderful.

  • You can talk about what you like and don’t like in a game, but you can’t really put a game to a definite scoring system.

    I really like the setup of TotalBiscuit’s first impressions of games “WTF series”, it let’s you judge whether you think you’ll like a game or not by showing you the early sections of a game then him giving his opinions on what he likes and does not like.

  • Hi Mark – I had to stop and re-read your sentence:

    “We have to start treating video games as an experience. Not a product.”

    At that point in the article I had concluded the exact opposite – that the subjective experience of a video game varies so much for everyone, which causes problems with ratings (the rest of the article goes on the explain what you really mean though regarding technical specs != good game).

    Given that games are designed to produce different experiences, how could the average punter believe ratings based on the reviewer’s subjective experience?

    I wonder if it’s the composite. aggregate score that’s the problem. What about a score for visuals, a score for glitchiness per gameplay hour, a score for story, a score for…

    • That is how a video game review should work…

      Interesting, because yoyogames.com, a amatuer game development site, actually has that system.

      The only problem is that people will put 1 star in the story a lot, even if it’s a game that doesn’t need a story, like Proun.

  • So basically… a Rotten Tomatoes, but for video games.

    I think the system is alright as it is, but a rottentomatoes-ish site would be a great additional resource to make choices as to what games to play.

  • I agree so much with you Mark. I’ve been thinking about this for a fair while and you pretty much said everything I was thinking. A game that has a 90 rating could be a game I seriously hate. Case in point for me is Duke Nukem Forever, copping a lot of flak but it’s something I’m really enjoying despite it’s flaws. Who’s to tell me a game is crap and I shouldn’t play it? That’s my choice and my choice only.

  • @Mark

    1.How does your vision differ from ‘New Games Journalism’ as espoused by Kieron Gillen (and others) around 2005? It seems to be the pretty much the same thing. I suppose I’ll have to watch to find out.

    “New Games Journalism rejects this, and argues that the worth of a videogame lies not in the game, but in the gamer. What a gamer feels and thinks as this alien construct takes over all their sensory inputs is what’s interesting here, not just the mechanics of how it got there. Games have always been digital hallucinogens – but games journalism has been like chemistry, discussing the binding reactions to brain sites. What I’m suggesting says what it feels like as the chemical kicks in and reality is remixed around you.”

    2. Your proposal retains the major problem that reviews aren’t a good buying guide, as you still have to know if you and the reviewer have similar taste to know how to weigh their review. How do you propose to solve this?

  • Something the Extra Credits crew over at Escapist Magazine brought up recently is that more often than not game reviewers will describe the experience, then apply a score to it. Not why they enjoyed or didn’t enjoy it, not what it’s similar or dissimilar to, or how it did/didn’t affect them – just what it is.

    “You kill vampires – 7/10”

    Whereas is film reviews the reviewer is more comfortable comparing a film to other, similar films in the same medium with the assumption that the person reading/viewing the review will have some familiarity with these examples and say “Ok, well I liked A, so maybe B will appeal to me!”.

    While I see the merit is such a system, I also immediately spotted the big flaw: fanboys. The main issue with video games is that it’s still an immature industry largely populated by an immature audience. Sure, the average gamer is 30, but that’s a whole lot of under 18s with a few 70 year olds dragging the average up. Us (allegedly mature) late 20s to early 30s gamers are still outnumbered by screaming 10 year olds, arrogant 16 year olds and 12 year olds who just discovered how to swear and ask for boob pics. I KNOW that doesn’t describe all 10, 12 and 16 year olds, so if you’re 10, 12 or 16 and think I am being unfair, be assured I am probably not referring to you.

    The point I am getting at is, if you were to say, put forward the hypothesis that someone who enjoys Modern Warfare 2 might enjoy Battlefield Bad Company 2 because they look and play similarly, you would have a handful of people who’d say “you know what? That makes so much sense I’m going to have sex with it follow that advice and buy the game I have not tried yet.

    You would also have a shitload of other people who would say “NO WAY, BAD COMPANY 2 IS WAY BETTER, IT HAS DOOBLEWHATSITS AND GONZJIBBERS. MODERN WARFARE 2 IS JUST A HERPY PILE OF DERPLE DOOP.”

    And vice versa.

    This is why game reviews end with some arbitrary score. The reviewers whether deliberately or unwittingly disconnects him/herself from his/her words of derision/praise and says “this is the value I place on the experience”.

    Now you still get a backlash from someone who is on his second pristine in Modo Waffle 2 on day 1 because he thinks Module Whacker 2 is a 15/10 and some jerpy tortlebort gave it a mere 8/10, but we’ve become a lot better at writing this off as a difference of opinion. Also, we still seem to believe 7/10 isn’t a bad score so we don’t rage too hard – only when a game is subject to universal vitriol does a reviewer dare to venture down towards a 6 or even a 5, lest he get fired. Only the very worst games like Duke Nukem Forever will be subjected to the misery of a 3.5, which is a step up from Big Rigs Over the Road Racing, but a step down from Pimp My Ride.

    Realistically though, we’re beginning to realise that when a reviewer things something is crap, they slap a 7/10 on it. So if we actually like a game that fairly earns a 7 because it’s not an 8 or 9, we assume the reviewer thought it was crap anyway.

    The way games are reviewed has to change, frankly I don’t think there’s a system that will actually work.

  • I actually prefer a review system that lists the highs and lows rather than the review score so my expectations aren’t skewed by the magic number. I’m not sure we should be so precious with games to review them differently to any other medium though. Yes games are an experience, so shouldn’t we rate them on that as well as the underlying game mechanics? I agree with the point about the clinical dissection of games, they’re not toasters as you say, and they should be reviewed for their ability to be a game. The reviewer’s experiences with the game should be distanced from the critique and the focus should be on “is this enjoyable regardless of my own preferences?”

  • Sorry but I have to disagree with you here. Reviews are a guide that should help you decide whether you might enjoy a game or not, not a method to tell yo whether a game is good or not, which is almost impossible as experiences are subjective as you have shown with your L.A. Noire example.

    This is true across all media, a good review will talk first technically using the language of the medium and then further expand on this as to whether the reviewer enjoyed it or not stating the reasons why.

    The power of this is that over time people will tend to form affinities with particular reviewers as they tend to find a similar fit to their tastes. This is not limited to traditional reviews as I find the same thing happens amongst my friends as some of enjoy particular types of games.

    For this reason I completely avoid aggregator site sites Metacritic as they basically dilute the views of the reviewers whose opinions I care about. While are agree that there are issues with the game reviews in general (publisher/editorial relationship is a huge issue) I do not think that the solution is to start from the ground up with a new way to review games but to provide an alternative to the aggregator sites that better connects people with reviews that will matter to them more.

  • OK.

    So you’ve delivered your verdict on videogame reviews.

    They suck, and that’s cool.

    So, what are you going to do about it?

    You, out of all the people who read Kotaku, are in the best position to change things, being the editor of a videogames website which frequently and obsessively refers to reviews and scores.

    So, what are you going to change things?

    • Fair call. On the other hand, we don’t do reviews here.

      Would you guys like to see Kotaku Australia reviews? I mean proper ones?

      • I have a fuzzy memory of Wildgoose doing reviews, I think.

        Or at least, he gave his opinions on new releases, sometimes through a “first hour” log of what happened in a game and what he thought about it.

        I’d love it, personally. Even the comments section would be a great place to crowd-source opinions on what worked and what didn’t, given the calibre of people who comment on this site 😛

      • Assuming you mean without a score attached – Yes. But not at the expense of the investigative / quality news articles you’ve brought to the site.

      • YES.

        I would also like to see some comment about the “Australian” experience. I mean specifically when a game has a multiplayer component – how’s the experience for an Australian gamer, taking into account latency and the obnoxiousness of hearing “rocket” pronounced “racket”.

      • However, you do frequently refer to other reviews and review scores, posting links from the Kotaku parent website which mention game scores.

        The Critical Reception posts with graphs showing the scores from the bigger sites comes to mind.

        And sure, the Kotaku reviews don’t have scores, but are still rather traditional, otherwise.

        It’s hard to consider change though. I know this comes topic pops up for us at xboxworld.com.au every few months. My last review gave up all pretence of being a review, and just analysed why movie tie-ins are so bad, considering I’ve been involved in making many of them in recent years 😀

        But ultimately it comes down to the audience. I write completely different reviews for XBW than I do for TimeOut Sydney. Feedback suggests the audience is appreciative of both approaches, and if they become less so, I’ll have to change my style.

        But having said all this, I would like to see Kotaku AU reviews 🙂

  • Problem is that unlike anything else.

    Video game producers put to much stock in the review score and as a result the metacritic average.

    They seem to view the number as all thats important from the review even though in most cases the review score is detached from what is actually mentioned in the article.

    With some games reviews mentioning no flaws and getting a 80% then another article by the same writer with game flaws pointed out in the review getting 90% with no clear distinction as to why there was any real difference.

    This wouldn’t really be an issue if it wasnt for the fact that those 10 points apparently determine the success or failure of a game.

    We have a system that for some reason sits on the cusp of a maximum score so that when something truely groundbreaking or deserving of the distinction a 5star rating would show it in the film industry instead it’s lost among the rest of the games that scored in the 90+ range either due to fandom or bought reviews

  • I find that I ignore the score and read what the reviewer actually says about the game. If they sound like they have similar tastes to mine, I can usually trust their judgement. That is, when I read reviews at all; I prefer to make up my own mind, play the game and then read the reviews. Same with movies, really.
    As for the trend of extreme ratings, maybe its because with most games, the experience lasts much longer than with a movie, so your opinion becomes more extreme one way or the other due to the extended exposure to the game. Ie. If you don’t like a game, playing it longer will likely make you like it less; if you are enjoying a game, playing more of it will make you like it more. Although, come to think of it, you rarely, if ever, see reviews in GameInformer (as an example) with a score of less than 50%, so maybe 75 is the new 50?

  • I always liked old school Hyper scoring.. A percentage for graphics, gameplay, sound and then an overall.. *shrugs* I don’t even really pay attention to scores anyway, I usually come on here and see what everyone’s been saying, then trawl googlge for some other opinions. Unless of course it’s a game i’ve been hanging out for, which i’ll buy no matter what..

  • This might be ever so slightly OT, so in advance, I’ll say that I thoroughly enjoyed the article, and I have to agree– VG reviews need a complete working over. The system just doesn’t seem to work right…
    In any case, I just have to say that most of the time, when I buy a gaming magazine expecting solid, researched reviews, I end up with a magazine full of blathering praise, rating every game an 8.5/10 at the very LEAST. It makes me wonder if they get paid to do so, and it really cheeses me off.

  • I agree – to an extent. There are games like Shadow of the Colossus where the experience is everything and by not commenting adequately on that then you are missing the best part of the game. But experience, as you told us, is subjective from user to user. Sometimes to make sure everyone understands your review or your opinion you have to throw in generic descriptors like “great combat” or “awesome soundtrack” (note the lack of jargon). While you want to convey your personal experience you also have to touch on a games broader appeal.

    Essentially I agree with you that reviewers are often more clinical (something I try to avoid on my site and I’m guessing Kotaku does too) but there is still a place for some of that in a review. It just can’t be the ENTIRE review.

  • I think your article makes several great points.

    Some reviews do make the mistake (well, a mistake in my opinion) of comparing games to a toaster precisely because they’re both commercial purchases and a certain type of consumer wants value for money.

    People buying a toaster want feature x, y and z – some people buying a game want to know that its graphics are this good, or that its multiplayer can do that amount of things.

    Video games are, indeed, an experience – the sum of all of the separate parts – and shouldn’t be judged based on the combination of how good their sound, graphics, gameplay, story, longevity, etc are.

    I guess what I’d take from this article, and forgive me if I’m wrong here, is a shift in the way games are reported and reviewed to reflect a lot more subjectivity and, I suppose, a much more nuanced approach to how people found the game. Similarly to how a novel or a film is reviewed, with a much greater emphasis on the opinion of the reviewer and their view of the game in aggregate rather than as a sum of a bunch of technical scores.

    It’s definitely an attractive goal, but would developers – and especially publishers – want to discard the Metacritic system, whereby they’re assured of a sort of herd mentality and can proudly claim to have a “90+ game”, or a “85+” game to their investors?

  • The way I see it, sites like IGN in particular are skewing people’s perceptions on review scores.

    You see a review from them and its always up there, between 8 and 9.5 most of the time, and when you look at people’s comments on reviews, they seem to think any game that gets under an 8 in any review is complete crap and worst game of the year.

    This is somewhat relevant: http://www.hejibits.com/comics/cash-cap/

    • exactly and it’s why metacritic went out of its way to give video games a different scoring system than any other media.

      a 60 for a movies is considered generally favourable

      for a game to be considered generally favourable it has to get a 75 or higher

  • I have no issue with reviews other than the fact that 98% of publications use the 8-10 scale instead of the 1-10. The only magazine I can think of that uses 1-10 is Edge; when Edge give a game which got a 9.5 on IGN a 7, people whinge. By the scale of 8-10, 9.5 is a 7.5 so I’d say a 7 is pretty accurate.

    Anywho, I shall be watching GG tonight.

    • Buckets of agreement. By most standards, if a game scores 6/10, it’s probably an abomination, yet the implication is that there must be some redeeming features to have bumped the score past half-way.

  • When a movie critic reviews a movie, I expect a critical appraisal of the component parts.
    When a game reviewer reviews a movie, I expect thinly veiled “ad” for the flavour of the month.

    I’ve got so much to say on this topic, I could go on for a while, so I’ll try my best to keep things concise.

    In any case, my first point is that there aren’t really any video game critics – except maybe Yahtzee, who hates everything with a burning passion and is very good a pointing out everything bad about what you love (a man after my own dark heart, in many ways). I think problem here is that the distributors have too great a control over the content of reviews. Write a bad review? Well, you can forget about getting a copy of our next game! Or you’ll just get fired if you work for GameSpot. And the driving force behind *this* is the apparent life-or-death-necessity for a 10/10 game to be released every month of the year.
    This gives me a great opportunity to offer some kudos to Kotaku. A big part of what has kept me here is the format of game reviews – a discussion of “What I Liked” vs “What I Didn’t Like”, and the lack of anything that can be boiled down to a raw number. The acknowledgement that personal bias will always enter into an opinion piece goes a long way in letting the reader use the information presented to make up their own mind.

    But more important than conjuring a scaled numeric representation of the quality of a given product, service and/or experience is the issue of distributor control I touched on above. I’m not even asking Mark, or anyone, to confirm or deny this because it does happen. In fact, I’m sure it happens in a variety of industries, and isn’t exclusive to games. Still, if a game is reviewed poorly (or even if one gets a middling review) it’s almost an indictment against the reviewer rather than the developers. In my perfect world, a bad review would encourage developers to go back to the drawing board and actually try and create something special. In reality, the reviewer is likely going to get a slap on the wrist, or the review may never even get published – why would you want to take a chance against a distributor than can buy out the side of every bus and building with advertising, grease enough palms for a few stunning sentences for the back of the box, and then tell the devs to make more of the same, but faster so we can get it out sooner, but we’ll pay you less than the first and we’ll charge retail outlets more than before for this “sequel to the #1 game”.

    I know, I know – it’s a dim, dark, pessimistic view of the world.

  • The issue is most reviews are critiques – not reviews. A critique is an opinion piece – a review should be factual eschewing comparisons and opinion as much as possible.

  • There is something fundamentally wrong with this article. It supposes that there is a “right” way to review something. Reviewing anything, whether it is a toaster, a game, a movie, or a piece of ‘art’, is ultimately a subjective exercise. You could argue that reviewing a product is easier because you would measure ‘fit for purpose’. So you’d be asking of toasters: Does it toast well?

    How do you ask this question of something designed for entertainment (assuming that’s what games are for, which can be debatable)? Easy. Does it entertain? Given that there are so many varying factors behind what would (or would not) entertain people, this is actually more easy to ask than to answer.

    I am mixed about LA Noire (still haven’t fully completed it, and can only play in short stages). But I’m not so arrogant as to assume that, if reviews do not accurately reflect my thoughts and impressions, then there is something wrong with them.

    A well written review is like a well constructed plot. It has to hold together and have internal logic. You can criticise a review if the reviewer contradicts themselves or fails to give reasons for their opinion. But you can’t criticise a reviewer just because they liked something that you didn’t.

  • I think it boils down to trust. If a certain game reviewer (particular person, magazine, blog etc) has generally agreed with your opinion on past games, it’s probably a good bet that in the future, his/her/their review will agree with yours too. Everyone has personal tastes and preferences, a friend of mine is mad about racing games, where as, another friend only plays sports games. Would I approach them for an opinion on Japanese bullethell game? No.

    Maybe a modified rating system where you would give your own weight to individual reviewers for a weighted average, instead of a straight average? So it would be a personalised rating, just for you. You’d spend half an hour on Metacritic giving your ‘trust factor’ or weight to each review based on how they agree with your past views, you may even give user rating higher than a published review. So when your brother view the rating for LA Noire, it will show a completely different number to your friend who’s playing it the 2nd time? Even though the rating is derived from the same sources, your weights will skew it to your view, hopefully giving you a more meaningful result.

    I work in finance, so I deal with weighted averages every day…

  • Writing reviews of games isn’t easy, least of all because many games have storied histories that most movies don’t. Games are judged by their company more often than films, that are easier judged on their individual merits. Reviewing a game is more like reviewing a TV show series than a film, which I think is a better comparison.

    In regards to LA Noire, I think, despite the diversion of opinion, the game does something that no game has done before, sets a new technological standard with MotionScan and as a result deserves a little more kudos than a 70 out of 100. I reviewed it for That Aussie Game Site and gave it 4 out of 5, after enjoying the detective sections but despising the action bits.

  • Just one more thing. I think reviews should really be 50% analyses and 50% opinion. People don’t read reviews to measure the quality of a game, they usually read them to decide if they should take the time to play it.

  • I prefer to let user reviews and word of mouth influence me much more than an arbitrary numeric rating.

    I just don’t trust a reviewer who makes their living actively playing games and picking them apart for a paycheck, because while someone would praise a game’s technical wizardry of bump mapped physx vertex particle HDR antialiased fire effects, I just wouldn’t notice that, and I don’t have the same mindset as a game reviewer.

    That, and the scoring systems are just weird. It’s like an 8 is a generally “average” game, and anything under that is below average. 7.2? B-? When their written review has nothing but praise for it except a few framerate hiccups?

    Please.

    • Example: Bulletstorm. I remember reading reviews that said the game was short, had too many dick jokes, not enough variety, poor level design… the list goes on.

      Me and my friend pretty much agreed that it was a good bit of fun – great for a weekend’s worth of entertainment and we enjoyed juggling fools with a shotgun and wrapping grenade-flails around dude’s necks.

      I just feel that most professional reviewers have a different view of fun that the average gamer.

  • The problem as I see it with reviews being more experience focused than product focused is that it’ll end up the same as it is now.
    Now the ratings predominantly all end up around the same mark. However if people’s experiences are used as a general average wouldn’t every game end up with approximately the same averaged out score of between 4-6 out of 10?

  • hmmm, good points, but I think it’s something wrong with game reviewers along with the review process. I find game reviewers are more prone to going along with general consensus rather than forming their own opinion.

    Bah, I don’t read reviews anyway. I’ve always thought they were biased and subject to publisher manipulation in order to help it sell more.

  • Isn’t there somewhat of a irony here when this segment is on Good Game? The show incapable of giving a review that isn’t 7-9.5?

    But I do agree with plenty of Serrels comments, and shall watch this segment in 2 hours time.

    • Completely disagree. How often do you watch Good Game? They handle their scores very well. Bajo raved about a game a few weeks ago, then gave it something around 7. I was surprised at first — until I realised that 7 is an appropriate score for a decent game.

  • Does anyone see irony in this?

    Duke Nukem Forever… the general consensus by the gamers has been positive, of course you have the trolls who hate it and the people who genuinely didnt enjoy it, in my opinion the graphics arent that bad, it has solid gameplay and yes it does have some issues but they can be sorted out. I have played worse games but i have played better. i would give it an 8. The thing i find funny is that Gamespot did a preview of DNF at PAX when it was revealed and was pretty positive, they liked the graphics, liked gameplay, humor and even liked the multiplayer modes but when it came to the full review they were negative on everything they were once positive on, its hypocracy…

    But the thing that worries me is the vast masses that actually rely on reviews by IGN, Gamespot etc… to tell them what games are goos and what games arent, its like people cant even think for themselves anymore.

    • Actually, according to metacritic gamers don’t like it either. The game play is bland, the graphics are definitely subpar, and the “humour” is too crass to be funny. It’s the scary movie of gaming.

  • The score at the end of a review is probably for those who either couldn’t be bothered reading the text, or cannot formulate a quantitative value for the game. I remember being 15 and going straight to the score before reading anything. These days I’m more interested in the execution of the idea (being software QA may have changed me…)

    What is a review, anyway? Some person you (likely) have never met, telling you about something they may or may not like.

    Time/value? If your time is that valuable to you, either you should be seeing the world, embracing your family – or perhaps just taking the $5 and an evening to hire the game yourself.

    Mark – why would you recommend this LA Noire toaster, when you did not enjoy the toast it gave you? Just because someone spent a lot of time and money to make it does not mean it has added value. Just remember, a lot of hate went into making that toaster. Sounds like this review was biased, then.

    In fact, it may do the same job as the Hello Kitty toaster I bought for my wife for $30, and Noire doesn’t burn a kitty face on your toast.

  • Thank you YES! I agree there’s almost a groupthink with reviews.

    The biggest offenders this generation IMO were two of the earliest, Gears of War and Mass Effect.

    I bought Gears of War and even managed to finish it (a year on) and it was in places an unforgiving drudge of a game. Worse still the multiplayer rendered the cover-mechanic useless and added insult to injury with it’s one death per round rules.

    Mass Effect I bought on the ebullient recommendations of friends and the gaming press – what they should’ve warned was ‘not for people who don’t like RPGs’

    Don’t get me wrong I loved chatting to aliens and making decisions, but choosing between a hundred similar guns for my team was not my idea of fun, not to mention when you actually got in a firefight you were usually killed hundreds of times due to your lack of XP (I reckon I died over 40 times tackling that Thorian level) The shooting had nothing to do with being a good shot and instead seemed to be based on random chance.

    By the time I started “exploring” planets in my moon buggy the whole thing turned from game to chore pretty quickly – I traded the damn thing and haven’t looked back.

    Yet when I go through the reviews for Mass Effect a lot of reviewers point out these same shortcomings but none of them dare mark the game down, I don’t think you’ll find a score under 9 anywhere.

    Remarkably I got Mass Effect 2 a few months ago and it is a vast improvement, the shooting works, the cover works and there’s no moon buggy that I can see – I’ve gotten further with that game in 2 months than I got with first in over a year, yet you don’t find much improvement in the score on review sites because they already rated the first one through the roof!

    • I take it you don’t like RPGs in general? The Mass Effect inventory system is grossly simplified compared to many other genre games. Of course, Mass Effect was much more of a niche back then.

      Then there are complaints about the ‘dumbing down’ of ME2 because it just becomes a shooter. The sequel was considerably more mainstream, but there are plenty who understandably take issue that one of the only high profile RPGs they get has been thrown to the masses.

      • No I don’t think I do, my idea of RPGs growing up was Legend of Zelda! I love the going around talking to NPCs and finding quests to do, but the whole stats, spells and inventory management thing is lost on me. Don’t even get me started on Turn Based – I rented out FFVII back when it was the hottest thing since sliced bread, played it for about 30 minutes, saw that it was turn based and to me it was like watching paint dry.

        Mass Effect was something different though – it promised the soul of an RPG with the mechanics of a shooter – ME2 is definitely – if I headshot a guy in ME2 he’s dead, not so in ME1 which may as well have been turn based if it was honest.

        I can understand hardcore RPG types being disappointed with the sequel, maybe this is one of the problems with reviews, they assume we’ll like all genres or something. I couldn’t care less about point & click or strategy games – maybe there needs to be reviews from different perspectives – something we didn’t get with Mass Effect.

  • Simple problem – What is considered “Average” seems to receive a 7, not a 5 when compared to other forms of media. So many reviews i have seen mention that a game is average, that has many flaws but some redeeming qualities yet they give it a 7. Anything below 7 is generally just below average, while anything like 4 and below is simply rated that for comedy value of how bad it is more often then not.

    Basically this is how i see many reviewers treat the rating system, even if it contradicts their own system guidelines as to what the value indicates:

    10 – As close to perfection as possible
    9 – Very Solid Game. Likely GOTY contender material.
    8 – Good Game that is worth a purchase but not GOTY material.
    7 – Average Game – May or may not be worth the purchase, depending on whether you like the genre etc..
    6 – Below Average, most likely avoid.
    5 – Pretty Dreadful – Avoid.
    4 and Below – They Steal Your Monies

    Personally i would prefer that we move away from scores entirely (like Kotkau) but in lieu of that, there needs to be a universal system that can be adopted by reviewers for their reviews and have strict guidelines enforcing what each score is.

    • ….have a read of Edge magazine’s game reviews. They work in exactly the manner as you outline – where 5 equals an average game. This makes their scores seem rather lower for games that are not the AAA hot-shit titles when compared to their contemporaries, but the ratings make much more sense and makes their judgements more standardized and apropriate across the board.

      They judge games on all kinds of merits taking into account relevant references and comparisons to other games. Also, way they articulate their opinions of a game is hard to argue with.

      If only all games reviews took their lead…

  • One thing I’ve noticed with video games reviews (notwithstanding the fannism that means people give things a 1 or a 10) is that just being a barely functioning game is usually enough to score at least a 6.
    Obviously movies are easier and less likely to have parts that are just fundamentally broken, but I really don’t think a game that just barely passes QA should be getting bonus marks based on that…

  • So essentially you want Yahtzees style of review?
    At least thats what I’m getting
    Honestly his experiences with a game refleect far more with what I experienced, than a review from IGN or other game reviewers give me

    In his Infamous 2 review he pointed out that Zeke became the emotional heart of the story and I found myself agreeing, because he had been written really well and I had actually experienced that.

    With IGN I got the story is decent, the gameplay is good etc, without it ever really telling me what was good in a substantial way or reflecting any experiences that happened in the game at all

    How sad is that really though, a guy who makes a comedic game reviewing show is more reliable and able to give better reviews on a weekly basis than people hired to professionally analyse games and find their flaws, while having much more time to do so.

  • It depends on the calculation. Which movie is better? The one that has 100% of reviewers giving mildly positive reviews, or the one which splits them right down the middle in a ‘love it or hate it’? Depending on how you present numbers, either can be 50% but still valid.

    Game reviews are skewed higher, I believe simply because reviewers don’t have quite the thick skin movie reviewers do, who’ve been shunned, mocked or attacked for their acerbic writing. In the film world, a run of the mill movie is a 5/10. In the game world, that’s a piece of garbage, with the average hovering around 6.5.

  • my number 1 gripe of any review nowadays is how they compare it to CoD, Halo or any successful franchise, and then pick out the floors as ones that are nowhere as good as them.

    Another point is inconsistentcy, and for this i’ll use MW2 and Gamespots reviews on PC and PS3. The review is a cut and paste job, yet the one difference is that online has less features than other FPS on the pc (negative) and all of a sudden, the pc version is 0.5 points lower, which although it doesnt mean anything, but ultimately it says “Because X did not have what other Y has on the console, it’s scored less”.

    Same deal with Duke Nukem Forever, the general consensus was that PC performed the best, then ps3, then the 360, with the 360 having visual problems galore, yet i look at the reviews, they are copied, same score, with the only difference being 0.5 and that was because apparantly the pc version did not have short levels with long loading screens.

    Nowadays i only use read an expert review to get a detailed look into the game, but i dont bother with the score, instead i’d look at the User Score average, as only gamers who arent influenced by work or the gaming industry, could give me a more down to earth score, because i am one of them.
    (example of Bajo feeling pressured with the score he gave DNF on the pc being too high when other reviewers scored it much lower)

    • Actually… totally agree!

      Perhaps instead of reforming reviews, we need reform to the marketing and selling of games themselves! Add more trials and demos, let people experience some of the game itself before committing their $$.

      Because thats all reviews are about… informing gamers before they open their wallet. I now watch more videos and play more demos these days, because reviews ARE subjective.

      I think we’re actually seeing some of this already in DL games.

      Not perfect of course, because marketers have an incentive to trial/demo what they consider to be the ‘best’ or most polished part of the game.
      But I’m sure there’s a middle ground somewhere.

  • The entire basis for this article is a small sample of anecdotal evidence. The range of reviews for Super 8 reflect the opinion on the 2 movie-goers you went with, and therefore you are happy with them. The range of reviews for LA Noire do not reflect the opinion of the small number of people you asked, therefore they are not accurate.

    Maybe you just happened to ask the people who have compeltely different views on LA Noire. Expand your sample size perhaps?

    • I should have used another more extreme example. Like a halfway decent movie scoring 55 on metacritic, and a game with mixed opinions among almost everyone I know – like GTAIV – getting 97.

      I think it’s a pretty obvious problem.

  • I almost think instead of giving a number for reviews, we should get reviewers to give a price.

    Like “I would be willing to pay $60 for this game because it’s given me 60 fun filled hours of time”
    or
    “don’t even pirate this game. The company should pay ME for it”
    Or the middle ground “buy it for $3.50 in a Steam sale.”

  • I couldn’t agree more with your article. And I’ve always had a particular problem with the way game reviews only seem to use the upper 20% to rate games… certainly AAA-rated popular games. If a game gets rated 8.0/10 on IGN for example, its probably a steaming pile-o-shite to be realistic.

    But one irony I’ve noticed with you is that you talked about this issue on ABC2’s Good Game… because as far as I’m concerned, they are very bad culprits in this regard when it comes to game reviews. I’ve always found that their reviews are very scientific, not taking into enough account the overall experience of the game, and they start a rating from a verrrry high percentile. Even an average game seems to get 80% plus. 80%+ should be for absolutely outstanding games, not average.

Show more comments

Log in to comment on this story!