Should Game Developers Speak Out On Reviews?

Uncharted 3 has done pretty darn well, going by the hive mind judgement of Metacritic. All but three outlets gave the game a score of nine or above. Of the sites to assign the game an 8 or less, Eurogamer is arguably the one with the highest profile — a profile significant enough for the game's lead designer, Richard Lemarchand, to speak about it.

Chatting to Eurogamer, Lemarchand expressed his disappointment with the review, stating that the team at Naughty Dog had "done even more than we did [in previous games] to keep the player in control from moment to moment." In the interview, he mentions he's not a fan of review scores, but accepts them as a fact of the business. Lemarchand also acknowledges that games journalism itself has matured, with reviews leaning towards useful criticism rather than glorified lists of features.

What caught my eye is that an 8/10 is now seen as an unacceptable score for a AAA game from a developer's point of view. I can tell you any Australian games developer back in 2007/2008 would have been ecstatic with a score so high, given the types of games we worked on. But expectations from publishers and consumers have gone through the roof, and it's a lot harder now for the likes of Naughty Dog, Blizzard, Gearbox and Valve to settle on anything less than flawless (or sub-flawless, in the case of nine out of 10s).

I think it would be grand if more developers spoke out about reviews — pointing out where it's clear the reviewer hasn't played the game for longer than 30 minutes, or simply clearing up misunderstandings. Reviews, by their nature, are a one-way street, but everyone would benefit from some discussion between critic and creator.

Developers, especially those with a hands-on role, tend to be more direct and honest than a PR or marketing mouthpiece, so what they say inherently carries weight. And who else is better qualified to answer perceived questionable design decisions than the person who made those decisions?

Are you interested in more back-and-forth between journos and developers when it comes to reviews?

Naughty Dog lead designer discusses Uncharted 3 review [Eurogamer]


Comments

    Trouble is the market is going to have trouble actually beleiveing a developers protests.

    Yeh, I definitely would love to see more discussion. Every now and then, you just get a review that raises eyebrows and it's nice to see what a developer has to say. Granted, it's open to developer bias though.

    And of course, the game developers would have to comment on any reviews in extremely muted tones, or it will just come across as a CliffyB-esque "my game is better than my previous game, so suck it" rant.

    I think maybe the reason the third game in a AAA trilogy slightly underperforms score-wise is because a lot of the time it seems like the reviewers won't give a 9 or 10 unless the developer re-invents the wheel, so to speak. In the case of Uncharted 2, that wheel was made of solid-gold and rimmed with diamonds and rubies, so I would be disinclined to play Uncharted 3 if they strayed too far from its predecessor.

      The problem though is that just because your game is better than the last doesn't mean that it deserves a higher score.

      Your team can do everything better than they did last time. But review scores aren't stagnant. An 8 this year is generally not equal to an 8 two years ago.

      Which means while your game might be better than the last. The reviews have only deemed that it grew as much as the rest of the games industry did. Your new game might look better than your old one. But so does just about every other game compared to the game they released 2 years ago.

      Thing that annoys me though is the statement that they put everything they had into it. It's as if they view the fact that they worked hard means they instantly deserve the score. I can put everything i have into something and it can still not be good enough.

        "An 8 this year is generally not equal to an 8 two years ago."

        That'd make the system incredibly flawed then, wouldn't it?

          Like how art, music and films that are panned at the time of their release are later considered to be brilliant?

    I don't think developers should really comment on them, but if they are inclined I am always interested in the expatiations for why X is Y in a game, which is often the kinds of things a reviewer dislikes/likes and mentions in their reviews.

    And yeah the whole 8 out of 10 being bad thing is just stupid.

    Meh, who cares what eurogamer thinks. They have no credibility in my mind.

    Love to see more back and forth. Discussion is always more enlightening than soliloquy. I'm always -more- inclined to look into something when I can see that the developers care.

    Just keep the PR/Marketing guys away. I hate their muting influence.

    Except for the superstar directors like Motomu Toriyama (FFX) and Markus Persson (Minecraft) developers shouldn't speak if only to have the company not seem whiny or arrogant. And definitely not in response to a review.

    They had their chance to put forward the best product they can. By all means share project developments with the community but don't criticize judgements of your work lest it escalate into a war of words leading ultimately to a defamation case.

    This is just the state of video game criticism for triple A titles. Anything under a 9 and you may as well have been awarded a zero.

    Im all for dialog between the two, but when PR tries to manipulate scores thats when I disagree

    Nothing wrong with opening up discussion if you keep the PR people away from it. They're skilled at spin not discussing legitimate concerns or criticisms.

    Developers should only be able to comment on factual errors or misleading statements. Who are they to tell anybody their opinion is wrong? Even if there is an unfair bias from a reviewer, that should be up to the people (and the site the review is on) to work out the reviewer not for the industry to censor them.

    "I think it would be grand if more developers spoke out about reviews — pointing out where it’s clear the reviewer hasn’t played the game for longer than 30 minutes, or simply clearing up misunderstandings. Reviews, by their nature, are a one-way street, but everyone would benefit from some discussion between critic and creator. "

    Honestly I think this is an awful idea. Developers should not be reviewing their reviews. You only have to see games like 'Too Human' to know why this would be a bad idea by design. It should not matter that someones played the game for only 30 minutes.

    "You don't have to eat the entire turd to know that it's not a crab cake."

      "Reviewing" reviews is definitely not what I'm talking about. Addressing errors, clarifying points and expanding on design decisions to provide both critics and players more insight into the development process? I don't believe these to be awful ideas.

      of course it should matter if they only played 30 minute's of the game.

      How can you elaborate on the scope of the story the development of characters the different environment's that exist within the game world. If there was a particularly ingenious physic's puzzle.

      http://www.neogaf.com/forum/showthread.php?p=32173792

      Hell that shows that some reviewers are so intent on rushing through games in the minimum amount of time(most likely because they are paid per review and not for hour's taken for review)

      And that's when there's an obvious flaw.

      Review's should if they are actually consistent be pointing out the same flaws across the board. Which should allow developer's to actually identify thing's to improve in the future.

      Instead way to much emphasis is placed on the precious score and not what's said in the review. Hell some reviews can come off seeming totally positive and then give the game a crap score for no real reason, and it happen's the other way. I remember reading a review in a magazine a couple of years back and it's like this this and this are broken, but if you can look past them it's a 8

        It's the same as a TV show. If a pilot episode doesn't get the reaction it needs then it's scrapped.

        The first 30 minutes of your average game is exactly the same. If it doesn't grab your attention in some way then why should you spend more time with it? A well made game hooks you in straight away, look at Bioshock, Gears or Assassin's Creed. For plenty of people they offered enough in the first 30 minutes for those people to keep playing and so did their sequels.

        Nothing needs more than it's pilot/first 30 minutes/opening chapters to determine if it's any good. If you have to view/play/read the ENTIRE product for it to be worthwhile then it wasn't made right at all.

          Oh yeah? Then why don't you just experience that 30 mins and then quite while you're ahead?

          If that was so then we would not have gotten Babylon 5, a show that basically needed an entire season to set the scene and mood for the series. The payoff is that in retrospect season 1 was more phenomenal than you were first led to believe, intertwining with the rest of the series to form something of awesome excellence.

          You cannot judge things too quickly, because of characters and background. Some stories can't be told in the space of a sunday comic strip, which in no way makes them less worthwhile or awesome.

    No, not really. There should be independence in these things. I don't think that sort of stuff happens with film reviews so why should it with games?

      I forgot to mention - they certainly should have a voice in these things, but they shouldn't be trying to affect review outcomes.

    Do you realise how many " game " sites are owned by Microsoft and its subsidiaries

    I usually disregard most reviewer scores to be honest. 7 out of 10 appears to be average for most reviewers which is completely wrong.

    Anything under that is just an arbitrary number given to a game they didn't like at all. A 4 or 5 out of 10 game may as well be a 2 or a 3.

    If you're going to review games and give them a numerical rating than you have to work within the confines of that system. 5 is average so an average game should be scored as such and an above average should be 6 and so on.

    The other thing is way too many games get 9 or 10 out of 10s these days. Certain publications, such as Eurogamer, give more accurate ratings but plenty of places still decide to give nearly every triple A game a 10 which is going to skew the data.

    Scores are an important aspect of reviews, I feel, but reviewers need to take them more seriously and readers need to take them less so.

    I hate how the rating system works for games. 10/10 is great, 9/10 is good, 8/10 is okay, 7/10 is bad, 6/10 is crap. It's like the numbers 0 to 5 don't exist!

      On the actual matter, I think it depends. Don't gloat, don't cry. Address the issues an outlet raises.

      I think this is partially because of the increasing creep in the standard of games. The quality of a modern game would be ahead of most games created a decade ago.

        But games nowadays shouldn't be rated on the same scale as games from the old days. If Goldeneye were released today, it'd be crap. Games have to be rated on a scale reflecting the age in which they are released.

          I think there's a reflection of both ideas in current game scores. Games in this period are judged against each other, somewhat on their own scale, but the numbers are influenced by what was given in the past. It could be a reviewer subconsciously compares modern games to older games, and then wants to push up the scores on modern games to reflect this.

          And you know, I think that's perfectly OK. It's nice to have a little scoring consistency over time as well.

        They might look better, but I'm struggling to think of many modern games that have had the replay factor and fun of something from the 80s or 90s.

    The fact that developers and publishers expect 9s and 10s and consider 8s to be sub-par just illustrates how completely useless and fundamentally broken the games review system and games "journalism" is.

    What annoys me about this review is not the score, they have every right to give it that score, but the whole idea that they gave it a score reflective of what the last game was. Now Naughty Dog are a relatively small developer, the only reason it is considered a AAA title because it has Sony publishing and marketing it. To expect a sequel to leap and bound over its award winning predecessor with such a small team working on it over two years is increadibly unfair of the reviewer.

    I think it in terms of the uc3 review and gaming in general, people would like to see more games based on the red dead redemption model.

    I hope it stings their pride and naughty dog strive for maybe a fantasy based rdr style game.

    cinematic linear games just don't do it for me anymore, I can see much better action in those brilliant BLUR game trailers.

    I'd still consider buying a game that scored a 1 out of 10. Which might be a tad off-topic, but I think we should first learn to value and use the full scale of the scoring system before we can argue with any sort of straight face that developers shouldn't have the mentality of anything bellow 8 or 9 is bad.

    But yes, more dialog is never bad. If it proofs to be pointless dialog you can ignore it and it if proofs to be useful dialog you can incorporate it. Good all around.

      Most Scoring systems take the average of a series of scores, that is why you will never see a really low score. i.e.

      Audio:
      Graphics:
      Fun:

      If you rate on fun-ness alone then you could use the scale, but then it would be even more biased considering the fact that everyone has a different taste in games.

      What we should really be doing is critiquing a game without giving a score as this allows both fans and developers to see what people did and did not like about a game while not giving publishers a score that they can use to cut funding to studios.

    It is not, by any stretch of the imagination, ok for game developers to interfere with the review process. That's my professional opinion as a game dev.

    If I was a media type, I'd institute an immediate 20% reduction in score for anyone getting whiny about the score they were given, particularly if its the difference between an 8 and a 9.

    I just wanna say to those who think this is a dispute over the score specifically,you're wrong.

    This is about the intricacies of that review that brought about the score. A game developer as highly regarded as Naughty Dog can just accept an 8/10 if it's about something technical that they can fix, but because it's 8/10 for the experiential reasoning that Simon gives, that's what's most disappointing for ND, because they built this game to be what it is, an experience-type rather than a 'gamey' game.

    Agreed! I would welcome more of a two-way discussion. All perspectives are valid IMO, meaning all subjective reviews represent a certain amount of consumers. I suppose you'd have to be careful to not come across as snarky as a dev. But this type of back and forth would lead to additional insights into the developers' goals & methods, as well as promoting transparency in critics and keeping them honest.

    And now we see what the mindless consumer has done to the game industry.
    Sigh.

Join the discussion!

Trending Stories Right Now