Apologies If We Wasted Your Time With That Preview

A couple of writers for this site recently proposed that we stop previewing games. We could still talk to developers before games come out, they suggested. We could still visit studios early or even play games before release, but somehow, some way, we'd no longer write previews.

Given that I prefer to be absolutist only about not being absolutist, I nixed this idea. If you declare you're not writing previews pretty much all you do is voluntarily roadblock one avenue of reporting while opening yourself up to criticism that the thing you did write about a game is a preview in masquerade.

The problem with previews though is that they mostly suck. You might agree. So let's talk about that and how to do previews right.

Previews mostly suck because they rarely serve any of the constituencies we want them to serve. There's you, the reader, who we figure would like to know about a new game. So we go to some hotel suite or development studio and play a game early. Or we watch someone play a game early. We chat with a producer who may or may not have been closely involved with the development of the game. We hope that 10 minutes of access to the game and a handful of questions that we politely or rudely throw at the producer can produce something we can share with you — something that will be edifying about the game.

Previews mostly suck because they rarely serve any of the constituencies we want them to serve.

Our inner critic worries that we're having to size up a game that won't be out for six months based on little more than a tease. One might more effectively predict the quality of some future meal by sampling one of those cubes of cheese they put on toothpicks at the supermarket.

Our inner reporter worries that whatever truth there is to know about the game we're previewing is not on display. A guy can check out 50 things that are in Assassin's Creed III, but of course, he won't know about the bugs that wind up being in the finished game.

***

How do you solve this problem? Ramp up the old ethics standards and declare that thou shall not eat any EA-crafted mini-burgers at the snack tables that seem to always be close to the preview builds we see? Demand exclusive previews from big, bad publishers and settle for nothing less? Write previews that are more negative? (Well, yeah.)

The obligation of any good reporter or critic is to be truthful. The reporter seeks to identify and convey the truth of the world around them. The critic strives to present the inner truth of what they make of the thing they have experienced. At Kotaku, we don't consider these roles to be in conflict. We accept that all reporting might be tinged with the subjective. We must simply disclose when opinion is entering into the reporting and be obvious when we're transitioning from dispassionate witness to chatty analyst. To preview a game, one must be a bit of both. When previewing a game, we're reporters and critics. And we're trying to find the truth.

When previewing a game, we're reporters and critics. And we're trying to find the truth.

The problem with previewing games is that many previews are themselves engineered to obscure the truth. There need not be any malicious intent. An EA or Activision or even an indie developer invites the press to swing by. We show up at a hotel suite. We're offered water or coffee, maybe a snack while we play. A psychological ploy or good manners? It doesn't matter. Whether we say yes or no, we're still only playing a sliver of a game. That's what matters. Slivers can always be good. Think about it. You could grab the worst game you own and mine a decent sliver out of it. So who's to say what we're playing six months before release is any good? It's just a sliver. We don't know about the whole game. It's not even done yet!

Ask the developers or producers about the game you're previewing and even the most honest creators aren't going to divulge the worst disasters their game is facing. Maybe they're showing you BioShock Infinite at E3 2011 and dazzling you with a demo. Are they hiding flaws or showcasing potential? Are they conducting a magic trick or are they showing you that they've at least made enough progress to have one awesome little thing to show off? These developers are not about to tell you about the struggles they and the makers of all great and horrible games are going through, years before the game is out. What is the truth of the game on display? It's nearly impossible to tell. Even the developers probably don't really know. They can't predict the disasters or successes to come.

***

The concern that we have at Kotaku about previews is that you, the reader, will feel misled by them. You'll read an optimistic preview in May and then wonder why the game we previewed seems, in November, to be junk. You'll wonder if the preview was de facto promotion, if it was the result of lazy, unskeptical reporting or even an honest effort made by people too foolish to identify the impossibility of assessing an autumn game in the spring.

To all that, I can say that we recognise the potential pitfalls more clearly than ever. To that end, we've been turning down an increasing number of invitations from game-makers big and small to see small pieces of their games. We skipped a chance to play the multiplayer in the new Luigi's Mansion and turned down an opportunity to try the new DLC for Battlefield 3, both at publisher-run showcases. When we're offered to see a single level of a game, we say no. Even when we are attending preview events — showcases where slivers of multiple games are on display — we've become more willing to spike boring interviews with producers whose comments would only generate interest when taken out of context.

We'll still gamble on some previews. We spent several hundred dollars to send one of our top editors across the country to Bungie to see their new game, Destiny. The fact that Bungie then showed so little is noted in our preview. We still dropped in at an EA event in New York City last week.

We've turned down more previews that offer eye-blinks of time with new games, favouring longer sessions like the one that got us 4.5 hours with BioShock Infinite (the game's second 4.5 hours might be dreadful for all we know; but it feels like it is worth your time — and ours — to assess the first 4.5 hours, which are very good).

It is our intent to present to you video game previews that resemble what we'd tell you about a game if you'd entered our chatroom or if you bought one of us a beer.

Some previews we write may still compare awkwardly with the quality of the game that is eventually released. We'll keep our reporting hats on to investigate the divide, as we currently are in order to reconcile the gulf between the 2012 Aliens Colonial Marines demo and the 2013 finished (!) product.

The main pitfall that I see with the video game preview is the propensity for the preview-writer to overstate what they've seen, to extrapolate from a fragment of the game or a fragmentary comment from a developer something grander about the whole game. That pitfall applies to a lot of criticism and reporting. Mountains are made of molehills. Small glimpses are puffed up as exposés. Taste-tests are reported as full meals. That kind of careless inflation is hyperbole. It may hype a company's new game, but perhaps more damaging, it hypes our own experiences, making more of them than is appropriate. The fat-free truth should be enough. Our gut reactions should suffice.

It is our intent to present to you video game previews that resemble what we'd tell you about a game if you'd entered our chatroom or if you bought one of us a beer. We'd tell you what we *really* thought, because what we *really* thought is what you, the Kotaku reader, deserves.

Going forward, one thing we'll be adding to previews is a footnote that states clearly how many minutes of the game we've played at a preview event and/or how many minutes of the game we've seen. That will help ground the lofty expectations writers and readers have of previews, I think. It will bring us one step closer, I hope, to taking you there with us and to presenting you what feels like — and what is — What Really Happened.


Comments

    That seems like a positive addition to reviews, going forward!

    I wonder if the writers' suggestions of nixing previews come as a result of Destructoid's Jim Sterling announcing that he's dropping previews after Aliens: Colonial Marines ended up as a complete farce?

      I think most everyone who saw the preview has had second thoughts about the veracity of them after seeing the actual game, but Sterling was one of the more vocal about it.

        This whole thing is pretty interesting. I think that Destructoid is one of the more reputable outlets - they already employ most of the policies you mention below, and have for some time.

        I think a big part of Sterlings disappointment is that he misled his readers because he was misled himself: Randy Pitchford lied about the product and presented 'fake' demos. At the time, it really wasn't an unusual preview - its just that the preview system is so gamed now that it makes it impossible to provide accurate information until they receive review copies. Hence, he's not doing previews anymore.

          Very much this.
          IIRC, a few reviewers have mentioned that what they played in the press review copies they received had content that was not included in the retail release.

          Unfortunately I don't think we'll ever hear the true story; "spiling the beans" on this will be effectively signing "do not hire" across your games industry CV.

    I think the best way to avoid the issues had with journalistic integrity questions when it comes to previews is to ensure full disclosure. Tell people what you saw, what was said, and who paid for lunch. If you felt like saying negative things would jeopardise future preview access, say so, and make it clear that's only your impression. It's probably true, because even from a non-objective perspective why should EA or whoever feel compelled to show you stuff if you have history of shitting all over it?

    If you start losing exclusives because you're not a suck up, maybe that will reflect well on you and lead more people to consider your work.

    That said, don't be negative about everything either. Some smaller blogs feel like people will only see them as honest if they are negative, and that's no way to build a reputation.

    But now that I think about it, to hell with it. At the end of the day, people whose opinions you reinforce will believe you, people who disagree with your opinion will consider you "bought" if you say nice things and "an asshole" if you say bad things.

    So hell, do whatever pays the bills, this is all pointless bickering over an entertainment product that'll probably sell pretty well anyway. Every year Call of Duty is critically panned as the worst one yet by a small contingent of indie blogs desperate for attention and advertising revenue, but the sales numbers keep going up.

    What really sells games is marketing, and a 10/10 in a magazine or website is just part of that. A clever publisher or marketer will find someone who'll give them one. Might as well be you.

    Last edited 19/02/13 1:12 pm

    I think the addition of this footnote is a smart move and an elegant solution. I like to think i will be able to gauge how much smoke and mirrors could be packed into a short playtime.

      We should still have previews, but writers should be asking the hard questions every single time. For example when TES 6 is announced, istead of gushing over how great TES series is writers should be asking if there will be a huge increase in bug catching and fixing, if the setting contradicts the settings, if the quests are actually engaging, if the gameplay will be balanced so that the the gameplay wont be broken because you leveled skills x,y,z aloong with skills f,i and q.

      Writers should be able to ask those types questions and not get bullshit responses.

        To be fair to TES, building stupidly overpowered characters from skills {f, i, q, x, y, z} is a feature for many players, including me.

        I agree up to the last few words. A writer can't control whether he's getting a bullshit response or not. I've done some interviewing before. You can word things as cleverly as possible to catch someone out, and even if 'caught' they'll quickly re-frame if they're on the ball. If they're REALLY on their game, they'll simply smile a sanguine smile, and you'll smile right back, and you'll both know that you're opponents and neither is going to get what they want, here. You can even break the fourth wall and outright say it. "No numbers, huh?" If you're lucky and the person is nice enough or seems to respect you, you might get a frank, "No numbers," back. (That's professional courtsey. If they're in full opponent mode, they'll just repeat a re-framing talking point again, about how numbers are misleading and a dangerous thing to talk about, when what's really important is the message...)

        I suggest re-phrasing. Writers should be able to ask those types of questions and make it very clear that they've received bullshit responses.

        Last edited 19/02/13 2:26 pm

        It's hinted at in the article, but sometimes the person present at the preview simply can't answer those questions. If it's a PR person, they will only have what information he has been provided by the team or what they have learnt chatting with them before the event.
        The limited access of a preview is what creates half of the problems addressed by the article.
        It can be very difficult to get answers to the difficult questions when the only person avaiable simply doesn't know.
        Which isn't to say they shouldn't try, but previews can be very tightly controlled.

    The simplest solution would seem to be writing previews in as objective a fashion as possible - This is what the game is. This is what we saw. Try to avoid editorialising. It's probably much easier said than done, but it seems like a more sensible solution than dropping previews entirely, which would seem to affect everyone involved - dev, publisher, blog and consumer, negatively.

    Please stop previewing all products and media.
    This site is much better suited to reporting on videogame decorated cakes, cosplay galleries and post-release concept art.

    I don't see the problem. This is how previews have always been and if you guys feel you're not getting the whole deal with a sliver of the game it not your fault, it's the developers and publishers that are determining what you get to see and as journalists you report what you get to see.

    I understand how you don't feel completely objective when playing a small part of an exciting new game but you think we don't know that? I mean who wouldn't be excited to go to the people who make our games and get to play a bit of an exciting sounding new game before anybody else does? E3 definitely isn't a representation of the full gaming experience but how many of us would gladly swap places with the journalists and play the latest games.

    "It is our intent to present to you video game previews that resemble what we’d tell you about a game if you’d entered our chatroom or if you bought one of us a beer."

    Drunken reviews. I'm looking forward to them. Probably the most useful to me as it matches how I play.

    "we’ve become more willing to spike boring interviews with producers whose comments would only generate interest when taken out of context."

    http://t.co/rmPiJmdl

    It seems like the problem is that previews are almost always positive as in the critic is looking from a gamers perspective in wanting the game to be good, whether that's just an industry trend or the publisher manufacturing the demo is another matter

    Doing previews well is the whole reason that you're a professional site and not a blogger. You have access to see these things before they ship, and we expect you to use your experience, intelligence, game-business savvy, and training to do a good review. Yes, it's HARD to judge a game from a preview, and balance the fact that you want your site to draw hits against the risk of making a bad call or unfairly damaging the sales of a game with real potential. It's hard. That's why you're paid money to do it. Or, more accurately, paid money to do it right. You should do previews. And when you get it wrong, you should be smacked in the face for it, the same way any other professional can get hauled over the carpet for failing to succeed in their core competencies.

    Here's an idea: Don't use bullshots in your reviews. Unfortunately every outlet seems to do it. Its ridiculous when you read a review that bags the graphics of a game and yet all of the screenshots are touched-up promotional shots.

    We definitely appreciate the fact that you want whats best for the readers.
    Not many game sites would give up the prince/princess treatment of being taken to nice hotels and sucked up to.

    Not enough commenters here are thanking you for that. So thanks again.

    Lets see how this new policy of yours go. Hopefully this makes at least a small impact on the preview industry. Yea I called it an industry of its own.

Join the discussion!

Trending Stories Right Now