Steam 'Review Bombing' Is A Problem

Steam 'Review Bombing' Is A Problem

In recent times, people have taken to flooding games' Steam pages with (typically negative) reviews and tags to protest, well, lots of things. New features, viewpoints of creators, messages in the games themselves. But why? And does it actually work?

Recently, games like Skyrim, with its controversial paid mods, Titan Souls, after a beef with popular YouTuber Totalbiscuit, and movies like Game Loading, a documentary about game developers groups like GamerGate can't stand, have come under fire on Steam. Their pages ended up looking like crime scenes, because in the eyes of some fans, they'd done just that: committed a crime — Skyrim with its jarring new approach to mods, Titan Souls and Game Loading by being linked to perspectives people didn't agree with.

(Disclosure: Game Loading features developer Zoe Quinn, who I once dated.)

Now, review bombing, as the practice is often known, is hardly a new thing. It happens frequently on review aggregate sites like Metacritic and storefronts like Amazon. But on Steam, user reviews — and most user-related/created content — are front and centre. Even before you click on a game's listing, you can find a thumbs-up or down icon that indicates general community sentiment toward the game. I've talked to people who will totally ignore a new Steam game simply on the basis that it doesn't have a thumbs-up.

Steam 'Review Bombing' Is A Problem

Beyond that, though, Valve has racked its throbbing hivemind brain collective to make Steam reviews useful, at least as a quick snapshot of what a game's about. Reviews are categorised as positive, negative, funny, and of course "helpful" — all voted on by users. The result? They are actually kinda handy sometimes. I know I check at least the first few before grabbing a game I've never heard of. And if they're overwhelmingly skewed in a certain direction, I know something is probably amiss.

Review bombs, however, often detonate because someone started snipping at the wrong wire, struck a nerve by adding an unwanted feature or saying or doing something certain people disliked. In some cases, they don't reflect the actual content of the game (or film or what have you) in question, despite directly affecting how people seem to perceive a game's overall quality. People just want to be heard. Sometimes they want something to change, sometimes they want a person to change, and sometimes they just want to make something or someone burn. Regardless, it's online protest, en masse.

When I asked people why they have participated in Steam review bombs, that's what most of them said: in their eyes, it's not only a form of protest, it's the only one that works for them. Everything else is just a tweet or forum post into the void, weightless words floating in empty space. Review bombing on Steam, though — a place where user reviews can actually matter sometimes — forces developers and publishers to take notice.

"Trying to negatively affect an entity's ability to make money is not only a valid way to protest," one Kotaku reader wrote to me, "as you can see by the removal of Skyrim's paid mods it is one of the most effective ways... I think defending the value-proposition of PC gaming is worth the protest as I get a lot of entertainment time I value out of PC gaming, and considering what a desert of quality mobile gaming is."

Steam 'Review Bombing' Is A Problem

Another reader echoed this sentiment, suggesting that during times of "war" there's no better way to hit home than a Steam review bombardment. Taking things even further, they voiced suspicion of shady (though unverifiable) practices game makers might employ to drown out dissent:

"I've only done it when for lack of a better term, war has been declared," they wrote in an email. "It's long been proven that sketchy publishers and developers are willing to pay (or otherwise offer some for of consideration) for better reviews and when you can afford to drown honest negative reviews with 10 paid for positive reviews, the only way the message will get out is via email mass reviewing. The same (but reversed) goes for games that have been targeted for whatever their message was, or if they were simply targeted for the lulz."

"I consider reviews in this forum as a type of public demonstration, be it a show of scorn or support."

Steam 'Review Bombing' Is A Problem

I found a similar set of viewpoints shared throughout hundreds of emails and comments: Steam page bombs — whether over big in-game issues or strong ideological disagreements with a game's creator — happen not because they're fair or easy, but because nobody will listen otherwise.

How much of an impact do they actually have, though? Do they work as well as people who've dropped everything else to drop some bombs seem to think they do? In light of recent events, the answer would appear to be yes. Valve and Bethesda did do a huge about-face on Skyrim's paid mods after a period during which Skyrim's percentage of positive reviews dropped from the high nineties into the low eighties. (That's quite a plummet over the course of only a few days for a game with so many years' worth of reviews.) However, correlation does not necessarily equal causation, and many other clashing ideals factored into Valve and Bethesda's decision to pull the plug.

Games that stand to get hardest by review bombs, however, tend to be the smallest. It's hard enough for a small game to gain traction on Steam as is, especially if it's a new release. However that early period is also crucial, one of a game's few shots at Steam front page glory before plunging into the chaotic black hole that is the rest of the service. I asked creators that were hit hard by irate Steam users in their early goings, Andrew Gleeson of Titan Souls and Game Loading director Lester Francois, how exactly they were impacted.

Steam 'Review Bombing' Is A Problem

"It's hard to say whether the reviews themselves affected traction," Gleeson replied to me via email. "For those who [went after Titan Souls because of the Totalbiscuit thing], they weren't going to be convinced otherwise. You only need 500 die-hard followers to screw up a Steam page like ours, which actually pales in comparison to the number of copies the game ended up selling anyway. A few people reached out to us, confused as to why negative reviews were pushed to the front of the page, when the actual -number- of positive reviews dwarfed the negative ones. I believe this highlights a flaw in Steam's current system."

Francois' response was more surprising. In the short term, he said, he felt like the resulting controversy actually benefited Game Loading. People swarmed on its Steam page like ants from an anthill, causing previously unaware parties on services like Twitter to come see what all the fuss was about.

"Of course the negative reviews do have an impact on sales, but we knew the barrage of criticism was coming and factored that into our marketing and expectations," he said. "All the noise started like clockwork when we launched. The online backlash worked in our favour, it was the best form of signal boosting. We are working with a tiny budget and to have that kind of exposure is priceless."

Steam 'Review Bombing' Is A Problem

The people who bombed their respective Steam pages, then, didn't exactly get the outcomes they wanted. Then again, what they wanted here is kinda difficult to judge. With Skyrim, it was easy: people wanted paid mods to go away. Eventually, they got that. But with Gleeson, what response were they hoping for? An apology to Totalbiscuit? And with Game Loading, did they want the film to simply stop existing, to get so thoroughly buried that it'd never see the light of day again? A protest without clear or achievable goals, one might argue, is not a very good protest at all.

As a result, neither Gleeson nor Francois are too worried about the damage Steam bombardments might have caused them. They both agreed that it was less about causing lasting harm and more about saying, "HEY, NOTICE US."

Steam 'Review Bombing' Is A Problem

"Luckily I don't think the bombardment has had much of a long-term impact," said Gleeson. "When our game was under fire, we all decided to try and not stoke the flames in hopes it would die out quickly, and it has. Again, it's impossible to say whether the whole drama has influenced sales; perhaps we lost a few thousand of them. Most likely not. I'm not worried either way."

Meanwhile, Game Loading got help directly from Valve, which definitely made the process more manageable. "Valve spotted a few people violating their rules and guidelines and deleted a few offensive comments," said Francois. "They were pretty quick to act on it."

A number of the less, er, game-related reviews of Titan Souls also seem to have disappeared, or at least gotten pushed down.

Still, existing in a system so easily manipulated by relatively small groups does give creators like Gleeson and Francois reason to worry. It's not just negative reviews, either (those are actually the hardest to manipulate, given that you have to own a game to write a review of it). People can also manipulate Steam pages by changing their tags — in the case Game Loading, "villain protagonist" and "dystopian" are the top two tags to this day — flooding discussion pages, and downvoting positive reviews so that only negative ones remain immediately accessible. It's those sorts of things, little tweaks that can do short and long-term damage, that Gleeson thinks Valve needs to work on fixing, stat.

Steam 'Review Bombing' Is A Problem

"It's kind of a joke, really," he said. "Allowing people who have not yet bought the game, or even have no plans to, be allowed to influence the way it's presented on a store, almost effortlessly, is absurd and short-sighted and harmful."

"I understand how Valve has set up the store to aid customers and that's great, but yes it can be easily manipulated," added Francois.

Gleeson opined that it's that ease — the lightning-fast clicks of a mouse or clacks of a few keys — that's making this an increasingly common occurrence. "It's effortless," he said. "It produces results. It's a call-to-arms to defend your values, to show dominance."

"It felt like clockwork," he added. "Twitter notifications flooding with harassment, fake Twitter accounts appearing and producing slander — the steam negative review up-voting, of course. The stuff we have all seen before. It's the only way these kinds of people have any influence at all because they're exploiting these systems, the same way, every single time, and not enough is being done to stop it from happening. Twitter needs to sort out their shit, and so do Valve. This stuff needs to be wiped out. It doesn't help anyone."

But others disagree with him. They want to be heard, even if it means going through some back doors and seedy alleys in order to arrive at their soapbox. It's symptomatic of an ever-growing tension between creators and people who consume (or at least exist in the same space as) the things they create. In this day and age, as a result of tools like Steam and Twitter and what have you, players feel like they're both right next to creators and a million miles away. Like they can scream and shout and nearly reach out and touch, yet elicit no reaction. It is a strange dissonance, to be interacting in a way that suggests it would be rude for someone to ignore you, yet share that same space with countless others — far too many for someone to pay attention to all at once. It prompts disconnection, dehumanisation, anger — especially when people don't share similar viewpoints.

Steam 'Review Bombing' Is A Problem

It's easy to say that these sorts of protests come from a place of immaturity — of an inability to cope with having people mess with your toys or hold differing opinions — and I think that's true some of the time. But the way it's expressed also suggests that modern communication is in a weird, frightening place. Millions of voices, all feeling equally valid, struggling to be heard. In truth, not all voices carry the same weight in all situations, depending on the situation, the context. But we've created a world where it feels like they're supposed to. It's hard not to be frustrated and confused when things don't pan out that way, when a flawed system shows its cracks. So people band together and take out their rage on others, even those who make creations they really love, because it's the only thing that will get any sort of reaction — positive, negative, effective or not — anything more than silence.

Picture: Bryce Durbin/TechCrunch

WATCH MORE: PC Gaming News


Comments

    Even a game like quake 1 gets it, due to the fact they removed the nine inch nails sound track.
    And please stop blaming gamer gate for people hating on somthing that you like. Belive it or not some are just bad and people see it. Not everything is a conspiracy.

      The Game Loading one's a weird one, though... I didn't realize it was some hate target from reading the reviews. But if people who care about that shit have been getting their twitter feeds bombarded by assholes, then I'll probably take their word on it. God knows there's too much precedent.

      Maybe it took the angry toolkits of the hate machine a while to get mobilized, but when I was refreshing the store page and checked the top-rated reviews of Game Loading after they turned up, they seemed pretty scathing from a film-critique point of view rather than a misogyny one.

      The general consensus (including even many of the positive reviews) seemed to be that 'Indie Game: The Movie' was the better film, being more in-depth and useful than simply a video-blog preview of someone's linked-in page (and free!). Frankly, I didn't even watch 'Indie Game' either; the subject REALLY doesn't interest me and I personally blame 'people listening to indie developers' for the clusterfuck that is Greenlight and the Steam store gradually turning into the fucking iTunes App store because every indie's precious game is an underrated/undiscovered diamond in the rough that 'doesn't get the attention it deserves'.

      ie: A bunch of people complaining about being stuck in traffic, not realizing they ARE traffic.

      For all that? I did like one positive, heartfelt review that was from an inspired indie dev from some developing nation who was expressing his thanks.
      For me, it only takes a couple of those touching, sincere thanks to cancel out the jaded critics who were turned off by the accused 'incestuous', nature of a vocal and opinionated self-serving indie clique.

        I sort of agree with you....
        I'm not saying GG assosiates are saints, but i'm pretty sure they won't pay 10$ just to negative a game.
        As you said; all of people just said indie movie was better.
        And that's just it: it's a opinion on a thing that has been done before and isn't all that spectacular, therefore it cops lower reveiws.

          Probably should stop stereotyping and typecasting and profiling people who support Gamergate. Those are the actions that Gamergaters get accused of.

            In what way am i doing that?

              I'm not saying GG assosiates are saints, but i'm pretty sure they won't pay 10$ just to negative a game.

              Remember every person is a special snowflake of individuality. Generalising like this is a symbol of patriarchy.........

                Let me say it differently; i don't think any one in there right, sane and individual mind would buy somthing to just down vote it. Especially if it means that money goes to the ones they disagree with.

          You have to have purchased a product to post a review, but any Steam user can then thumbs up or thumbs down that review. If a group of people did want to create negative perceptions about a product, they would only need a handful of individuals to actually purchase it and post reviews, and then others could simply upvote that review, pushing it to the top.

            Ahh. I see what you mean about that. But even so it looks like people just didn't agree with the video and people are upset about it.

            It's a good point.

            I don't see it being abused as much as you'd suspect it could be, though... just as an example, I remember seeing the Mortal Kombat X negative reviews on 'launch' day before it was made clear that the game wasn't actually available and only the menu was 'launched'.

            There were some pretty unambiguously fair reviews which were essentially just warning, "Look, it doesn't matter how good or bad the game is, right now I don't recommend you buy it because it's broken and no-one knows why. I'll update when it's fixed with an actual review."

            Rated 45% helpful. Just so that you're aware, that percentage is not based on how many people viewed it, it's based on upvote vs downvote.

            Two people read a review and both upvote it, it's 100% helpful. Two people read it and only one bothers to upvote it, it's 100% helpful. Two people read it and one upvotes it and one downvotes it, it's 50% helpful.

            For all the people who said, "Yup, that helped me with my purchasing decision," or, "Yeah, those fuckers releasing a broken game!" about this review, there were more who for whatever reason didn't think that was a fair review.

            For all the folks bombing reviews for a political agenda, it seems that there are those willing to balance them out, who have exactly the same access to upvoting or downvoting the review.

            Edit:
            After reading a LOT of reviews in Steam (like... almost daily. What can I say? I'm curious about every game that pops up!) something I've noticed is that in general, sincere positive reviews tend to get rated as 'helpful' a lot more than sincere negative reviews that point out real issues. It seems like negativity isn't as popular in the review system as positivity, in general. I'm not sure whether to attribute this to a general desire in the community to disapprove of negativity, or if it's the one and only places that developers can push back against criticism without it being traceable back to them, opening them up to criticisms of 'censorship'.

            It's certainly true of my own reviews.

            I've reviewed games like Portal 2 with nothing more useful than, "Holy shit why haven't you picked this up already, what the hell is wrong with you?" and that's been rated as being more helpful than a grudgingly negative or disappointed review that advises I can't recommend the game that I've played due to observable flaws. To my mind, baseless, unqualified fanboyism on my part (no matter how justified) with no explanation should not be rated as more helpful than when I've taken the time to measure out some criticism and decide that it should serve as a warning against what I consider a bad purchase. But there it is!

            Last edited 01/05/15 1:12 pm

      A decent portion of the negative reviews of GameLoading are not discussing the film itself, but complaining about the developers featured, pushing Gamergate talking points as they do so.

    "I’ve only done it when for lack of a better term, war has been declared,” they wrote in an email. “It’s long been proven that sketchy publishers and developers are willing to pay (or otherwise offer some for of consideration) for better reviews"

    What war was declared? There's war going on in the Middle East; people are being brutally killed or displaced from their homes, war crimes and genocide are being committed, refugees are fleeing on dangerous journeys to other parts of the world (and tragically losing their lives on the way, as 700 people did in one recent incident). Comparing a gaming issue, in any way, to the massive humanitarian crisis going on is ridiculous.

      War were declared.

        Just out of curiosity, we could use the cards to buy gum, then immediately quit the army, right? You know, playing you all for chumps?

      Its language. No one is comparing those two things. May as well get annoyed that you said "they wrote in an email" How can you compare writing with typing?! Completely different levels of communication!

        Writing/typing? Very similar. War, displacement, mass murder/entitled gamers being sooks because something isn't the EXACT way THEY want it? Not even close to a decent analogy.

          Defined: a state of armed conflict between different countries or different groups within a country.

          Analogy stands. Quite obviously. If you can't see that, go elsewhere.

            It took 3 days for you to think of that?
            And fact 1: this ain't no armed conflict.
            Fact 2: different groups in a country? Pretty sure GG doesn't have borders.
            Analogy is still horrible.

              No, it took three days for me to revisit Kotaku.

              Clearly you have no idea what an analogy is or how it is used. For example, likening a heart to a pump is an analogy, even though one is an organ and one is a machine. The point of the analogy is to draw comparison through similarities. You can't see the validity of the analogy in this circumstance. So, once again, go elsewhere. /thread

                There are good analogies, and then there are bad ones. Your analogy was a bad one. Deal with it. And as for your "go elsewhere" comment, well that's just the typical GG supporter attitude, isn't it? Someone has an opinion you don't like? Drive them out of town! Punish the heathens! After all, isn't that exactly what they did to Gamasutra and Kotaku by pressuring companies who advertised on their sites?

                  /thread

                  /rant

                  /ignorance

                  /stupidity

                  /strawman

                  run goelsewhere.exe

                  @keiranj

                  The level of immaturity you are displaying has done nothing but lose you respect. Constantly down voting someone just for having a different opinion, as well as trying to make them leave a forum because you aren't adult enough to have an intelligent debate with them, is just plain sad.

                  I'd agree with you but then we'd both be wrong.

    Entitled gamers throw tantrums when small things don’t go their way? I’m shocked!
    Mass Effect 1,2 and 3.- “Only loved the first 75 hours, they should change the end so it’s more to my liking. 0/10”

    There really isn’t much you can do about these kinds of things. You give everybody a voice and idiots will inevitably be the most vocal.

      Just the other day I found a thread which pretty much sums up how little faith I have in the community of 'gamers'....

      http://steamcommunity.com/app/310560/discussions/0/611704730326958644/

      tldr: someone gets very angry at Codemasters because they.......... put an achievement in Dirt Rally which is given to those who bought the game in Early Access.
      The conversation then of course devolves into hated, emotionally charged debate about the matter. It goes on for 8 pages.

        Should we tell them they can use SAM to unlock the achivement? ;)

        Thankyou for showing me this, it just became the most stupid, ridiculous and pointless thing I've seen this year. Honestly, it's a fucking pre-order achievement, of course you cant get it if you don't pre order it, why would you want it if you didn't pre order it and why should nobody else get to have it if they did pre order it?

        oh and it's just a fucking steam achievement! Who cares!

        Sorry....I just hate stupid people

        Well, initially I did sort-of agree with the OP. The tactics the industry is using to get people to pre-order games (digital ones at that!) is getting pretty ridiculous. PEOPLE NEED TO STOP PRE-ORDERING.

        Then I read on. The developer gave a fairly believable reason for why they did it. But yeah, it escalated fairly quickly after that.

    Long and short of it is that lots of people are cry baby jerks and can't possible stand the idea of someone getting enjoyment out of something they don't like themselves.

      Devils advocate here; Perhaps they see it as a frustration -- something that annoyed them so much that it ended up giving them a negative outlook on the game. They gave that negative review as a warning to others so they don't feel the same as they did saving them from it.

      It's a different veiw point but the end result is that same - it all comes down to what you think that they thought, which is a silly notion in it's own right.

    I don’t think there’s a problem with the data being presented, or even necessarily how it’s being presented. But people need to know how to interpret it.
    Interpreting data is something that people – in general – do very poorly.

    You can look at the ‘overwhelmingly positive’ or ‘overwhelmingly negative’ but if there’s only 15 reviews, it doesn’t mean shit and you need to look closer.

    You’re only reading them if you’ve already visited the page because you’re interested - that’s a win right there. I learn more from negative reviews than I do from positive ones. I’m viewing the page because the screenshots and general tags (RPG/strategy/TD) have interested me and don’t feature any of my red flags (Multi-player only, PVP, platformer).
    What I want to know is: “I’m interested. Are there any other dealbreakers not covered her?”

    Mortal Kombat X for example was flooded with negative reviews on launch day because it was, well… not fucking launched yet. That’s a pretty useful thing to know, and when all fifty reviews at the top of the page are saying it, you know it’s not isolated.

    Negative reviews will warn me if there’s only a couple hours content for this $25 game and I might want to wait for a sale. They might warn me that the localization is horrible and English translation very dodgy. They reduce my expectations in a way that prevents me from receiving an unpleasant surprise. I’ll put up with a lot of failings in a game (or movie), as long as I know about them up-front and can go into it, expecting it. It’s the nasty surprises that turn me off and make me regret my purchase.

    And there are certain types of reviews that I’m pretty sure EVERYONE knows to discount out of hand immediately.

    If the only negative reviews you can find are that the developers broke some kickstarter promises, or pissed off a youtuber, or didn’t keep beta-testers/early access rewards exclusive, then as a new-comer to the game… who gives a fuck? You can usually ignore those. If 300 reviews are positive and 10 are negative, people complaining about bugs that only happen on their SLI – which I don’t have – then I summarily ignore those as well.
    Others are: ‘Shitty mobile port’. Watch a let’s play and you’ll soon see if it’s a style that you’re willing to play on your desktop, though.

    Frankly, the fact that you can only review paid games that you personally have paid for is an excellent protection against the horde of flying internet monkeys that some asshole ‘activist’ has sent screeching toward the target of his or her ire. It’s a great equalizer, and sometimes you just really want to express your buyer’s remorse and make sure that friends and family don’t buy something that isn’t quite what it represented itself as.

    I like the system as it stands because I’ve learned how to usefully, meaningfully interpret the data on display.
    A clusterfuck of complaints is useful not always for turning me off a purchase, but if the worst anyone can complain about is minor or something I don’t care about, then I’m more likely to trust the negative reviews than a million glowing reviews from fans who I can’t necessarily trust not to have some rose-tinted goggles on.

      I'm not likely to trust anyone who doesn't know their shift. Includes people wearing rose tinted goggles but shockingly, negative reviews as well. For whatever reason I haven't made any excuses for unfounded criticism the way some do with praise.

        Now, see, that's where I'd say you're probably looking in entirely the wrong place, or looking for the wrong thing. There's a bit of a difference between finding a dataset useful vs finding its individual data-points 'praiseworthy'.
        Unless you don't give a shit about utility, or aggregate data, in which case I guess large scale data-sets aren't going to be much use to you and you should continue relying on the opinions of an individual reviewer you trust.

      You can look at the ‘overwhelmingly positive’ or ‘overwhelmingly negative’ but if there’s only 15 reviews, it doesn’t mean shit and you need to look closer.
      This! I have to admit, that seeing mostly negative, or overwhelmingly negative marked against a game sure does make me a little more cautious when it comes to purchasing games, but I always check to see how many reviews there are, and then actually find and read some of the both positive and negative ones, to give myself a better idea of whether it's worth the risk. Most of the time, I look, and they only have a few reviews, or the negative ones are just whiney babies crying because their feedback went unanswered or something. Always worth more investigation! Use your brains people! That's what you've got em for!

    I only do positive reviews. If you have nothing nice to say, then don't say anything at all.

      I dunno. I've had friends buy Infestation: Survivor Stories not realizing it was The War Z re-branded. If only I could've warned them! Or warn them about Colonial Marines... but then I would've had to buy it. There are sometimes good reasons to review negatively. To save the people you care about the heartbreak of buyer's remorse. :)

        I suppose, but I'm just one guy so the number doesn't really matter. Though for some games there aren't many reviews, there it would matter. Also some reviews I find I don't really read much in to. Depends also on the quality of the review. Too lengthy and detailed; forget about it. Too short and a joke or a wide overview/detaileless on things; skip that too... maybe mark as funny.

        Last edited 01/05/15 1:43 pm

          What I've noticed is that it shows up on the activity feed for your Steam friends. This is probably the only reason I've left reviews. Like you said... just one guy. I don't expect anyone aside from my friends to read them, and that influences what I write and how.

          Also WHAT I review. Generally I won't negatively review something unless I want to warn my friends. They'll click the store page, they'll see that I reviewed it, they'll have a chance to avoid a really bad purchase.

          But this is also why I generally only review the things that I quite like, that I'm pretty sure no-one else owns and could benefit from taking a look at. Some of the reviews I think are most important are for stuff I really want them to play, like Sequence (now renamed to 'Before The Echo'), or Card City Nights, or Zen of Sudoku.

    (Disclosure: Game Loading features developer Zoe Quinn, who I once dated.)
    I find this act of disclosure titillating.

    Eh. To me it's one of those 'What can you do about it eh?' issues.

Join the discussion!

Trending Stories Right Now