Over the past few years, review bombs — people organising en masse to post negative reviews to a game’s store page to tank its review score — have become one of Steam’s most visible issues. Last year alone, review bombing happened in Steam reviews over everything from women as generals to sales that happened too early.
Developers have cited this sort of toxicity as a reason they’re excited about the Epic Games Store, which plans to address the issue with an opt-in review system. Today, Valve announced it will take steps to defuse Steam’s review bomb problem.
In a news post, Valve described a series of planned changes to Steam’s review system aimed at minimising the impact of review bombs. “That change can be described easily,” the statement reads. “We’re going to identify off-topic review bombs, and remove them from the Review Score.”
Valve says it will do this by using its chart-based review bomb detection system — which previously only served to make review bombs more visible — to identify “anomalous review activity”. At that point, Valve says a team of people will investigate those anomalies, and, if they determine that something fishy is afoot, they’ll “mark the time period it encompasses and notify the developer”.
If Valve finds that coordinated review bombing has indeed occurred, any reviews posted during that time period won’t count toward the game’s review score. This will unfortunately include reviews posted from non-nefarious individuals during that time period, though, because it “isn’t feasible for us to read every single review”.
According to Valve, however, data has shown that review bombs are “temporary distortions”, so the overall review score will remain accurate even if some well-meaning reviews get caught up in Valve’s net.
It’s unclear whether Valve will be selectively removing reviews left during a review bombing time period, hiding them, or doing nothing at all to them. On one hand, the company says that “the reviews themselves are left untouched — if you want to dig into them to see if they’re relevant to you, you’ll still be able to do so.”
But then, one sentence later, the company says, “To help you do that, we’ve made it clear when you’re looking at a store page where we’ve removed some reviews by default, and we’ve further improved the UI around anomalous review periods.”
Kotaku has reached out to Valve for clarification on this point, but has not yet heard back.
In the statement, Valve offered examples of subjects that will be deemed “off topic” in the form of DRM and EULA changes.
“We had long debates about these two, and others like them,” Valve said. “They’re technically not a part of the game, but they are an issue for some players. In the end, we’ve decided to define them as off-topic review bombs. Our reasoning is that the ‘general’ Steam player doesn’t care as much about them, so the Review Score is more accurate if it doesn’t contain them.”
Valve also noted that players will still be able to dig into “removed” reviews if they’re interested in those issues.
On top of that, Steam users will be able to opt out of this new system entirely by using an option that’ll keep review bombs in games’ review scores. And, again, people will apparently still be able to look at reviews that have been removed.
Review bombers won’t have as much power to affect games’ standing with the Steam algorithm, but this could also just encourage review bombers to find other ways to evolve their tactics and get through what sounds like some still worryingly large loopholes. Time will tell.
Comments
17 responses to “Valve Says It Will Remove ‘Off-Topic Review Bombs’ From Steam Scores”
This does sound troubling. some ‘review bombs’ are large scale critics of very valid issues with a game. take no mans sky for instance. the inital reviews could be construed as review bombs and yet it had a lot of very valid criticisms of things. Not to say that unjustified review bombing doesn’t happen but, I really hope this system isn’t entirely automated.
Another problem is review content. how many reviews are just a few sentences? quite a few from memory. will they be caught up? after all the content of a review bomb is irrelavent. Youtubers and other sources will voice the same criticisms so the steam review page just needs a flood of negative reviews disguised either with little content or made up generic problems such as “the story is flat. the controls are clunky” and so on.
You may have missed the multiple references in the article in which Valve is quoted as considering removing only “off-topic review bombs”, not review bombing in general.
and what exactly is considered off topic? is it anything not core gameplay? is it the message the game spread? can I not criticise developer practices?
Yeah my first thought was of the Diablo mobile release. Though to be fair people weren’t reviewing the video as a mobile game but as a PC game. So they may still consider that to be review bombing.
It’s an interesting problem. It’s going to suck if they nuke legit reviews at the same time as dodgy ones.
Yeah. It’s a very complex issue and no broad strokes are going to deal with it appropriately. It’s something that needs careful consideration.
I’m all for it so long as they also wipe out the stupid “Did x in game. 10/10 would buy again.” reviews as well. But…
Uh, why? This is actually useful information. If the game has awful DRM or changes their EULA in a way that might affect me, then I definitely want to know about it and reviews highlighting this should be permitted. Alternatively there needs to be some other mechanism to allow these things to be highlighted to prospective buyers, if they don’t want it to affect the review score.
I don’t agree with your reasoning, but I do agree that its a bit of a stretch to say that those topics are ‘off topic’. There directly related to the game, and to claim that there ‘off topic’ shows they need to rethink shit.
Upvoted because that clown @kethic is rolling through comments downvoting everybody for no reason.
Was about to upvote yours for the same reason, as you cant upvote your own comment, but Jagji beat me to it 🙂
😛
Upvoting because all the cool kids are doing it.
Yes that’s a reason why I hate Metacritic – the 10 scores are almost as bad as the 0 scores. You need to know what a game’s flaws are and if that will affect your gameplay.
Honestly, its a problem with every review system. Buy something from ebay and it’s the same.
Not really sure how you can make a “perfect” review system because human nature will mess it up. Lazy people not reviewing, or doing half assed reviews and using the default score, or just voting max. People voting low because they don’t like the company, even though the product itself was perfect.
Makes it hard to design and hard to interpret results 🙁
I’d absolutely want to know if there are DRM or EULA changes that are draconian. This just highlights why I dislike Steam.
I think the biggest problem on Steam (all online stores really), is over pricing. And having negative reviews is actually good at this point in time. I typically read through whatever is on the program page in Steam to see what people say and if it has some info about the game is helpful. In the last year I’ve only bought Dragon Quest 11 and Assassin’s Creed on Steam, everything else has been played for no more then a few hours and been disappointing to me. And they’ve finally converted to the AU dollar, which means we’re paying more then we used to in general. I don’t think my medical issues would have caused these feelings in previous years (2016->). Gaming in general is just a disappointing thing at the moment.
It’s kinda sad that I can buy a physical copy for a console for less than a digital copy on PC.
Steam reviews are about as much use as IMDB reviews. 90% of them are written by semi-literate imbeciles. The proliferation of “did x. 10/10 would y again” rubbish is testament to this.