Valve has spent this year slowly revamping Steam reviews, and the latest change is a biggie: By default, reviews written by people who obtained a game with a key (or anything else not directly within Steam) don’t factor.
Valve explained that they made the change to counter fraudulent reviews and other attempts at score-boosting:
An analysis of games across Steam shows that at least 160 titles have a substantially greater percentage of positive reviews by users that activated the product with a cd key, compared to customers that purchased the game directly on Steam. There are, of course, legitimate reasons why this could be true for a game: Some games have strong audiences off Steam, and some games have passionate early adopters or Kickstarter backers that are much more invested in the game.
But in many cases, the abuse is clear and obvious, such as duplicated and/or generated reviews in large batches, or reviews from accounts linked to the developer. In those cases, we’ve now taken action by banning the false reviews and will be ending business relationships with developers that continue violating our rules.
It’s understandable that Valve would go after this kind of behaviour, but many developers are worried that this is less of a surgical strike and more of a carpet bombing. Kickstarters, especially, stand to take a big hit in the reviews department, and reviews have a huge impact in terms of who buys games, as well as in smaller, more intangible ways dictated by Steam’s discoverability algorithms.
Jonas Kyratzes, a writer and designer on games like The Infinite Ocean, The Sea Will Claim Everything, The Talos Principle and Serious Sam 4 said:
Hey look, half the reviews of TSWCE no longer count. And none of the people who backed The Council of Crows will be able to support it.
— Jonas Kyratzes (@JonasKyratzes) September 13, 2016
Kieron Kelly, a writer at Divinity studio Larian, explained:
The new Steam Review policy will hurt. As a kickstarter dev, your most passionate fans are now silenced.
— Kieron Kelly (@Kurnster) September 13, 2016
Simon Roth, creator of colony-building game Maia, lamented:
That’s me probably going out of business then. My crowdfunders and direct sales were how I hoped to improve my score pic.twitter.com/pvxXGQhweL
— Simon Roth (@SimoRoth) September 13, 2016
Between direct sales, Kickstarter, indiegogo I have 16000+ users who will never be able to leave me a review that matters to the score.
— Simon Roth (@SimoRoth) September 13, 2016
Death Ray Manta’s Rob Fearon said:
For some perspective, out of 120 odd reviews. I’ve just lost 40. I’m fortunate that I’ve retained ‘very positive’ because they’re all legit.
— Rob (@retroremakes) September 13, 2016
David Pittman, who worked on Eldritch and Neon Struct, offered some perspective in regards to Valve’s priorities:
…”let’s stop players abusing reviews” is disingenuous. Way more of the latter, in my experience. (2/2)
— David Pittman (@dphrygian) September 13, 2016
Dave Gilbert of adventure game studio Wadjet Eye gave anecdotal evidence of Valve’s changes in action:
Today those reviews dropped to 4. And this was WELL DESERVED. That said, however… /3
— Dave Gilbert (@WadjetEyeGames) September 13, 2016
These hardcore fans aren’t able to post reviews. E.g., the ones most likely to praise the game and give us the best reviews CAN’T.
— Dave Gilbert (@WadjetEyeGames) September 13, 2016
That said, this kind of behaviour isn’t exactly atypical of Valve, for better or worse. Alexis Kennedy, who wrote games like Fallen London and Sunless Sea (and now helps out at places like Paradox and BioWare), broke it down:
Historically they’ve often made a dramatic change, watched the effects, and then pulled back , calibrated or added a refinement. 2/4
— Alexis Kennedy (@alexiskennedy) September 13, 2016
When I think of Valve I think of the HG Wells line about ‘intellects vast and cool and unsympathetic’. Stupid, they’re not. 4/4
— Alexis Kennedy (@alexiskennedy) September 13, 2016
Sure enough, in a statement to Gamasutra, Valve said, “We are hearing lots of positive response to this update, and some criticisms. Like all updates we issue to our games and services, we will be monitoring the community reaction and incorporating that feedback into the next set of changes we make to improve the service for everyone.”
Still, it’s a rough situation for many. Sure, Valve will probably refine the system over time, but some developers stand to lose sales and standing right now. I can understand their ire.
It’s also just a weird decision all around. For one, Valve cited 160 instances of games that “have a substantially greater percentage of positive reviews by users that activated the product with a cd key”. That is, by many metrics, A Problem. The thing is, Steam is now home to thousands upon thousands of games. 160 is a relatively small proportion of them, and a change like this stands to impact more legitimate developers than sketchy ones. I feel like Valve could’ve hired a small team of people to monitor games with suspicious review activity and, as Valve put it, “end business relationships” on a case-by-case basis. Apparently, however, they chose to nuke everything from orbit and, perhaps, Make A Statement.
The underlying message of this change is also bizarre. Back when Valve first opened the floodgates to Steam by way of programs like Steam Greenlight, they encouraged developers to cultivate communities outside of Steam. In recent times, Valve has discouraged developers from doing things like offering keys to people who boost their game on Greenlight in order to close loopholes, which makes sense. This change to reviews, however, directly flies in the face of philosophies that form modern-day Steam’s building blocks. Why go through all the trouble of cultivating a community outside Steam, only to watch their influence wither when it actually counts?
Then, of course, there’s the cynical take on the review brouhaha, which goes something like this: Prior to these changes, developers would often point their most dedicated fans to direct purchase options or other stores (as opposed to Steam) because they took a bigger cut of sales that way. Now they have an incentive to say, “Please make sure to buy our game on Steam,” which would naturally drop more coins in Valve’s pocket. I’m wary of viewing Valve that way, if only because — more so than most companies in existence — they’re not in any way hurting for cash. But who knows? Steam is dominating the market right now, but they have competition. When you’re on top, all you can do is strengthen your stranglehold or, eventually, lose your grip and fall.
Comments
19 responses to “Valve Makes Big Changes To Steam Reviews, Angers Developers”
I can see why they’d want to remove weight from reviews from people who redeemed a key because of the possibility that the dev gave away copies in exchange for positive reviews, but it really doesn’t factor in the Kickstarter factor, where keys are offered in exchange for financial backing. In the event of successful Kickstarters this is effectively a preorder.
I don’t think this will last.
I could do without seeing Kickstarter and/or Early Access reviews, really, or at least some way of filtering them out. It’s always puzzling/frustrating when you read these reviews from people who’ve apparently invested hundreds of hours in a title that only launched on the store-front yesterday, telling you all their bullshit drama about the multi-months-in endgame or ‘how things used to be’, or the fact that their Kickstarter stretch goals/backer rewards weren’t met or some other irrelevant bullshit that doesn’t matter to me as someone who just wants to know what the game’s going to be like for the first handful to couple dozen hours, depending.
Does Steam not tell you when a review was left when the game was pre-release or in early-access? I honestly don’t know and can’t check but I feel like I’ve seen this as a flag in reviews when games have gone full release (IF they do, see: recent Ark DLC controversy).
I guess it would make sense to only see reviews from people who paid for the game. There is a valid argument that people who backed the Kickstarter have something of a vested interest in seeing the game succeed, if only so they feel vindicated.
Currently there’s a specific label at the top of a review which tells you if the review was submitted in Early Access (and I think those are weighted lower when viewing, after release?), and a note either at the top or bottom which tells you if the reviewer was playing a copy from a developer-generated key, rather than purchased through the Steam Store. I have no idea if it also includes 3rd-party distributor keys, such as from Humble Bundle or physical copies, or if it includes Kickstarter backer copies. The tag very specifically mentions that the code was provided by the developer – I’m not sure if this is an option that people can select when writing a review, using some magical code or tags, or if it’s an automatic process which flags specific keys which are generated on request by the developer, presumably for use in promotions and advertising.
Steam and paid mods: Surprise announcement, utterly broken implementation. Rightly derided, at the time, but there was a pocket of users that gave Valve the benefit of the doubt and said to not be too hasty with criticism.
Steam and refunds: Surprise announcement, utterly broken implementation. Rightly derided at the time, but there was a pocket of users that gave Valve the benefit of the doubt and said to not not be too hasty with criticism.
As the greater minds than mine above state, Valve’s tendency to carpet bomb its way to finding a solution to what it believes to be a problem usually ends up with it taking on board feedback from the user-base. Whether Valve acts (if at all) on what we offer, is another story.
Perhaps a poll to ask the reader-base here to gauge how often Aussies use keys?
I literally bought a boxed PC game this week. Steam seems to want to categorise that as a key from a reseller? Am I reading that right?
It’s not a game that’s redeemable through Steam anyway, but I do wonder.
Wouldnt this also remove the reviews of some game critics, curators and streamers who get review copies ???
That said there are some who game the system, moderating is hard especially when you dont hire community staff and get bots to do it. 😛
Shit man, no respect for the customers here either. It doesn’t matter where I get the key from, a Steam key is tied to the Steam platform and you can’t even play the game without using it. Why should these users be prevented from reviewing a game they legitimately purchased?
My understanding is that the developer can generate new keys for their game at little or no cost. The allegations seem to be that some developers are giving away keys in exchange for good reviews, and some might even be redeeming keys to sock puppet accounts that post positive reviews.
I understand that, my concern is that just like any blanket fix (DRM for example) it unfairly punishes those who are legit customers.
Simple factor that ppls are not taking into account.
Steam is becoming as bad and greedy as Sony and Microsoft.
They truly need to pull thier heads in.
One word to the Dev’s: GOG
Well its not like Steam Reviews really matter to anyone these days anyway. Well beyond acting as a cesspool for anyone who feels the need to one-up each other for the best insults possible or those who use it is a platform of protest for whatever crime its devs made along the way, instead of you know, reviewing the game itself (see the way that Witcher 3 reviews were tanked on launch because the devs tried to launch the game in its best possible light, which meant optimising lead to a graphics downgrade.
Up to that point Steam reviews were stating to really mean nothing, now they dont even remotely matter in my gaming decisions or online surfing.
Enh, I always find Steam reviews useful.
What people don’t say is often just as important as what they do say. And how they choose to say it. And how many of them are saying it.
Think of it like a survey on whether the earth is flat or not. Just because a large number of people respond that the earth is flat doesn’t mean that it is… but it tells you an awful lot about education in the area surveyed, or how much people in that area like to have a laugh.
Steam reviews are similarly useful.
I’d say a better approach to stop devs flooding reviews with fake accounts would be to require a minimum level of activity on the account before letting the review count. Account has to have X amount of games etc, would help wheedle out the small time devs with one game creating fake reviews which seems to be the goal.
This is gonna hurt Humble Bundle a bit too.
Really? I don’t think I’ve ever made purchasing decisions on whether my review will be included in some aggregate score should I decide to write a review down the road.
It’s up to the individual user whether or not to filter out non-Steam purchases. I can see why some devs are upset, but ultimately it’s the purchaser who is deciding whether to trust the external key buyers. I do wonder how this will impact handling of reviews of boxed games which require a steam registration.
One option is they could follow what they’re doing with “recent” reviews and include internal/external purchasers separately – but I can see where this might confuse some buyers.
Oh, so we can opt-in to view 3rd-party-‘purchased’ reviews? Well that’s interesting.
As much as Early Access/Kickstarter-based reviews shit me for their unique flavour (all too often not focused on the game and more on its politics/dev-relations/kickstarter promises, etc), every bit of data provides useful information. Even incorrect data (see points above).
Thanks for the tip. I might go turn those back on.
I assumed this was so from the screenshot shown at the top of the article, but it looks like these settings can only be viewed when looking at a particular game – you can’t change the default setting.
If you click on the review score you get a panel that allows you to change how review scores are integrated and change the settings for that game. I haven’t found any way to change the way the defaults are handled.
So yes, you can dig deeper into the review scores, but at the moment there doesn’t seem to be any way of handling how ALL games are filtered.
This is absolutely ridiculous. I have several hundred games in my library, all legally purchased, but only a handful of them came directly from their store.