Steam Review Scores No Longer Count People Who Didn't Pay

Steam reviews are changing again. This time, it's a subtle thing that could have big reverberations. If you got a game for free — say, as a gift, during a free weekend, or what have you — your review no longer counts toward that game's overall score.

Valve, uh, didn't really explain their rationale in-depth, but here's the standout bit:

In September, we made some adjustments to how the review score was calculated for each product. You can read about those changes and the reasoning behind it here. We're continuing with a few more changes in this direction to improve the relevance of the score by better reflecting the sentiment expressed by invested, paying customers.

With the changes we are making now, the review score (shown at the top of store pages and in various places throughout the store such as search results) will no longer include reviews by users that received the game for free, such as via a gift, or during a free weekend. Reviews can still be written by customers that obtained the game in any of these ways, but the review will not count toward the overall review score.

So basically, Valve thinks people who pay will be more "invested" in their games, and therefore more honest. There may or may not be some false equivalence going on there (paying for something doesn't necessarily make somebody care about it, and investment does not necessarily lead to honesty; also, if you're going for accuracy, removing certain kinds of users just creates a more skewed sample), but that's the reasoning, anyway.

A lot of Steam users are up in arms over this decision, given that not everybody can afford to directly purchase a ton of games, and they still want their reviews to count. I can understand the sentiment. The last thing we need is yet another ecosystem in which money is the sole arbiter of influence, small though said influence might be. If I had to guess, though, Valve did this in part to curtail sketchy promotions and gifts from developers that might lead to disingenuous or inflated scores. Are those things a big enough problem to justify leaving legit users high and dry? Valve seems to think so.

What do you think? Do you receive a lot of Steam games for free? Does this change directly affect you?


Comments

    So basically, Valve thinks people who pay will be more "invested" in their games, and therefore more honest.

    I don't think so. It's more likely Valve wants to minimise fake reviews (eg. by people who have never played the game), because it's a lot easier to mob reviews for a game when you can get the game for free.

      basically, Valve thinks people who pay will be more "invested" in their games

      Ah, so they're going for a different phenomenon instead: sunk cost fallacy. See also; Star Citizen.

        Well no. That quote was from the article, and my post was saying why I don't think it's true.

          Yeah, I got that. I was also quoting that part of the article and making a different point about how it doesn't actually guarantee "accurate" reviews.

            Sure, I understand what you mean. Thing is, there are no guarantees, especially with subjective things like reviews. The best you can do is try to promote honesty and filter out dishonesty as much as possible. Developers giving free copies to a bunch of people with the understanding they'd leave positive reviews is more on the dishonest side, and also something Valve has had to deal with a fair bit recently (eg. cash for comment scandals reported here recently). In context this move seems to be targeting that kind of behaviour.

    An even better idea is too scrap them all together. They are nothing but a cesspool. If I want reviews for games will watch my favourite youtubers just play the thing for a bit. The last place I go for purchasing advice is steam.

      Thats more difficult when you're playing pretty niche games like the mass of VR titles coming out. Those Steam reviews are often the only way I'll find out what glaring flaws a game might have ahead of time, since online reviews on youtube take way longer to come out, if at all.

      Whilst taken individually, any individual review is essentially worthless short of watching someone play the entire game several times, what I do is look only at the negative reviews and look for trends in what people are saying, rather than looking at any one review and giving it credence over the others. This way, you are more likely to get a better understanding of what is actually wrong with the game. The game devs do a good enough job of extolling their own virtues, so that side is already covered.

      So if you only watch your favourite youtubers for reviews. Why does the steam review system matter to you then?

      Steam reviews are quite accurate. If there is a trend towards negative or positive reviews its for a reason.

      What you are exhibiting is confirmation bias. You only want reviews than confrom with your views. If people hate a game you like they are clearly making up their reviews.

      Your comments on NMS are evident of that.

      The steam reviews system is an important part of steam.

    I'm fine with it. Yes, it's a dud that people who get games for free can't review them (well, they can't on steam anyway). That being said, the new-releases section on steam is plagued by asset flipping, shovelware rubbish lately, and this is the main avenue that shit games like that get to the front page.

    About 1/4 of my steam library (280games) is from 'gifts' because group buys are cheaper

    Couldn't they just limit the users to only have access to reviewing a game after x amount of hours play?

      Though I've read reviews for a game where most people had only played for a couple of hours. They were negative reviews about how broken the PC port were and they were true.

      How many hours is enough to write a review varies too much between games for there to be a one-size-fits-all solution. Something like DLC Quest or McPixel could be finished in less than an hour, while anything less than five hours probably isn't reliable for Witcher 3.

      They do show how many hours the reviewer has played though, which I think is a great idea.

        Very true, both you and @g-man make excellent points about why that wouldn't be such a viable idea.

        The broken ports problem might be able to be fixed though, if Steam had a pre-approval testing process before the game is released to ensure it's actually working without being broken, but that wouldn't be fair on Steam because the game companies should really make sure their product works for themselves.

        Also yeah it is a great thing that they show how many hours play someone has, perhaps they could sort reviews based on how much play time someone has had as a ratio compared to what others have played, so if it's a game where most people played for 2 hours, and someone plays only for 30 minutes, their review wouldn't count towards the overall score as much as someone who has played for 2+ hours.

        I'm not sure how viable that is, but it sounds like a good idea to me. Thoughts?

    There may or may not be some false equivalence going on there (paying for something doesn't necessarily make somebody care about it, and investment does not necessarily lead to honesty...There's actually a little truth in the thinking that people who receive a game for free review it differently to those who paid. Studies have shown (yeah I know, citations or get out) people tend to review a bad game more positively if they received it for free or paid very little for it. This is because in their mind their enjoyment of the game balances with their investment.

    By the same token though, people who paid a lot for a bad game are going to review it more harshly because that value balance isn't there so Valve should, to be completely unbiased, be removing those reviews as well. Basically, the moral of the story is that people are inherently bad at reviewing things and all review scores should be removed.

      Certainly a lot of the No Mans Sky negative reviews (and several positive ones too) mentioned the price as unreasonable, so the amount paid certainly seems to influence people's opinions.

      Then again, is it really a bad thing to include that? "It's not worth $60, wait for a $20 sale" seems like a pretty reasonable thing to factor considering that's what most people would have to pay. It's not like reviews aren't subjective anyway so including a subjective value assessment seems par for the course.

    Either way i dont care about this. If it stops people who are paid by the devs to review it all the better. Without the steam reviews system companies like digitial homicide would still exists and be able to peddle crap.

    Last edited 11/03/17 11:30 pm

    It's not like the reviews disappear, from my reading, so yeah, your review counts... just not toward the aggregate 'overall positive/negative' thing. Which is important, because that's what influences whether it turns up on peoples' screens as featured.

    It's very important to eliminate sources of developer/hate-mob uninvested bias for that aggregate, and I think losing a tiny element of your 'voice' because you got it as a gift is a small price to pay for the overwhelming benefit of hamstringing the manipulative assholes out there.

Join the discussion!

Trending Stories Right Now