Steam Spy Is Back, But Not As Accurate As Before

Steam Spy Is Back, But Not As Accurate As Before

Steam Spy, the PC gaming sales-tracking mainstay loved by some developers and loathed by others, was recently presumed dead by its creator, Sergey Galyonkin, after an update to Steam’s privacy defaults. It is now coming back, but it will be different.

The old Steam Spy estimated a PC game’s sales by sweeping publicly viewable Steam profiles and taking stock of their game collections. That technique was blocked in early April when Steam’s architects at Valve made it so that users’ profiles no longer automatically show which games players own. The new version of Steam Spy will get around that by embracing machine learning to predict sales based on “coincidental data”.

In a blog post, Galyonkin explained that he’s in the process of switching Steam Spy over to a model rooted in machine learning. He confessed, however, that despite a wealth of data to feed his algorithm, it’s “not very accurate” at this point.

“I have the data for around 70 games from different developers, and for 90% of them, the new Steam Spy is within 10% margin of error,” he wrote, pointing out that the site’s estimate for Frostpunk (252,000 units sold) was basically on the money (250,000 units sold).

“But I also saw some crazy outliers, where the difference between the estimates and the real data could be fivefold.”

If there’s a chance the data could be that inaccurate, why bring Steam Spy back at all? Galyonkin credited his decision to “over 200” messages from developers explaining how Steam Spy impacted their operations.

He mentioned an indie company from Berlin that got government funding for its game with the help of Steam Spy’s data, and though he didn’t name the developer in the post, he told Kotaku that it was inbetweengames, creator of noir tactical game All Walls Must Fall.

Praise for Steam Spy has always been buffeted by wariness. Earlier this month, many developers were relieved to see it seemingly go, because it empowered armchair analysts (and some harassers) with data they perceived to be accurate – even when it wasn’t. In theory, a new, less accurate model could exacerbate that problem.

Galyonkin, though, still thinks Steam Spy is worth bringing back from the brink.

“Armchair analysts do not need Steam Spy data,” he said in an email. “They’re fine with whatever is available: [Concurrent players], the number of reviews, even a user score.”

He noted that, in his opinion, the most important number for most players is concurrent users in a given game, because that helps them decide if multiplayer games are worth purchasing. Valve, he added, already makes that number public.

As for why developers can still benefit from Steam Spy’s reanimated bones, Galyonkin claimed that they’re already happy to shell out serious cash for services that he thinks are even less accurate.

“Relatively low accuracy doesn’t prevent developers from paying $30,000/year for [an] AppAnnie subscription,” he said, referring to a popular app analytics service, “because having an estimate is usually better when making decisions than having no data at all. Analysts can work with data when they know it’s not entirely accurate as long as they are aware of the limitations and caveats.”

Galyonkin is still fine-tuning the new algorithm, and many of Steam Spy’s old features aren’t even back online yet. Country stats, for example, are off the table right now. Things are early, in other words, and there are still options on the table – some of which involve collaboration with other interested parties.

“I have some offers from people with more recent knowledge in the field of machine learning,” he said. “I hope they will help me improve the algorithm.”

The Cheapest NBN 1000 Plans

Looking to bump up your internet connection and save a few bucks? Here are the cheapest plans available.

At Kotaku, we independently select and write about stuff we love and think you'll like too. We have affiliate and advertising partnerships, which means we may collect a share of sales or other compensation from the links on this page. BTW – prices are accurate and items in stock at the time of posting.