Benchmarking new hardware is part of the fun. But when that hardware happens to be a brand new CPU, which isn't from the same family of CPUs that have enjoyed years and years of optimisations and familiarity with developers and operating systems, things can get tricky.
While there's a lot to like about the Ryzen 7 series of octa-core CPUs, there's also been a quietly understood reality that Ryzen's full performance will take some time to bear. Developers have been making games on Intel hardware for a long time. Intel's CPUs have had a lot more optimisations and affinity with Windows 10 than Ryzen has.
And that affects the end results. In a post on their community site, AMD's Robert Hallock outlined a discovery with F1 2016 that saw the average FPS of a Ryzen 7 1800X increase (sometimes, quite noticeably).
Now you have to take the above graph with a slight grain of salt. AMD's trying to make the point that your system setup matters, which it does. But when you have effectively six different sets of settings - overclocked, not overclocked, different memory timings, HPET enabled, a different power profile for Windows 10 selected - it can be a little misleading.
But, that's not the curious part. Here's Hallock talking about F1 2016.
"Throughout this process we also discovered that F1 2016 generates a CPU topology map (hardware_settings_config.xml) when the game is installed. This file tells the game how many cores and threads the system’s processor supports," Hallock wrote. The problem: the topology map is stored in the Steam Cloud, not on the player's hard drive.
"This settings file is stored in the Steam Cloud and appears to get resynced on any PC that installs F1 2016 from the same Steam account. Therefore: if a user had a 4-core processor without SMT, then reused that same game install on a new AMD Ryzen™ PC, the game would re-sync with the cloud and believe the new system is also the same old quad core CPU."
After editing the file directly to force F1 2016 to treat the Ryzen machine as an 8-core, 16-thread system (instead of the game thinking it was a 16-core CPU, courtesy of the freshly created topology map), the average FPS (1080p, High preset, clear weather, no anti-aliasing) jumped by almost 3%.
The process shows why you should always test things on a fresh install, if possible. And it raises a real problem: how many other games automatically build a topology map and store that information in the Steam Cloud - rather than on the users' hard drive - and what other games and benchmarks are affected?
F1 2016 isn't the only title that builds a profile of a computer's CPU. A quick search of my local settings folder found that every recent Codemasters game has an XML configuration file with settings for the amount of cores, hyperthreading and settings for that computer.
Shots of the XML files from F1 2016 and DiRT 3
There were also CPU settings in XML files for other games - TERA, Dragon Age: Origins, Remember Me, as well as Mass Effect 2 and 3 - that specified the amount of cores the CPU had. It's not known whether those settings were downloaded from Steam (or Origin, in ME3's case), whether the information is overridden if the game detects a change in hardware (or whether graphics/profile settings are reset) and whether changing that setting would have an impact on performance for Ryzen CPUs.
But at a minimum, AMD has shown that you can get a performance bump from Codemasters games (DiRT Rally and the F1 series; I don't think anyone is really anticipating any FPS issues with the upcoming Micro Machines). I've asked their local team whether AMD is aware of other games that store CPU settings in the cloud, and whether those settings are correctly identifying the number of cores on Ryzen CPUs. They didn't respond at the time of writing, but if more info comes out I'll let you know.