Benchmarking games has come a long way since the days of a simple FRAPS framerate comparison. This is partly because we have a better understand of what makes a game feel "bad" when the hardware can't keep up, but also because the hardware itself has changed. The rise of VR headsets is the most significant shift in the way we consume games in recent times, but testing methodology and software is still playing catch-up.
To help out, NVIDIA has released a new version of its Frame Capture Analysis Tool (FCAT), designed specifically for VR demands.
There's more going on behind the scenes when you don a virtual reality headset -- there's a software middle man that has a perceptible effect on performance:
VR software for delivering frames can be divided into two parts: the VR Game and the VR Runtime ... The job of the Runtime becomes significantly more complex if the time to generate a frame exceeds the refresh interval. In that case, the total elapsed time for the combined VR Game and VR Runtime is too long, and the frame will not be ready to display at the beginning of the next scan.
Thanks to vertical synchronisation, this hiccup results in the "redisplay [of] the prior rendered frame from the Runtime". This isn't great on a normal screen, but it can be headache-inducing -- literally -- with VR.
The company's goal with FCAT VR is to provide a baseline tool for all interested parties, be they reviewers or manufacturers themselves, to diagnose VR-specific performance issues.
According to NVIDIA, the industry has lacked a consolidated application for this purpose:
...Virtual Reality testing relied on general benchmarking tools, synthetic tests, and hacked-together solutions, which failed to reveal the true performance of GPUs in VR games. With FCAT VR, we read performance data from NVIDIA driver stats, Event Tracing for Windows (ETW) events for Oculus Rift, and SteamVR’s performance API data for HTC Vive to generate precise VR performance data on all GPUs.
While very technical, the program is available for download by anyone from NVIDIA's website, though you'll want to read the accompanying guide for diving in.