Nvidia’s Broadcast tech is one of the coolest uses of AI in gaming, if only for its ability to remove stupidly loud keyboards from everybody’s microphone on Discord. It’s capable of much more, of course, and overnight Nvidia announced it’s being upgraded with more powerful, anti cat/dog filtering capabilities.
The AI model underneath Nvidia Broadcast has been updated in the latest patch to deal with a ton of annoying issues. While the original RTX Voice and Nvidia Broadcast apps were good at dealing with painfully loud sounds, like vacuum cleaners, it wasn’t so good at handling things like room echo.
That’s one of the biggest updates in Nvidia Broadcast 1.2, which is available now from the Nvidia website. But it’s not just echo that can be cut from your microphone or any other audio input: the updated AI model is now intelligent enough to remove dogs, cats (do cats make that much noise?) and insects, cicadas included.
Audio noise removal in general has been improved, but the AI model is now capable of better removing noise from webcams, too. “This is a complex challenge, one that will take some time to perfect — but the Video Noise Removal beta feature is a first step, helping to reduce video static to deliver a cleaner image,” Nvidia said.
The “auto frame” tool now has more flexibility. Previously, it would always shift the camera view to keep the user completely in the centre of the frame. Now, more allowance has been built in so that you can move your head slightly, and the AI auto frame will only shift your camera view whenever your head moves out of a certain zone (like when your head might be out of shot).
In a slightly more useful tool for video calls and streamers, the Broadcast app — which has been natively integrated into OBS, if you prefer to use it that way — can now stack effects on top of each other. Previously, you were only capable of choosing one effect (like the fake green screen or auto frame) to apply onto a camera source. Now, multiple effects can be applied, so you can automatically have the AI zoom and continuously track your face while blurring out the background, or removing it entirely. Or you could combine the video noise removal and background blur techniques, which can work really nice for regular streamers.
But other companies are now incorporating these tools directly. AverMedia, makers of some very good capture cards, has added Nvidia’s noise removal and virtual background effects into their software too.
If you’ve got any RTX 20 or 30 series card, a Titan RTX GPU or a Quadro RTX card, you’ll be able to use the full suite of the Nvidia Broadcast features by downloading the app here. Owners of 10 series GPUs can still get the benefit of the noise cancellation for microphones by downloading RTX Voice instead.
It’s worth noting that Nvidia didn’t provide any guidance on the performance impact of stacking effects, or if there were any performance optimisations when all of these tools are applied. Nvidia Broadcast can actually tank your FPS by quite a bit once the camera effects are applied, more so if the game you’re playing has ray-tracing enabled. And it’s worth noting your GPU usage will increase depending on the resolution of your webcam: if you’re running at 4K instead of 720p, the impact on your GPU will be much greater. (It’s usually worth it to lower your webcam’s resolution anyway: most webcams are either used in Google Hangouts, Zoom or Microsoft Teams, where you’re not getting the highest quality, or during a stream where many minimise the webcam to a smaller window, which reduces the benefits you’d get from higher resolution webcam footage.)
As before, these kinds of features are completely free for RTX graphics cards owners. It’s a massive upgrade for anyone with a crappy webcam, whether you’re gaming or just using the webcam for work calls. If you have a 20-series or 30-series GPU and want to give it a whirl, check out the app here.
Leave a Reply