So, Sony's sci-fi flick Passengers didn't exactly set the box office on fire, but at least it has some neat special effects. If you've seen the movie, you may have noticed something a bit different about the ship's touch-based user interfaces — rather than being pre-made clips, they were built in the game engine Unity and designed to actually work.
The usual practice is to add user interfaces in post-production or, if you want to do something more interactive, use controllable videos. However, with the accessibility and availability of tablets with high-definition displays and hardware powerful enough to run details visuals in real-time, VProFX's Vince Parker saw an opportunity to leverage a game engine — in this case Unity — to make a proper, usable interface for the actors:
While great actors, such as Chris Pratt, can make nearly anything they do believable even if all they’re interacting with is a green screen, it is my opinion that the more realistic and interactive we can make the technology they’re performing with on set, the easier it is for them to stay fully immersed in their character’s world.
And it works pretty much like you'd imagine. The "interface" is loaded onto tablets, which are installed into the set itself. You exchange the cost of post-production CG for a bunch of tablets and the creation of the UI.
Parker also points out that by maintaining a network connection with the devices, they can be controlled and updated in real time:
Whenever color, text, sizing, positioning or speed needed to be changed, it could be done quickly and remotely either by using a game controller or interactively on the tablet itself. This kept us from causing the kind of delays in the shooting schedule that would have resulted if we’d had to rip devices out of the walls every time a change was made.
If you'd like to read more, there's a detailed article by Parker on Local 695.