A couple of days ago, I felt like I was standing in a shark tank. Some time before that, Sony's head of PlayStation game development, Shuhei Yoshida, fed a dinosaur and the company's top PlayStation researcher, Richard Marks, stood next to the Mars rover — all in virtual reality, of course. With any luck, you will eventually be able to as well.
We're getting awfully close to a moment when virtual reality is going to be relatively easy for all gamers to experience. That moment is coming thanks, first, to the pioneering Oculus Rift and also to the Project Morpheus, the VR headset from Sony that debuted this week at Game Developers Conference in San Francisco.
Oculus has been selling increasingly good versions of the Rift, with a commercial release on the horizon. And now Sony wants to make VR the latest gaming peripheral to stack up with the likes of the Wii Fit balance board or the Microsoft Kinect. In fact, Yoshida told me earlier this week that he believes that VR has "bigger potential" than either of those multi-million-selling peripherals.
I've tried Sony's new VR and had a good conversation with Yoshida and Marks about it. More of that in a bit, but I write about VR with the understanding that it's one of those things, like the 3D effect on Nintendo's 3DS, that you need to see yourself to really appreciate.
To help you out some, here are the basics of what it's like: in all of these VR experiences, you put on a headset that blocks out any view of the outside world, immersing your field of view with three-dimensional graphics that stretch out as far as a computer can draw them. Your eyes only see a virtual world. If you have headphones on, you only hear that world, too. Instead of seeing a game world in front of you on a TV, you feel like you're inside of it. That's what it looks like. If you tilt your head and look up, you see the virtual sky or a virtual ceiling, perhaps. If you look down, you might see your virtual body and the ground. Wherever you turn your head, that's what you see. It's as if you've been transported to another place. It's how I was able to stand in Sony's booth at GDC but, with the Morpheus on, feel like I was standing inside a shark tank.
Here's our own Kirk Hamilton in the virtual shark tank.
If just one company was pushing VR — or if the experiences that people like me have experienced were bad — then you could dismiss all this as hype. You could write it off as another of those little-requested tech innovations that seem to be foisted on consumers in the interest of finding something new for people to buy. Remember 3D TVs?
VR, I'm happy to report, is impressive. It's cool and worth wanting.
It's also not quite ready for everyone to play, Sony's Yoshida and Marks told me in an interview this week, but it's getting close.
"What we have today is good as a devkit for developers to be working on," Yoshida said after I tried some Morpheus demos, "but we don't think that's good as a consumer product as yet."
Don't Get Sick
Sony has been working on the physical tech leading to the Morpheus headset since 2010, Marks and Yoshida said. They have been improving the displays for both eyes, calibrating movement sensors and working toward a design for mounting the thing comfortably on a user's head. Along the way, they have been running into a variety of presumably solvable problems, making improvements so that their virtual reality experiments can become a real thing for everyone to play.
"The biggest thing is the latency," Yoshida said, zeroing in. "It's currently about 40 milliseconds from when you move to what you see changes. We know we have to go lower. That's one big thing we want to continue to improve." That 40-millisecond number is something of an acceptable minimum, according to some experts, though lower is better and is preferred by both Sony and rival Oculus. With more lag, you find that what you see doesn't quite line up with how you're turning your head. Your brain gets confused. You feel uncomfortable and even nauseous.
Yoshida, Marks and members of Sony's research teams have been learning about that VR-induced nausea the hard way as they test virtual reality experiments being designed around the world by various PlayStation teams.
"We have a lot of our internal dev teams trying to connect their game to some level of VR," Marks said. "Some games don't work at all, because they're not designed for that. Some games worked surprisingly well. Standing in a shark cage looking around? That worked well. Sitting in the cockpit of a spaceship, dogfighting with other fighters? That worked well, too. Getting in a car? Mixed results.
"We had a demo for a driving game with the tech, and that was really great when I drove slowly," Yoshida said. "I could view the scenery. It's no longer racing. It's driving, driving through scenery. But when I put on the gas and started driving [fast], it was too much." He said that "many people" who tried it felt ill.
"Mostly, the biggest learning is what makes people sick," Yoshida said. "That's something we definitely want to eliminate." Latency, he reiterated, "is killer." Get that number down and the comfort of VR goes up.
Marks reports that there's been great progress there. "It's much better now, because the hardware is way better," he said. "But, two years ago, it was a very difficult thing." It was also difficult because Marks' team would intentionally mess with core settings, testing different widths between the displays for each eye, for example, to see what feels better or worse. As they have gotten closer to the perfect settings, they have experienced less nausea. "Once we had all the settings and the lenses well-tuned we don't have that so much any more."
For Gamers First
As cool as VR is, using it is a bit goofy. You basically strap a brick to your face and blot out the world. Given that there are millions of people in the world who are still intimidated by the prospect of picking up a game controller, VR could be a tough sell. There's a community of people, however, who Sony thinks are less worried about how cool they look while using it. That's me and probably you: gamers.
"We are focused on the gamers on PS4 because those people will be excited to try and not be as design-conscious as the mass market," Yoshida said, "but we are really hoping to reach mass market in the future with broader applications."
For the moment, Sony has tapped its internal studios to make both game and non-game applications, assuming that games will be exciting but that some of the non-game stuff might feel even more wonderful.
Sony's London studio, which created mini-games for Marks' team's Eyetoy camera many years ago, made the shark cage VR experience that I tried. Yoshida said that studio is "experimenting and prototyping many kinds of things." As Yoshida noted, there was a driving/racing experiment under way as well though it's not clear if that's still being worked on. While Yoshida and Marks didn't mention the genre, first-person shooters are an obvious one to dabble in, too.
I asked about non-first-person experiences, and Marks mentioned that "there are studios I know that are looking at third-person over-the-shoulder games in VR... there are some interesting things with that. It's not exactly obvious how it would work."
"Remember Mario 64," Yoshida said, referring to Nintendo's pioneering 1996 game that moved Super Mario from the world of side-scrollers to a three-dimensional polygonal world. "The camera was actually a character. So you [would] become that [in a third-person VR game]." Needless to say, the odds of us getting a Super Mario 64 port for Project Morpheus are about as low as the odds that this sentence will end with a question mark. Sorry.
VR might also work for top-down strategy games — so-called god games that are played from a view from the heavens. Both Yoshida and Marks mentioned a non-Morpheus VR demo from PC studio Valve that involved having more of a god-game-like overhead view of a playing field. That's not a Sony project but it seems like it might be a source of inspiration. "They have a demo like a diorama," Marks said, adding that within that diorama there can be guys playing soccer or fighting or whatever. You can look around it and through it.
"Nintendogs might be interesting," Yoshida riffed, referencing Nintendo's pet simulator. "Or something like Playroom," he added, referring to the augmented-reality play space for PlayStation 4.
"Or The Sims," Marks suggested, laughing.
All of this brainstorming was infectious. VR is that tantalising. I started rambling about how good a batch of VR sports games would feel. Anything familiar to gamers could suddenly feel magical again.
While Yoshida didn't talk about any specific games being made for Morpheus he did at least articulate Sony's Morpheus gaming philosophy. He said they're "totally anti-port." He doesn't want his teams making games that can be played with or without the headset. He doesn't want old games brought back with headset controls. He wants games built for Morpheus or for non-Morpheus games to have a Morpheus-only side mode..
3D Controllers For 3D Displays
The last time Sony tried to push a new PlayStation peripheral, it didn't set the world on fire. It wasn't a disaster, but it wasn't a phenomenon either. Not only could things be different this time, but that last peripheral, the Move, might get some new life.
The 2010 PlayStation Move was technologically more impressive than the 2006 Wii Remote. And like the Morpheus, its development was overseen by Marks. Sony made games for it but also allowed developers to treat it as an optional control scheme.
The Move just wasn't a hit, maybe because it cost too much, maybe because motion control gaming's fad moment had passed, maybe because peripherals have always been a hard sell to people who've already spent hundreds on a console.
All that said, the Move was a great controller that might be about to get a second chance thanks to Morpheus. The wand-shaped Move is capable of allowing a PlayStation 3 (or 4) and a PlayStation camera to track arm movement and wrist rotation and also the Move's position in a room: forward, back, high, low, etc. In other words, the Move is detectable within three dimensions of real space. That turned out to be a problem, of sorts, for which Morpheus is an unexpectedly exciting solution.
"We have a 3D input device, the Move," Marks said, as he recalled the experience of playing a Move-based tennis game in front of a conventional TV, "but you have a 2D display to see what you're doing. The feedback you're getting back is 2D for all of your 3D actions. You can do things with shadows [in the game] to help, but it's really hard, when you have a 2D display, to know with the tennis racket where you are in space and stuff. It's actually hard to hit a ball because the feedback is 2D."
Enter Morpheus: "Put on the 3D glasses and all of a sudden the Move is a way different controller," Marks said. "
It actually matches the rest of everything going on. I believe, actually, that in a way we've been working on interfaces and input devices so much over the last 10 years, because graphics had gotten ahead of input. Now we actually made an input device that was kind of a little bit ahead of display. And now display is here to catch up."
There was an example of what Marks was talking about at GDC this week. One of Sony's VR demos was set in a castle and required a Move in each of the player's hands. In the virtual reality view of things, the Moves are displayed as hands sheathed in gauntlets. As you swing the Moves in real space, the idea is, you see your virtual hands move in sync. You can pick up a sword and chop at a target dummy. That demo wasn't working when I was at Sony's booth, but we're hoping to get some impressions by week's end. Word from Polygon's Ben Kuchera is that there are some sync issues, but that it feels pretty good.
Even if you've used VR or can imagine it, there are things about it that might catch you by surprise. Marks and Yoshida marvelled at two qualities of VR that I hadn't noticed but that make the enterprise all the more interesting. One involved multiplayer. The other involved a sense of scale.
The multiplayer thing came from an exchange between my colleague Kirk Hamilton and the two Sony guys. Kirk was asking about whether using VR might make gaming feel more isolated and less social. Marks wasn't so sure. He shared an observation from having used VR in a multiplayer environment in which each user has a headset that is tracking their head movements and, to a limited extent, how they move their body.
"We have done some tests," he said. "You can almost tell who is who just from their body language, the tracking. You add the hands into that and you get a very rich kind of interaction with somebody when you're in a multiplayer kind of game."
The other non-obvious idea was about scale, and that brings us back to the dinosaur that Yoshida was feeding and the Mars rover that Marks stood next to. Both of those VR experiences seemed designed to let the user feel like they were doing some virtual tourism.
First the dinosaurs. Yoshida explained that one of Sony's European studios had run some tests by converting an augmented-reality PS3 game called Walking With Dinosaurs for Project Morpheus. The game includes a bit where you can reach out and seemingly feed the animals. "I was able to feed the dinosaur," Yoshida said. "The dinosaur was like this tall." He indicated the height of a person. And I said, 'I thought dinosaurs were much bigger,' and they said, 'No, no, no, that's the actual size.' I said, 'OK, the movies are not exactly right.'"
And Mars: "I was working on this demo," Marks said "and there was the Mars rover, which I've seen pictures of so many times. I even know the dimensions of the rover. It's the size of a Mini Cooper...You go into VR and it's next to you and it's, 'whoa, that is a real piece of hardware that is on Mars.' I know all this but somehow to be next to it is a different thing."
In both cases, the Sony men said, TVs and pictures hadn't done the things they'd seen justice. VR did. It gave them a sense of size and scale. They felt the presence of things they will probably never stand next to in real life.
Virtual Reality... Reality?
At previous GDCs, Sony has shown technology that it's never released. Even at this one, they're showing a way to control games with your eyes that seems unlikely to ever be a commercial PlayStation experience. The Morpheus is a real headset. Sony really is releasing devkits to developers so they can make VR projects for PS4. It's still not guaranteed, though, that Morpheus will ever actually come out.
Yoshida did clarify that there's no chance Morpheus will go on sale for gamers in 2014. But beyond that? Will gamers be able to use them on PS4s of their own? Yoshida chuckled. Tough question, he said.
I wasn't asking to trick him, wasn't trying to get him to spill a release date he wasn't prepared to offer. I just wanted to know how real this was.
He sounded determined and said, "we really, really, really, really want this to be a possible project."
Let's hope so.