Portal, Microsoft Flight Sim Touchscreen Controlled With Surface

Can touchscreen controls ever replace - or even closely match - control input from a gamepad or keyboard and mouse? This demonstration of Valve's Portal and Microsoft Flight Simulator played on Microsoft's experimental Surface platform offers hope that it might.

Building on Mark Micire's DREAM (Dynamically Resizing Ergonomic and Multitouch) Controller, University of Massachusetts Lowell Robotics Lab student Eric McCann hacked together an onscreen joystick driver that makes touch-based controls feasible for flight sims and first-person shooters. A perfect replacement? Not exactly. Even at double the playback speed, it's clear that Surface controls could easily be outperformed by more familiar interfaces.

But hardly shabby for a week's worth of adapting the impressive DREAM Controller's touchscreen-based interface. Cooler still is that the DREAM interface dynamically resizes itself to a user's hands and can be moved anywhere on the Surface's screen.

The UMass Lowell Robotics Lab folks are teasing "more fun to come" that's likely StarCraft-related. Can't wait to see what they come up with next.

Joystick Emulation using the DREAM Controller on the Microsoft Surface [YouTube - thanks Justin!]


Comments

    The flight sim demo was pretty encouraging. This surface won't be a replacement for first person mouse movement.

    Essentially first person games allow you to control where the protagonist is looking. This movement needs to be speed sensitive and accurate. No 'touch' input will be able to duplicate what the mouse is currently capable of.

    Move your mouse cursor from one side of the screen to the other as fast as you can, then back again.

    Put your finger ON the screen and move it from one edge to the other and back again.

    Which was faster? Which one fatigued you more?

    This is only exacerbated when you try to stop the cursor on a specific point.

      I don't think it's a very good control for flight-sims, seeing as you can currently fly flight-sims with actual flight controls. But for FPS, it will be an improvement of "click on head" gameplay: you control your view with a scaled interface, and just touch enemies' heads with your finger on a 1:1 scale.

        I'll qualify that, the tech would be neat for aircraft panel operations.

    As a video game designer, I really don't see this method as a possible replacement for the external, solid controller. I am happy people are experiment and, as always, I'm sure some nice results and/or other applications will arise from this type of prototyping.

    However, I don't think that it's ideal, from a design standpoint, and as an end user, to have so much of the screen real estate occupied with the control scheme. Part of the Player's vision is also impaired by their hands and arms extending into their cone of vision - this is clumsy and undesireable as it also means that the Player needs to be extremely close to the screen. This would also present more strain on the human eye.

    You also lose the sense of tactility now given by rumble features and also the frction oriented casing of analogue sticks and the actual controller casing themselves. Controllers also have a certain amount of weight associated with them, adding even more tactility for games such as FPS.

    While I do support the innovation and eventual removal of artificial interaction by way of 'second hand' means, I don't think interacting directly with the screen, for all video games, will work. We need to tamper with the human brain (non destructively and non invasively of course hehe) if we ever wish to truly surpass the use of conventional controllers to the fullest and most immersive extent.

    Nice video, though... and I would love to play something akin to an RTS with a surface computer - however I do feel it is quite limited with the general presentation and gameplay of video games.

    Wow, that's a *brilliant* idea for gaming!

    Especially for titles where you don't want to know what's happening on the screen, and you won't... BECAUSE YOUR BLOODY HANDS ARE NOW OBSCURING THE ACTION.

    #fail

      What I find weird though is that they're demonstrating a virtual controller on a screen. Why do we need virtual when physical is already working? Physical can provide physical feedback, it can be ergonomic and comfortable to use (which I'm sure this touchscreen alternative ISN'T)... so why directly emulate it? It all kind of reminds me of the jump from CD players with big speakers and nice amps to crappy microsystems running super compressed MP3s. The microsystem sounded terrible in comparison to a proper stero... but it still became the popular choice because it was new and more convenient.

      My prediction: We're going to end up with more gesture and touch screen alternatives with their own platform specific games, while traditional controllers will become a less popular method of playing games. There will always be the purists who find ways of making games work on controllers... just like there are valve purists out there right now building their own stereos, buying massive speakers and playing flac files or old records.

      So True almost half the screen is covered by your hands and virtual buttons.

      Yeah perhaps all Real planes should be controled like this, hmmmm NO.
      Why not just use a flight stick like THE REAL THING?

      This is technology looking for an application rather than a need to drive the technology.

    and here I thought it was a cross between portal and MS Flight sim - which would be awesome btw, and you'd never crash if you were a decent shot.

    If anyone's used the on-screen controls that a lot of iphone games rely on, you'll know how much having your fingers all over the screen sucks.

    Cool concept though. Put it on one of those see-through, double-sided TVs I saw at Big Boy's Toys years ago, and you're a step closer to Minority Report!

    Tabletop games have seen a massive resurgence over the last couple of years, and this surface is perfect for it.

    Along with anything even remotely turn based.

    Civilization on this thing would be mad fun. Wouldn't need to rotate desk chairs when playing hot seat either.

    Not to mention the plethora of board games that would easily be portable to such a device.

    Operation would be interesting on this.

    Make it cost effective enough and this thing will be what an iPad should of been.

Join the discussion!