Why The Apple Trackpad Might Be The Best Video Game Controller Ever Made

I was designing a user interface for someone else's social game the other day, and I was completely oblivious that I'd just recommended a feature that Just Would Not Do.

Usually, I'm pretty good at never recommending these features -- that's my Area Of Expertise: I recommend Features That Will Definitely Work and then suggest tweaks until they feel Good Enough For Me. I am a fairly critical person (single at thirty-two, I have not had a girlfriend in literally 10 years, I eat the exact same meal six times a day, I often will listen to the same song on repeat for an entire day), and the giving-in of my critical faculties usually manifests itself in what I like to call "supertolerance". My "joy" is finding things I can tolerate, "loving" them when I do find things I can tolerate, and being constantly mindful of how best to avoid the things I can't.

One of those things I can tolerate: Apple products. That's not to say I am a devout Apple fan-zombie. It's just that everything else is so.. ugly and clunky. When I saw what Vizio's Apple-aping personal computer lineup looks like, as a person who has read all of Steve Jobs's biography and is thus familiar with the sort of suffering he'd feel if he saw them -- I'd wish that on no one -- I honestly felt sort of glad for a couple of seconds that he was dead.

So: I am so good at tolerating Apple products that you could definitely say I prefer them, and I suppose you could say I like them.

So we were talking about a user interface.

"We could make that a swipe with two fingers -- " I started to say.

" -- swipe? What, are we using a trackpad, here?"

My face went red. Luckily, this was an audio-only Skype call. (Skype is number one with a bullet on my list of worst interfaces in essential software.)

"Oh. Oh... well, I was thinking -- for a trackpad interface, yeah, we could build that in."

"Not everybody has a Mac, you know."

"In a perfect world!" one guy chuckled (he owns two golf carts).

"And definitely not everybody has a MacBook or newer."

"And certainly not everybody has a Mac with one of those new multi-touch trackpads."

"That doesn't mean we could just put that in there... in addition to whatever other control we add."

"Let's not confuse people."

"They won't be confused if they don't know it exists."

The Macbook is where Apple introduced the two-finger swipe. I got a 17-inch MacBook Pro in 2006, and literally right up until Apple announced the redesigned MacBook Pros and the MacBook Air, with their wide, luxurious glass trackpads, I had Mac users express amazement whenever I'd use two fingers on my trackpad to scroll down a webpage or a document.

"How did you do that?"

"You just use two fingers."

"How did I not know this?"

The unspoken answer was that "Maybe you, unlike me, have some degree of tolerance for situations which require effort."

Though the two-finger swipe was a trackpad gesture I used no less than a thousand billion times per day, it never came up in casual conversations with women at parties or hobos on the bus. I especially never brought it up during interactive entertainment software user interface experience design documents, presentations or meetings. Even when the iPhone was finished being The Next Big Thing and was, in fact, The Current Big Thing, I never ventured a millimetre toward recommending software designers relegate an input to a motion involving two fingers of the same hand moving together in a straight line. One finger on each hand, sure -- man,let's just leave it at two-finger involvement maximum, and two-hand involvement minimum (they'll need to be holding the phone (another rule is to never expect or ask the user to set the phone on a hard surface (what if they're on a train or waiting at a bus stop?))).

Friendly as I myself was with the two-finger swipe, I considered it Definitely Not A Thing. And since I knew not everyone owned a MacBook Pro with a two-finger-enabled trackpad, the two-finger swipe certainly was Not A Thing I'd accidentally recommend in a meeting.

Then, two things happened:

First, social game company executives must have all read the same article in The Secret Wall Street Journal, because they started all individually demanding that the number of icons displayed on screen in their interfaces be cut at least in half.

Second, this year I received The Best Christmas Present Ever: it was a thirteen-inch MacBook Air. We'll ignore that it might have been a present from myself (so lonely), though I tell you what: it sure was Real Joy I felt when that box arrived at my parents' house in Indiana the week before Christmas. I never believed in Santa Claus as a kid, and I still don't, though I think I might have cracked the code re: surprising myself as much as I possibly can by creatively arranging a UPS delivery.

Say what you will about the Apple Computer Corporation: they know how to make you feel good about opening a box and booting up one of their products for the first time. I'd needed a new computer for a while, and here it was. Now I faced the hurdle I knew I'd have to face: every time I ever tweaked around with a newer MacBook Pro or MacBook Air in a Best Buy or Apple Store, my lips inadvertently screwed up as I tried to use the trackpad to manipulate the cursor. It was always too slow. Tap to click was always (inexplicably?) disabled. And the way it's all just one smooth piece now, with no tactile divider between the trackpad and the mouse button, didn't sit well with my old-habit-weary fingertips.

Within 45 seconds of starting up my impressive new electronic toy, I had enabled tap to click and increased the pointer speed to one notch away from maximum. Thinking I'd use this thing the exact way I used my old MacBook Pro (that is: all day, every day, for five years), I set "hot corners" -- the lower-left corner would show me my windows, and the lower-right would show me my desktop.

This is where I encountered the "trackpad gestures" configuration menu. "Huh," I thought. "I remember hearing something about this."

Apple is so proud of "Multi-Touch gestures" that they capitalise the "M" and "T" in "Multi-Touch", and feature these gestures right at the top of their "What's New in OS X Lion" index page.

Using gestures, I can scroll a browser window up and down with a two-finger swipe -- hey, I know that one already -- or I can look a word up in the dictionary by double-tapping it with three fingers. I can also "right-click" -- Apple calls it a "secondary click" -- by touching something with two fingers. Or I can --

"Son, what the hell is that?"

"It's a computer."

"It's an Apple computer."

"That's what it is."

My dad screwed up his face. My mum was looking over his shoulder and into the living room. She was eating a handful of little crunchy pretzels.

"You've got one of those iPhones too, don't you?"

"Yeah."

"Your brother's wife just got one of those," my mum said. She lowered her voice: "She's probably going to try to talk to you about it."

"She also got one of those things you got -- a Mack Book? A Mack Pro? A Pro Book?"

"A MacBook Pro?"

"That's it," my mum said.

"Boy, look at that. That thing's slick," my dad was saying.

"Whatever happened to that friend of yours -- you remember that friend of yours? He was your roommate in college."

"Yes."

"Yes, that nice little gay boy. Do you still keep in touch with him?"

"We're Facebook friends."

"He was nice. Remember, honey? He had an Apple computer."

My dad was saying, "I will always -- always prefer an IBM. Remember when your brother had to borrow that Apple Macintosh from school? Damn thing only had one button on the mouse. What the hell is with that?"

"It's not about how many buttons are on the mouse," I was about to say, when my dad peered over the lid of the display and eyed the finer details of my MacBook Air.

"Boy, that screen is bright. What in the hell -- it doesn't have any buttons on the mouse."

"It doesn't."

"You can't right-click, and you can't even left click. You can't even click."

"It's got a clicker," I was saying. "The whole thing is clickers -- "

He walked away, shaking his head. I thought it over: this is a guy who didn't professionally use a computer until he was already 40 years old -- at which point he'd started using one every day.

I remembered friends of mine, in Japan, lifelong Windows users, in the age before Wi-Fi was everywhere (like serial killers): "Can I check my email for a second on your computer?"

"Sure, man."

An instant later: "Whoa, what the hell did I do -- hey, nice desktop wallpaper."

"Sorry, the cursor speed is high."

"How do I get the internet back?"

"Here."

"OK, thanks. Whoa -- it's the desktop again."

"I've got hot corners enabled."

"I don't know what that means."

And they would go on not knowing what it meant, even after I explained, "It's set so that when you move the mouse to a corner, it shows the desktop, or all your open application windows," and they replied, "Oh, okay".

Christmas, 2011: I was a Multi-Touch gestures evangelist.

"Look at this." I showed my dad: "Use two fingers to scroll. Touch with three fingers to move something. Grab the file here; slide it here. One touch. Grab a window at the top: move it around. Spread two fingers to zoom out. Pinch two fingers to zoom in. Flick with three fingers and a thumb to show the desktop. Scoop with three fingers and a thumb to show all applications. Slide four fingers up to see all open desktops. Slide four fingers to the right and I have a calculator and movie showtimes. Slide to the left -- and I've got my desktop. Touch here and I can maximise this web browser. Now it's full-screen. Now look at this: swipe this way to go back to the desktop. See this photo? Pinch here, and twist -- and it rotates. Just like that. Slide three fingers down and -- hey there! -- I've got an overview of all my open apps, windows, and desktops at a glance."

My dad was silent. "Well, shit," he said.

"It's not a mouse with no buttons -- it's a mouse with a million buttons."

My dad was silent for a couple of seconds, and then said, "You should get a job at the Apple store."

I'm sure he meant that as a compliment, though it only made me sit there, sigh for a second, and realise that I am, in fact, not yet rich. Here I am with a MacBook Air and no car. Such is life. On top of that, I'm on a sheet-covered sofa (big brother's eldest daughter is allergic to cat hair) in Indiana during a sleety, frozen holiday, wondering every third second if I left a stove burner on back in Oakland, California, if my apartment exploded, if Pixar was damaged in the explosion -- and my little brother just got married the day before at a justice of the peace, he and his new wife have a five-month-old son, my big brother has the same three children as last year (only they've been slightly enlarged), and I have a fancy computer, fantastic hair, nice jeans, and no car. I have, at least, in the past week, learned to love Multi-Touch trackpad gestures.

My dad was silent for a couple of seconds, and then said, "You should get a job at the Apple store."

My old friend Doug Jones comes over after a shift at The Factory. Me and him are going to go to Perkins in his Subaru WRX and inhale some omelets. He'll talk about his kids and I'll talk about user interfaces. Before we leave the house, I show him some trackpad gestures. Unlike my dad, Doug Jones is not particularly Mac-averse. He's opinionless. He's my truest Gamer Friend From Way Back. He's the only adult I think I know who actually plays video games instead of obsessively reading reviews and previews and deciding that nothing is worth it anymore. When my company has enough money to secure his and his family's future, I want to hire him and pay him six figures just to sit at a desk next to mine and tell me when I'm being too much of a jerk.

"I seriously played every game this year," I tell him. "I have them all in a stack on my desk. And I have to tell you -- this right here has amused me so stupidly."

I get on the internet. I spend 30 seconds looking for a blog post longer than two browser window-lengths. I can't find one. Aha -- I decide to navigate to one of my articles.

"Look at this," I say. I whip two fingers upward on the trackpad. The browser window accelerates, careens downward, and then slows to a gradual stop. "Look at this," I say, again. I whip my fingers downward. The text accelerates upward. I drop my fingers lightly onto the trackpad. "Look at that -- I can stop it right there."

"Oh... kay."

"Look at it -- look." I do a slow, deliberate sweep of the height of the trackpad, from top to bottom. "At a lower speed -- see this? At a lower speed, one swipe equals exactly one scroll."

"Alright?"

"Now, if I really whip the thing, it cranks up and jets down there. It's weighted, like an old stereo knob. I lightly touch it to stop it. It's fantastic."

"OK."

"Think about it!" I said. "Now look at this -- here. You try it. Swipe with two fingers."

He gave it a shot.

"Alright."

"Now swipe with one finger. Yeah, see that -- it just moves the cursor. Now put down three fingers. Put them anywhere. Now move those three fingers around. See? You're highlighting text."

"OK."

"Now slash four fingers downward. See that? It opens up a dashboard of windows. Slash them back up. Now slash to the right or the left."

"Huh. Can it do diagonals?"

"I'll -- I'll look into that."

"It'd be neat if it could do diagonals."

(It can.)

"Seriously, really just whip that scroll bar around. Watch the text flip by. Now flip in the other direction. Look how smooth that is! See how it turns on a dime! That's literally thousands of hours of interface tweaking that went on there. It's magnificent. This is the Bugatti of web browser scrolling."

"Hah."

"So I was thinking -- what if the turnaround wasn't so instant? What if touching one finger onto the trackpad was, like... a brake? What if it had to slow down? What if I whipped two fingers and then whipped two fingers in the opposite direction, and the scroll bar, rather than turning around automatically, slid to a stop and then turned around."

"You must be the only person who can have more fun with an operating system than a video game."

"Actually!" I started to say. Then I stopped. I recall all the game designers I'd seen sitting in board rooms in 2011 with brand-new MacBook Pros. Not a single one of them had ever said, between pizza slices or granola bars, "These trackpads sure are sweet."

"...Maybe you're right."

We were at Perkins. Doug Jones asked the waitress for "A cup of regular to start, and decaf after that."

"So, like -- imagine you're using that trackpad to play Super Mario Bros.."

Doug Jones took a sip of his coffee. How do people do that, when it's so hot?

"I'm imagining."

"So, like -- imagine you're using that trackpad to play Super Mario Bros.."

"Like I said, about the whipping two fingers up or down. Let's say up makes him run right and down makes him run left."

"That wouldn't confuse people?"

"Nah -- if 10 million 13-year-olds can get their heads around up being down and down being up in a first-person shooter."

"Fair enough."

"So you whip it hard to run. Whip it in the other direction to skid and turn around. Plant two fingers to slow your guy down. While you're scrolling with two fingers, touch with one finger to jump. Hold to jump longer."

"How do you duck? How do you throw fireballs?"

"You duck by slipping two fingers to the right. You throw fireballs by tapping three fingers anywhere on the pad."

Doug Jones blinked.

"That might work."

I played through 1-1 in my head.

"It does work."

"How well does it work?"

I played through 8-3 in my head.

"It works better than a Nintendo controller."

"How much better?"

I played through 8-2 in my head. Then I played through 2-3.

"Way, way better."

"How much way better?"

I tried Super Mario Bros. 3, world 3-8.

"Incredibly better."

"I'm still not sold. Like, how would you do a first-person shooter with just a trackpad? How would you do that without buttons, or without a keyboard? What about a 3D action game?"

I opened my mouth. I closed my mouth. "I'm -- I'm sure you could do it. 3D action games rarely even use the full versatility a 3D space can afford, anyway. You'd probably use a bunch of pinches and rotates and spreads."

Doug Jones swallowed some coffee.

"How about Super Mario 64?"

"Are you asking if Super Mario 64 used 3D space well, or are you asking me to redesign it for trackpads?"

"Redesign it for trackpads."

I opened my mouth; I forgot to close it for a couple of seconds.

My final answer was: "Hmm."

Since then, I haven't been able to figure it out more than 99 per cent of the way. Unfortunately, if it's not 100 per cent, it's not enough.

And besides, how many people using Macs with Apple Multi-Touch trackpads want to play games on their computer in the first place? The PC Gamer Demographic -- rife with individuals who have embraced hotkeys and scroll wheels since two seconds after their invention -- no doubt includes thousands of intrepid users who would leap right on board with Multi-Touch trackpad controls. So, of course: if you build it, they will come -- and at first, it has to be optional.

You'd need a Different Sort Of FPS for trackpad-only controls to work. For what it's worth, I can conceive of a 3D platformer that works sort of excellently -- sort of. The epiphany hit me while I was in a Vietnamese restaurant, showing someone how I could use a pair of chopsticks in each hand. This, of course, lead me back around to my hypothetical design for an FPS: yes, it would certainly have to be a Different Sort Of FPS -- probably something like GunValkyrie, not something about headshots and turbo-sprinting down corridors. It'd have to be more about finesse. It'd have to more like checkers than what FPSes are currently like (the current FPS gamedesignosphere, for those keeping score, is like Connect Four where you have 0.7 seconds to make your move; it's about strategy, though it's also about how fast you can move your hands -- and I mean that as a compliment).

I tried to explain my Multi-Touch trackpad gesture-controlled 3D platform action-adventure game to a half-dozen people. I couldn't make any of them get it. It must not have been good enough.

It could be because of how I devised it while showing someone I could use a pair of chopsticks in each hand simultaneously. I mean, how many people can do that? It's something wise old men do in kung-fu movies. I should add that no one ever thinks it's impressive, unless they try to do it themselves and find their brain promptly snapped in half. I guess crushing someone's self confidence in their motor skills isn't the best way to invent a bold new user interface.

As I developed and redeveloped my idea for Trackpad Super Mario 64, my idea for Trackpad Super Mario Bros. grew in depth and became a perfect jewel in my mind. In a few short weeks, I'd possibly solved platform games on iOS.

Many of the platform game on iOS has those atrocious on-screen buttons. They both get in the way of the action and are unresponsive as heck. Either one of these reasons is two strikes in one -- that's four strikes these games have, and that's one more strike than something needs before I press the "home" button and promptly delete the app.

The reason on-screen buttons don't work is -- well. If you ask a room full of hardcore gamers, they'll tell you it's because "buttons are better". If you ask a room full of game designers, they'll tell you that it's because iOS games requiring on-screen buttons are either direct ports of classic games which were Designed For Buttons, or else slaves to archetypical game designs which required buttons.

Let's not get ahead of ourselves: the point is, we shouldn't port exact remakes of Megaman or Contra -- we should make new and better games in the same style, one which actually makes use of the potential of an analogue device.

I had a look around for successful "hardcore" gamers' games on iOS which implement platforming or shooting. Note that, in games like Canabalt or Jetpack Joyride, the player is moving automatically. Even in Halfbrick's nifty Monster Bash, the player is running automatically, with screen regions designated for jumping or shooting. Why is this? Because movement is tricky; because the world's paradigm is currently a couple inches away from where it should be.

THE FUTURE OF MOTION CONTROLS "Here's a Kinect."

"OK."

"I want you to design a game for this which is actually fun."

"OK."

"I'll leave you to it."

The next morning:

"How was that Kinect yesterday?"

"Oh."

"Did you come up with anything?"

I looked over the lip of my MacBook Pro display.

"Nope."

"Have another crack at it?"

"Nope."

That's my review of the Kinect.

I suppose I could also review Child of Eden in one sentence: that game is about as fun as an accountant's birthday party.

The Kinect is dumb. I'm sorry.

A girl I was thinking of dating asked me if I had a Kinect, and I said I didn't. Then she saw that I did have a Kinect.

"Hey, you do have a Kinect. Let's play this."

"Nope."

"Come on -- it'll be fun. Do you have that dancing game?"

"No."

"We can go to Target and buy it."

"I'd rather we just skip that and go straight to having sex."

"You're an arsehole. You're -- you're such an arsehole -- "

"Also, I don't have a TV."

"Oh."

"Yeah. See?" I pointed to the TV stand, atop which rested no TV.

"Why do you have a Kinect and no television?"

"I needed an Xbox, so I figured I'd get the one with a Kinect."

"Oh."

"I figure someone might make a Kinect game that is sort of decent."

"That dancing game looks nice."

"I don't want a game that looks nice. I want a game that is fantastic."

"I want to play that dancing game."

"That's how They get you."

"Why wouldn't you just buy the cheaper Xbox if you hate the Kinect? Can we at least try it out? Can't you control the menus like in ‘Minority Report'?"

"Well, if I had a TV, maybe we could try the Kinect out on the Xbox dashboard menus. I assure you it's about as much fun as making a banana smoothie using a coffee stir instead of a blender. And that ‘Minority Report' user interface is a joke. Why would anyone want that? Why would anyone seriously want that?"

"It looks neat."

"I don't want a computer interface that looks neat. I want it to do its job. I want to Get Work Done -- I don't want to become a professional fashion model while doing it."

"What about the Wii?"

"What about it?"

"I bet you sit down while playing Wii Tennis."

"First of all, I don't play Wii Tennis. Second of all, if I did, yes, I would sit."

"You have no imagination."

"That's not true: I'm just a realist. Realists can have imaginations."

"I bet you pee sitting down, too."

"I do! Sometimes it's the middle of the night, and I don't want to turn the light on."

"What about that PlayStation thing?"

"That thing would be great, if it actually existed."

"I saw it in Best Buy."

"That's not what I mean."

She blinked.

How was I having a conversation about motion-controlled video-gaming with an adult female? What sort of awful sitcom had I stumbled into?

"What about iPhone games?" she asked.

"What about them?"

"Do you hate them, too?"

"I sure don't -- some of them are great, in fact."

"Like which ones?"

"Like this one I made."

I showed her the one I made. She played it. She handed the iPhone back.

"I'd play that."

"You just did."

"No, silly, I mean if I had an iPhone."

"It has to be released first."

"Maybe I'll have an iPhone by then."

A few months later, I had this MacBook Air. I was finally alone with the glass Apple Magic trackpad, fully familiar with its Multi-Touch Gestures. Here I am, on it right now, sweeping one, two, three, and even four fingers -- pinching and rotating and squeezing and spreading like a pro, sitting in my same old slacker-jerk posture, in my Calvin Kleins and bathrobe, yawning on the stairs of my own apartment -- I figure, I have these stairs here in my house: I might as well sit on them, sometimes, even while computing. I'm sure my face, if isolated from my surroundings and activity, would not appear, to a Hollywood blockbuster summer movie audience, to be the face of a man hip-deep in the highest of high-tech. The casual popcorn-muncher would have no reason to suspect that this is the face of a man computing -- and computing hard -- with efficiency, productivity and aplomb far surpassing Tom Cruise's futurecop in "Minority Report". He's catching killers -- so what? I'm running a business. I sure am cool. Yeah, I'm great: I'm just going to go on sitting here with this bored look on my face. (I wish I was rich (I wish I had a girlfriend (I wish I didn't need a root canal (I sort of wish I had a TV (I bet I'd have six-pack abs already if I played Dance Central every day))))).

This thin sheet of glass beneath my fingers feels better than maybe any electronic device I have ever touched.

It occurs to me quite gently this thin sheet of glass beneath my fingers feels better than maybe any electronic device I have ever touched. I wonder why that is. I am particularly sensitive to friction: for example, I prefer the Hori EX2 wired Xbox 360 controller to the official Microsoft one. That Hori is a company that knows how to pick a delicious plastic.

Here's this glass trackpad, and it's like butter -- like micro-stubbly butter, like what the world's smartest mad scientist would want his robot dolphin's fuzzy hide to feel like.

The responsiveness -- the lightning speed with which the trackpad identifies the number of fingers you've laid atop it -- is extraordinary. If you've never sincerely laid hands on such a trackpad, I ask you to try it out the next time you're in Best Buy or an Apple Store. It's not the same as an iPad or an iPhone. There's none of that sluggish, syrupy friction of finger against glass. The smoothness of the material and transparency and instancy of contact to input to output is remarkable.

I get on the internet, and I find the keynote speech at which Steve Jobs introduced the glass Magic Trackpad for Apple's desktop computers. I want to see the words he used.

Here is an exact quote:

"We've optimised the coefficiency of the friction on the glass, so it's just really beautiful."

Optimised! Coefficiency! Friction! Beautiful!

These are the sorts of words I was born to follow. ("Coefficiency" isn't even a word. (Neither is "instancy".))

I had a really good think about it: they have these things for desktops, now. I read and watched CNet.com's review of the trackpad, in which the editor asks if the device is meant to replace the mouse. His answer is "not exactly". He asks: "Will the mouse ever really die?" He says "this is maybe another step toward [the death of the mouse]". I looked at the new issue of Mac|Life while at the drug store the other day; they've got a feature that's trying to predict the future of Apple products. They've got something called the "iDesk", where the desk is a multi-touch glass surface. I saw this and immediately thought that the logical step between right now and this multi-touch desk surface is an Apple trackpad that's twice as wide as the one we have now.

Here's where I start wondering.

"IT'S GOTTA BE A MOUSE AND A KEYBOARD" The first time I heard a gamer say he refused to play a first-person shooter without a mouse and a keyboard was at an Electronics Boutique several months after Goldeneye's release on the Nintendo 64.

I heard it again -- this time from my little brother -- when Perfect Dark was released.

As Halo and the Microsoft Xbox loomed, "there is no better way to play a first-person shooter than a mouse and a keyboard" and "I don't play FPSes on consoles" had become mantras among the hardcore LAN-party FPS crowd.

As recently as September, 2011, people as intelligent and prominent as Kotaku.com's editor-in-chief Stephen Totilo casually expressed the common-knowledge "fact" that "Trackpad = Diablo death". This statement was with regard to his MacBook Air trackpad not being as good for playing Diablo III as a mouse would probably be.

I won't say Totilo is wrong. In its current state, I'd be a pathetic fool to insinuate that a mouse isn't a better tool for plumbing the depths of Diablo. Same thing for Starcraft. For what it's worth, we might as well go to the Wikipedia article for "Mouse (computing)" and edit the intro paragraph to read, "The mouse was invented by Counterstrike, Diablo, and Starcraft."

I could not imagine a professional e-sports gamer playing Starcraft 2 with an Apple Magic Trackpad.

Though why not? Why the heck not? Can we redesign Starcraft 2 to feel amazing with trackpad gestures?

I just thought about it for three high-octane seconds, and the answer is: "Duh."

Starcraft would take too long to describe, and I'd hurt my head in the process, so let's do Diablo instead.

Note from the video in the above-linked Stephen Totilo piece the faint sound of a mouse click. Aha -- is Totilo one to disable tap-to-click? Tap to click is the soul of a gesture-based complex game interface. We'll need it on for our example. That way we can use the mouse click regions for other things -- nothing fancy, of course.

At the very least, our ideal Diablo would use all the Mac OS X Lion three- and four-finger gestures: spread with three fingers and a thumb to open your inventory, touch three fingers on an item the cursor is resting on to move it. Set potions in your belt; slash four fingers up, down, left or right to use an item in a particular belt slot. (I'd set "down" to be my trusty little potions.) Touch an enemy to attack it or a spot on the ground to move there. Touch an enemy with three fingers to use a queued technique.

This isn't perfect, though it's a good-enough launchpad.

Going deeper: the challenge of Diablo often stems from Clicking The Right Thing Quickly Enough. You don't want to click the wrong monster, for example. You might not want to click on the loot that a monster just dropped -- you might want to click on the monster next to him. Or you might want to run away -- yes, running away is a very good thing to do, sometimes.

A common method of player movement in Diablo is to click a spot on the ground and hold the mouse button down. The player will now run in the direction of that arrow, and that arrow stays in the same position relative to the centre of the screen.

Trackpads "equal Diablo death" because, until the Apple glass trackpads, with their "inertial scrolling", they weren't so deliciously sensitive to movement and finger acceleration.

So let's try this: if you put two fingers on the trackpad, the game will immediately place the cursor on a part of the screen relative to where you put your fingers. Now you can tap to click, and hold it there, and... you're running in that direction.

That solves one problem.

Now for the others. I've mentioned before that the trackpad is genius at immediately recognising how many fingers you've just put down, and what you're doing with them. A quick three-finger swipe to the right on the centre of the trackpad could be one skill. Left could be another. You could pinch or spread to switch between different skill configurations. How many configurations could you have? Six or seven, maybe. Pinch three times to get from configuration number four to configuration number one. Spread three times from configuration four to get to configuration seven.

In the inventory screen, you can set up different equipment configurations. Just use two fingers to rotate -- either clockwise or counter-clockwise -- anywhere on the trackpad to cycle between equipment configurations.

I think you get the idea.

What I'm saying is... yeah, I could design a trackpad control scheme for Diablo III that allowed at the very least the same amount of control expression as a mouse and an entire keyboard. (Note that I didn't even mention that the Magic trackpad allows multiple "clicking zones" for use of the pad as a mouse button -- we would have gotten silly really quickly.)

Now let's get ridiculous and talk about designing thought-speed trackpad controls for Starcraft 2. It won't be hard if we cheat a little bit and start with: "Two track pads, one for each hand."

I'll just leave it at that.

A TRACKPAD TO THE PAST I asked followers of my Twitter to suggest me classic games to redesign for single-trackpad gesture-based controls. Most of the submissions were "Zork", "Typing of the Dead", "Steel Battalion", "God Hand" and "Street Fighter III", so most of my responses would be: "-_-".

I've sat here and thought this over with my MacBook Air on my lap. I've filmed a little video of my hands "playing" a few games with the trackpad. See if you can get what I'm saying. (I'm particularly proud of Katamari Damacy (note the "chopsticks" motions).)

BACK TO MARIO So we come back to Super Mario 64. How would you control Super Mario 64 with a trackpad? What checkers-worthy FPS could one conceive with Apple's little glass-and-aluminium square? My mind reels; I'm giving it a lot of (too much) thought. I'm thinking if you put two of these things on the desks of the right people (Will Wright, et al (make sure he has a comfortable chair)), and we'd have something better than Portal 2 knocked out in six months.

In summary: is the Apple Magic trackpad better than a Dual Shock 3? It's better than a Rock Band guitar, that's for sure. Whether it's better than a Dual Shock 3 or not: I can't really say, because there aren't any games specifically for it. It's better than an iPhone, because you're not touching the screen.

Tim Rogers is a person you can follow on Twitter; he also sometimes reviews video games here.


Comments

    I can't believe I just read that much text about trackpads.... and enjoyed it.

    Using Apple and Video game in the same sentence still makes me giggle.

      Not just because barely anyone offers mac support, but also, because anyone who plays computer games is generally smart enough to know that buying a mac is like buying an alienware computer, a huge ass markup for no reason other than brand.

        Except my mac mini was dirt cheap at $500 when I got it and while not as powerful could play the same games as all my mates, while I paid $500+ less ;)

        Not sure about the new ones, or how the pricier machines go either, but you certainly do get good value. Mac hardware in my experience tends to be, well, less likely to kark it randomly, no heating problems despite having no airconditioning (my friends' computers seem to practically set on fire, despite all the cooling systems they've invested in), and I've had very few problems with things like adapters and cords and other peripherals making strange vibrating noises or outright sparking, another benefit (I won't lie though, my ancient iBook's cord eventually karked it sparking, but I figured being dropped, pulled, trodden on, and semi-eaten, it probably had a good excuse).

        Of course, I'm always willing to try new things, or go with non-mac things (Acer monitor, old dell laser mouse from when they first came out, altec lansing speakers), but every time I ask my mates to show me some good deals or show me where I can get some good cheap stuff that's more powerful than my "overpriced mac", I'm often taken around to find that I'd be paying the same price for a mac anyway, only not prebuilt and without the reliability of the brand I've found for myself.

        The real problems though are in *software* support. That I can say is truly terrible! Thank god for the porting team! Another problem is that Apple portable devices like the iPad are good at gaming (as much as they can be with only a screen anyway), so the Mac store is full of ports from *those* rather than PC games =(

          Being able to play the same games as your friends on your 500 dollar mac probably just means you guys were all playing games that didn't require much technically. My point was that you pay more, for worse. Which is still true, regardless.

    I'm a buttons man, myself...
    http://www.toplessrobot.com/sb.jpg

      A fellow toplessroboteer?

        I stop by the site every so often, but not enough to knowingly portray myself as something as awesome-sounding as a toplessroboteer, unfortunately...

    Some interesting ideas in this article. Though for my part, I hope they're largely academic. Using a trackpad for games would be an incredibly complicated thing to learn (especially for more complex control schemes) and the Diablo one is impractical because it requires you to look away from the screen at the trackpad to place your fingers correctly.

      Only in the beginning. Muscle memory simply from practice would have the dimensions and areas of a trackpad worked out in no time, rendering your objection moot very quickly.

      At least in my experience that's how it goes :)

    Christ almighty that was a glorious piece (as expected from ABDN), with proper laugh out loud moments and beautiful descriptions of conversations.

    I would love to sit in a room with you Tim.

    Pretty though-provoking article. I'm still pretty set in the ways of keyboard and mouse, though recently I've become more accepting of console controllers (after half a decade of separation). I hope this means I can become more flexible for future interface shifts. Having said that, I would love the opportunity to give dual multi-touch controls a go.

    I’d rather we just skip that and go straight to having sex.
    Could not stop laughing.

      i made several particularly loud hyena sounds while writing that part.

    The track pad is cool for the same reason the Wii-mote should have been. Games are never made to utilise the high concept controls that are possible because of the steep learning curve... and the valley that lies at its base. I seldom think developers get controls right with a standard controller!

    Why is this in "PC" news?

    Also: All the ideas mentioned in the article are terrible. Games with any gameplay depth require the ability to issue simultaneous commands. No amount of gestures on a track pad can compensate for this because you can only do one at a time.

    Also, console FPS's (Goldeneye etc...) were successful DESPITE the terrible controls. It says more about the game's content that we struggled with things like the awkward 3 prong controllers simply to play them than anything else.

    In order for a game to utilise a multi-touch/gesture track pad successfully it would have to be built from the ground up with that particular input method in mind and nothing else.

      +1
      Your comment came up while I was writing mine: Invention and innovation should suit reality, not the other way around.

        Funnily enough, my entire job is built around matching design to reality. No offense to the author, but these sorts of "the user will learn eventually" ideas are what keep me in the money.

          My job is just the opposite, I try to bend reality to suit better design. Users don't know what's good for them :)

        So the keyboard and the mouse were invented for FPS games, and not the other way 'round?

      It's in PC news because it's an issue that deals with Personal Computers. Is that not what the acronym stands for?

      "...would have to be built from the ground up with that particular input method in mind and nothing else."

      Well, yeah... that was pretty much his argument, except for the nothing else part, that's stupid. Considering games today are being built for console controls, motion controls, Wii controls (which are a combination of both, i guess), and keyboard and mouse, why would one more input method be a problem? It wouldn't affect the others...

      He was saying that with the more complex games, you'd need more controls, using a trackpad for both hands.

        It's really not stupid. If you actually want a method of control to be utilised properly and not just be a gimmick, you're going to have to base the fundamental game mechanics around that input. Touch pad gestures are a VERY specific and limited type of control and is not something you can really generalise with.

        And two touch pads gives you a whole... TWO simultaneous commands. Nothing suggested here is an improvement on what already exists as an extremely versatile method of input (KB + Mouse) and is simple trying to fit a square block in a round hole because it's covered in glitter and looks prettier.

        Also PC = Any computer not running a Mac OS. Obviously.

          100%

          Trackpad games will need to built from the ground up for trackpad to work well. Just looking at a bunch of iphone games proves this, for the best games on the iphone are ones that make the most efficient and comfortable use of the touch-screen. We're at a state in gaming where people still argue whether gamepad or kb and mouse are the best controls: There is not going to be some "best" magic device.

          Lastly, apples trackpad only feels so good because you guys were using a laptop where (until you put in a mouse) it is the best input system for pointer GUIs. Apple's trackpad is not even exclusive or special, every modern laptop I've seen has the same gestures and actions in their trackpads now.

      i tried to be pretty clear that what i am saying here is that the trackpad would be a fantastic controller IFFFFFFF we had games built from the ground up for that specific input method -- not that we should shoehorn old games into this new method.

      hence the "might be" in the title. it's a "might be" because there aren't enough games that use the multi-touch trackpad at all, much less extensively.

      here's one, though:

      http://bitpilotgame.com/

        Isn't that a bit like saying the DDR floor mat would be the best video game controller ever made if only people made more games for it?

          Not everyone has a dance mat, but a large portion (99%?) of laptop owners have trackpads. It would be nice for games to be able to take advantage of their extra features as an option, instead of just being there trying to emulate a mouse.

          no, it isn't like saying the DDR floor mat would be the best controller if people made more games that used it.

          it's actually nothing like saying it at all!

          the DDR floor mat has four buttons! they have two states: pressed, and not pressed.

          the apple trackpad, on the other hand, Does A Lot Of Stuff! cool stuff, even!

          TONS of cool stuff!

      Because we don't have a "MAC" category.

        Yes. Sorry, I was being a dick. It was meant rhetorically.

          try putting "RHETORICAL QUESTION" in all capital letters and parentheses after your rhetorical questions on the internet, or start expecting

          1. answers
          2. answers you don't want to hear
          3. hostility
          4. lol

            I was really impressed it was in the PC category. Macs are PC's. The fact most of us have forgotten this or are too lazy too use the proper terminology just makes me think how successful that Mac vs PC campaign was.

            It reprogrammed so many of us, so that we think of Macs as being special, it makes it seem that they are more than just a mere personal computer, that somehow they are in their own realm.

            It also draws another line in the sand, making us more likely to associate ourselves with a chosen side, hence the "vs" in Mac vs PC. These versus battles just fuel more sales based on fervent consumer loyalty.

            Apple marketing is so effective because of these subtle psychological tricks.

            The sent from my iPhone, includes an advertisement at the end of every email. I can understand if it simply said "Sent from my phone", to indicate that you are probably going to be short.

            And the fact that Jobs used to blatantly lie in keynotes, and ads would greatly exaggerate the truth, creates the most brilliant form of marketing, which I am falling prey to right now. It fuels arguments based on defending and attacking these elusive half truths.

        Kotaku has no need for a "MAC" category. "MAC" generally refers to an operating system (OS X). "PC" is an abbreviation for Personal Computer, Windows 7 is the operating system that most current computer models run. Some computers (mine included) run both OS X and Windows 7 :). No matter its operating system, a PC is a PC.

    TL;DR

    Trackpads are terrible. Always.

      nope!

      maybe give something a try before you type a comment on the internet.

      Man, posting "TL;DR" doesn't contribute to discussion in any valuable way at all. It's about as useful as posting "First!".

        It contributes by letting you know he thought the article was too long.

          Or that he's too lazy to read the whole thing in which case he can't really justify contributing to the comments section...

    This guy sounds like he has a mild form of Aspegers. That is neither a bad thing, or here, nor there.

    In any case, this would be an interesting idea. I remain sceptical; especially with all the patents that make the track-pad special held in the harder-and-tougher-than-iron-fist of Apple.

    I also think this is more a case of making reality fit the invention, rather than the invention fit reality. The simplicity of a hard and smooth yet gently textured surface with fewer moving parts, seems seductive in replacing oddly-shaped and button covered lumps of plastic... But since when are ergonomics the "pretty" option?

      as i have advised others, so i advise you:

      try reading my articles in a Jerry Seinfeld Voice.

    Oh my God who the hell wrote that. This is meant to be a news article, not Odyssey......

      while you are attacking someone for what you consider to be superfluous prose length, it might interest you to know that three dots suffices for an ellipsis. no need to put six on there! six would be like you were pausing to reflect for thirty whole seconds before clicking "submit comment" -- which is something i'm pretty sure you didn't do, because if you did that you surely would have noticed that you missed a comma and should have put the word "The" before "Odyssey".

        Starting sentences with a capital letter helps too.
        If we're going to nitpick.

          You sure took that particular piece of bait, there! :-/

    I'm a fan of the MT gestures but totally disagree with Mario being played in the current touch environment. Mario and Super Meat Boy are best with digital input. They are twitch games that need precision and speed. Touch is everything BUT that.

    Read a third before I couldn't stand the Apple gushing any longer.

    "When I saw what Vizio’s Apple-aping personal computer lineup looks like, as a person who has read all of Steve Jobs’s biography and is thus familiar with the sort of suffering he’d feel if he saw them — I’d wish that on no one — I honestly felt sort of glad for a couple of seconds that he was dead."

    Give me a fucking break!

      well, you seem to hate apple a fair deal, so is it not at least Sort Of Enough that that sentence ends with me saying i'm "sort of glad" steve jobs is dead?

    I'd love to see a good implementation of a dual trackpad 3d game...sounds very promising.

    Interesting idea...

    By interesting I mean it's interesting in the way I could train myself to eat using just my feet instead of hands to operate my knife and fork. This would only be useful for games with the most simplistic of control schemes. Like those 99 cent puzzle games or simple flash platformers. But for an FPS, MMO, MOBA, Third-Person Action whatever, I don't see this working at all without some ridiculous assists thrown in to make up for the lack of control this gives the player.

      Well, I must add that FPS games on a touch screen can be heavenly, simply because you have a combination of precision (where your finger is is where you aim), in conjunction with motor memory etc (analogue sticks do well here, but require a more abstract concept of aiming).

      That makes a nice little compromise. The minor problem is that on devices of certain sizes, your hand is suddenly quite big and takes up the entire screen. But if you alleviate that problem (tablets for example), and the device has enough guts for smooth, scripting assisted gameplay (e.g. so the most minor of muscle movements doesn't send the crosshair waving around), you could theoretically get some great 1st person shooters.

      Aim assist will always be a dark area of debate, partially because it is about skill vs making technology do what the analogue human body wants it to do (when a person aims a gun, the use of the arms is no where near as rigid and mechanical as using a mouse to aim on an FPS is), but with various prejudices put aside I reckon tablets at least and other touching devices could do quite well FPS wise, or at least Gameloft's trademark ripoff "Modern Combat" series does a good job of demonstrating how to do it ;)

        Yeah but let's be honest here. Using a touchscreen or a touchpad you are NOT going to ever be as accurate as a mouse. You may enjoy one form over the other but to say that touchscreen or touchpad is more accurate than a mouse is nothing but an outright LIE. As it stands touch technology can not replicate the accuracy of using a mouse. I wasn't even bringing analogue sticks into the argument because that method requires aim assist to be able to slightly create the accuracy of a mouse.

        Your response relies solely on poorly considered "what if's". Which is nothing but a waste of time for actual gamers.

    While I only read the first few paragraphs, where does apple come in? My eeePC and laptop both have multi-touch track pads, and my eeePC is now a few years old.
    The only relevance I can see that track pads have with apple, is they try to get people to use the damn things on a desktop...

      The multi-touch trackpads on the unibody Macbook Pros and Macbook Airs is different. Much larger, made of glass. Whole thing is a giant high-resolution multi-touch button basically. Your fingers slide on it just right. It's hard to describe really, it's worth giving one a try for a bit (with tap-to-click enabled) and the difference is quite obvious.

      Negative Zero is right, that's kind of like saying 'whats all this fuss about the Ferrari Enzo gearbox, my Daewoo has one as well" Same principal, very different experience!

    Fantastic article, thanks! I shall have to read again in a Jerry Seinfeld voice ^_^

      I tried: it's exhausting!

    I was initially skeptical of the Macbook Pro trackpad when I first got mine back in 2009, but within seconds of enabling tap to click (why is this not on by default?!) I was hooked, and now it actually prevents me from migrating back away from Apple hardware for laptops. There's no viable Windows equivalent and there isn't even a good Windows driver for the touchpad so I can't even use it via boot camp.

    Why is this in the PC section?

    "Interesting idea…

    By interesting I mean it’s interesting in the way I could train myself to eat using just my feet instead of hands to operate my knife and fork."

    My thoughts exactly. If it ain't broke don't fix it, and we've enough mediocore touch input games/software with no depth anyway.

    At the end of the day mouse/keyboard is the quickest most efficient input method ever (atm)- everything you said you could do with a swipe this way or that with how ever many fingers I can do with a series of extremely simple SHORCTUCTS AND HOTKEYS, that with no doubt in my mind are faster than a touch pad. See all those silly key things above the touch pad?

    I just feel sorry for this poor guy... somehow...

    The biggest problems with trackpads that I can see is accuracy. If I'm using 2-3-4 fingers, it only knows the target/cursor must be somewhere within the area specified by those 4 fingers, and possibly the zone they cover when swiping, a much larger area for error than just a single finger.
    Not to mention every time you are swiping to perform a swipe/action, you can no longer be controlling the cursor/movement etc.

    I just can't see a touchpad working for anything that requires simultaneous or fast precise control until they come up with the iDesk, allowing you to use both hands. This is the sort of scale one would require to allow quick and accurate movements to be translated onto 13" to 28" monitors

    Gestures are really nothing more than fancy hot keys; Swipe direction to movement = WASD, 4 finger swipe for desktop = Windows Key + D, only the latter leaves me with control of the cursor still.
    Accuracy is what I demand of my control schemes, and there is far less room for error in a button press, than a gesture.

      I should probably add since there seems to be more than enough vitriol here, that the article was still a thought provoking (I wouldn't have replied with the above otherwise!) and enjoyable read.

      well, you use one finger to move the pointer, remove the finger from the trackpad, and then touch -- anywhere -- with the four, three, or two fingers, and it'll immediately register as a two-, three-, or four-finger click on the exact spot indicated by the cursor. and i should indicate that the super-thin glass of the trackpad allows for a neat-enough level of touch sensitivity that the recognition of contact is really super-immediate.

      that's the default setting, anyway. application developers can tweak these settings as they see fit.

      anyway, there's that! thanks for the comment~~~

    I confess I TL;DR'd this at about the point where the writer started nattering on about the Kinect and not having a TV. This seriously needed an editor.

    Oh, and the writer needs therapy. He's not "critical", he's seriously OCD.

      It is not nice to make a personal insult like that to a complete stranger. :-/

        Tracey, it's also not nice to repost simple Apple fanboyism hiding under the guise of being a "journalist". Tim Rogers proved nothing more than being an Apple fanboy despite his claims to the contrary and proposed an idea that is not clever in any way, nor is it thinking outside the box. An Apple trackpad or any trackpad for that matter will never be a better control scheme for the vast majority of games. It's preposterous. Like I said, his idea of using the trackpad for gaming is like using a knife and fork with your feet. It will NEVER be as good as current control standards.

        Also, I love his defensive nature. It's like watching a Nazi try and defend being racist in that it's nothing but pure stupidity. Then again, that is the requirement for purchasing a laptop that is more than twice the cost of a windows laptop that will do exactly what you need. Or in his case what Tim needs, which is spreading hipster stupidity like it's AIDS.

        He look I own an Apple laptop...sure... a Windows laptop with twice the hardware capabilities was cheaper but I'm now qualified to be a twat....

        Yeah....this is totally what KotakuAU needs.

          Really? Somebody mentions Apple in a good light and it's time for Godwin's law? Mac users are hipster twats? What?

          The article was interesting, funny (at least to me - not everybody's the same) and I enjoyed the read. I wish there were more articles like this on Kotaku. You of course may not agree, but just please try and keep the insults to a minimum, eh? The authors are part of the reason we come to Kotaku, and there are ways to get your message across instead of using the gag-inducing term 'fanboy'.

            How does paying 40% (at best) more for less work for you? Would that be a better way to say it?

            Even those who use Mac's on a professional level have no recourse as the software used is either now available on the Windows platform or equivalents are available. There is no reason whatsoever to own Apple towers or laptops anymore.

            The only possible reason for purchasing an Apple computer is purely due to fanboyism or stupidity...it's also not really that surprising how often those coincide.

              It depends on what you mean by 'less'. If you're saying that Mac's give you less in regards to being a pure gaming machine (3D graphics power) then yes, I totally agree. That's why I own a Windows machine for 3D games.

              For everything else, I have a MacBook Pro. Why? Many reasons (OS X, Logic Pro, hardware design, can also run Windows if necessary, to name a few). I'm pretty sure I'm not stupid (my Mum says so - perhaps I should get another opinion), and I couldn't care less of what people think about the stuff I buy.

              It's also got Steam on it for the odd casual game if I'm not at the Windows beast, so it's not completely useless in regards to gaming. But it would be nice to have some games take advantage of the trackpad, which is what the author was getting at. FPS shooters ain't gonna be great on it (KB/M for a while yet), but not everything is an FPS (yet).

              I'm not saying everything Apple does is fantastic (and I don't think the author did either), but I give credit where credit's due. I like their computers. You may hate Apple and everything they do, and that's fine, just understand that not everybody has the same needs/wants/points of view/priorities.

                So what can your mac do that a PC equivalent (laptop or tower) can't?

                  It runs OS X for a start (which I prefer to Windows), as well as Logic (it has some awesome built in synths and is a pretty complete package).

                  As for the other reasons, it's hard to put a finger on it. It's the whole package, really. The actual design of the laptop and the materials used, the battery life, instant wake (so many Windows machines I've used don't wake up!), the power cord thingy, the feel of the keyboard, quality of the display, the glass trackpad... Windows laptops have these things, though usually not all in the same laptop.

      you know what? you might need therapy, too! feeling the need to get on the internet and inform the public that you were not interested in reading a particular thing is a textbook symptom of narcissistic personality disorder -- and also a symptom of You're An Asshole.

      unless by "OCD" you were abbreviating "obsessive-compulsive disorderly" or "obsessive-compulsive-disordered".

      i don't think you meant that, though, because by all the indications you've given me in your post you don't have a sense of humor and are Not Clever.

      (i should point out that i'm just messing with you, by the way. i just enjoy insulting people. someday i want to be as cool as don rickles.)

        That's like saying you want to be as cool as Han Solo. You're just setting yourself up for massive disappointment. Unless your name is Dirk Benedict, in which case you'd actually come close.

          With his proud adidas style wardrobe and tween moustache are you really expecting anything worthwhile? This is a guy living in a dreamland of "expensive = good" who was hired by KotakuUS in their true tabloid fashion. He just your typical trash news writer who uses "clever" analogies and such to try and be cool when he's nothing more than another useless piece of upper middle class trash who wouldn't know the first thing about life or having to actually work for a living.

            For a few seconds I thought you were referring to Dirk Benedict :-P
            We were gonna have words... But then I realized you just meant Mr. Rogers. Crisis averted.

    So, you recorded these conversations? Recalling them from memory? Or just making shit up?

    Hey Tim, loved the article content and style. Also loved your comment responses. Keep up the good work!

Join the discussion!

Trending Stories Right Now