How do you make music visual? How do you make it interactive? And how do you make the abstract tangible? These are just a few of the questions we wanted to ask Tetsuya Mizuguchi, the celebrated creator of Rez and Lumines. Turns out he has answers.
Tetsuya Mizuguchi stands in the middle of the room, he’s holding an Xbox 360 controller, two seconds away from giving a public demo of his latest title Child of Eden.
“You can use this if you want,” he smiles, with the controller in his hand. “It vibrates with the music. Or if you want to use Kinect you can just, you know, put it in your pocket instead!”
Mizuguchi laughs, but he doesn’t follow his own advice - instead he puts the controller down, before raising his hands like an expert conductor. The show is about to begin.
In ancient Greek Synesthesia is a combination of the words ‘together’ and 'sensation' - a neurologically-based condition where the stimulation of one cognitive pattern elicits an involuntary experience in another sensory pathway.
Like hearing colours. Or seeing music.
As Mizuguchi plays in front of us, using his hands to direct the sound - exploding in vibrant, abstract colour - we almost grasp the concept. Child of Eden, if anything, is an attempt to make music visual, an interactive experience over which you have control. You are constantly being rewarded through sound, through colour.
“It’s a difficult question,” claims Mizuguchi, when we ask him precisely how one goes about making music visual, “but it’s important.
“It starts from a concept. For example, evolution. In Child of Eden we start the game from a simple cell in the sea, and things slowly become more organic and complex. We go from darkness, then up to the sea, and into space – that sort of concept influences the music, it starts off ambient and grows.”
Watching Child of Eden in motion, we start to wonder – how is this built? Clearly the music and the visuals evolve in unison - but how does this process begin? How does it work?
“Well,” begins Mizuguchi, “we have meetings with the sound designers, the mixers, the DJs who create the music for this game. To be honest, it’s not so easy. We look at the sound design and try and look at what changes – there’s a lot of trial and error. We have an architecture to the game - windows, doors, walls colours – which we change with the music. It’s that kind of process.
“It’s very easy with simple beats,” he continues. “It’s easy to fit that into the game. But just using tracks like that is boring - we have to try and use different types of music. Sometimes we experiment with polyrhythms to get that human, organic touch. And we try and create a game that represents that.
“There is a voice orchestra in Bali, where they use just human voices – it’s very interesting, very trance, human. But it has an architecture. They have seven types of voices, and all the voices are different – so we try and add that sort of touch into the game. It’s like an orchestra. We have to make this all work behind the scenes.”
Traditionally rhythm games, like Rock Band and Guitar Hero, exist as representations of something concrete. A guitar is placed in our hands and from that we create sound. This is the easily comprehesible rule - you are tethered to a song you know well and hope to replicate correctly. The reward comes from emulating something you’ve heard in another context, a song you grew up with, or a song you fell in love to - that connection is already there, it already exists. With Child of Eden that connection is far more abstract, and is evolving constantly – how does one help the audience make that connection? How do you make something that is almost completely abstract tangible?
You get the sense that this is Tetsya Mizuguchi’s greatest challenge.
“It’s like a new frontier of creation and imagination,” he says, smiling. “It’s like walking into a field of fresh snow - a tough process, but fun also. I really love the new challenge, using new technology, new ideas - a new sense of wonder.
“We have to have some sort of rules within the game. The enemies in the game are viruses, and we try to make them red, purple, or orange. These are passionate colours, and we use them to represent danger. Then, when you purify the viruses, you have blues, whites, greens – those kinds of colours.”
But how does Mizuguchi decide what colour represents what? Instinct?
“Yes,” he agrees, “I think following your instincts is important. With colour we have a common code for all people, all races. Sometimes I feel the process of making this game is the process of finding that common human code - with colours, sounds, everything.”
Clearly we all have a common concept of what colour represents what – red equals danger, blue tranquillity. Is this a learned behaviour or something we are inherently born with.
“It’s a mix I think,” says Mizuguchi. “We inherently have both – I don’t know what the border is but we have to think about both.”
There has been an influx of new, creative video game designs that attempt to fuse visual audio rewards – games like Flower and de Blob. We asked Mizuguchi if he enjoyed seeing similar ideas in different types of games – does he find that inspiring?
“I don’t really find inspiration from other people’s games,” he claims. “I don’t play them so much. If I play someone else’s game, it’s difficult to focus on creation.”
Then what does he take inspiration from?
“Anything but games,” he laughs!
“Music, obviously, but travelling really inspires me," he continues. "You know, if you walk around the markets in Istanbul you can hear many different types of sounds – different languages, people moving and talking – you can see different colours. That kind of stimulation is really important in the act of creation.”
This isn’t Mizuguchi’s first time in Australia, but we wondered what he might take from this trip, despite the fact that, on this day, Sydney is coated in a sheet of rain, and the Harbour Bridge obscured by fog and drizzle.
Sydney is very sophisticated,” he says, struggling to find the words, “and natural. It’s rich in nature – I find it futuristic in a good and positive way. It’s a very special place. It’s not like the US or the UK, more like a hybrid, and I mean that in a positive sense.
“When I come to Sydney I feel like I can get a sense of the future.”
Sounds almost exactly how we’d describe Child of Eden.