A Conversation With A Game? Devs Seek To Break The Ice

Language recognition is not a new concept to video games - the first text adventures had to understand commands somehow. But researchers are trying to integrate it in more open-ended ways - allowing for dynamic conversations between players and characters.

BBC Radio profiled the efforts, lately highlighted in the game 221B, a movie adaptation of the recent Sherlock Holmes film. In it, players must interrogate witnesses and suspects to gain answers that advance the story.

"Rather than attempting to create an exhaustive list of possible questions and the appropriate response, the characters in the game are capable of making a 'fuzzy interpretation' of what is said to them," the BBC reports. "The intention is to remove the frustration, familiar to any who played the old text-based adventure games, of having to guess the right way of asking a question or giving an instruction."

Other games based on open-ended use of language, spoken or written, include Facade and, of course, Scribblenauts - and even Left 4 Dead. "Each of the characters has a set of voice samples which can trigger based on events, situations and other dialog lines," Rockstar's Alex Champandard said of L4D. "This results in completely emergent short conversations depending on the situation."

The BBC calls it one of "the last uncracked problems" in games design. It's a good read, especially for the humorous kicker paragraph. AI Aims to Solve In-Game Chatter [BBC Radio]


    One of the major problems I see with language recognition as it stands is that it lacks one of the defining factors of language: intonation. There are so many things you can communicate just through grunts and non-committal noises - anger, sadness, confusion etc... - that can't be interpreted purely through words. A text-based interface might be easier at the moment, but in the long-term it's going to severely limit this technology.

    I have to say, though, this has perked my interests.

    Tom Clancy’s Endwar was a game that had fully functional voice controls. It seemed to work pretty well in that game. And besides if parrots could do it then so can we.


    I could see someone like Hideo Kojima doing something special with this in the future, he’s always been an advocate for including all the senses in a game, just imagine letting out a tiny squeak to distract a nearby soldier, or having to be absolutely silent at a certain part to save your skin.

    With the general trend towards more interactivity by the person, it seems only logical that video games now want you to use your voice, I’m looking forward to seeing this in more games.

      Neat, I had overlooked that one. Another that jumps to mind is SOCOM, although I've never played it. There's still a massive jump from simply following commands to recognizing emotional patterns, though.

      I seem to remember the demo of natal suggesting it could do something akin to recognizing voice patterns, but I'm not sure

        You have a point, the common feature of all of the current games that incorporate voice commands is that it can only recognise words from a pool of pre-determined answers. Like Endwar for example, it can recognise a lot of things but it’s not really total control in a sense that you can say things in your own special way. If you break down the technology, it’s a lot like voice recognition on Windows, It’s not really reading your voice at all, it’s just keeping its year open for catchwords. For example I can literally say a line like “blah blah blah blah open kotaku” since i have Kotaku in my favourites it will open it, but according to windows that line was just as coherent as an actual sentence. I think soon enough the technology will be advanced enough that games will be able to tell what you want no matter how convoluted you say it. But this is probably not until the next generation.

          Unfortunately, Andrew, it's a bit further off than that, I'm afraid. These aren't new problems, robotics has been dealing with them for some time now, and they're difficult to solve well. I do share your optimism that when the technology happens, it will go straight into games - how cool will that be?!

            I remember a while ago I was on a website talking to one of those “AI bots” that can talk back to you when you type in text. It could keep up with the most basic conversation like what is your name or what are your interests, but as soon as I asked anything somewhat complex I just got the same generic response it went something like this “That’s really interesting please tell me more about (whatever topic you wrote about)”. As soon as I asked the robot anything that would require it to have an opinion on anything it just told me to re ask the question basically. I know this is a long way off, but I don’t think it will be too long before its seamless enough to be included into video games.

              That's one of the main limitations with chatbots, and can conceivably be solved with larger databases. But another more subtle problem is for a system to know when you're asking it garbage. For example, even the best AIs have trouble with questions like "What time can I catch an elephant ride to Jupiter?" They usually respond that they're not sure of the schedule, but ignore the fact that catching an elephant to Jupiter is ridiculous, which a person would surely notice and remark upon.

    would certainly make games more interesting if done right

    That's an interesting discussion on voice recognition, guys, and I agree with you about its shortcomings, but I think you may have missed the point of the article somewhat - there's no indication that 221B incorporates voice commands, and I'd expect them to be advertising it if they were, because that would be quite a feat. Any vocalised elements mentioned in the article are done by the games themselves.

    They're talking about the old-fashioned Turing test - basically, the idea is to program an artificial intelligence so well that a person can communicate with it and not be able to tell the difference between the AI and a real person. It's the AI equivalent of the Uncanny Valley - when you're talking to an AI and you have to treat it differently to a person, the result is a major loss in immersion.

    This is really difficult stuff to solve, and lots of scientists have been trying to build such programs since the days of the first computers and not yet succeeded, but I reckon games are actually in a great position to get a jump on some of the more formal efforts. That's because devs can control the subject of the conversation to a large extent, drastically reducing the number of questions that you can ask. They can really focus on covering HOW you will be asking questions, rather than what you are asking about, to hopefully minimise the number of times you ask a question and get that that "I just phrased it wrong for the damn machine" moment.

      Someone only has so many ways they can say “Take cover” or “move forward” so once they iron out issues with different dialects and recognising a vast variety of words I think it could definitely work in video games. Like you said. Video games won’t need to recognise the fact that in a historical epic game you’re talking about what songs are on your ipod, but as for this being a fully realised feature in games, I can see that happening in the next 5 years.

        True, the sort of commands to move forward and attack and such are pretty limited, especially if it's in the context of a military game where the devs can ensure it makes sense for you to be issueing orders and expecting them to be followed. True voice conversation with the NPCs in an RPG could be a bigger challenge.

        And what about localisation? Sure, there are only so many ways to give an order in English, but do the devs have to code for all languages or is that the job of (sometimes incompetent) local publishers? A game like this will be more expensive to sell around the world, for sure. I hope that doesn't stop them trying though...

          I think it’s just one step closer to virtual reality, let’s face it that’s the path that all games will go down eventually. That’s why the Wii sold so well IMO because it was the most significant step towards VR in mainstream gaming in a long time. People want that experience of being totally immersed in the video game world and seamless voice recognition is something developers will have to work out eventually.

          As for the other languages that is just something the developers will have t change in the different regions. I don’t actually think it will cost that much more money, maybe at first but after a while i predict it will be cheap to configure this for other languages, that’s just the price you have to pay with the changing times really, for example I don’t think it costs any more or less to make a good game now than it did 10 years ago (adjusting for inflation)

            Actually, it's much more expensive to make a good game these days - the higher investment price is why so many smaller studios have gone belly up this generation, as well as why we're seeing much less new IP. Developers have the new ideas, but it's very difficult to get investment money (especially since the GFC) if they can't point to sales figures for the game they're sequelising.

            And I think that the Wii certainly promised that kind of step towards a virtual reality, but it just as certainly hasn't delivered on it - the tech won't live up to our expectations for longer than you think... There's more to solving these problems than seeing a demand and deciding to fill it.

    As an adventure game enthusiast I've always wondered what games would be like if you could say more than just 'ASK ABOUT PLANT'.

    I suppose it's about finding a blend between what is logical to ask a character within the game world, and what isn't.

    It would also most likely be the end of big hollywood stars doing voice acting - because the amount of dialog that would be required is insane. Unless they figured out some (decent) way of synthesizing the human voice by then. I suppose they could use a similar system to what is being used in GPS systems, or Tom Baker being used as the voice of London Telecom.

Join the discussion!

Trending Stories Right Now