It’s always been a question for strategy games: if machine learning monsters like AlphaGo exist, how long will it be before RTS games have an AI so good that it defines how humans play the game? Age of Empires 4 has been thinking about that, as it turns out.
In a new interview with World’s Edge franchise creative director Adam Isgreen and Relic game director Quinn Duffy, I asked the pair about an age-old annoyance: AI. When you up the difficulty to get a real challenge, the dev answer has simply been to let the AI cheat — or give them absurd buffs to resources and other tricks to paper over the cracks of the AI not being smart enough.
Age of Empires 4, gracefully, won’t resort to either of those old-school methods. But Isgreen and Duffy both revealed that Age of Empires 4 has been toying around the idea of a Merciless AI. It’s inspired by some of the work that went into Killer Instinct’s Shadow AI, which learnt from the moves players made.
Age 4 already used machine learning to train its AI, but after the game launches this October, the developers will probably add this harder difficulty mode that will just continually learn from whatever the current meta is. “We use machine learning in training the AI right now, but we want to take that even further. Down the road, not at launch, we’ll probably look into having a merciless AI that keeps learning the more people play against it, to the point that it’ll be unbeatable,” Isgreen said. “But we’re OK with that, because if you opt into that difficulty, if you want to opt into making the AI better at beating you, go right ahead.”
“We can use that in different aspects in terms of how we cut it up for the different difficulties. Like, we did a lot of that in Killer Instinct 2, we had AIs that would learn how to play like players down to even the taunts, you know for a game like KI, it’s like when did [the player] teabag, how did they move around while they were waiting for a player to get up,” Isgreen added. “I think we can start applying a lot of that to a real time strategy game. But the AI it does things that players do, we try to imitate and add in strategies that players use.”
For lower difficulties, however, Isgreen emphasised that the AI is tuned to create exciting matches for players. He recalled a GDC talk on real-time strategy balancing years ago that was run by industry heavyweights including Chris Taylor. “I was like: AI should only cheat to make the game experience more enjoyable for the player, and I got booed by people,” he said.
“The guy from Total Annihilation was there — Chris Taylor was running the thing but one of the other guys from Total Annihilation was like, ‘AI should be perfect, it should never cheat’, and everybody was applauding.”
“And I was like no, you don’t get it: we make entertainment. I don’t want AI to cheat in terms of getting buffs and bonuses — that isn’t satisfying for me. But when difficulties are lower, it’s important to realise that drama is almost more important than perfect balance. You want highs and lows emotionally. An AI that can do that and generate a fun fight that people walk away from [saying], ‘Wow that was great, I just pulled it off’ — even if that was fudged a little bit, that entertainment is so valuable. But as you turn up the difficulty, that’s when all of that stuff diminishes.”
Right now, the AI learns by playing against itself, throwing unit combinations against each other, trialling different scenarios and making different battle calculations. This trial-by-AI combat actually ended up discovering a major bug in the game, one that the developers didn’t spot themselves — until they saw the AI randomly fleeing from battles. The strategies and tactics that players invent for Age 4 will be re-fed into the game’s AI post launch, which will help in the balancing process especially in the post-launch years as more civilisations and factions are added.
“The one with pikemen where they’d age up to Age Three, and then they’d run away? Basically it turned out there was a bug in the data that when you aged up to Age Three, the pikemen’s attack values dropped to a tenth of what they were supposed to be,” Isgreen recalled. “It accidentally got in there and the AI found out first, and the AI was like, ‘I’m out of here,’, and it’d turn around and run away with all its units. It’s because it knew it couldn’t win — it was like, well the values are this, I can’t win — so it’d run the entire army [away], and you’d be like, ‘What are you doing? You could have won that!’”
“It’s a great tool for helping us find crazy stuff like that. You’re just like: What is its decision making–oh, because it ran more iterations and caught this bug that no-one had caught in mainline yet, because the build went live and the AI started running through all of this stuff automatically.”
As for people who don’t want an AI that’s impossible to beat? Well, Age 4 has something there too. Isgreen and Quinn explained that the historical RTS will ship with a Story Mode that effectively removes almost all of the loss conditions from the singleplayer campaign. It’s one of the additions designed to make Age 4 more accessible, with the devs understanding that Xbox Game Pass will result in a much wider range of players giving AOE 4 a go than they might have otherwise.
“It’s almost like a no-lose to play through the story, where you can just keep trying and build your forces. You can just play it, and there’s no real penalty. I know Quinn and I have both made games where it’s you’re dead, you’re over, you lose all your forces, it’s over, start over. Maybe more than one mission even, like in the case of something like Homeworld,” Isgreen said. “That’s great and I’m glad we can still offer things like difficulty and really hard difficulties for people who want that, but it’s also awesome to have something like Story Mode.”
Age of Empires 4 launches on PC and Xbox Game Pass for PC on October 28.
Leave a Reply