If you've ever tried to learn Dota 2 from scratch, you'll be able to appreciate how many different elements and decisions there are to juggle at any given moment. That's partially why it's so much fun watching the progress of OpenAI Five, a variant of the AI that beat pro players in 1v1 games at last year's International. The makers of the bot have announced that it's now beating regular teams of humans with abandon, including a 5500 MMR-rated group of humans and a flawless victory against a group of Valve employees.
The bot is produced by the same AI lab that's funded by Elon Musk, and while they're straying away from any AlphaGo comparisons for now the hardware — and results — are impressive.
While the 1v1 bot showcased at last year's International ran on 60,000 CPU cores (through Microsoft's Azure cloud platform), the team-based bot commands 128,000 preemptible cores through the Google Cloud Platform (GCP). On top of that, OpenAI Five has 256 Tesla P100 GPUs at its disposal, Nvidia GPUs that are designed specifically for AI supercomputing.
So far, the bot has been able to win a full series, sometimes flawlessly, against the following teams:
• Best OpenAI employee team: 2.5k MMR (46th percentile)
• Best audience players watching OpenAI employee match (including Blitz, who commentated the first OpenAI employee match): 4-6k MMR (90th-99th percentile), though they’d never played as a team.
• Valve employee team: 2.5-4k MMR (46th-90th percentile).
• Amateur team: 4.2k MMR (93rd percentile), trains as a team.
• Semi-pro team: 5.5k MMR (99th percentile), trains as a team.
There is a huge caveat, however: the bot has been playing with some pretty gnarly restrictions. For one, couriers for both sides are invulnerable. OpenAI can't deal with warding at this stage, a critical part of Dota 2 play at all levels of the game, and there's no scanning, summons, illusions, invisibility, Bottle, Divine Rapier, Boots of Travel, Roshan doesn't exist, no Quelling Blade, and both teams have to play with the same lineup: Mirror match of Necrophos, Sniper, Viper, Crystal Maiden, and Lich.
So, OpenAI still has a long way to go.
The researchers also made notes about the bot's playstyle and current limitations, which aren't too dissimilar from an average-level human:
- The bot averages around 150-170 actions per minutes. That's higher than most average players, and while it's not as high as professional Dota or Starcraft folk, it's high enough to be effective.
- OpenAI's average reaction time is 80ms, significantly faster than the regular player, but slightly slower than the 67ms reaction speed of the 1v1 OpenAI bot.
- The bot managed to learn creep blocking with no input at all. "One of our team members left a 2v2 model training when he went on vacation (proposing to his now wife!), intending to see how much longer training would boost performance. To his surprise, the model had learned to creep block without any special guidance or reward," the researchers said.
- OpenAI tends to give support heroes more gold and experience early on, instead of favouring the carries or damage dealers. "OpenAI Five’s prioritisation allows for its damage to peak sooner and push its advantage harder, winning team fights and capitalising on mistakes to ensure a fast win."
- The OpenAI team also echoes a strategy seen in higher level play, by sacrificing its own safe lane (bottom for Radiant, top if playing as Dire) to take control of the enemy's safe lane. It's a basic tactic that changes the location of the team fight to an area that's more difficult for the opposing team to defend — if you're sending three heroes in Radiant's safe lane as Dire, for instance, there's more room to gank and move around.
- The bot's teamwork essentially runs on a value called "team spirit", which tells each bot hero how much they should favour the benefit for themselves versus the benefit for the team.
The OpenAI researchers will be holding a stream on July 28 (international time), where their AI squad will square off against "top players". It sounds like it'll be a mix team of notable and pro players rather than a full team, although the eventual goal still hasn't changed: to beat a proper team using the restrictions outlined above (although the team says they're going to add Roshan and wards "as soon as possible").
For a more granular breakdown of how OpenAI works, including a browser-based demonstration of how it identifies and processes information, head to the OpenAI blog.