Smerity

... the real focus should be on how these bots are better team players than humans. Humans pride themselves on this. Every darn Disney film focuses on this. Society is based on this. Yet these "dumb" bots used self play and coarse objectives to learn better team work than humans.

For me this registered as a distinctly different beast as to what ML/AI is traditionally superhuman at. I was a heavy DotA player (started WC3 era and have played 830 hours of Dota 2) and these bot games are best described "restricted gameplay but with a sophisticated team plot".

I honestly believe the "restricted gameplay" part is irrelevant as I see no major obstacles (other than immense engineering which @OpenAI are well equipped) that would prevent near bot perfection. It's the team work angle that is distinctly unique and the new addition for me.

The bots knew when to take the fall to set their team up, when to waive their local reward for the team's global reward, and had no sense of a hero complex (I'm going to attack as I have a time limited Double Damage rune even though my team isn't ready!).

Humans can't learn this properly. At the amateur level, amateurs can't rely on their team mates. At the professional level, there are only so many hours in a day to perfect co-operation and players suffer from rage quits / going full tilt / ... even at the highest tier of sports.

This is an iterated prisoner's dilemma but humans will never spend enough time in this jail to work out how to optimally play the game through team work and collaboration. Conversely, we can happily lock our friendly @OpenAI Five team into hundreds of years of gameplay per day.

Why is this exciting for me? This has implications far beyond DotA. These bots never see traditional human strategies, they only play the game as the rules and objectives dictate it should be played. If there's a positive sum way to play a "human" zero sum game, it'll find it.

I would argue humans have forced themselves into a corner in many places due to our limits - slowly evolving rules that have high overheard to set up and poor interpretability. Ex: Loop holes are ~when laws aren't updated to account for changed context or adversarial intentions.

These bots weren't perfect at most of the skills humans optimize for. In a million ways they appear to have suboptimal behaviour to humans (getting gold (last hitting, jungling, farming, ...), strategy (creep blocking, pushing towers, ...), ...) - but that really didn't matter.

The bots, working as a team, saw the objective pipeline and optimized for the parts that mattered. They don't get all the gold+items if it means their team members are underpowered. They don't get the ultra-quadruple-mega-kill for glory when it'll make your team lose the game.

We could see a future where many of society's patchwork forced complexities fall away as these automated systems reveal the premature optimizations we've tricked ourselves into and the obvious blindspots we've missed that until now are only visible with hindsight.