This is just so tough for robots. The balance, ball control, collision recovery, opponent modelling, locomotion, and strategy all have to run at once.

here the humanoid is producing joint-angle commands 50 times a second, not choosing from a neat discrete action space.

The training setup is even more interesting, it basically spent 140 simulated years getting cooked by itself. it played millions of accelerated virtual soccer games, repeatedly competing against saved earlier versions of its own policy and learning from which actions helped it score.