MuZero beats AlphaZero

[sorin]

DeepMind published a papar about MuZero, a new approach to learning, which they evaluated on several board games and Atari video games: https://arxiv.org/pdf/1911.08265.pdf

From what I understand from a quick browse of the paper, the innovative part compared to AlphaZero type of approach is that MuZero doesn't "know" the rules in advance, therefore is a more general learning algorithm, which can be used in more open-ended domains.

They tested it against AlphaZero for go and MuZero won, this is an exact quotation:

"In Go, MuZero slightly exceeded the performance of AlphaZero, despite using less computation per node in the search tree (16 residual blocks per evaluation in MuZero compared to 20 blocks in AlphaZero)"

Very interesting news, I hope they will publish some game records too!

[Bill Spight]

DeepMind published a papar about MuZero, a new approach to learning, which they evaluated on several board games and Atari video games: https://arxiv.org/pdf/1911.08265.pdf

From what I understand from a quick browse of the paper, the innovative part compared to AlphaZero type of approach is that MuZero doesn't "know" the rules in advance, therefore is a more general learning algorithm, which can be used in more open-ended domains.

Actually, learning the rules is not innovative.

Very interesting news, I hope they will publish some game records too!

Very interesting, indeed.

[EdLee]

Hi sorin, thanks.

Nice to see the classic Atari games.
Mr. Aja Huang (relayer in AlphaGo-LSD match) not listed in this paper.

Too bad the "casual" readers of these papers would have no idea of the etymology of Atari and its connection to Go. (Unless they accidentally wikipedia it up.)

[Uberdude]

I don't think we ever got game records of AlphaZero for Go did we? Also AlphaZero was only stronger than the 20 block version of AlphaGo Zero (which was between AG Lee and AG Master), not the 40 block version, see viewtopic.php?p=239589#p239589. So these games would be interesting to see from a "what style does this new bot from an independent training run of self discovery of rules have" perspective but will likely be weaker than AG0 40b.

[Uberdude]

Now the real challenge for MuZero is can it play Mao?

[Kirby]

Next step: AI to decide to play go when it doesn’t know the rules, and also doesn’t know it can use board or stones.

[Yakago]

Yes, we should eagerly anticipate the day that the AI learns Go out of sheer interest

[Gomoto]

Mu Zero, can you tell us more about Go?

I don't care. I just win.

[Bill Spight]

Mu Zero, can you tell us more about Go?

Mu.

[MikeKyle]

Now the real challenge for MuZero is can it play Mao?

I played Mao in college and genuinely thought it was just made up by a small group of bored Yorkshiremen.
I guess it's your point, but Mau is kind of the only game Muzero seems to play.

[pookpooi]

Surprised not to see Aja Huang in this, but he appears in AlphaStar paper.

Anyway, I just love the name, Zero is nothing, and Mu is also nothing in Japanese and Korean (Wu in Chinese), something like that.

I'm wondering if they manage to also play StarCraft at AlphaStar level in their next project, the AI name could be MuZeroNova, Nova is 'new' in Latin and also 'star explosion' in astronomical term. Though I might consider adding another 'nothing' in the name if the AI manage to win even without being tasked to win/winning reward.

[jlt]

For the next name of a Deepmind product, I suggest EpsilonZero (vacuum permittivity).

[EdLee]

μ

[Bill Spight]

無

[sorin]

DeepMind published a papar about MuZero, a new approach to learning, which they evaluated on several board games and Atari video games: https://arxiv.org/pdf/1911.08265.pdf

From what I understand from a quick browse of the paper, the innovative part compared to AlphaZero type of approach is that MuZero doesn't "know" the rules in advance, therefore is a more general learning algorithm, which can be used in more open-ended domains.

Actually, learning the rules is not innovative.

Right. And this is not about "learning the rules", but learning to act in an environment where there are no clear rules.

They used it for go as well just as proof-of-concept I guess, but go (or board games in general) is not the main target for this family of algorithms. Nevertheless, I think it's very cool, I am mostly interested about the learning trajectory for go, whether it ended up learning in a different way, or did it converge to AlphaZero style, etc.