Facebook release Zero bot, open source code/weights

[Uberdude]

https://research.fb.com/facebook-open-s ... lf-opengo/

https://facebook.ai/elf

https://github.com/pytorch/ELF/releases

Based on DeepMind's work but running on their machine learning framework, trained on 2000 GPUs for 2 weeks, beat LeelaZero 200-0, and some top Korean pros 14-0 running on 1 GPU with 50s a move with unlimited time for the humans (no game records yet).

Discussion on LeelaZero github, including from facebook devs about running another test match against LeelaZero with better configuration. A welcome difference in attitude compared to AlphaZero chess vs stockfish.

Hopefully we can see this new facebook bot in some computer go competitions in future (the strong bots on computer go server named after American presidents were test versions of it).

[tchan001]

Waiting for the game records to be released

[Uberdude]

For now, here is a game from monroe which topped the CGOS vs Zen (but their best version wins 80% vs monroe):



And one vs "Leela-yao", no idea what that is but I guess some version of Leela Zero and was rated above Zen above.

[Uberdude]

Someone has managed to convert the Facebook ELF OpenGo weights to a LeelaZero compatible format, so a match against latest LeelaZero is now scheduled using the Leela Zero project's distributed clients, see it progress here, OpenGo's hash is b99ff...

Update: match cancelled as some problems with those converted weights (fb got destroyed).

[sleepyEDB]

From the article at https://research.fb.com/facebook-open-s ... lf-opengo/

Update May 3, 2018: An earlier version of this post stated a 200:0 record against LeelaZero. However, we discovered a flaw in our setup that resulted in LeelaZero using a tighter time budget than what we had expected. After correcting this issue and giving 50 seconds per move to both sides, we reran the series of games. The 198:2 record stated above is from this latest run. We thank the LeelaZero team for bringing this to our attention.

Even with allowing for equal time on both sides, LZ only won twice. Not to flaunt my naivete, but how is this even possible if LZ was the current reigning champ of AI? And what does this mean for the LZ team? Back to the drawing board? Pack it in and mine Dogecoin?


sleepy

[Uberdude]

Even with allowing for equal time on both sides, LZ only won twice. Not to flaunt my naivete, but how is this even possible if LZ was the current reigning champ of AI?

LZ is not the current reigning champ of AI, but is probably the strongest free and open source one (overtaken AQ now) so was easy for the facebook team to download and play against as a test (without having to arrange with the creators). AlphaGo is still widely seen as the reigning champ and most respected, but as it is retired and doesn't play other bots it won't be clear if/when others surpass it. FineArt would probably be #2 until recently, maybe BensonDarr/PhoenixGo has just overtaken it.

[sleepyEDB]

Thanks for the clarification! Pretty awesome how the world of AI Go has exploded recently.


sleepy

[Uberdude]

12 of the 14 pro games are released: https://github.com/pytorch/ELF/releases ... 19-2018-04 (2 withheld, I guess too embarrassing loss!). Names anonymized, but at least 2 each from Kim Jiseok, Choi Cheolhan, Park Yeonghun, Shin Jinseo, maybe we can recognise their styles?

Game 3 (first 2 weaker bot):


Game 4:


Game 5:

[EdLee]

Could someone unzip them so everyone can view them here ?

zipped

[Uberdude]

Someone has uploaded the games vs Leela Zero to OGS at https://online-go.com/library/433376/2431. 980 wins for elf, 18 for LZ.

Move 13 here is a fun wedge. Reminds me of this one from AlphaGo teaching tool.

Screenshot_2018-05-04-06-44-14.png (435.96 KiB) Viewed 15536 times

An LZ win because Elf misread a ladder: https://online-go.com/game/12727726.

Game 4 vs pro Elf also ran out a ladder once that it was captured in. Maybe you could argue it was a probe to make a ladder breaker more effective, but it's pretty costly and the large swings in black value metric suggest it is confused not a genius move. The pro captured the 2 stones with tortoise shell and elf thought it was behind, but human was soon behind again. Facebook dev confirmed they have ladder troubles but didn't add any special fixes to keep true to Zero approach. So although LZ has ladder problems in some situations it's better than Elf, but Elf is overall much stronger. So we don't need to bother teaching beginners about ladders if instead they can somehow develop superhuman intuition and judgement!

[Bill Spight]

Facebook dev confirmed they have ladder troubles but didn't add any special fixes to keep true to Zero approach. So although LZ has ladder problems in some situations it's better than Elf, but Elf is overall much stronger. So we don't need to bother teaching beginners about ladders if instead they can somehow develop superhuman intuition and judgement!

Oh, it's good to teach raw beginners about ladders. It quickly gives them a feeling of power. https://themathlab.com/writings/short%2 ... eeling.htm

[ez4u]

How many training games to reach this point?

[Uberdude]

How many training games to reach this point?

I saw 20 million mentioned (LZ is at 7 million), but can't find official source.

Btw: I only just realised you can move and zoom the graph on http://zero.sjeng.org/ with click/drag and mouse wheel, makes it much more readable!

[Mike Novack]

probably the strongest free and open source one (overtaken AQ now) so was easy for the facebook team to download and play against as a test (without having to arrange with the creators). ....

Excuse me please, but open source SOFTWARE does not imply all DATA used with the program would be open.

A neural net is implemented by a program, but the neural net must then be TRAINED to evaluate the desired function, which process accumulates DATA to be used with the program to perform THIS task well (the same neural net could have been trained to do something else).

Given the data, a clone of the neural net need not be trained from scratch, that data just provided to it. But I see no reason to think that the DATA would necessarily be public domain. Not any more than if I write a book using LibreOffice Write as the word processor and save it to the .odf format it thereby become not copyright.

Since training one of these powerful neural net programs requires significant resources, I see no reason why one of these PROGRAMS couldn't be open source while somebody providing the data form training did so for a price. IMO, no violation.

[Uberdude]

Excuse me please, but open source SOFTWARE does not imply all DATA used with the program would be open.

Did I say it was? I said "free and open source", the order was not an accident.