https://research.fb.com/facebook-open-s ... lf-opengo/

https://facebook.ai/elf

https://github.com/pytorch/ELF/releases

Based on DeepMind's work but running on their machine learning framework, trained on 2000 GPUs for 2 weeks, beat LeelaZero 200-0, and some top Korean pros 14-0 running on 1 GPU with 50s a move with unlimited time for the humans (no game records yet).

Discussion on LeelaZero github, including from facebook devs about running another test match against LeelaZero with better configuration. A welcome difference in attitude compared to AlphaZero chess vs stockfish.

Hopefully we can see this new facebook bot in some computer go competitions in future (the strong bots on computer go server named after American presidents were test versions of it).

Waiting for the game records to be released

_________________
http://tchan001.wordpress.com
A blog on Asian go books, go sightings, and interesting tidbits
Go is such a beautiful game.

For now, here is a game from monroe which topped the CGOS vs Zen (but their best version wins 80% vs monroe):



And one vs "Leela-yao", no idea what that is but I guess some version of Leela Zero and was rated above Zen above.

Someone has managed to convert the Facebook ELF OpenGo weights to a LeelaZero compatible format, so a match against latest LeelaZero is now scheduled using the Leela Zero project's distributed clients, see it progress here, OpenGo's hash is b99ff...

Update: match cancelled as some problems with those converted weights (fb got destroyed).

From the article at https://research.fb.com/facebook-open-s ... lf-opengo/


Even with allowing for equal time on both sides, LZ only won twice. Not to flaunt my naivete, but how is this even possible if LZ was the current reigning champ of AI? And what does this mean for the LZ team? Back to the drawing board? Pack it in and mine Dogecoin?


sleepy

_________________
Playing since: March 2018
The road of life is rocky, and you may stumble too. So while you point your finger, someone else is judging you. --Bob Marley

LZ is not the current reigning champ of AI, but is probably the strongest free and open source one (overtaken AQ now) so was easy for the facebook team to download and play against as a test (without having to arrange with the creators). AlphaGo is still widely seen as the reigning champ and most respected, but as it is retired and doesn't play other bots it won't be clear if/when others surpass it. FineArt would probably be #2 until recently, maybe BensonDarr/PhoenixGo has just overtaken it.

Thanks for the clarification! Pretty awesome how the world of AI Go has exploded recently.


sleepy

_________________
Playing since: March 2018
The road of life is rocky, and you may stumble too. So while you point your finger, someone else is judging you. --Bob Marley

12 of the 14 pro games are released: https://github.com/pytorch/ELF/releases ... 19-2018-04 (2 withheld, I guess too embarrassing loss!). Names anonymized, but at least 2 each from Kim Jiseok, Choi Cheolhan, Park Yeonghun, Shin Jinseo, maybe we can recognise their styles?

Game 3 (first 2 weaker bot):


Game 4:


Game 5:

Could someone unzip them so everyone can view them here ?

zipped

Someone has uploaded the games vs Leela Zero to OGS at https://online-go.com/library/433376/2431. 980 wins for elf, 18 for LZ.

Move 13 here is a fun wedge. Reminds me of this one from AlphaGo teaching tool.


An LZ win because Elf misread a ladder: https://online-go.com/game/12727726.

Game 4 vs pro Elf also ran out a ladder once that it was captured in. Maybe you could argue it was a probe to make a ladder breaker more effective, but it's pretty costly and the large swings in black value metric suggest it is confused not a genius move. The pro captured the 2 stones with tortoise shell and elf thought it was behind, but human was soon behind again. Facebook dev confirmed they have ladder troubles but didn't add any special fixes to keep true to Zero approach. So although LZ has ladder problems in some situations it's better than Elf, but Elf is overall much stronger. So we don't need to bother teaching beginners about ladders if instead they can somehow develop superhuman intuition and judgement!

Oh, it's good to teach raw beginners about ladders. It quickly gives them a feeling of power. https://themathlab.com/writings/short%2 ... eeling.htm

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

How many training games to reach this point?

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21

I saw 20 million mentioned (LZ is at 7 million), but can't find official source.

Btw: I only just realised you can move and zoom the graph on http://zero.sjeng.org/ with click/drag and mouse wheel, makes it much more readable!

Excuse me please, but open source SOFTWARE does not imply all DATA used with the program would be open.

A neural net is implemented by a program, but the neural net must then be TRAINED to evaluate the desired function, which process accumulates DATA to be used with the program to perform THIS task well (the same neural net could have been trained to do something else).

Given the data, a clone of the neural net need not be trained from scratch, that data just provided to it. But I see no reason to think that the DATA would necessarily be public domain. Not any more than if I write a book using LibreOffice Write as the word processor and save it to the .odf format it thereby become not copyright.

Since training one of these powerful neural net programs requires significant resources, I see no reason why one of these PROGRAMS couldn't be open source while somebody providing the data form training did so for a price. IMO, no violation.

Did I say it was? I said "free and open source", the order was not an accident.

They are similar indeed, the AlphaGo wedge is more "contextual" since it has more stones nearby but still very surprising.
Either way - it is one thing that AI shows us more ways to think about the game "strategically" (shoulder-hits on high lines, or early 3x3 invasions, or seeing something weak that used to be thought of as strong before by humans) - but innovating on a micro-level such as these wedges (assuming here that FB Elf's wedge is a good move too) is really shocking to me...

_________________
Sorin - 361points.com

Err... or maybe we should respect their wish and let them be anonymous

_________________
Sorin - 361points.com

Old bridge joke:

Bridge player, having butchered a hand: How should I have played that hand?

Partner: Under an assumed name.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Looked through half of the pro vs. elf games. Facebook's bot is just frighteningly powerful (to my untrained 1k eyes at least). If you trust the "black value" evaluation (I assume black value goes from -1 to 1 meaning almost certain loss to almost certain win) in the comments, it seems most games are nearly lost after 30-50 moves.

I also had a look at one of the games with Lizzie, and Leela doesn't give as strong edge (the scales are different of course) to FB bot as quickly, taking maybe 10-20 moves extra for pro chances to drop below 35 %, but still it's showing the bot is crushing most games by move 50, with pro odds usually around 25 %.

I'm hoping the Leela Zero project will gain from Facebook's contribution. Not sure if they should "borrow" the network weights, use ELF OpenGo to help in the training, or just look for ways to improve LZ's training indirectly, though.

My comments on gh:

So… overall ELF is at least >500+ Elo stronger than current best LZ network and possibily up to 800 elo stronker. But it appears to have some latent issues such as being even more prone to ladder fallability and other certain fragilities at less than superhigh playouts, which ironically are things that aren’t as prelevant in the current LZ net arch. By using the approach of hybrid mix 50% LZ/ 50% ELF self-games and continuing to train the LZ net but using ELF as a part-time strength-gainer games-generator (at least until which point LZ catches up to ELF or even surpasses it) may patch the current weak spots in both net archs and set up a framework that is repeatable into the future, such as if AGZ weights get released or facebook teams releases a (new) and even stronger ELF OpenGo network sometime in the future, etc this can be repeated again. And it gives the LZ project the chance to figure out how to resolve two of the biggest issues of superhuman Go AI, that is ladders, large group deaths, and things like high handicap and variable komi and whatnots. Since the current 15b still has enough capacity left, the adding of ELF games will allow LZ to catch up enough to give the devs a way to figure out how to lower gating or remove gating etc. For sure the continuation of “things as usual” for the LZ project is no longer an option, if for no other reason than the strength gap being too great for volunteer clients to still be willing to go the business as usual path, and as gcp stated, its not an option to train the ELF network directly either. (not to mention that switching directly to the ELF network would be in essence killing off the Leela Zero itself) So looks like the roadmap going forward is to train LZ hybridized with ELF, so it seems like there is a way to make good on the fb post about ELF helping project like Leela Zero after all!

1000 games of Facebook OpenGo vs Leela Zero on Nvidia Tesla V100 at 80,000 playouts per move.
(acuatlly its 998 games and Leela Zero had a 18:998 winrate, which means the ELF is ~700 elo stronger)

What is not in debate is this, in terms of open source Go, we have well passed without a shadow of any doubt into “superhuman” territory. ELF is estimated to be 4400 real elo on max hardware while top players are about 3700.