Page 1 of 2

DeepMind give AlphaGo strength graph, and it will shock you!

Posted: Thu Mar 24, 2016 11:11 pm
by pookpooi
Image

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Thu Mar 24, 2016 11:16 pm
by RobertJasiek
They continue the same mistake of setting amateur 9d below professional 1p. If AlphaGo v.18 were much stronger than human 9p, it would not have lost any game against Lee Sedol.

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Thu Mar 24, 2016 11:22 pm
by pookpooi
RobertJasiek wrote:They continue the same mistake of setting amateur 9d below professional 1p. If AlphaGo v.18 were much stronger than human 9p, it would not have lost any game against Lee Sedol.

Agree, this is my version of the graph, I don't claim it's 100% accurate though cause professional dan work in different way than kyu - amateur dan.

Image

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 12:17 am
by Uberdude
Gosh, beating Nature/Fan Hui AlphaGo with 4 stones is impressive, but I reserve my judgement about the meaning of that in actual strength given bots may have trouble playing with/against handicap. A better test would be could/did it give 4 stones to Fan Hui? If AlphaGo v18 were 4000 Elo and Lee Sedol 3500 then it should win a single game 95% of the time and the match 5-0 with 78% chance so it doesn't look 4000 let alone 4500. Having said that it only lost game 4 due to one move far below the usual high standard of its moves (and subsequent monte carlo nonsense which can sort of be dismissed, though not entirely as the ability to play sensibly and well when behind is important, though perhaps not so much if AlphaGo fixes that bug and is then never behind in any game against a human ever!) so if they could fix that bug/blunder/weakness then such high Elos might be plausible. However, I expect humans to improve how they do against AlphaGo with practice if Deepmind make it more widely available, though of course AlphaGo will also likely keep improving. We can already see than in just 5 games Lee Sedol was able to learn and adapt how to play against AlphaGo with some success, whilst AlphaGo v13 needed 60 million positions and games to get to its level (plus scores of clever developers/researchers) and AlphaGo v18 probably used far more, so humans are still pretty good at this learning thing.

P.S. ajahuang was on KGS yesterday testing the gtp interface for an AlphaGoBot to play on KGS, subject to Google's permission. Said it would likely first play in some arranged matches against some pros / 9ds before (if) being more widely available. Exciting!

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 12:35 am
by pookpooi
Uberdude wrote:P.S. ajahuang was on KGS yesterday testing the gtp interface for an AlphaGoBot to play on KGS, subject to Google's permission. Said it would likely first play in some arranged matches against some pros / 9ds before (if) being more widely available. Exciting!

I was also there, very exciting time even it's a false positive

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 8:32 am
by uPWarrior
I think that graph is believable is you treat "9d" as "average 9d". In that case, Lee Sedol himself would not be 3500 but probably 3800+.

Remember that these scales are arbitrary anyway.

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 9:55 am
by yoyoma
Well I can certainly see why they were "quietly confident" going into the Lee Sedol match!

BTW it played 7 games on Tygem today and won all 7.

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 10:10 am
by Krama
yoyoma wrote:Well I can certainly see why they were "quietly confident" going into the Lee Sedol match!

BTW it played 7 games on Tygem today and won all 7.


What nickname is it on tygem?

The "deepmind" account actually has something like 40 wins 20 loses.

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 10:36 am
by yoyoma
Krama wrote:
yoyoma wrote:Well I can certainly see why they were "quietly confident" going into the Lee Sedol match!

BTW it played 7 games on Tygem today and won all 7.


What nickname is it on tygem?

The "deepmind" account actually has something like 40 wins 20 loses.


Yes the deepmind account. I just meant it's most recent 7 games.

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 10:40 am
by Krama
yoyoma wrote:
Krama wrote:
yoyoma wrote:Well I can certainly see why they were "quietly confident" going into the Lee Sedol match!

BTW it played 7 games on Tygem today and won all 7.


What nickname is it on tygem?

The "deepmind" account actually has something like 40 wins 20 loses.


Yes the deepmind account. I just meant it's most recent 7 games.


But in last 7 games it won 2 and lost 5 or was it won 3 lost 4

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 10:51 am
by yoyoma
I'm at work now so I can't check with the client, I'm going off this reddit post https://www.reddit.com/r/baduk/comments ... ns_so_far/ which has this image linked: http://postimg.org/image/rvxwwof93/

There is has seven red circles in a row at the end, and below you can see those games played on March 25.

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 11:01 am
by sorin
yoyoma wrote:I'm at work now so I can't check with the client, I'm going off this reddit post https://www.reddit.com/r/baduk/comments ... ns_so_far/ which has this image linked: http://postimg.org/image/rvxwwof93/

There is has seven red circles in a row at the end, and below you can see those games played on March 25.


Given that Aja Huang said he didn't get Google approval to play on KGS yet, I doubt that the deepmind account on Tygem is AlphaGo, at least it won't be the latest version.
Why would they give approval for Tygem, but not for KGS?

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 11:23 am
by Kirby
sorin wrote:Given that Aja Huang said he didn't get Google approval to play on KGS yet, I doubt that the deepmind account on Tygem is AlphaGo, at least it won't be the latest version.
Why would they give approval for Tygem, but not for KGS?


Didn't Aja Huang also say that 'deepmind' on Tygem was his account [1]? Is it possible that 'deepmind' on Tygem is not "pure AlphaGo", but rather Aja Huang + AlphaGo - maybe experimenting a bit, or just playing around?

Just a theory, but it could explain things.

---
[1] https://www.reddit.com/r/baduk/comments ... _on_tygem/
KillerDucky wrote:It's Aja Huang. He said so himself in the youtube stream, and denied that AlphaGo played on the account.

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 11:30 am
by yoyoma
They aren't exactly straightforward about it. Yes earlier Aja Hwang had said that deepmind was his own personal account. But later there was a Korean article with a picture of Demis Hassibis and the CEO of Tygem. The title was something like "Thanks Tygem!". Even in that article it's sorta like Demis is saying it's Aja's account, but he sorta "indirectly" confirmed it. It's like they are saying it's Aja's personal account, but they are no longer directly denying that he used it for AlphaGo anymore.

I want to be careful about this too but at this point is just seems overwhelmingly likely that AlphaGo is playing on that account, just for some reason Deepmind just doesn't want to publicize this. That's my take at least.

Re: DeepMind give AlphaGo strength graph, and it will shock

Posted: Fri Mar 25, 2016 11:39 am
by Kirby