Page 1 of 2
How strong is Alphago (blitz mode) ?
Posted: Thu Jan 05, 2017 2:09 am
by Vargo
In a 60 games match, to have a decent chance at winning 60-0 (let's say a 50% chance to achieve this feat), you need to have a 99% probability of winning against each opponent.
So, very roughly, it would seem Alphago has something like a 99% prob of winning a blitz game against a top pro...
How does it translate in terms of Elo (several hundreds ??) or handicap stones (is Alphago blitz 2 stones stronger than a top pro ???)
What do you think ?
Re: How strong is Alphago (blitz mode) ?
Posted: Thu Jan 05, 2017 3:27 am
by HermanHiddema
According to the Elo rating formula, the winning chances a player (A) against another player (B) are:
________1________
1 + 10(dR/400)
Where dR is the rating difference between the players (RA - RB)
This means the winning chances of a player increase tenfold for every 400 rating points. So at 400 difference you have roughly 90% chance, at 800 you have roughly 99% chance, at 1200 you have roughly 99.9% chance, and you can keep adding 9's behind that for every 400 rating points more.
So to have a 99% chance, you need a rating difference of roughly 800.
Re: How strong is Alphago (blitz mode) ?
Posted: Thu Jan 05, 2017 3:42 am
by Uberdude
Measuring strength difference in handicap stones versus winning percentages is rather tricky: I would expect a top pro taking 2 stones handicap to beat another top pro the vast majority of the time (maybe 95%+) whilst I would beat another European 4d taking 2 stones maybe 75% of the time, and for 14 kyus even less (maybe 60%). I think the EGF's modified Elo system does take something like this into account with parameters in the basic Elo formula that change depend on players' rating. Also on OGS I played 32 games against an AGA 5d and won all but the first game (when I was a bit weaker, I improved from UK 2d to 4d over those years which is probably something like AGA 3.5d to 6d), but there were quite a few close games. So whilst I had a near perfect win-rate I wouldn't say I could give him 2 stones and be confident to win. Anyway, although Master also got some close wins maybe it was always in control and didn't play its best. We'd really need to see some 2 stone games to see if it could win giving handicap. But for sure it is a lot stronger than humans in blitz, but I won't define "a lot"
.
P.S Zhou Junxun tried mirror Go which was interesting.
Re: How strong is Alphago (blitz mode) ?
Posted: Thu Jan 05, 2017 4:12 am
by pookpooi
Facebook user Trent D Carroll post this on Go (Weiqi) Players on Facebook page
Look like handicap stones is a better measurement than Elo in this situation.
Re: How strong is Alphago (blitz mode) ?
Posted: Thu Jan 05, 2017 6:58 am
by Mike Novack
Does anybody know how much "iron" has been made available to this bot?
While the absolute strength of play is probably very non-linear with increased crunch power, the amount of time required per move IS inversely linear.
What I am saying is that what we are seeing MIGHT not be an increase in the strength of the program. Let's say that the program is strong enough that when given a certain amount of hardware power it can play evenly against these humans (win 50%) at a time control of 1 minute per move. Give it hardware with six times the crunch power and make the time control 10 seconds per move, and while the program isn't playing any better, the human opponents are playing worse. So the program would now be winning more than 50%, maybe a lot more.
Re: How strong is Alphago (blitz mode) ?
Posted: Thu Jan 05, 2017 9:18 am
by Bill Spight
Uberdude wrote:Measuring strength difference in handicap stones versus winning percentages is rather tricky: I would expect a top pro taking 2 stones handicap to beat another top pro the vast majority of the time (maybe 95%+) whilst I would beat another European 4d taking 2 stones maybe 75% of the time, and for 14 kyus even less (maybe 60%).
The two approaches are different. You could also say that measuring strength difference by win rate versus handicap stones is rather tricky. One advantage of using handicap stones is that they add linearly fairly well to make reasonably even games. I once gave a 41 stone handicap (!) based upon rank difference. I really was not expecting a close game, but I won by only 10 points.
Re: How strong is Alphago (blitz mode) ?
Posted: Thu Jan 05, 2017 9:44 am
by skydyr
Bill Spight wrote:Uberdude wrote:Measuring strength difference in handicap stones versus winning percentages is rather tricky: I would expect a top pro taking 2 stones handicap to beat another top pro the vast majority of the time (maybe 95%+) whilst I would beat another European 4d taking 2 stones maybe 75% of the time, and for 14 kyus even less (maybe 60%).
The two approaches are different. You could also say that measuring strength difference by win rate versus handicap stones is rather tricky. One advantage of using handicap stones is that they add linearly fairly well to make reasonably even games. I once gave a 41 stone handicap (!) based upon rank difference. I really was not expecting a close game, but I won by only 10 points.
Out of curiosity, what was the presumed rank of your opponent?
Re: How strong is Alphago (blitz mode) ?
Posted: Thu Jan 05, 2017 9:50 am
by Uberdude
Bill Spight wrote:I once gave a 41 stone handicap (!) based upon rank difference. I really was not expecting a close game, but I won by only 10 points.
Just a few days ago Sai and I lost the final of the London Open Pair Go by half a point against a 10kyu pair giving them 11 stones. I made the last losing play of prematurely defending inside our territory to prevent an upcoming snapback when I could have connected the last ko
. In the lightning final Oh Chi Min 7d gave 21 stones and won against his girlfriend Zoe!
Re: How strong is Alphago (blitz mode) ?
Posted: Thu Jan 05, 2017 10:56 am
by Bill Spight
skydyr wrote:Bill Spight wrote:Uberdude wrote:Measuring strength difference in handicap stones versus winning percentages is rather tricky: I would expect a top pro taking 2 stones handicap to beat another top pro the vast majority of the time (maybe 95%+) whilst I would beat another European 4d taking 2 stones maybe 75% of the time, and for 14 kyus even less (maybe 60%).
The two approaches are different. You could also say that measuring strength difference by win rate versus handicap stones is rather tricky. One advantage of using handicap stones is that they add linearly fairly well to make reasonably even games. I once gave a 41 stone handicap (!) based upon rank difference. I really was not expecting a close game, but I won by only 10 points.
Out of curiosity, what was the presumed rank of your opponent?
Weak 35 kyu. He was not a rank beginner, but had not played for many years, and had been a DDK when he had played before.
Re: How strong is Alphago (blitz mode) ?
Posted: Thu Jan 05, 2017 11:28 pm
by Anzu
On a good computer, only one second is necessary for a chess computer to win against me :D
Re: How strong is Alphago (blitz mode) ?
Posted: Sat Jan 07, 2017 4:16 am
by Vargo
Thank you for these answers.
P.S Zhou Junxun tried mirror Go which was interesting.
Yes, interesting, thank you.
I think it is game F51 , but Zhou Junxun quit mirroring at move 71, does someone know why ?
For the strength of Alphago-Blitz, couldn't we at least have a lower bound estimate ?
What could be the minimum Elo of a player winning 99% of his games against 3500-3600 Elo opponents ?
Re: How strong is Alphago (blitz mode) ?
Posted: Sat Jan 07, 2017 4:32 am
by Uberdude
Vargo wrote:
I think it is game F51 , but Zhou Junxun quit mirroring at move 71, does someone know why ?
You can't keep mirroring as black forever, or else white wins by komi. So you only keep mirroring whilst you think your tengen stone is worth more than komi, so presumably he thought something like if they kept mirroring the centre would become dame so white wins. Or white's move was a bit slack so he didn't want to copy it. Mirror go is actually very hard as you have to continually make these judgements which in 30 seconds is tough. I actually think mirror Go as white is more interesting, and that's what Fujisawa Hosai did. There are some nice articles in GoGoD's New in Go about that.
Vargo wrote:
For the strength of Alphago-Blitz, couldn't we at least have a lower bound estimate ?
What could be the minimum Elo of a player winning 99% of his games against 3500-3600 Elo opponents ?
But we don't have Elo ratings for KeJie-Blitz, ParkJunghwan-Blitz etc.
Re: How strong is Alphago (blitz mode) ?
Posted: Sat Jan 07, 2017 6:59 am
by pookpooi
This page is quite interesting
49 of 60 finised games, we know the identity of opponent, add one game that Master didn't win to give a lower boundary to calculate (that disconnect Choi Cheolhan's game). The average of of 50 game's pro goratings.org's rating is 3447, the winrate is 98% (49/50) giving Master's elo as 4123. And yes, I know that many people will quote this method as completely mathematically wrong. But I hope it is somewhat fulfilled some of the OP's question.
And I think we still have to use overall ranking since there's no blitz ranking (like chess) in go yet. (Also, there's no ranking based on Color, Rule, Komi, etc.)
Re: How strong is Alphago (blitz mode) ?
Posted: Sat Jan 07, 2017 7:42 am
by Krama
pookpooi wrote:This page is quite interesting
49 of 60 finised games, we know the identity of opponent, add one game that Master didn't win to give a lower boundary to calculate (that disconnect Choi Cheolhan's game). The average of of 50 game's pro goratings.org's rating is 3447, the winrate is 98% (49/50) giving Master's elo as 4123. And yes, I know that many people will quote this method as completely mathematically wrong. But I hope it is somewhat fulfilled some of the OP's question.
And I think we still have to use overall ranking since there's no blitz ranking (like chess) in go yet. (Also, there's no ranking based on Color, Rule, Komi, etc.)
We need to wait for normal games to be played. However it's not that strange that AlphaGo could be ~4000 in normal games. Look at chess engines, they are quite higher in rating compared to top GMs.
Re: How strong is Alphago (blitz mode) ?
Posted: Sun Jan 08, 2017 3:21 am
by Vargo
You can't keep mirroring as black forever, or else white wins by komi.
Crystal clear, thank you, you must be be a very good go teacher
pookpoi, thanks, the page you linked to is in chinese (?)
hereunder is google-translate's version...
What is the new AlphaGo Master's Elo Rating? I use goratings.org information irresponsible chaos count.
2016-12-29 to 2017-01-04 of the 60 Bureau Master fast chess, opponents have goratings.org combat effectiveness of the value of 49, the opponent's real name is not clear 11 Bureau. This 49 Bureau opponents fighting the highest for the Ke Jie 3627, the minimum is strict in the 3109, an average of 3447 (assuming Chenyao Ye + Meng Tai Ling's combat effectiveness is equal to the higher Chen Yaoye combat effectiveness).
The most rough algorithm is to put this 49 Bureau opponent as the same person, its combat effectiveness = 3447. In order to count the fighting must lose a Board, so hard to plug the Master to lose the next Council, the winning percentage is 49/50 = 0.98, converted in the fighting 676 points higher than the opponent. Therefore, the combat effectiveness of the Master> = 4123.
DeepMind before that from the new version of AlphaGo and the old version of each stroke's winning rate is estimated that the new combat is 4500, then I suspect that will not be over-fitting? In other words, the new version caught the old version of the bug, each with the bug to win, so against the 3600 AlphaGo winning rate will be much higher than the 3600 players to deal with the winning percentage. Master 60 from this chess view, there are more than 4100 to determine. If AlphaGo really have 4500 combat, the world's first person on the 3627 rate of 99.3% Kejie Sheng, which is a thousand war seven defeats, strong to unimaginable.
The following is the number of Master 49 chess opponents in the game and goratings.org combat effectiveness and ranking, according to the order of appearance sequence:
Go-AI is now better than man, yes, definitely.
But translation-AI , hummmmmm..... there's still work to be done, it seems
Am I right to think that the mentionned 4500 Elo is Deepmind's estimate of the new Alphago strength ?
