How strong is Alphago (blitz mode) ?

Vargo · #1

In a 60 games match, to have a decent chance at winning 60-0 (let's say a 50% chance to achieve this feat), you need to have a 99% probability of winning against each opponent.
So, very roughly, it would seem Alphago has something like a 99% prob of winning a blitz game against a top pro...
How does it translate in terms of Elo (several hundreds ??) or handicap stones (is Alphago blitz 2 stones stronger than a top pro ???)
What do you think ?

HermanHiddema · #2

According to the Elo rating formula, the winning chances a player (A) against another player (B) are:

^________¹^________
1 + 10^(dR/400)

Where dR is the rating difference between the players (R_A - R_B)

This means the winning chances of a player increase tenfold for every 400 rating points. So at 400 difference you have roughly 90% chance, at 800 you have roughly 99% chance, at 1200 you have roughly 99.9% chance, and you can keep adding 9's behind that for every 400 rating points more.

So to have a 99% chance, you need a rating difference of roughly 800.

Uberdude · #3

Measuring strength difference in handicap stones versus winning percentages is rather tricky: I would expect a top pro taking 2 stones handicap to beat another top pro the vast majority of the time (maybe 95%+) whilst I would beat another European 4d taking 2 stones maybe 75% of the time, and for 14 kyus even less (maybe 60%). I think the EGF's modified Elo system does take something like this into account with parameters in the basic Elo formula that change depend on players' rating. Also on OGS I played 32 games against an AGA 5d and won all but the first game (when I was a bit weaker, I improved from UK 2d to 4d over those years which is probably something like AGA 3.5d to 6d), but there were quite a few close games. So whilst I had a near perfect win-rate I wouldn't say I could give him 2 stones and be confident to win. Anyway, although Master also got some close wins maybe it was always in control and didn't play its best. We'd really need to see some 2 stone games to see if it could win giving handicap. But for sure it is a lot stronger than humans in blitz, but I won't define "a lot"

.

P.S Zhou Junxun tried mirror Go which was interesting.

pookpooi · #4

Facebook user Trent D Carroll post this on Go (Weiqi) Players on Facebook page

Look like handicap stones is a better measurement than Elo in this situation.

Mike Novack · #5

Does anybody know how much "iron" has been made available to this bot?

While the absolute strength of play is probably very non-linear with increased crunch power, the amount of time required per move IS inversely linear.

What I am saying is that what we are seeing MIGHT not be an increase in the strength of the program. Let's say that the program is strong enough that when given a certain amount of hardware power it can play evenly against these humans (win 50%) at a time control of 1 minute per move. Give it hardware with six times the crunch power and make the time control 10 seconds per move, and while the program isn't playing any better, the human opponents are playing worse. So the program would now be winning more than 50%, maybe a lot more.

Bill Spight · #6

Uberdude wrote:

Measuring strength difference in handicap stones versus winning percentages is rather tricky: I would expect a top pro taking 2 stones handicap to beat another top pro the vast majority of the time (maybe 95%+) whilst I would beat another European 4d taking 2 stones maybe 75% of the time, and for 14 kyus even less (maybe 60%).

The two approaches are different. You could also say that measuring strength difference by win rate versus handicap stones is rather tricky. One advantage of using handicap stones is that they add linearly fairly well to make reasonably even games. I once gave a 41 stone handicap (!) based upon rank difference. I really was not expecting a close game, but I won by only 10 points.

skydyr · #7

Bill Spight wrote:

Uberdude wrote:

Measuring strength difference in handicap stones versus winning percentages is rather tricky: I would expect a top pro taking 2 stones handicap to beat another top pro the vast majority of the time (maybe 95%+) whilst I would beat another European 4d taking 2 stones maybe 75% of the time, and for 14 kyus even less (maybe 60%).

The two approaches are different. You could also say that measuring strength difference by win rate versus handicap stones is rather tricky. One advantage of using handicap stones is that they add linearly fairly well to make reasonably even games. I once gave a 41 stone handicap (!) based upon rank difference. I really was not expecting a close game, but I won by only 10 points.

Out of curiosity, what was the presumed rank of your opponent?

Uberdude · #8

Bill Spight wrote:

I once gave a 41 stone handicap (!) based upon rank difference. I really was not expecting a close game, but I won by only 10 points.

Just a few days ago Sai and I lost the final of the London Open Pair Go by half a point against a 10kyu pair giving them 11 stones. I made the last losing play of prematurely defending inside our territory to prevent an upcoming snapback when I could have connected the last ko :-?

. In the lightning final Oh Chi Min 7d gave 21 stones and won against his girlfriend Zoe!

Bill Spight · #9

skydyr wrote:

Bill Spight wrote:

Uberdude wrote:

Measuring strength difference in handicap stones versus winning percentages is rather tricky: I would expect a top pro taking 2 stones handicap to beat another top pro the vast majority of the time (maybe 95%+) whilst I would beat another European 4d taking 2 stones maybe 75% of the time, and for 14 kyus even less (maybe 60%).

The two approaches are different. You could also say that measuring strength difference by win rate versus handicap stones is rather tricky. One advantage of using handicap stones is that they add linearly fairly well to make reasonably even games. I once gave a 41 stone handicap (!) based upon rank difference. I really was not expecting a close game, but I won by only 10 points.

Out of curiosity, what was the presumed rank of your opponent?

Weak 35 kyu. He was not a rank beginner, but had not played for many years, and had been a DDK when he had played before.

Anzu · **#10**

On a good computer, only one second is necessary for a chess computer to win against me :D

Vargo · **#11**

Thank you for these answers.

Quote:

P.S Zhou Junxun tried mirror Go which was interesting.

Yes, interesting, thank you.
I think it is game F51 , but Zhou Junxun quit mirroring at move 71, does someone know why ?

For the strength of Alphago-Blitz, couldn't we at least have a lower bound estimate ?
What could be the minimum Elo of a player winning 99% of his games against 3500-3600 Elo opponents ?

Uberdude · **#12**

Vargo wrote:

I think it is game F51 , but Zhou Junxun quit mirroring at move 71, does someone know why ?

You can't keep mirroring as black forever, or else white wins by komi. So you only keep mirroring whilst you think your tengen stone is worth more than komi, so presumably he thought something like if they kept mirroring the centre would become dame so white wins. Or white's move was a bit slack so he didn't want to copy it. Mirror go is actually very hard as you have to continually make these judgements which in 30 seconds is tough. I actually think mirror Go as white is more interesting, and that's what Fujisawa Hosai did. There are some nice articles in GoGoD's New in Go about that.

Vargo wrote:

For the strength of Alphago-Blitz, couldn't we at least have a lower bound estimate ?
What could be the minimum Elo of a player winning 99% of his games against 3500-3600 Elo opponents ?

But we don't have Elo ratings for KeJie-Blitz, ParkJunghwan-Blitz etc.

pookpooi · **#13**

This page is quite interesting
49 of 60 finised games, we know the identity of opponent, add one game that Master didn't win to give a lower boundary to calculate (that disconnect Choi Cheolhan's game). The average of of 50 game's pro goratings.org's rating is 3447, the winrate is 98% (49/50) giving Master's elo as 4123. And yes, I know that many people will quote this method as completely mathematically wrong. But I hope it is somewhat fulfilled some of the OP's question.

And I think we still have to use overall ranking since there's no blitz ranking (like chess) in go yet. (Also, there's no ranking based on Color, Rule, Komi, etc.)

Krama · **#14**

pookpooi wrote:

This page is quite interesting
49 of 60 finised games, we know the identity of opponent, add one game that Master didn't win to give a lower boundary to calculate (that disconnect Choi Cheolhan's game). The average of of 50 game's pro goratings.org's rating is 3447, the winrate is 98% (49/50) giving Master's elo as 4123. And yes, I know that many people will quote this method as completely mathematically wrong. But I hope it is somewhat fulfilled some of the OP's question.

And I think we still have to use overall ranking since there's no blitz ranking (like chess) in go yet. (Also, there's no ranking based on Color, Rule, Komi, etc.)

We need to wait for normal games to be played. However it's not that strange that AlphaGo could be ~4000 in normal games. Look at chess engines, they are quite higher in rating compared to top GMs.

Vargo · **#15**

Quote:

You can't keep mirroring as black forever, or else white wins by komi.

Crystal clear, thank you, you must be be a very good go teacher :clap:

pookpoi, thanks, the page you linked to is in chinese (?)
hereunder is google-translate's version...

Quote:

What is the new AlphaGo Master's Elo Rating? I use goratings.org information irresponsible chaos count.

2016-12-29 to 2017-01-04 of the 60 Bureau Master fast chess, opponents have goratings.org combat effectiveness of the value of 49, the opponent's real name is not clear 11 Bureau. This 49 Bureau opponents fighting the highest for the Ke Jie 3627, the minimum is strict in the 3109, an average of 3447 (assuming Chenyao Ye + Meng Tai Ling's combat effectiveness is equal to the higher Chen Yaoye combat effectiveness).

The most rough algorithm is to put this 49 Bureau opponent as the same person, its combat effectiveness = 3447. In order to count the fighting must lose a Board, so hard to plug the Master to lose the next Council, the winning percentage is 49/50 = 0.98, converted in the fighting 676 points higher than the opponent. Therefore, the combat effectiveness of the Master> = 4123.

DeepMind before that from the new version of AlphaGo and the old version of each stroke's winning rate is estimated that the new combat is 4500, then I suspect that will not be over-fitting? In other words, the new version caught the old version of the bug, each with the bug to win, so against the 3600 AlphaGo winning rate will be much higher than the 3600 players to deal with the winning percentage. Master 60 from this chess view, there are more than 4100 to determine. If AlphaGo really have 4500 combat, the world's first person on the 3627 rate of 99.3% Kejie Sheng, which is a thousand war seven defeats, strong to unimaginable.

The following is the number of Master 49 chess opponents in the game and goratings.org combat effectiveness and ranking, according to the order of appearance sequence:

Go-AI is now better than man, yes, definitely.
But translation-AI , hummmmmm..... there's still work to be done, it seems :wink:

Am I right to think that the mentionned 4500 Elo is Deepmind's estimate of the new Alphago strength ?

pookpooi · **#16**

It's DeepMind estimated of v18, which is not truly 4500 since it lose one game to Lee Sedol.

Vargo · **#17**

Reddit/baduk has a very interesting thread about all this :

https://www.reddit.com/r/baduk/comments/5mp7x9/proper_handicap_between_alphago_and_top_humans/?sort=old

Many people seem to agree that AG is 2-3 stones better then a top pro, and... probably getting better as we speak ;-)

Mike Novack · **#18**

Vargo wrote:

Many people seem to agree that AG is 2-3 stones better then a top pro, and... probably getting better as we speak ;-)

Not so clearly true. What we have learned is that AG is maybe that much better AT THIS TIME CONTROL. Especially for a process which allows parallel processing, the amount of time required is a function of the processor power applied.

It is possible that in absolute terms, AG would not be any/much stronger given more time. But the human player is helped significantly by given enough time to think.

Let's go back a decade or so when the strongest go programs (pre MCTS) were at about 6 kyu using an AI capable of finishing their processing in about 10 seconds on a machine very modest by today's standards. But more to the point, modest in terms of a really powerful machine of that day. Suppose that 6kyu program were running on a machine powerful enough that it could make its move in just under 1 second and we made the time control 1 second per move. Do you think we would STILL consider it 6 kyu against a human trying to play at 1 second/move? I don't think so. In absolute terms, that AI would be no stronger, but given just 1 second to make each move, the human would be much weaker.

It would not be difficult to conduct an experiment to determine the true state of affairs. Reduce the amount of processing power available to AG so that it requires (for example) 1 minute per move and see how the pros do at that time control. Then maybe conduct the experiment a little differently, using a time control where the human has ten minutes to make ten moves << but can distribute that time as he or she wishes >>

How strong is Alphago (blitz mode) ?

Who is online