Humans vs. Alpha Go Zero

[Vargo]

According to THIS POST, Leela Zero #125, with 3000-4000 playouts, was already much stronger than 2p.

It usually got to 3000-4000 playouts before hitting the time limit.

Baolong also thinks that if he could exploit bugs, he could maybe win one or two games out of a hundred. He thinks this configuration could already be stronger than AlphaGo Lee.

Here is a 10 game test at H4, ~ time parity.
(gogui 1.5.1 twogtp, average length 240 moves, av. time B : 420s, av. time W, 460s)

LZ#125 -v 3500
v.
KG 1.8.1 network: kata1-b40c256-s7306904576-d1772668053.bin.gz -v 1200 (KG always W)

Result :
LZ wins 6-4 (4 times W+R, 3 times B+R, 3 times B+xx)

[gennan]

Was that H4 with 0.5 komi or 7.5 komi?

LZ doen't understand anything else than 7.5 komi and chinese rules and it's not trained to play with handicap or against handicap. So I think LZ would have significant disadvantages in handicap match, because of weakness in handicap games and ladder issues, both of which are exploitable issues that pros won't have. So I don't think handicap matches between LZ and KataGo are reliable to determine AI ranks.

I think it would be more informative to establish handicaps between different versions of KataGo.

[Vargo]

It was komi 0 and chinese rules, but you're right about LZ and handicap.

[And]

@gennan
thanks for such a detailed answer! (viewtopic.php?p=264114#p264114)

What do you think, if conditionally 0 Elo is assigned to 30 kyu, we can assume that with each rank the same Elo value is added and which value is better suited? and it is possible to derive an approximate formula for calculating the probability of winning between rank A and B?

[gennan]

@gennan
thanks for such a detailed answer! (viewtopic.php?p=264114#p264114)

What do you think, if conditionally 0 Elo is assigned to 30 kyu, we can assume that with each rank the same Elo value is added and which value is better suited? and it is possible to derive an approximate formula for calculating the probability of winning between rank A and B?

Assigning 0 Elo to 30 kyu is possible, but it doesn't really matter where you anchor an Elo scale, because Elo ratings are relative. Only Elo differences matter (within the same system). Absolute Elo ratings only matter when comparing different Elo systems that use different anchors (similar as comparing Centigrade temperatures to Kelvin tempatures).

With the conversion formulae and the anchor I gave in that post, EGF 30k(-900 GoR) would correspond to about 1000 Elo. Shifting it to 0 Elo as you suggest, would just shift all ratings down by 1000 Elo, without changeing any win% prediction made by the system.

Analysis of the EGF historical data shows that using the same Elo gap between all ranks does not match observed winrates. A 15k has a much higher win% against a 14k (~43%), than a 6d against a 7d(~25%). A 1d against a 2d is somewhere in between (~36%). So by definition of Elo (based on probability in even game win%), the Elo gap between 15k and 14k (~50 Elo) is smaller than the Elo gap between 6d and 7d (~200 Elo) and the Elo gap between 1d and 2d is somwehere in between (~100 Elo).

To calculate the predicted winrate between 2 players with given EGF GoR, this will be used by the EGF rating system after the coming update:

Code: Select all

Se = 1 / (1 + exp(β(GoR2) - β(GoR1)))
where β = -7 * ln(3300 - GoR)

This β function can be seen as yet another rating scale, similar to Elo, but using a different unit to avoid redundant conversions.

[And]

@gennan
I know the Elo ratings are relative, but thanks a lot for the other part of the answer!

[shinchane101]

Thanks for sharing this with us.