LZ's progression

[Vargo]

the less games, the less reliable result. “Official” result obtained from 158 games.

Yes, but in the original post about this network, a lot more than 158 games have been played by 14a3a, and the results were 50+% vs 226

I wonder why you chose gnugo for handicap tests?

GnuGo is 5 kyu, it's a reference point.
Ideally, I'd choose Crazy Stone DL for handicap games, it can be set anywhere from 13 kyu to 7 Dan , but CSDL can't be used in Sabaki...

But you're right, these networks are so strong at handicap, that I have only two choices :
1) Forget GnuGo and find another reference point
2) Keep on using Gnugo and ask Sabaki's authors to add the possibility to play at H10, H11, etc.

[And]

Vargo,
and if to use gtp4zen or cs dl directly through gtool2? (tool for auto-playing between two igo softs. but a lot of games will of course be tedious)

PS I checked gtp4zen 0.3.5 (zen7), gogui 1.5.1, H9, settings zen7: 5K -t1 -r5 -s800 -n1 -o2.4 -p0.3 (https://github.com/breakwa11/GoAIRating ... /AIcmds.md). gtp4zen playing white, then black - all ok

[Vargo]

I tried LZ017_14a3a v gtp4zen(7) at H9

First, with --noponder -v 200 -r 1 for LZ
But... -r 1 makes it resign too soon, for example, in this position. I think a 5kyu can be invaded by W.

toosoon.jpg (140.44 KiB) Viewed 13768 times

After that, I tried --noponder -v 200 -r 0 for LZ
with the hope that LZ wouldn't resign so soon.
These settings work when LZ_14a3a wins, for example in these games against gtp4zen.exe (-t4 -r5 -s800 -n1 -o2.4 -p0.3, corresponding to ~5 kyu)

or

But against (gtp4zen.exe -t4 -s1000 -n1 -o2.0 -p0.3) , corresponding to ~3k,
H9 is too much, and 14a3a loses heavily, but thinks it has won (the correct score is RES_B)

#GAME...RES_B....RES_W
0.......B+92.5...W+12.5
1.......B+102.5..W+3.5



Go figure !


EDIT : The two posts with various handicaps contained 2 errors in gtp4zen parameters (3d and 5d parameters were wrong) , I removed them.

[And]

What will happen to the new network LZ? search to win? or will something change?

[And]

False promotion lz227? https://github.com/leela-zero/leela-zero/issues/2421

[And]

Extensive tests of LZ improvements: true ELO gain is ~5 ELO per net since LZ174 https://github.com/leela-zero/leela-zero/issues/2425

[And]

new rules ? 923 games!? http://zero.sjeng.org
2019-08-06 16:14 88765231 VS a4f5d99a 390 : 318 (55.08%) 708 / 400 PASS
2019-08-06 16:14 88765231 VS a4f5d99a 498 : 425 (53.95%) 923 / 400 fail

[jokkebk]

new rules ? 923 games!? http://zero.sjeng.org
2019-08-06 16:14 88765231 VS a4f5d99a 390 : 318 (55.08%) 708 / 400 PASS
2019-08-06 16:14 88765231 VS a4f5d99a 498 : 425 (53.95%) 923 / 400 fail

I recall that the gating test is a probability one, it might be that if you're very close to the treshold, it keeps playing until certain probability is reached.

Still, 900 games sounds like tossing a coin twenty time to get heads. Another plausible explanation is some server configuration/connectivity issue that gave out a lot of match games to clients but couldn't receive them, and once that was resolved, finally got them all (or most) back.

[Aram]

Nothing to see here.. it was an error and it has been fixed in the server code so it would not happen again..

[And]

LZ#239 VS LZ#230 53.00% ???
2019-08-21 17:07 3bd93d6e VS a9543f21 212 : 188 (53.00%) 400 / 400 fail
http://zero.sjeng.org

[iopq]

LZ240 still beat LZ230
0e17a181 VS a9543f21
225 : 176 (56.11%) 401 / 400 PASS

Then from discord jio tested

lz_240 v lz_230 (400/400 games)
board size: 19 komi: 7.5
wins black white avg cpu
lz_240 233 58.25% 106 53.00% 127 63.50% 214.76
lz_230 167 41.75% 73 36.50% 94 47.00% 216.98
179 44.75% 221 55.25%

cmdline options: "-g -t 6 --batchsize 5 --noponder -v 1600 -r 5 --precision half"

this is in line with previous results, slower progress, but progress nevertheless

[hydrogenpi7]

LZ240 still beat LZ230
0e17a181 VS a9543f21
225 : 176 (56.11%) 401 / 400 PASS

Then from discord jio tested

lz_240 v lz_230 (400/400 games)
board size: 19 komi: 7.5
wins black white avg cpu
lz_240 233 58.25% 106 53.00% 127 63.50% 214.76
lz_230 167 41.75% 73 36.50% 94 47.00% 216.98
179 44.75% 221 55.25%

cmdline options: "-g -t 6 --batchsize 5 --noponder -v 1600 -r 5 --precision half"

average of 3 real elo per net?

this is in line with previous results, slower progress, but progress nevertheless

[nbc44]

LZv17 #219 vs LZ (MiniGo training data) (https://github.com/leela-zero/leela-zero/issues/2509)
2x1080ti, v 2400

C:\APPS\l0gpu17\validation.exe -k lz247-minigo17 -s "0:1" -g 5 -n C:\APPS\net\901e04de.gz -o "-g -v 2400 --gpu 0 --gpu 1 -r 5 -t 2 --batchsize 1 --noponder -q -d --timemanage off --precision single -w" -n C:\APPS\net\leelaz-model-swa-16-768000.gz -o "-g -v 2400 --gpu 0 --gpu 1 -r 5 -t 2 --batchsize 1 --noponder -q -d --timemanage off --precision single -w " -- C:\APPS\l0gpu17\leelaz -- C:\APPS\l0gpu17\leelaz

Code: Select all

#247 v MG17 ( 400 games)
          wins        black       white
#247  178 44.50%   72 43.64%  106 45.11%
MG17  222 55.50%   93 56.36%  129 54.89%
                  165 41.25%  235 58.75%
400 games played.
Status: 0 LLR -0.128628 Lower Bound -2.94444 Upper Bound 2.94444