LZ's progression

[Vargo]

20 game match :#194 v. #157 at time parity
5min per side per game, komi 7.5, -r 10 for both, LZ 0.16, twogtp 1.5.0
PONDERING ENABLED for both (#194 using gpu 0, and #157 using gpu 1 , gpu : 2x1080Ti)

#194 wins 12-8 (7 wins as W)

If someone wants the games or the stats, I'll upload them

[splee99]

Another strong opponent is the Leela master. I tested the 15b LM_GX88 which is indeed stronger than #193 in time parity games. However #194 is now much stronger than LM_GX88.

[nbc44]

Forgotten test of "home-made" LeelaMaster_GX88 vs #157a net.

By visits (standart 3200):

Code: Select all

The first net is better than the second
LeelaMaster_GX88.txt v fc5e0a50.gz ( 231 games)
                          wins        black       white
LeelaMaster_GX88.txt  136 58.87%   58 61.05%   78 57.35%
fc5e0a50.gz            95 41.13%   37 38.95%   58 42.65%
                                   95 41.13%  136 58.87%

C:\APPS\l0gpu16\validation.exe -n C:\APPS\net\LeelaMaster_GX88.txt -o "-g -v 3200 --gpu 0 --gpu 1 --noponder -t 12 -q -d --timemanage off --precision single -w" -n C:\APPS\net\fc5e0a50.gz -o "-g -v 3200 --gpu 0 --gpu 1 --noponder -t 12 -q -d --timemanage off --precision single -w" -- C:\APPS\l0gpu16\leelaz -- C:\APPS\l0gpu16\leelaz -k lmgx88-157a

By time-parity (3s per move):

Code: Select all

The first net is worse than the second
LeelaMaster_GX88.txt v fc5e0a50.gz ( 308 games)
                          wins        black       white
LeelaMaster_GX88.txt  147 47.73%   53 46.90%   94 48.21%
fc5e0a50.gz           161 52.27%   60 53.10%  101 51.79%
                                  113 36.69%  195 63.31%

C:\APPS\l0gpu16\validation.exe -n C:\APPS\net\LeelaMaster_GX88.txt -o "-g --gpu 0 --gpu 1 --noponder -t 12 -d --precision single --logfile log -w" -n C:\APPS\net\fc5e0a50.gz -o "-g --gpu 0 --gpu 1 --noponder -t 12 -d --precision single --logfile log -w" -- C:\APPS\l0gpu16\leelaz --gtp-command "time_settings 1 4 1" -- C:\APPS\l0gpu16\leelaz --gtp-command "time_settings 1 4 1" -k lmgx88-157a

P.S. 2x1080ti

[Vargo]

20 game match between #195 and #157
1x1080, 5 min per side per game, no pondering, komi 7.5 , for both matches.

#195 wins 12-8
(average length 248 moves, average time used 243" and 239")

20 game match between #195 and Leela Master GX88
(average length 258 moves, average time used 242" and 237")
#195 wins 12-8


250+ moves... it seems a lot, maybe it would have been less if I'd used -r 10 ?

If someone wants the game, I'll upload them.
For both matches, I used -alternate, so, there are automatic color changes which make the reports look weird (because of the "Colors Exchanged" column in the stats)
The stats :

157_195.jpg (160.13 KiB) Viewed 12362 times

195_gx88.jpg (154.48 KiB) Viewed 12362 times

[Vargo]

Rn, with the LZ weights #191 got 2nd, just behind Golaxy, at the recentAI Ryusei 2018.
(see HERE , where there's a link to download the binaries)

20 game match between Rn (with network #195) and LZ0.16 with #195
--device-id 0 --weights C:\...\195.gz --komi 7.5 --const-time 2 for Rn
--noponder --weights C:\...\195.gz for LZ
-time 1s+3s/1 -auto -komi 7.5 for both

twogtp1.5.0, 1x1080, LZ used almost exactly 2sec/move, and Ray used 1.8sec/move
Result 10-10 No duplicate games or errors, All games by resignation, but for two games I discarded because I didn't understand their result ... (see image)

hummmm.jpg (31.34 KiB) Viewed 12165 times

Rn is B.zip

(9.63 KiB) Downloaded 590 times

Rn is W.zip

(9.71 KiB) Downloaded 577 times

[jlt]

Can you explain a bit more? You said this is a 20-game match but you only posted the results of 11 games (games #0 --#10). On the other hand, the zip files you provided do contain 20 games (essai2-0...essai2-9, essai3-0...essai3-9).
What is the correspondence between the table you provided and these 20 games?

Also, concerning the two games whose result you don't understand, if you post them we could check manually to see if there is something wrong.

[Vargo]

Can you explain a bit more?

You're right, it's not very clear...
I ran 20 games, 2 of them with dubious results, so I ran 2 more, and the match result (10-10) is with unequivocal games, all won by resignation.
I just discarded the two strange games.
The zip "Rn is B" contains 10 games, with 6 wins for Rn as B, and
The zip "Rn is W" contains 10 games, with 4 wins for Rn as W.

Concerning the curious (discarded) result in the picture, I think
B sees itself as winner (Result_B : B+13.5) and
W sees itself as winner too (Result_W : W+10.5) and
the 3rd column is Result_Referee : ?

I'll run some more games tomorrow, and I'll post the sgf of the dubious ones.

[Vargo]

Here we are... Another 40 game match Rn(#195) v. LZ0.16(#195)

36 games won by resignation, I've looked at some of them, and, they look ok.

Three games where both sides agree on the winner, but not on the score :
B sees B as winner (Result_B : B+9.5) and
W sees B as winner too (Result_W : B+5.5) and
the 3rd column is Result_Referee : ?

B sees W as winner (Result_B : W+4.5) and
W sees W as winner too (Result_W : W+6.5) and
the 3rd column is Result_Referee : ?

B sees W as winner (Result_B : W+48.5) and
W sees W as winner too (Result_W : W+6.5) and
the 3rd column is Result_Referee : ?
File : "rn195_lz195_2sec_c-5.sgf" in the zip

To my untrained eye, they look like unfinished business, I could understand a resignation, but not a precise score.

One game where Rn and LZ don't agree on the winner:

B sees B as winner (Result_B : B+11.5) and
W sees W as winner too (Result_W : W+8.5) and
the 3rd column is Result_Referee : ?
File : "rn195_lz195_2sec_c_LZ_B-5.sgf" in the zip

I'm calling the referee (that's you ) on these games, how can you count points ?

Anyway, discarding the 6 dubious games (out of 62 in all), it's a 56 game match, at 2sec/move
and the score is a perfect draw, at 28-28

The 40 games :

rn195_lz195_2sec.zip

(38.18 KiB) Downloaded 520 times

[jlt]

The problem is that the games are unfinished. Consider the last game rn195_lz195_2sec_c_LZ_B-5.sgf. Edit: I made a confusion with rn195_lz195_2sec_LZ_B-5.sgf

Black was leading by 1.5 point. When Black played P3, White should have protected at P1, but instead connected at G6, so Black played the last move P1. If the game had continued normally, White would have protected at Q2, then Black O2, White O1 and Black captures 6 stones with O2 (snapback), White Q1, Black O1, White R6 and Black would have won by 14.5 points (or 15.5, I don't know which rules are used to calculate the score).

So to summarize:


1) Black was leading
2) White made a gross blunder (but anyway had a lost game).
3) The game is not finished, but if continued Black would have won anyway.
4) The two players probably don't agree on the life-and-death status of the stone P1, so calculate the score differently (but still I don't understand how White can come up with a result like W+8.5).

[Vargo]

Thanks ! You're absolutely right about your analyse of rn195_lz195_2sec_LZ_B-5.sgf,
but LZ and Rn both agreed it was B+

The weird one is rn195_lz195_2sec_c_LZ_B-5.sgf (sorry for the confusing names...)

[jlt]

Oops, I indeed made a confusion between two names. The game rn195_lz195_2sec_c_LZ_B-5.sgf was unfinished, so it's impossible to count.

[zakki]

Sometime RLO plays PASS at losing game, because they cannot find any good move, but win rate is bigger than a resign threshold.

[And]

new network 15x192 from bjiyxo is stronger lz157: 92297ff2 VS d351f06e 260 : 144 (64.36%)
http://zero.sjeng.org

[Vargo]

100 game match between #196 and #157 at short time parity
1 min per side per game, komi 7.5 no pondering, 1x1080, LZ0.16, twogtp V1.5.0
0 duplicate game, 0 error, all games by resignation.
#196 wins 55:45
196 is now clearly stronger than 157, even at short time settings (both networks spending 0.5-0.6 sec/move)

The 100 games :

157_196_1minute.zip

(81.87 KiB) Downloaded 525 times

Detailed stats :

Pas de nom.jpg (215.24 KiB) Viewed 11037 times

new network 15x192 from bjiyxo is stronger lz157: 92297ff2 VS d351f06e 260 : 144 (64.36%)

Very interesting !

I've launched a 100 game match at 1 min per game between #196 and this new network 92297ff2, result later.

[Vargo]

Here we are :
#196 (40x256) vs 92297ff2 (15x192) at short time parity : 1 min per side per game, komi 7.5, no pondering, 1x1080, LZ0.16, twogtp V1.5.0
0 duplicate game, 0 error, all games by resignation.

This time, #196 is not the winner :

92297ff2 v 196 : 63:37

It was to be expected, because 92297ff is stronger than #157.

The 100 games :

196_15x192black_1m.zip

(82.36 KiB) Downloaded 524 times

The stats :

Pas de nom.jpg (685.42 KiB) Viewed 10988 times