LZ's progression

[moha]

Single visit games are completely different imo. The value net is not used at all, and policy changes are multiplied. If we expect the net to be slightly stronger at 3000 visits, that includes both policy and value improvements in unknown proportions - and as I mentioned the result only applies to that visit range. At much more visits the new net will appear even stronger, at less visits it can be slightly less stronger, and single visit games would be hard to predict.

But the variance anomaly does seem strange, I wonder if there was a problem with your setup. Or maybe the results are correlated - how many unique games were there in each set?

[Vargo]

A last match : 20 x 5000 games between 9e88 and 057a with --visits=1 ( 2-3 hours of computer time, 10x5000 as B, 10x5000 as W)
Overall result : 9e88 wins 48631 out of 100000 games (48.631%)
For 9e88, min number of wins for a 5000 games match was 1920 (as B) , and the max was 3321 (as W)

I don't think there was a problem in these matches, the .dat files generated are OK, there was no crash or hangup.

Parameters were the same as before :
--gtp --weights=xxx --visits=1 --noponder -r 10 and
-games xxxxx -sgffile C:\... -auto -komi 7.5

In conclusion, I think you're right, "single visit games would be hard to predict"

[moha]

I meant, did you check that each set of 5000 games actually contain 5000 different games, and not a lot of duplicates?

[Vargo]

Ah, but you're right, some of the games are duplicates.
Is there a LZ or twogtp command to prevent that ?

[moha]

Ah, but you're right, some of the games are duplicates.
Is there a LZ or twogtp command to prevent that ?

IIRC there is a command line option for noising even the policy, but then the strength will be different.

[Vargo]

What is the influence of --visits=xxxx on the Elo or dan level of a given network ?
10 twogtp matches for network #145 (b691)
Each match is 100 games --noponder -komi 7.5 (no duplicate game in the reports)
For example, in the table below, b691 at 6401 visits won 64% of its games against b691 at 3201 visits.

b691.jpg (67.78 KiB) Viewed 15346 times

CF. Uberdude's comment and his link about win% (https://senseis.xmp.net/?EGFWinningStatistics)

and this site, https://www.reddit.com/r/cbaduk/comment ... _of_lzero/ , according to which, b691 is 10 dan at 1601 playouts (visits ?)
At 6400 visits, b691 wins 93% against a 10 dan, woaw !
If someone is interested in the .dat reports, I can upload them.

[Vargo]

What about the ELF weights (62b54 , or ELF V0)
How does it scale with the number of visits ?
At 1600 playouts, its strength is around 12 dan (cf. same site : https://www.reddit.com/r/cbaduk/comment ... _of_lzero/)

100 games match between 62b54 (6400 visits) and 62b54 (1600 visits)
twogtp, --noponder --visits=1601 (--visits=6401) -komi 7.5 , there was no duplicate game in the .dat report, all games won or lost by resignation.

Result : 90-10 (6400 visits won 41 games out of 50 as Black, and 49 as White)

Sample is small, but still... at least 1 stone stronger with 4 times more visits.

There was no ladder games in the 30-40 I've looked. If someone is interested in the games or the .dat report, I can upload them.

[ez4u]

Very interesting stuff. Thanks for all your efforts!

BTW, how long did it take your machine to run the ELF match?

[Vargo]

Thanks !
The 10 matches with b691 were run on 2 computers, one with 1x1080 and one with 2x1080Ti.
It took around 2 days, on and off.
For example, using both gpus, it takes about 3-5 minutes per game for 6400 visits against 3200 visits. (cf left of IMAGE)

The Elf match (6400 vs 1600) was with 1x1080, it took around 16 hours, each game 6-12 minutes (cf right of IMAGE)
Each match was two times 50 games (50 as B and 50 as W)

IMAGE

[Vargo]

The ELF network (62b54) is stronger than #147 (10bc1) at the same visits count.
After some games to estimate the difference, I tried visits=12801 and 1601.

Result of a 100 games twogtp-match (LZ0.15 for both , komi=7.5) between

10bc1 (--visits=12801 --noponder) and
62b54 (--visits=1601 --noponder)

10bc1 wins 51-49 (23 as B and 28 as W, no duplicate game)

Average game length : 218 moves ; min=91, max=384
10bc1 takes 3.65 times more time (for 8 times more visits)


Again, if someone wants the two .dat reports or the games, I can upload them.

[ez4u]

A zip of the games would be interesting.

[Bill Spight]

{Never mind.}

[Vargo]

Sorry for the delay, I was away.

Here is the zip of the 100 games ELF network (62b54 at 1600 visits) vs network #147 (10bc1 at 12800 visits)

100 games.zip

(91.79 KiB) Downloaded 511 times

[Vargo]

Thanks to alreadydone, LZ can now handle high handicap (HERE)

Some H6, H7, and H8 matches between network #153 (e1d46) and network #80 (e1156)
According to this site , #153 is 10.4D, and #80 is exactly 4D

Sabaki matches, with 3200 visits, no pondering

at H6, #153 wins 2-0
at H7, #153 wins 2-1
at H8, #153 loses each time, playing first line moves,

So... new H8 matches with 12800 visits for #153 (#80 still at 3200 visits)
#153 manages to win sometimes !

Playing (and winning) H7 and H8 games against 4 Dan... Wow !


H7 # 153 loses
H7 # 153 wins
H7 # 153 wins

H8 # 153 wins

[jokkebk]

Thanks to alreadydone, LZ can now handle high handicap (HERE)

Playing (and winning) H7 and H8 games against 4 Dan... Wow !

Winning against "4d" LZ #80 might not be such an achievement, considering the LZ network is probably very bad playing as black in high handicap. It would be more interesting to see this against humans.

KGS Leela bot Petgo author added the functionality to the bot, there seem to be a couple of high handi games there alread, although it seems +Forf wins for some opponents -- might be related to the comments of some people that the patched version of Leela sometimes hangs. But the wins will be interesting to check out:

https://www.gokgs.com/gameArchives.jsp?user=petgo3

Hint: KGS archives with color highlight of wins/losses of given user with my Tampermonkey (Chrome plugin) script: http://joonaspihlajamaa.com/data/kgs_graphs.user.js