LZ's progression

[And]

I wonder if 92297ff2 is stronger than elf? (time parity)

[splee99]

Here we are :
#196 (40x256) vs 92297ff2 (15x192) at short time parity : 1 min per side per game, komi 7.5, no pondering, 1x1080, LZ0.16, twogtp V1.5.0
0 duplicate game, 0 error, all games by resignation.

This time, #196 is not the winner :

What is (maybe) unexpected, is that #196 is not as strong as #195 in some aspects. I had run some handicap games on my computer and got this feeling.

[splee99]

Just run two time parity games with 20 seconds per move between ELF and #196 and #196 won both games. It looks like to me that the 40b weight is indeed superior in handling large groups and taking advantage of a ko fight. However, I don't understand why #196 lost to the new 15b weight in the above tests. Maybe the game time is too short ( 1 minute )?

[Vargo]

I don't understand why #196 lost to the new 15b weight in the above tests. Maybe the game time is too short ( 1 minute )?

I thinnk you're right.

I've run two more matches #196 v 92297ff at time parity :
the three results are :

1min/game (corresponding to visits=~300 and visits=~1100) #196 wins 37% (100 games, 1x1080)
5min/game (corresponding to visits=~1000 and visits=~3600) #196 wins 60% (20 games, 1x 1080)
5min/game with 2 GPU (corresponding to visits=~3000 and visits=~10800) #196 wins 65% (20 games, 2x 1080Ti)

I've done several tests to determine time-visits ratio, the results vary a lot, these correspondences are approximate.

games and stats for 2 gpus :

196_v_92297_5min_2gpu.zip

(19.19 KiB) Downloaded 509 times

2gpu.jpg (247.26 KiB) Viewed 10877 times

[And]

20 games is enough for a confident conclusion? I doubt, especially considering the slight advantage

[Vargo]

20 games is enough for a confident conclusion?

20 games is not nearly enough, but each of these 20 game match is almost 3 hours...
After several such matches, we can hopefully see a trend.

Ideally, this kind of match should be... 400 games, for example
To do this, one possiblity would be to find 20 people OK to run a 20 game match with the same parameters including -v 1000 for #196 and -v 3600 for #92297ff, so, we would have 400 games at time parity, and at the same strength, I think -v 1000 and -v 3600 should be at near time parity on any machine (?)
But it would probably be more useful for people to donate their GPU time to autogtp.

Anyway, I ran 40 more games on each machine, and after 60 games, #196's winrate has gone down.

5min/game (corresponding to visits=~1000 and visits=~3600) #196 wins 52% (31:29, 1x 1080)
5min/game with 2 GPU (corresponding to visits=~3000 and visits=~10800) #196 wins 52% too (31:29, 2x 1080Ti)

I'll upload stats and/or games if someone wants them.

[And]

thanks! after such a result (52%) is not the fact that #196 is stronger

[And]

Strange, why there is no new network 92297ff2 in the table "Best Network Hash"? (http://zero.sjeng.org) for example, under the number 197. it is logical and convenient

[Tryss]

Because this network is not trained the same way.

[dfan]

Strange, why there is no new network 92297ff2 in the table "Best Network Hash"? (http://zero.sjeng.org) for example, under the number 197. it is logical and convenient

It's a 15-block network, which played a test match against the previous best 15-block network, not against the current best 40-block network.

[And]

In the table there are other networks 15x192 (weaker) and from bjiyxo. where then to look for this network, if not in this table? the strongest 15x192!

[dfan]

Each entry in the table is stronger than the entry before it (reading from bottom to top). To put this latest 15-block network one in, they'd have to either break that rule or change all the numbers or call this network 157a or something. Maybe they should have used a 2-dimensional numbering scheme.

I agree that it would be nice to have a place to easily find the best 15-block network.

[And]

the table is made for the convenience of people, not for the report! besides, it has a Hash, Size, Elo, Games, Training. if a person came to this site, then this is not a random visitor, and he will understand what's what

[dfan]

I was just trying to answer your original question of why it was not in the table. If you would like the format of the table to be changed, you could try opening an issue on the Leela Zero GitHub page: https://github.com/gcp/leela-zero/issues

[And]

Formal reasons are clear to me, but thanks anyway for your answers!