LZ's progression

[hydrogenpi7]

[admin]
Off-topic link removed.
-JB
[/admin]

[johnsmith]

https://www.gokgs.com/gameArchives.jsp?user=petgo3

Hint: KGS archives with color highlight of wins/losses of given user with my Tampermonkey (Chrome plugin) script: http://joonaspihlajamaa.com/data/kgs_graphs.user.js

This script is just excellent, thank you!

[Vargo]

I've run a 100 games twogtp match between regular LZ 0.15, with network #155 and LZ Variable Komi V0.3, with network #155.
Even games, with komi 7.5 , --noponder --visits=3201 -r 10 for both.

Regular LZ wins 52-48 (50 games as W, 50 as B)

It looks like LZVK is as strong as Regular LZ on even games (I should run more games to confirm this...)

[Vargo]

Four 10-games matches between LZ variable komi V0.31 (network#156) and gnugo 3.8 at various handicaps (I used twogtp V1.4.10)

komi 7.5, playouts=1000 for LZvk
H6 : LZvk wins 10-0

komi 7.5, playouts=2000 for LZvk
H8 : draw 5-5

komi 7.5, playouts=2000 for LZvk
H9 : gnugo wins 8-2

komi 7.5, playouts=4000 for LZvk
H9 : gnugo wins 9-1 (a bit curious, compared to 8-2 with less playouts)


Two 10-games matches between LZ variable komi(#156) and Leela 0.11.0 OpenCL

komi 7.5, playouts = 1000 for both
H5 : Leela OpenCl wins 6-4

komi 7.5, playouts = 3000 for both
H5 : Leela OpenCl wins 7-3

I didn't use -r 0 , so, in some games, LZvk resigns rather early.
If someone wants to see the games, I can upload them.

[Vargo]

A while ago, I ran H6, H7 and H8 mini matches between LZvk(#153) and LZ#80.
LZvk won at H6 and H7, but jokkebk's remark is true...

Vargo wrote:
Thanks to alreadydone, LZ can now handle high handicap (HERE)
Playing (and winning) H7 and H8 games against 4 Dan... Wow !

jokkebk wrote :
Winning against "4d" LZ #80 might not be such an achievement, considering the LZ network is probably very bad playing as black in high handicap.

So, what about a rematch, but with LZvk for both #157 and #80 (they're supposed to be 10.6D and 4D)

visits=3201, komi 7.5, noponder for both, 10 games match with twogtp 1.4.10, at a more reasonable H5.

LZvk with network #157 wins (only) 6-4
If someone wants the games, I'll upload them.

twogtp v1.4.10 with handicap seems to work well with LZvk , with regular Leela OpenCl and with gnugo, but not with gtp4zen, I don't know why.

[Vargo]

In his last witty post (the "#xxx Ten Game Match Report" series at reddit, cbaduk), HERE, Dontbteme writes :

Actually, maybe LZ#157 could already give LeelaElf a run for its money with Time Parity... Too bad I didn’t think of setting a match between them before, and REALLY meant it when I said I’d go to bed early.)

He's right, maybe it would show LZ's progress.

So, here are three 20 games matches at 5 minutes (per game and per side), at 10 minutes, and at 15 minutes, no pondering, komi 7.5.
The matches are run with twogtp 1.4.10 (win10, i7, 1x1080)

LZ15#157 vs LZElf :
at 5 minutes , LZElf wins 13-7 (65%) (6 wins as B, and 7 as W)
at 10 minutes, LZElf wins 13-7 (65%) (5 wins as B, and 8 as W)
at 15 minutes, LZElf wins 12-8 (60%) (6 wins as B, and 6 as W)

So, we're not there yet, but it's coming nearer


Edit : I just saw that Dontbteme has posted an Elf vs #157 match , but with very different result... HERE

[jokkebk]

LZ15#157 vs LZElf :
at 5 minutes , LZElf wins 13-7 (65%) (6 wins as B, and 7 as W)
at 10 minutes, LZElf wins 13-7 (65%) (5 wins as B, and 8 as W)
at 15 minutes, LZElf wins 12-8 (60%) (6 wins as B, and 6 as W)

Very nice! It means one can likely get already quite on par analysis with Lizzie on Leela Zero #157 as LZElf. If there's still 2-3 promotions with 192x15b net, it might be 50-50 already on time parity, but my bets would be that it'll take too long and Leela will switch to larger net before that.

[Vargo]

There's been some controversy about the passage to 20 block networks.

Three 10 games matches between LZ15 with #157 (192x15) and LZ15 #159 (256x20)

twogtp V 1.4.10 , komi 7.5, no pondering, 5 minutes per side and per game , 5 games as B, 5 games as W. (id with 10 min, and with 15 min.)

All games won or lost by resignation, all 30 games between 125 and 347 moves. Time used is consistently around two thirds of the alloted time.

LZ#157 vs LZ#159 :
at 05 minutes , LZ157 wins 6-4 (4 wins as B, and 2 as W)
at 10 minutes , LZ157 wins 6-4 (3 wins as B, and 3 as W)
at 15 minutes , LZ157 wins 7-3 (5 wins as B, and 2 as W)

Seems like #157 is still stronger at equal time setting...

Edit : hardware : 1xGTX1080, i7, win 10

[Uberdude]

Roughly how many playouts per move are the 15- and 20-block networks getting at these settings?

[Vargo]

Same kind of results with #160 at 5' per side and per game

LZ15#157 vs LZ15#160 :
at 5 minutes , LZ157 wins 7-3 (4 wins as B, and 3 as W)

@Uberdude
I don't know. At the end of an automatically played n-game match, twogtp gives reports like this

rep.jpg (32.76 KiB) Viewed 14926 times

with : winner, number of duplicate games (hopefully none) , length of game, time for B and for W, number of errors, but nothing about playouts.
I'll run some tests, and I'll let you know.

[Vargo]

@Uberdude :
#160 at 1600 visits takes a little more time than #157 at 3200 visits, and the matches take about the same time as the 5' games.
cf this :

rep2.jpg (179.06 KiB) Viewed 14939 times

@Marcel Grünauer
I've just downloaded the new ELF network, looking forward to trying it, thanks !

[Uberdude]

I'm not very knowledgable about how to interpret the results, but did note that with the 20-block networks there is a wild fluctuation of win rates. For example, yesterday and today some 20-block candidates only had a 30% win rate against the current #160.

So maybe getting a high enough percentage to be promoted is more of a fluke than a steady progression. With the 15-block networks and earlier, there were lots of candidates just below the promotion threshold of 55%.

Yes, I noticed this too. There was a test match of #159 (the 2nd 20-block against #157 (best 15-block) and it got 50% win vs 45% for #158. I've not yet seen a test match for the new #160 vs #157. Given all these 20-30% candidates there is more danger you are not progressing but just jumping around about the same level networks that randomly sometimes do well; so these test matches against #157 as a more external benchmark are important. In fact during normal 15-block training I think doing some occasionally e.g. #157 vs #147 would be a good idea to see how inflated the incremental self-improvement Elo is (based on the incremental Elo differences from the promotions 147->148->149 etc it went from 11401 -> 11806 which predicts #157 would beat #147 91% of the time, but I would bet it would be quite a lot lower than that in reality).

[Vargo]

20 games match between ELF1 and ELF2 at 1600 visits.
(twogtp 1.4.10, komi 7.5, noponder, all games by resignation)

ELF2 wins 14:6 (70% , 9 wins as W, 5 wins as B)


If someone wants the games and/or the reports...

[dfan]

There's a test match going on right now between the ELF1 and ELF2 weights, imported into Leela Zero. As I write this ELF2 is up 102 to 51.

Edit: the final result was 265 to 145 (64.6%).

[Uberdude]

Instead of win ratios amongst all these super-strong bots, I'd kind of like "average number of moves a strong/weak Tygem/Fox 9d lasts until only 10% winrate"