Life In 19x19
http://lifein19x19.com/

LZ's progression
http://lifein19x19.com/viewtopic.php?f=18&t=15718
Page 13 of 21

Author:  roy7 [ Mon Dec 24, 2018 8:16 am ]
Post subject:  Re: LZ's progression

For those who might prefer it, there's an experimental "new" home page at http://zero.sjeng.org/home.

Author:  Vargo [ Mon Dec 24, 2018 9:24 pm ]
Post subject:  Re: LZ's progression

400 is the magic number ;-) , so, I extended the match between #196 (40x256) and 92297ff2 (15x192) to 400 games (more than 10 hours!).

time parity : 1 min per side per game, komi 7.5, no pondering, 1x1080, LZ0.16, twogtp V1.5.0
0 duplicate game, 0 error, all games by resignation, no contested scores, the games look normal and seem to have varied beginnings.


The first 10 results were
1, 0, 1, 0, 0, 0, 0, 0, 1, 0 .... (1 if #196 wins)
So, the cumulative winrate for #196 at move n, begins like this :
1, 0.5 , 0.66 , 0.5 , 0.4 , 0.33 , 0.29 , 0.25, 0.33 , 0.3 , ....
you can see these numbers in the left of each graph.


There are huge swings for the winrate in the first few games, and then, the winrate gets more and more stable with the number of games increasing.
The first picture is after 10, 50, and 100 games, with a dotted red line at 50%.
The second one is after 400 games, the dotted red lines are at 0.31 and 0.36, and the "final" win percentage is 0.33

I'd be very interested if someone could duplicate this experiment, I bet the graph after 400 games would look the same, but who knows...
Attachment:
3gr.gif
3gr.gif [ 11.67 KiB | Viewed 10327 times ]
Attachment:
400.gif
400.gif [ 9.82 KiB | Viewed 10327 times ]

Author:  And [ Tue Dec 25, 2018 3:00 am ]
Post subject:  Re: LZ's progression

thanks! do you think that with more time control 92297ff2 will also be stronger than lz196?
please upload the games 1min/game and 5min/game

Author:  And [ Tue Dec 25, 2018 10:18 am ]
Post subject:  Re: LZ's progression

accidentally discovered a shortened version of an elf - 136mb (same values, only rounded, original - 248mb) http://gridmaster.tengen.nl/engines/armv7l/. I launched a benchmark with the original, then with a shortened one. shortened runs about 12 percent faster. I wonder how this rounding affects the strength of the game? (LZ cpuonly, Athlon x4 950, leelaz.exe -w theNetworkfile.txt --benchmark -v 300, results: 90 sec and 80 sec)

Author:  Vargo [ Tue Dec 25, 2018 1:05 pm ]
Post subject:  Re: LZ's progression

I wanted to see if another 400 game match would end up with a graph similar to the first one, so, I ran a second 400 game match with the same parameters.
The answer is yes,the graphs look the same, but... the two final winrates seem very different (33% and 39%), particularly if you consider it's 400 games. Go figure :scratch: ;-)

The .bat file contains :
Code:
gogui-twogtp -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\196.gz --noponder" -black "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\92297ff_15x192.gz --noponder" -time 1 -games 400 -sgffile 196_15x192_1min -auto -komi 7.5 -alternate
Because of -alternate, in the even-numbered games #196 is always W, (and always B for the odd ones)
Again, there was 0 error, 0 duplicate game, and no contested score, all games by resignation.

The dotted red lines are at 0.38 and 0.42
Attachment:
400.gif
400.gif [ 9.84 KiB | Viewed 10152 times ]
The zip contains the 400 games, and the detailed stats (the html file)
Attachment:
196_15x192_1min.zip [331.61 KiB]
Downloaded 392 times

Author:  Vargo [ Wed Dec 26, 2018 8:18 am ]
Post subject:  Re: LZ's progression

Third and last 400 game match between #196 and #92297ff at 1 min per side per game.
Same parameters.

This time, there were 3 duplicate games among the 400, but no error or unagreed score.

The three final results are 33%, 39%, and 39%

Attachment:
gr.gif
gr.gif [ 11.89 KiB | Viewed 10025 times ]

The zip contains the 400 games of this last match, and the detailed stats (the html file)
Attachment:
196_15x192_1min_bis.zip [343.21 KiB]
Downloaded 433 times

Author:  Uberdude [ Sun Dec 30, 2018 10:20 am ]
Post subject:  Re: LZ's progression

Just one more data point for the 9229 network: I just used it against petgo3 on KGS which was using #197 on similar hardware to my own, 10x20s byoyomis, I/9229 won, see viewtopic.php?p=240161#p240161. So at 20 seconds per move (more playouts than the minutes per game here) on a medium GPU those extra playouts were still more important than the stronger intuition of the deeper network.

Author:  And [ Fri Jan 04, 2019 5:45 am ]
Post subject:  Re: LZ's progression

new network 15x192 from bjiyxo: 8b23a0ea VS 92297ff2 229 : 191 (54.52%). the strongest 15x192?
http://zero.sjeng.org/networks/8b23a0ea ... 8cf3b3a.gz

Author:  Vargo [ Thu Jan 10, 2019 11:45 pm ]
Post subject:  Re: LZ's progression

20 game match : #200 v. #157 (newest 40x256 v. last 15x192)

Time parity, 5min per side and per game, GPU: 1x1080, no pondering, komi 7.5, twogtp 1.5.0, ~205 moves/game (min=114, max=324), ~2sec/move
All games by resignation, no duplicate, no error.

Result : #200 wins 17-3
WOAW !

Stats :
Attachment:
200v157_5min.zip [11.33 KiB]
Downloaded 399 times

Author:  nbc44 [ Fri Jan 11, 2019 1:25 am ]
Post subject:  Re: LZ's progression

Vargo wrote:
20 game match : #200 v. #157 (newest 40x256 v. last 15x192)


Why not the latest 15x192 199-96k quantized?

Code:
2019-01-08 07:01   f438268e VS  92297ff2 230 : 174 (56.93%)   404 / 400   PASS
2018-12-22 02:42   92297ff2 VS  d351f06e 260 : 144 (64.36%)   404 / 400   PASS

Author:  Vargo [ Fri Jan 11, 2019 5:09 am ]
Post subject:  Re: LZ's progression

nbc44 wrote:
Why not the latest 15x192 199-96k quantized?
Why not, indeed !

20 game match : #200 v. #f438268

LZ0.16, Time parity, 5min per side and per game, GPU: 1x1080, no pondering, komi 7.5, twogtp 1.5.0, ~198 moves/game (min=106, max=304), ~2sec/move
All games by resignation, no duplicate, no error.

Result : #200 wins 11-9

STATS:
Attachment:
200_f438_5min.zip [11.22 KiB]
Downloaded 381 times


The games (200 is Black in the games 200_f438_5min_2-XXX, 200 is White for 200_f438_5min-XXX )
Attachment:
200_f438_5min_SGF.zip [17.01 KiB]
Downloaded 405 times

Author:  Mike Novack [ Fri Jan 11, 2019 7:18 am ]
Post subject:  Re: LZ's progression

Vargo wrote:
Result : #200 wins 11-9


Because it is likely that the majority here have never had a course in statistics, it might be useful, when posting something like this to add "no meaningful difference".

In other words, if you flipped an honest coin 20 times you would expect a result at least as unbalanced as 11-9 << 10-10 might be the single most likely outcome BUT there are so many other outcomes possible that we expect something else. For example, 11-9 or 9-11, one of these or the other, are together more likely than 10-10>>

Author:  Vargo [ Fri Jan 11, 2019 8:30 am ]
Post subject:  Re: LZ's progression

You're absolutely right. As the result was very close, I immediately ran a second 20 game match, with...
the same result.
STATS
Attachment:
200isB.gif
200isB.gif [ 96.6 KiB | Viewed 9345 times ]
STATS
Attachment:
200isW.gif
200isW.gif [ 95.09 KiB | Viewed 9345 times ]
Attachment:
200vf438_5min.zip [17.55 KiB]
Downloaded 382 times

But I completely agree with you, it could very well have been 8-12, 9-11, 10-10, 11-9, or 12-8.
What it (probably) means, is they're about the same strength.
On the other hand, even with only 20 games, 17-3 probably means that the winner IS stronger (if they were the same strength, it would be very infrequent to get 17-3)

Author:  nbc44 [ Fri Jan 11, 2019 3:35 pm ]
Post subject:  Re: LZ's progression

Vargo wrote:
...I immediately ran a second 20 game match, with...
the same result.

Now then:
Code:
2019-01-11 08:30  #200 VS f438268e 22 : 18 (55.00%)   40 / 40   PASS

But this is a modest result for bigger network. Long live 15x192 :lol: .

Author:  Vargo [ Fri Jan 11, 2019 10:33 pm ]
Post subject:  Re: LZ's progression

nbc44 wrote:
Long live 15x192 :lol: .


:D I'm not rooting for or against #200, I just like to run this kind of matches, and I always give the results exactly as they come.
I don't know your parameters and hardware.

Here is my command line for a 20 game match (twogtp 1.5.0)
Code:
gogui-twogtp -white "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\200.gz --noponder" -black "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\f438268.gz --noponder" -time 5 -games 10 -sgffile ***NAME OF SGF*** -auto -komi 7.5
gogui-twogtp -black "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\200.gz --noponder" -white "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\f438268.gz --noponder" -time 5 -games 10 -sgffile ***NAME OF SGF*** -auto -komi 7.5

If someone wants to run the "same" match at the same strength, but with a different GPU, it can be done :
time parity at 5min per side per game with my GTX1080 is around 2sec/move, and it corresponds to something like -v 1600 for #200 and -v 5000 for #f438.

Long live LZ ;-)

Author:  splee99 [ Fri Jan 11, 2019 11:30 pm ]
Post subject:  Re: LZ's progression

Personally I like #199 more. The games of #199 are so much more enjoyable to watch because #199 likes to attack. #200 maybe better than #199 in some aspects, but it always plays some boring moves such as solid defense, rarely tenuki or sacrifice few stones to gain influence.

Author:  And [ Wed Jan 16, 2019 11:33 am ]
Post subject:  Re: LZ's progression

Vargo,
in the latest version AhQ Go (2.4.2) in the list of recommended weight appeared new network 192x5 Ms5. much stronger than f438268e. could you test it?

Author:  Vargo [ Sun Jan 20, 2019 2:02 am ]
Post subject:  Re: LZ's progression

Do you have a link to a network file usable with LZ0.16 ?

Author:  And [ Sun Jan 20, 2019 8:01 am ]
Post subject:  Re: LZ's progression

https://drive.google.com/file/d/11uBKRz ... sp=sharing
when using LZ0.16 and Ms5-192x15 or f438268e sometimes LZ0.16 freezes. better lz0.15

Author:  Vargo [ Sun Jan 20, 2019 9:23 am ]
Post subject:  Re: LZ's progression

Thanks.
f438268 is OK with LZ016 and twogtp on my computer, but the combination Ms5, LZ(015 or 016), and twogtp is not OK, it crashes every time :sad:

Page 13 of 21 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/