Life In 19x19
http://lifein19x19.com/

LZ's progression
http://lifein19x19.com/viewtopic.php?f=18&t=15718
Page 21 of 21

Author:  splee99 [ Fri May 17, 2019 7:16 pm ]
Post subject:  Re: LZ's progression

For higher playouts, for example 2500 vs 5000, ELF becomes weaker than 226. It's just my initial observation.

Author:  SHKD [ Sat May 18, 2019 4:34 am ]
Post subject:  Re: LZ's progression

nbc44 wrote:
Сity legend check (l0XXX 1600 visits vs elfv2 3200 visits):

P.S. If someone wants the games, I can upload them.


I'd like to have these games, interesting to take a look at the next LZ vs ELF episode)

Author:  Vargo [ Fri May 24, 2019 4:04 am ]
Post subject:  Re: LZ's progression

Interesting topic HERE about a new LZ network.

The provided network (14a3a) looked very promising...
I've tested it, using regular LZ017, first at H9 (komi 0.5) against gnugo (5 kyu)
it won 2 games out of 3 !
(laptop with gtx965, ~3 sec/move effectively used)

Sabaki settings for GnuGo:
Code:
--mode gtp  --level 10
Sabaki settings for for 14a3a
Code:
C:\...\leela-zero-0.17-win64\leelaz.exe   AND   --gtp -w C:...\14a3a5f70.gz --noponder -r 0  AND  time_settings 0 5 1    -komi 0.5

One of the H9 games : here




Second test : 50-game match between 14a3a and #226 (--noponder -v 1600)
14a3a won 52% (gogui 1.5.1, no error, no duplicate game, ~200 moves/game, ~170s/game for #226 or for 14a3a, all games by resignation)

the stats :
Attachment:
stat.gif
stat.gif [ 160.02 KiB | Viewed 9791 times ]
the games :
Attachment:
lz_226v14a3.zip [42.17 KiB]
Downloaded 442 times

So, a very interesting network, despite the poor "official" result against 226...

Author:  And [ Fri May 24, 2019 7:44 am ]
Post subject:  Re: LZ's progression

Vargo wrote:
Second test : 50-game match between 14a3a and #226 (--noponder -v 1600)
14a3a won 52% (gogui 1.5.1, no error, no duplicate game, ~200 moves/game, ~170s/game for #226 or for 14a3a, all games by resignation)

the stats :
Attachment:
stat.gif
the games :
Attachment:
lz_226v14a3.zip

So, a very interesting network, despite the poor "official" result against 226...

the less games, the less reliable result. “Official” result obtained from 158 games.
99a8d3a3 - 226:174 and 115:132 in the official test. maybe even 400 games is not enough

Author:  And [ Fri May 24, 2019 8:01 am ]
Post subject:  Re: LZ's progression

I wonder why you chose gnugo for handicap tests? the program is quite old. Now programs play differently. possible choose something newer with adjustable strength. By the way, have you tried gnugo with the --cache-size option? (e.g. 256mb)

Author:  Vargo [ Fri May 24, 2019 8:45 am ]
Post subject:  Re: LZ's progression

And wrote:
the less games, the less reliable result. “Official” result obtained from 158 games.
Yes, but in the original post about this network, a lot more than 158 games have been played by 14a3a, and the results were 50+% vs 226

And wrote:
I wonder why you chose gnugo for handicap tests?
GnuGo is 5 kyu, it's a reference point.
Ideally, I'd choose Crazy Stone DL for handicap games, it can be set anywhere from 13 kyu to 7 Dan , but CSDL can't be used in Sabaki...

But you're right, these networks are so strong at handicap, that I have only two choices :
1) Forget GnuGo and find another reference point
2) Keep on using Gnugo and ask Sabaki's authors to add the possibility to play at H10, H11, etc. ;-)

Author:  And [ Fri May 24, 2019 11:24 am ]
Post subject:  Re: LZ's progression

Vargo,
and if to use gtp4zen or cs dl directly through gtool2? (tool for auto-playing between two igo softs. but a lot of games will of course be tedious)

PS I checked gtp4zen 0.3.5 (zen7), gogui 1.5.1, H9, settings zen7: 5K -t1 -r5 -s800 -n1 -o2.4 -p0.3 (https://github.com/breakwa11/GoAIRating ... /AIcmds.md). gtp4zen playing white, then black - all ok

Author:  Vargo [ Sat May 25, 2019 7:15 am ]
Post subject:  Re: LZ's progression

I tried LZ017_14a3a v gtp4zen(7) at H9

First, with --noponder -v 200 -r 1 for LZ
But... -r 1 makes it resign too soon, for example, in this position. I think a 5kyu can be invaded by W.
Attachment:
toosoon.jpg
toosoon.jpg [ 140.44 KiB | Viewed 9638 times ]



After that, I tried --noponder -v 200 -r 0 for LZ
with the hope that LZ wouldn't resign so soon.
These settings work when LZ_14a3a wins, for example in these games against gtp4zen.exe (-t4 -r5 -s800 -n1 -o2.4 -p0.3, corresponding to ~5 kyu)



or




But against (gtp4zen.exe -t4 -s1000 -n1 -o2.0 -p0.3) , corresponding to ~3k,
H9 is too much, and 14a3a loses heavily, but thinks it has won (the correct score is RES_B)

#GAME...RES_B....RES_W
0.......B+92.5...W+12.5
1.......B+102.5..W+3.5




Go figure :scratch: !


EDIT : The two posts with various handicaps contained 2 errors in gtp4zen parameters (3d and 5d parameters were wrong) , I removed them.

Attachments:
z7_5kv14ar0-0.sgf [1.86 KiB]
Downloaded 1710 times
z7_5kv14ar0-1.sgf [1.48 KiB]
Downloaded 1723 times

Author:  And [ Sun Jun 02, 2019 8:05 am ]
Post subject:  Re: LZ's progression

What will happen to the new network LZ? search to win? or will something change?

Author:  And [ Mon Jun 03, 2019 3:04 pm ]
Post subject:  Re: LZ's progression

False promotion lz227? https://github.com/leela-zero/leela-zero/issues/2421

Author:  And [ Tue Jun 11, 2019 9:05 am ]
Post subject:  Re: LZ's progression

Extensive tests of LZ improvements: true ELO gain is ~5 ELO per net since LZ174 https://github.com/leela-zero/leela-zero/issues/2425

Author:  And [ Tue Aug 06, 2019 10:55 am ]
Post subject:  Re: LZ's progression

new rules ? 923 games!? http://zero.sjeng.org
2019-08-06 16:14 88765231 VS a4f5d99a 390 : 318 (55.08%) 708 / 400 PASS
2019-08-06 16:14 88765231 VS a4f5d99a 498 : 425 (53.95%) 923 / 400 fail

Author:  jokkebk [ Fri Aug 16, 2019 6:19 am ]
Post subject:  Re: LZ's progression

And wrote:
new rules ? 923 games!? http://zero.sjeng.org
2019-08-06 16:14 88765231 VS a4f5d99a 390 : 318 (55.08%) 708 / 400 PASS
2019-08-06 16:14 88765231 VS a4f5d99a 498 : 425 (53.95%) 923 / 400 fail


I recall that the gating test is a probability one, it might be that if you're very close to the treshold, it keeps playing until certain probability is reached.

Still, 900 games sounds like tossing a coin twenty time to get heads. Another plausible explanation is some server configuration/connectivity issue that gave out a lot of match games to clients but couldn't receive them, and once that was resolved, finally got them all (or most) back.

Author:  Aram [ Fri Aug 16, 2019 9:26 am ]
Post subject:  Re: LZ's progression

Nothing to see here.. it was an error and it has been fixed in the server code so it would not happen again..

Author:  And [ Wed Aug 21, 2019 12:09 pm ]
Post subject:  Re: LZ's progression

LZ#239 VS LZ#230 53.00% ???
2019-08-21 17:07 3bd93d6e VS a9543f21 212 : 188 (53.00%) 400 / 400 fail
http://zero.sjeng.org

Author:  iopq [ Fri Aug 23, 2019 3:27 pm ]
Post subject:  Re: LZ's progression

LZ240 still beat LZ230
0e17a181 VS a9543f21
225 : 176 (56.11%) 401 / 400 PASS

Then from discord jio tested

Quote:
lz_240 v lz_230 (400/400 games)
board size: 19 komi: 7.5
wins black white avg cpu
lz_240 233 58.25% 106 53.00% 127 63.50% 214.76
lz_230 167 41.75% 73 36.50% 94 47.00% 216.98
179 44.75% 221 55.25%

cmdline options: "-g -t 6 --batchsize 5 --noponder -v 1600 -r 5 --precision half"


this is in line with previous results, slower progress, but progress nevertheless

Author:  hydrogenpi7 [ Sun Oct 06, 2019 2:21 am ]
Post subject:  Re: LZ's progression

iopq wrote:
LZ240 still beat LZ230
0e17a181 VS a9543f21
225 : 176 (56.11%) 401 / 400 PASS

Then from discord jio tested

Quote:
lz_240 v lz_230 (400/400 games)
board size: 19 komi: 7.5
wins black white avg cpu
lz_240 233 58.25% 106 53.00% 127 63.50% 214.76
lz_230 167 41.75% 73 36.50% 94 47.00% 216.98
179 44.75% 221 55.25%

cmdline options: "-g -t 6 --batchsize 5 --noponder -v 1600 -r 5 --precision half"



average of 3 real elo per net?

this is in line with previous results, slower progress, but progress nevertheless

Author:  nbc44 [ Fri Oct 11, 2019 7:46 pm ]
Post subject:  Re: LZ's progression

LZv17 #219 vs LZ (MiniGo training data) (https://github.com/leela-zero/leela-zero/issues/2509)
2x1080ti, v 2400
C:\APPS\l0gpu17\validation.exe -k lz247-minigo17 -s "0:1" -g 5 -n C:\APPS\net\901e04de.gz -o "-g -v 2400 --gpu 0 --gpu 1 -r 5 -t 2 --batchsize 1 --noponder -q -d --timemanage off --precision single -w" -n C:\APPS\net\leelaz-model-swa-16-768000.gz -o "-g -v 2400 --gpu 0 --gpu 1 -r 5 -t 2 --batchsize 1 --noponder -q -d --timemanage off --precision single -w " -- C:\APPS\l0gpu17\leelaz -- C:\APPS\l0gpu17\leelaz

Code:
#247 v MG17 ( 400 games)
          wins        black       white
#247  178 44.50%   72 43.64%  106 45.11%
MG17  222 55.50%   93 56.36%  129 54.89%
                  165 41.25%  235 58.75%
400 games played.
Status: 0 LLR -0.128628 Lower Bound -2.94444 Upper Bound 2.94444

:clap:

Attachments:
lz247-minigo17.zip [368.65 KiB]
Downloaded 409 times

Page 21 of 21 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/