It is currently Fri Nov 15, 2019 6:44 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 425 posts ]  Go to page Previous  1 ... 18, 19, 20, 21, 22  Next
Author Message
Offline
 Post subject: Re: LZ's progression
Post #401 Posted: Tue May 07, 2019 7:25 am 
Dies in gote

Posts: 53
Liked others: 3
Was liked: 33
Rank: KGS 2k
There was a new 40b network promoted today, and it was also tested officially against a other 40b network from 10 networks ago.

The result is a 69% winrate against the older 40b network from 10 networks back. Rather impressive!
The 40b network is clearly still growing and learning and getting stronger at a good pace!

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #402 Posted: Wed May 08, 2019 3:52 am 
Judan

Posts: 6175
Location: Cambridge, UK
Liked others: 353
Was liked: 3335
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Aram wrote:
There was a new 40b network promoted today, and it was also tested officially against a other 40b network from 10 networks ago.

The result is a 69% winrate against the older 40b network from 10 networks back. Rather impressive!
The 40b network is clearly still growing and learning and getting stronger at a good pace!


I wonder what the improvement in winrate of this latest 40b compared to 10 networks ago would be versus a non-LZ bot (for fairest comparison would be time parity but even visit parity I expect same effect). Eg maybe #215 beats Elfv2 at 1600 visit parity 65% of the time (and maybe <50% at time parity), and #225 beats it only 67%. I expect quite a lot less difference than 19% (or equivalent ratio difference).

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #403 Posted: Wed May 08, 2019 6:42 pm 
Dies with sente

Posts: 94
Liked others: 2
Was liked: 15
Rank: KGS 2 D
Here is a game between LZ-225 (B) + LZ0.17 and ELF (W) + LZ0.16 in time parity (20 seconds per move).
The ELFv2 was far ahead with winrate close to 90%, until move 135, when LZ-225 suddenly started a complicated fight. After that ELF still had lots of chance to win the fight, according to LZ-225, but eventually ELF gave up.



Attachment:
225-elf-20s.sgf [3.77 KiB]
Downloaded 1553 times

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #404 Posted: Thu May 09, 2019 12:20 am 
Dies in gote

Posts: 53
Liked others: 3
Was liked: 33
Rank: KGS 2k
One shouldnt use ELF winrates or ELF in general for analysis.

They sharpened the network a rather huge amount to make it stronger in self play and play versus other bots.
(Probably because they plateud and needed to get some more strength out of the relatively small 20-ish block network)
So basically ELFs possibility to grow and explore new and different moves was terminated and it became stronger on a narrow path of moves.

This also sadly led the network to believe that every match is either a 100% win or a 100% loss.

This is quite visible for anyone using both ELF and LZ networks. ELF will always be very black and white and whats good or bad, while the LZ networks provide several moves which are playable.


LZ could do the same and would probably see a boost in strength, but at the same time it probably wouldnt improve anymore and it wouldnt explore either.
Since the LZ network is larger, it still has capacity to grow and improve by searching and exploring, and as such its not needed to try to force it to its full strength right now.

Same thing with learning rates, one could probobly get a small increase in LZ strength by just dropping the learning rate, but that would kill any further increase in playing strength from exploration.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #405 Posted: Thu May 09, 2019 10:09 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
20 game match LZ017#225 v LZ017#215
no pondering, komi 7.5, -v 1600
twogtp 1.5.1 , no error, no duplicate game.

#225 wins 17-3
Wow ! It confirms the 69% , and it's only a 10 networks difference... :clap:

The stats :
Attachment:
225.gif
225.gif [ 46.02 KiB | Viewed 2717 times ]
The games (225 is B in the odd-numbered games) :
Attachment:
225_215_v 1600.rar [15.32 KiB]
Downloaded 23 times

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #406 Posted: Fri May 17, 2019 3:36 am 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
Сity legend check (l0XXX 1600 visits vs elfv2 3200 visits):

2 x 1080ti

[Engine_Master]
enginedir = C:\APPS\l0gpu17\
engine = leelaz.exe -g --gpu 0 --gpu 1 -t 24 --batchsize 12 -r 5 --noponder --timemanage off --precision single
weightdir = C:\APPS\net\
weight = 05dbca15.gz
name = ELFV2

[Engine_01]
enginedir = C:\APPS\l0gpu17\
engine = leelaz.exe -g --gpu 0 --gpu 1 -t 24 --batchsize 12 -r 5 --noponder --timemanage off --precision single
weightdir = C:\APPS\net\
weight = a20c31da.gz
name = LZ226

[Engine_02]
enginedir = C:\APPS\l0gpu17\
engine = leelaz.exe -g --gpu 0 --gpu 1 -t 24 --batchsize 12 -r 5 --noponder --timemanage off --precision single
weightdir = C:\APPS\net\
weight = d42f950a.gz
name = LZ225

[Engine_03]
enginedir = C:\APPS\l0gpu17\
engine = leelaz.exe -g --gpu 0 --gpu 1 -t 24 --batchsize 12 -r 5 --noponder --timemanage off --precision single
weightdir = C:\APPS\net\
weight = f3be3bf8.gz
name = LZ224

Code:
===================================================================================================================
From:May-07-19 07:12:33 to May-08-19 05:08:00. Spend 78927.58483s
                         Game << 400 >>
LZ224                    1600 visits  win(All) = 249  62.25%        win(B) = 115  57.50%        win(W) = 134  67.00%
ELFV2                    1600 visits  win(All) = 151  37.75%        win(B) =  66  33.00%        win(W) =  85  42.50%

===================================================================================================================
From:May-09-19 07:42:53 to May-10-19 06:46:29. Spend 83015.170982s
                         Game << 400 >>
LZ224                    1600 visits  win(All) = 212  53.00%        win(B) = 102  51.00%        win(W) = 110  55.00%
ELFV2                    2000 visits  win(All) = 188  47.00%        win(B) =  90  45.00%        win(W) =  98  49.00%

===================================================================================================================
From:May-08-19 05:08:00 to May-09-19 05:47:02. Spend 88741.358819s
                         Game << 400 >>
LZ224                    1600 visits  win(All) = 180  45.00%        win(B) =  91  45.50%        win(W) =  89  44.50%
ELFV2                    2400 visits  win(All) = 220  55.00%        win(B) = 111  55.50%        win(W) = 109  54.50%

===================================================================================================================
From:May-10-19 07:37:42 to May-11-19 07:16:09. Spend 85106.834626s
                         Game << 400 >>
LZ225                    1600 visits  win(All) = 207  51.75%        win(B) =  94  47.00%        win(W) = 113  56.50%
ELFV2                    2000 visits  win(All) = 193  48.25%        win(B) =  87  43.50%        win(W) = 106  53.00%

===================================================================================================================
From:May-11-19 07:16:09 to May-12-19 07:23:49. Spend 86860.492488s
                         Game << 400 >>
LZ225                    1600 visits  win(All) = 172  43.00%        win(B) =  81  40.50%        win(W) =  91  45.50%
ELFV2                    2200 visits  win(All) = 228  57.00%        win(B) = 109  54.50%        win(W) = 119  59.50%

===================================================================================================================
From:May-14-19 06:55:16 to May-15-19 06:12:54. Spend 83858.237752s
                         Game << 400 >>
LZ226                    1600 visits  win(All) = 212  53.00%        win(B) = 100  50.00%        win(W) = 112  56.00%
ELFV2                    2000 visits  win(All) = 188  47.00%        win(B) =  88  44.00%        win(W) = 100  50.00%

===================================================================================================================
From:May-15-19 06:12:54 to May-16-19 05:59:42. Spend 85607.877284s
                         Game << 400 >>
LZ226                    1600 visits  win(All) = 196  49.00%        win(B) =  94  47.00%        win(W) = 102  51.00%
ELFV2                    2200 visits  win(All) = 204  51.00%        win(B) =  98  49.00%        win(W) = 106  53.00%


P.S. If someone wants the games, I can upload them.


This post by nbc44 was liked by: SHKD
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #407 Posted: Fri May 17, 2019 11:16 am 
Beginner

Posts: 5
Liked others: 0
Was liked: 0
Rank: KGS 3d
KGS: CuTy
Can I have the games, nbc44?
For studying, of course! :study:

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #408 Posted: Fri May 17, 2019 7:16 pm 
Dies with sente

Posts: 94
Liked others: 2
Was liked: 15
Rank: KGS 2 D
For higher playouts, for example 2500 vs 5000, ELF becomes weaker than 226. It's just my initial observation.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #409 Posted: Sat May 18, 2019 4:34 am 
Beginner

Posts: 1
Liked others: 3
Was liked: 0
nbc44 wrote:
Сity legend check (l0XXX 1600 visits vs elfv2 3200 visits):

P.S. If someone wants the games, I can upload them.


I'd like to have these games, interesting to take a look at the next LZ vs ELF episode)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #410 Posted: Fri May 24, 2019 4:04 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
Interesting topic HERE about a new LZ network.

The provided network (14a3a) looked very promising...
I've tested it, using regular LZ017, first at H9 (komi 0.5) against gnugo (5 kyu)
it won 2 games out of 3 !
(laptop with gtx965, ~3 sec/move effectively used)

Sabaki settings for GnuGo:
Code:
--mode gtp  --level 10
Sabaki settings for for 14a3a
Code:
C:\...\leela-zero-0.17-win64\leelaz.exe   AND   --gtp -w C:...\14a3a5f70.gz --noponder -r 0  AND  time_settings 0 5 1    -komi 0.5

One of the H9 games : here




Second test : 50-game match between 14a3a and #226 (--noponder -v 1600)
14a3a won 52% (gogui 1.5.1, no error, no duplicate game, ~200 moves/game, ~170s/game for #226 or for 14a3a, all games by resignation)

the stats :
Attachment:
stat.gif
stat.gif [ 160.02 KiB | Viewed 2146 times ]
the games :
Attachment:
lz_226v14a3.zip [42.17 KiB]
Downloaded 18 times

So, a very interesting network, despite the poor "official" result against 226...


This post by Vargo was liked by: SHKD
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #411 Posted: Fri May 24, 2019 7:44 am 
Lives with ko

Posts: 161
Liked others: 31
Was liked: 10
Vargo wrote:
Second test : 50-game match between 14a3a and #226 (--noponder -v 1600)
14a3a won 52% (gogui 1.5.1, no error, no duplicate game, ~200 moves/game, ~170s/game for #226 or for 14a3a, all games by resignation)

the stats :
Attachment:
stat.gif
the games :
Attachment:
lz_226v14a3.zip

So, a very interesting network, despite the poor "official" result against 226...

the less games, the less reliable result. “Official” result obtained from 158 games.
99a8d3a3 - 226:174 and 115:132 in the official test. maybe even 400 games is not enough


Last edited by And on Fri May 24, 2019 8:22 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #412 Posted: Fri May 24, 2019 8:01 am 
Lives with ko

Posts: 161
Liked others: 31
Was liked: 10
I wonder why you chose gnugo for handicap tests? the program is quite old. Now programs play differently. possible choose something newer with adjustable strength. By the way, have you tried gnugo with the --cache-size option? (e.g. 256mb)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #413 Posted: Fri May 24, 2019 8:45 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
And wrote:
the less games, the less reliable result. “Official” result obtained from 158 games.
Yes, but in the original post about this network, a lot more than 158 games have been played by 14a3a, and the results were 50+% vs 226

And wrote:
I wonder why you chose gnugo for handicap tests?
GnuGo is 5 kyu, it's a reference point.
Ideally, I'd choose Crazy Stone DL for handicap games, it can be set anywhere from 13 kyu to 7 Dan , but CSDL can't be used in Sabaki...

But you're right, these networks are so strong at handicap, that I have only two choices :
1) Forget GnuGo and find another reference point
2) Keep on using Gnugo and ask Sabaki's authors to add the possibility to play at H10, H11, etc. ;-)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #414 Posted: Fri May 24, 2019 11:24 am 
Lives with ko

Posts: 161
Liked others: 31
Was liked: 10
Vargo,
and if to use gtp4zen or cs dl directly through gtool2? (tool for auto-playing between two igo softs. but a lot of games will of course be tedious)

PS I checked gtp4zen 0.3.5 (zen7), gogui 1.5.1, H9, settings zen7: 5K -t1 -r5 -s800 -n1 -o2.4 -p0.3 (https://github.com/breakwa11/GoAIRating ... /AIcmds.md). gtp4zen playing white, then black - all ok

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #415 Posted: Sat May 25, 2019 7:15 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
I tried LZ017_14a3a v gtp4zen(7) at H9

First, with --noponder -v 200 -r 1 for LZ
But... -r 1 makes it resign too soon, for example, in this position. I think a 5kyu can be invaded by W.
Attachment:
toosoon.jpg
toosoon.jpg [ 140.44 KiB | Viewed 1993 times ]



After that, I tried --noponder -v 200 -r 0 for LZ
with the hope that LZ wouldn't resign so soon.
These settings work when LZ_14a3a wins, for example in these games against gtp4zen.exe (-t4 -r5 -s800 -n1 -o2.4 -p0.3, corresponding to ~5 kyu)



or




But against (gtp4zen.exe -t4 -s1000 -n1 -o2.0 -p0.3) , corresponding to ~3k,
H9 is too much, and 14a3a loses heavily, but thinks it has won (the correct score is RES_B)

#GAME...RES_B....RES_W
0.......B+92.5...W+12.5
1.......B+102.5..W+3.5




Go figure :scratch: !


EDIT : The two posts with various handicaps contained 2 errors in gtp4zen parameters (3d and 5d parameters were wrong) , I removed them.


Attachments:
z7_5kv14ar0-0.sgf [1.86 KiB]
Downloaded 1136 times
z7_5kv14ar0-1.sgf [1.48 KiB]
Downloaded 1149 times
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #416 Posted: Sun Jun 02, 2019 8:05 am 
Lives with ko

Posts: 161
Liked others: 31
Was liked: 10
What will happen to the new network LZ? search to win? or will something change?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #417 Posted: Mon Jun 03, 2019 3:04 pm 
Lives with ko

Posts: 161
Liked others: 31
Was liked: 10
False promotion lz227? https://github.com/leela-zero/leela-zero/issues/2421

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #418 Posted: Tue Jun 11, 2019 9:05 am 
Lives with ko

Posts: 161
Liked others: 31
Was liked: 10
Extensive tests of LZ improvements: true ELO gain is ~5 ELO per net since LZ174 https://github.com/leela-zero/leela-zero/issues/2425


This post by And was liked by: SHKD
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #419 Posted: Tue Aug 06, 2019 10:55 am 
Lives with ko

Posts: 161
Liked others: 31
Was liked: 10
new rules ? 923 games!? http://zero.sjeng.org
2019-08-06 16:14 88765231 VS a4f5d99a 390 : 318 (55.08%) 708 / 400 PASS
2019-08-06 16:14 88765231 VS a4f5d99a 498 : 425 (53.95%) 923 / 400 fail

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #420 Posted: Fri Aug 16, 2019 6:19 am 
Dies in gote

Posts: 41
Liked others: 2
Was liked: 14
Rank: EGF 1 kyu
KGS: finity
And wrote:
new rules ? 923 games!? http://zero.sjeng.org
2019-08-06 16:14 88765231 VS a4f5d99a 390 : 318 (55.08%) 708 / 400 PASS
2019-08-06 16:14 88765231 VS a4f5d99a 498 : 425 (53.95%) 923 / 400 fail


I recall that the gating test is a probability one, it might be that if you're very close to the treshold, it keeps playing until certain probability is reached.

Still, 900 games sounds like tossing a coin twenty time to get heads. Another plausible explanation is some server configuration/connectivity issue that gave out a lot of match games to clients but couldn't receive them, and once that was resolved, finally got them all (or most) back.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 425 posts ]  Go to page Previous  1 ... 18, 19, 20, 21, 22  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: Google [Bot] and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group