It is currently Thu Mar 28, 2024 2:42 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 418 posts ]  Go to page Previous  1 ... 10, 11, 12, 13, 14, 15, 16 ... 21  Next
Author Message
Offline
 Post subject: Re: LZ's progression
Post #241 Posted: Mon Dec 24, 2018 8:16 am 
Dies in gote

Posts: 41
Liked others: 0
Was liked: 7
OGS: roy7
Universal go server handle: roy7
For those who might prefer it, there's an experimental "new" home page at http://zero.sjeng.org/home.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #242 Posted: Mon Dec 24, 2018 9:24 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
400 is the magic number ;-) , so, I extended the match between #196 (40x256) and 92297ff2 (15x192) to 400 games (more than 10 hours!).

time parity : 1 min per side per game, komi 7.5, no pondering, 1x1080, LZ0.16, twogtp V1.5.0
0 duplicate game, 0 error, all games by resignation, no contested scores, the games look normal and seem to have varied beginnings.


The first 10 results were
1, 0, 1, 0, 0, 0, 0, 0, 1, 0 .... (1 if #196 wins)
So, the cumulative winrate for #196 at move n, begins like this :
1, 0.5 , 0.66 , 0.5 , 0.4 , 0.33 , 0.29 , 0.25, 0.33 , 0.3 , ....
you can see these numbers in the left of each graph.


There are huge swings for the winrate in the first few games, and then, the winrate gets more and more stable with the number of games increasing.
The first picture is after 10, 50, and 100 games, with a dotted red line at 50%.
The second one is after 400 games, the dotted red lines are at 0.31 and 0.36, and the "final" win percentage is 0.33

I'd be very interested if someone could duplicate this experiment, I bet the graph after 400 games would look the same, but who knows...
Attachment:
3gr.gif
3gr.gif [ 11.67 KiB | Viewed 10339 times ]
Attachment:
400.gif
400.gif [ 9.82 KiB | Viewed 10339 times ]


This post by Vargo was liked by 2 people: And, Waylon
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #243 Posted: Tue Dec 25, 2018 3:00 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
thanks! do you think that with more time control 92297ff2 will also be stronger than lz196?
please upload the games 1min/game and 5min/game

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #244 Posted: Tue Dec 25, 2018 10:18 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
accidentally discovered a shortened version of an elf - 136mb (same values, only rounded, original - 248mb) http://gridmaster.tengen.nl/engines/armv7l/. I launched a benchmark with the original, then with a shortened one. shortened runs about 12 percent faster. I wonder how this rounding affects the strength of the game? (LZ cpuonly, Athlon x4 950, leelaz.exe -w theNetworkfile.txt --benchmark -v 300, results: 90 sec and 80 sec)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #245 Posted: Tue Dec 25, 2018 1:05 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
I wanted to see if another 400 game match would end up with a graph similar to the first one, so, I ran a second 400 game match with the same parameters.
The answer is yes,the graphs look the same, but... the two final winrates seem very different (33% and 39%), particularly if you consider it's 400 games. Go figure :scratch: ;-)

The .bat file contains :
Code:
gogui-twogtp -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\196.gz --noponder" -black "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\92297ff_15x192.gz --noponder" -time 1 -games 400 -sgffile 196_15x192_1min -auto -komi 7.5 -alternate
Because of -alternate, in the even-numbered games #196 is always W, (and always B for the odd ones)
Again, there was 0 error, 0 duplicate game, and no contested score, all games by resignation.

The dotted red lines are at 0.38 and 0.42
Attachment:
400.gif
400.gif [ 9.84 KiB | Viewed 10164 times ]
The zip contains the 400 games, and the detailed stats (the html file)
Attachment:
196_15x192_1min.zip [331.61 KiB]
Downloaded 394 times


This post by Vargo was liked by: And
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #246 Posted: Wed Dec 26, 2018 8:18 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Third and last 400 game match between #196 and #92297ff at 1 min per side per game.
Same parameters.

This time, there were 3 duplicate games among the 400, but no error or unagreed score.

The three final results are 33%, 39%, and 39%

Attachment:
gr.gif
gr.gif [ 11.89 KiB | Viewed 10037 times ]

The zip contains the 400 games of this last match, and the detailed stats (the html file)
Attachment:
196_15x192_1min_bis.zip [343.21 KiB]
Downloaded 434 times

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #247 Posted: Sun Dec 30, 2018 10:20 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Just one more data point for the 9229 network: I just used it against petgo3 on KGS which was using #197 on similar hardware to my own, 10x20s byoyomis, I/9229 won, see viewtopic.php?p=240161#p240161. So at 20 seconds per move (more playouts than the minutes per game here) on a medium GPU those extra playouts were still more important than the stronger intuition of the deeper network.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #248 Posted: Fri Jan 04, 2019 5:45 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
new network 15x192 from bjiyxo: 8b23a0ea VS 92297ff2 229 : 191 (54.52%). the strongest 15x192?
http://zero.sjeng.org/networks/8b23a0ea ... 8cf3b3a.gz

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #249 Posted: Thu Jan 10, 2019 11:45 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
20 game match : #200 v. #157 (newest 40x256 v. last 15x192)

Time parity, 5min per side and per game, GPU: 1x1080, no pondering, komi 7.5, twogtp 1.5.0, ~205 moves/game (min=114, max=324), ~2sec/move
All games by resignation, no duplicate, no error.

Result : #200 wins 17-3
WOAW !

Stats :
Attachment:
200v157_5min.zip [11.33 KiB]
Downloaded 401 times

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #250 Posted: Fri Jan 11, 2019 1:25 am 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
Vargo wrote:
20 game match : #200 v. #157 (newest 40x256 v. last 15x192)


Why not the latest 15x192 199-96k quantized?

Code:
2019-01-08 07:01   f438268e VS  92297ff2 230 : 174 (56.93%)   404 / 400   PASS
2018-12-22 02:42   92297ff2 VS  d351f06e 260 : 144 (64.36%)   404 / 400   PASS

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #251 Posted: Fri Jan 11, 2019 5:09 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
nbc44 wrote:
Why not the latest 15x192 199-96k quantized?
Why not, indeed !

20 game match : #200 v. #f438268

LZ0.16, Time parity, 5min per side and per game, GPU: 1x1080, no pondering, komi 7.5, twogtp 1.5.0, ~198 moves/game (min=106, max=304), ~2sec/move
All games by resignation, no duplicate, no error.

Result : #200 wins 11-9

STATS:
Attachment:
200_f438_5min.zip [11.22 KiB]
Downloaded 382 times


The games (200 is Black in the games 200_f438_5min_2-XXX, 200 is White for 200_f438_5min-XXX )
Attachment:
200_f438_5min_SGF.zip [17.01 KiB]
Downloaded 406 times

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #252 Posted: Fri Jan 11, 2019 7:18 am 
Lives in sente

Posts: 1037
Liked others: 0
Was liked: 180
Vargo wrote:
Result : #200 wins 11-9


Because it is likely that the majority here have never had a course in statistics, it might be useful, when posting something like this to add "no meaningful difference".

In other words, if you flipped an honest coin 20 times you would expect a result at least as unbalanced as 11-9 << 10-10 might be the single most likely outcome BUT there are so many other outcomes possible that we expect something else. For example, 11-9 or 9-11, one of these or the other, are together more likely than 10-10>>

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #253 Posted: Fri Jan 11, 2019 8:30 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
You're absolutely right. As the result was very close, I immediately ran a second 20 game match, with...
the same result.
STATS
Attachment:
200isB.gif
200isB.gif [ 96.6 KiB | Viewed 9357 times ]
STATS
Attachment:
200isW.gif
200isW.gif [ 95.09 KiB | Viewed 9357 times ]
Attachment:
200vf438_5min.zip [17.55 KiB]
Downloaded 383 times

But I completely agree with you, it could very well have been 8-12, 9-11, 10-10, 11-9, or 12-8.
What it (probably) means, is they're about the same strength.
On the other hand, even with only 20 games, 17-3 probably means that the winner IS stronger (if they were the same strength, it would be very infrequent to get 17-3)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #254 Posted: Fri Jan 11, 2019 3:35 pm 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
Vargo wrote:
...I immediately ran a second 20 game match, with...
the same result.

Now then:
Code:
2019-01-11 08:30  #200 VS f438268e 22 : 18 (55.00%)   40 / 40   PASS

But this is a modest result for bigger network. Long live 15x192 :lol: .

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #255 Posted: Fri Jan 11, 2019 10:33 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
nbc44 wrote:
Long live 15x192 :lol: .


:D I'm not rooting for or against #200, I just like to run this kind of matches, and I always give the results exactly as they come.
I don't know your parameters and hardware.

Here is my command line for a 20 game match (twogtp 1.5.0)
Code:
gogui-twogtp -white "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\200.gz --noponder" -black "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\f438268.gz --noponder" -time 5 -games 10 -sgffile ***NAME OF SGF*** -auto -komi 7.5
gogui-twogtp -black "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\200.gz --noponder" -white "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\f438268.gz --noponder" -time 5 -games 10 -sgffile ***NAME OF SGF*** -auto -komi 7.5

If someone wants to run the "same" match at the same strength, but with a different GPU, it can be done :
time parity at 5min per side per game with my GTX1080 is around 2sec/move, and it corresponds to something like -v 1600 for #200 and -v 5000 for #f438.

Long live LZ ;-)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #256 Posted: Fri Jan 11, 2019 11:30 pm 
Dies with sente

Posts: 101
Liked others: 2
Was liked: 16
Rank: KGS 2 D
Personally I like #199 more. The games of #199 are so much more enjoyable to watch because #199 likes to attack. #200 maybe better than #199 in some aspects, but it always plays some boring moves such as solid defense, rarely tenuki or sacrifice few stones to gain influence.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #257 Posted: Wed Jan 16, 2019 11:33 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
Vargo,
in the latest version AhQ Go (2.4.2) in the list of recommended weight appeared new network 192x5 Ms5. much stronger than f438268e. could you test it?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #258 Posted: Sun Jan 20, 2019 2:02 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Do you have a link to a network file usable with LZ0.16 ?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #259 Posted: Sun Jan 20, 2019 8:01 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
https://drive.google.com/file/d/11uBKRz ... sp=sharing
when using LZ0.16 and Ms5-192x15 or f438268e sometimes LZ0.16 freezes. better lz0.15

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #260 Posted: Sun Jan 20, 2019 9:23 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Thanks.
f438268 is OK with LZ016 and twogtp on my computer, but the combination Ms5, LZ(015 or 016), and twogtp is not OK, it crashes every time :sad:

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 418 posts ]  Go to page Previous  1 ... 10, 11, 12, 13, 14, 15, 16 ... 21  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group