It is currently Thu Dec 05, 2019 7:21 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 425 posts ]  Go to page Previous  1 ... 10, 11, 12, 13, 14, 15, 16 ... 22  Next
Author Message
Offline
 Post subject: Re: LZ's progression
Post #241 Posted: Mon Dec 24, 2018 6:12 am 
Lives with ko

Posts: 173
Liked others: 31
Was liked: 14
the table is made for the convenience of people, not for the report! besides, it has a Hash, Size, Elo, Games, Training. if a person came to this site, then this is not a random visitor, and he will understand what's what

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #242 Posted: Mon Dec 24, 2018 6:52 am 
Gosei

Posts: 1429
Liked others: 716
Was liked: 474
Rank: AGA 3k KGS 1k Fox 1d
GD Posts: 61
KGS: dfan
I was just trying to answer your original question of why it was not in the table. If you would like the format of the table to be changed, you could try opening an issue on the Leela Zero GitHub page: https://github.com/gcp/leela-zero/issues

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #243 Posted: Mon Dec 24, 2018 7:06 am 
Lives with ko

Posts: 173
Liked others: 31
Was liked: 14
Formal reasons are clear to me, but thanks anyway for your answers!


This post by And was liked by: dfan
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #244 Posted: Mon Dec 24, 2018 8:16 am 
Dies in gote

Posts: 41
Liked others: 0
Was liked: 7
OGS: roy7
Universal go server handle: roy7
For those who might prefer it, there's an experimental "new" home page at http://zero.sjeng.org/home.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #245 Posted: Mon Dec 24, 2018 9:24 pm 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
400 is the magic number ;-) , so, I extended the match between #196 (40x256) and 92297ff2 (15x192) to 400 games (more than 10 hours!).

time parity : 1 min per side per game, komi 7.5, no pondering, 1x1080, LZ0.16, twogtp V1.5.0
0 duplicate game, 0 error, all games by resignation, no contested scores, the games look normal and seem to have varied beginnings.


The first 10 results were
1, 0, 1, 0, 0, 0, 0, 0, 1, 0 .... (1 if #196 wins)
So, the cumulative winrate for #196 at move n, begins like this :
1, 0.5 , 0.66 , 0.5 , 0.4 , 0.33 , 0.29 , 0.25, 0.33 , 0.3 , ....
you can see these numbers in the left of each graph.


There are huge swings for the winrate in the first few games, and then, the winrate gets more and more stable with the number of games increasing.
The first picture is after 10, 50, and 100 games, with a dotted red line at 50%.
The second one is after 400 games, the dotted red lines are at 0.31 and 0.36, and the "final" win percentage is 0.33

I'd be very interested if someone could duplicate this experiment, I bet the graph after 400 games would look the same, but who knows...
Attachment:
3gr.gif
3gr.gif [ 11.67 KiB | Viewed 2824 times ]
Attachment:
400.gif
400.gif [ 9.82 KiB | Viewed 2824 times ]


This post by Vargo was liked by 2 people: And, Waylon
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #246 Posted: Tue Dec 25, 2018 3:00 am 
Lives with ko

Posts: 173
Liked others: 31
Was liked: 14
thanks! do you think that with more time control 92297ff2 will also be stronger than lz196?
please upload the games 1min/game and 5min/game

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #247 Posted: Tue Dec 25, 2018 10:18 am 
Lives with ko

Posts: 173
Liked others: 31
Was liked: 14
accidentally discovered a shortened version of an elf - 136mb (same values, only rounded, original - 248mb) http://gridmaster.tengen.nl/engines/armv7l/. I launched a benchmark with the original, then with a shortened one. shortened runs about 12 percent faster. I wonder how this rounding affects the strength of the game? (LZ cpuonly, Athlon x4 950, leelaz.exe -w theNetworkfile.txt --benchmark -v 300, results: 90 sec and 80 sec)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #248 Posted: Tue Dec 25, 2018 1:05 pm 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
I wanted to see if another 400 game match would end up with a graph similar to the first one, so, I ran a second 400 game match with the same parameters.
The answer is yes,the graphs look the same, but... the two final winrates seem very different (33% and 39%), particularly if you consider it's 400 games. Go figure :scratch: ;-)

The .bat file contains :
Code:
gogui-twogtp -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\196.gz --noponder" -black "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\92297ff_15x192.gz --noponder" -time 1 -games 400 -sgffile 196_15x192_1min -auto -komi 7.5 -alternate
Because of -alternate, in the even-numbered games #196 is always W, (and always B for the odd ones)
Again, there was 0 error, 0 duplicate game, and no contested score, all games by resignation.

The dotted red lines are at 0.38 and 0.42
Attachment:
400.gif
400.gif [ 9.84 KiB | Viewed 2649 times ]
The zip contains the 400 games, and the detailed stats (the html file)
Attachment:
196_15x192_1min.zip [331.61 KiB]
Downloaded 41 times


This post by Vargo was liked by: And
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #249 Posted: Wed Dec 26, 2018 8:18 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
Third and last 400 game match between #196 and #92297ff at 1 min per side per game.
Same parameters.

This time, there were 3 duplicate games among the 400, but no error or unagreed score.

The three final results are 33%, 39%, and 39%

Attachment:
gr.gif
gr.gif [ 11.89 KiB | Viewed 2522 times ]

The zip contains the 400 games of this last match, and the detailed stats (the html file)
Attachment:
196_15x192_1min_bis.zip [343.21 KiB]
Downloaded 33 times

Top
 Profile  
 
Online
 Post subject: Re: LZ's progression
Post #250 Posted: Sun Dec 30, 2018 10:20 am 
Judan

Posts: 6220
Location: Cambridge, UK
Liked others: 355
Was liked: 3363
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Just one more data point for the 9229 network: I just used it against petgo3 on KGS which was using #197 on similar hardware to my own, 10x20s byoyomis, I/9229 won, see viewtopic.php?p=240161#p240161. So at 20 seconds per move (more playouts than the minutes per game here) on a medium GPU those extra playouts were still more important than the stronger intuition of the deeper network.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #251 Posted: Fri Jan 04, 2019 5:45 am 
Lives with ko

Posts: 173
Liked others: 31
Was liked: 14
new network 15x192 from bjiyxo: 8b23a0ea VS 92297ff2 229 : 191 (54.52%). the strongest 15x192?
http://zero.sjeng.org/networks/8b23a0ea ... 8cf3b3a.gz

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #252 Posted: Thu Jan 10, 2019 11:45 pm 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
20 game match : #200 v. #157 (newest 40x256 v. last 15x192)

Time parity, 5min per side and per game, GPU: 1x1080, no pondering, komi 7.5, twogtp 1.5.0, ~205 moves/game (min=114, max=324), ~2sec/move
All games by resignation, no duplicate, no error.

Result : #200 wins 17-3
WOAW !

Stats :
Attachment:
200v157_5min.zip [11.33 KiB]
Downloaded 29 times

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #253 Posted: Fri Jan 11, 2019 1:25 am 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
Vargo wrote:
20 game match : #200 v. #157 (newest 40x256 v. last 15x192)


Why not the latest 15x192 199-96k quantized?

Code:
2019-01-08 07:01   f438268e VS  92297ff2 230 : 174 (56.93%)   404 / 400   PASS
2018-12-22 02:42   92297ff2 VS  d351f06e 260 : 144 (64.36%)   404 / 400   PASS

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #254 Posted: Fri Jan 11, 2019 5:09 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
nbc44 wrote:
Why not the latest 15x192 199-96k quantized?
Why not, indeed !

20 game match : #200 v. #f438268

LZ0.16, Time parity, 5min per side and per game, GPU: 1x1080, no pondering, komi 7.5, twogtp 1.5.0, ~198 moves/game (min=106, max=304), ~2sec/move
All games by resignation, no duplicate, no error.

Result : #200 wins 11-9

STATS:
Attachment:
200_f438_5min.zip [11.22 KiB]
Downloaded 28 times


The games (200 is Black in the games 200_f438_5min_2-XXX, 200 is White for 200_f438_5min-XXX )
Attachment:
200_f438_5min_SGF.zip [17.01 KiB]
Downloaded 24 times

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #255 Posted: Fri Jan 11, 2019 7:18 am 
Lives in sente

Posts: 936
Liked others: 0
Was liked: 159
Vargo wrote:
Result : #200 wins 11-9


Because it is likely that the majority here have never had a course in statistics, it might be useful, when posting something like this to add "no meaningful difference".

In other words, if you flipped an honest coin 20 times you would expect a result at least as unbalanced as 11-9 << 10-10 might be the single most likely outcome BUT there are so many other outcomes possible that we expect something else. For example, 11-9 or 9-11, one of these or the other, are together more likely than 10-10>>

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #256 Posted: Fri Jan 11, 2019 8:30 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
You're absolutely right. As the result was very close, I immediately ran a second 20 game match, with...
the same result.
STATS
Attachment:
200isB.gif
200isB.gif [ 96.6 KiB | Viewed 1842 times ]
STATS
Attachment:
200isW.gif
200isW.gif [ 95.09 KiB | Viewed 1842 times ]
Attachment:
200vf438_5min.zip [17.55 KiB]
Downloaded 25 times

But I completely agree with you, it could very well have been 8-12, 9-11, 10-10, 11-9, or 12-8.
What it (probably) means, is they're about the same strength.
On the other hand, even with only 20 games, 17-3 probably means that the winner IS stronger (if they were the same strength, it would be very infrequent to get 17-3)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #257 Posted: Fri Jan 11, 2019 3:35 pm 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
Vargo wrote:
...I immediately ran a second 20 game match, with...
the same result.

Now then:
Code:
2019-01-11 08:30  #200 VS f438268e 22 : 18 (55.00%)   40 / 40   PASS

But this is a modest result for bigger network. Long live 15x192 :lol: .

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #258 Posted: Fri Jan 11, 2019 10:33 pm 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
nbc44 wrote:
Long live 15x192 :lol: .


:D I'm not rooting for or against #200, I just like to run this kind of matches, and I always give the results exactly as they come.
I don't know your parameters and hardware.

Here is my command line for a 20 game match (twogtp 1.5.0)
Code:
gogui-twogtp -white "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\200.gz --noponder" -black "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\f438268.gz --noponder" -time 5 -games 10 -sgffile ***NAME OF SGF*** -auto -komi 7.5
gogui-twogtp -black "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\200.gz --noponder" -white "C:\***PATH TO LZ***\leelaz.exe --gtp --weights=C:\***PATH TO NETWORK***\f438268.gz --noponder" -time 5 -games 10 -sgffile ***NAME OF SGF*** -auto -komi 7.5

If someone wants to run the "same" match at the same strength, but with a different GPU, it can be done :
time parity at 5min per side per game with my GTX1080 is around 2sec/move, and it corresponds to something like -v 1600 for #200 and -v 5000 for #f438.

Long live LZ ;-)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #259 Posted: Fri Jan 11, 2019 11:30 pm 
Dies with sente

Posts: 94
Liked others: 2
Was liked: 15
Rank: KGS 2 D
Personally I like #199 more. The games of #199 are so much more enjoyable to watch because #199 likes to attack. #200 maybe better than #199 in some aspects, but it always plays some boring moves such as solid defense, rarely tenuki or sacrifice few stones to gain influence.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #260 Posted: Wed Jan 16, 2019 11:33 am 
Lives with ko

Posts: 173
Liked others: 31
Was liked: 14
Vargo,
in the latest version AhQ Go (2.4.2) in the list of recommended weight appeared new network 192x5 Ms5. much stronger than f438268e. could you test it?

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 425 posts ]  Go to page Previous  1 ... 10, 11, 12, 13, 14, 15, 16 ... 22  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: Google [Bot] and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group