It is currently Thu Jun 20, 2019 4:34 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 418 posts ]  Go to page Previous  1 ... 16, 17, 18, 19, 20, 21  Next
Author Message
Offline
 Post subject: Re: LZ's progression
Post #361 Posted: Tue Apr 09, 2019 5:44 am 
Lives in gote
User avatar

Posts: 430
Liked others: 39
Was liked: 135
LeelaZero counted the score as territory + prisoners (+komi for White)

Crazystone counted 182 points for Black, which corresponds to (black living stones)+(black territory).
My guess is that the 189.5 points for White correspond to (white living stones)+(white territory)+(komi)+(number of handicap stones).

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #362 Posted: Tue Apr 09, 2019 10:09 am 
Dies in gote

Posts: 58
Liked others: 12
Was liked: 5
H3 game with komi 7.5 : Crazy Stone DL (5 Dan) - zen 7 (5sec), zen win, score CS W+35.5, zen and sabaki W+34.5. Crazy Stone possibly mistaken?
ps I figured it out: CS shows area + handicap, and sabaki and zen - territory

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #363 Posted: Thu Apr 11, 2019 9:01 am 
Lives with ko

Posts: 236
Liked others: 4
Was liked: 57
And wrote:
CS shows area + handicap, and sabaki and zen - territory
jlt wrote:
LeelaZero counted the score as territory + prisoners (+komi for White)
Crazystone counted 182 points for Black, which corresponds to (black living stones)+(black territory)
Thx :tmbup:


49 games at H2…H9 played byLZ017#10…#190 v. LZ017#220

time parity, 2 sec/move, 1x 1080, official LZ0.17#220 v.#xxx komi 7.5 , -r 1 for W (to avoid resigning too soon) –r 30 for B (to avoid very long games)
Attachment:
1.jpg
1.jpg [ 111.02 KiB | Viewed 1690 times ]
It gives an idea of the handicap skills of #220.

In the interesting zone (bold frames) mini 3 game matches :
Attachment:
2.jpg
2.jpg [ 85.56 KiB | Viewed 1690 times ]

According to THIS SITE, and in KGS rankings
#40 is very approximately around 4K
#70 is around 3D
#100 is around 6D
#130 is around 9D
(it seems a lot, and these rankings weren't based on 2sec/move)

You can play handicap go at this excellent site

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #364 Posted: Thu Apr 11, 2019 7:04 pm 
Beginner

Posts: 13
Liked others: 0
Was liked: 0
If anybody hasn't used their free $300 from Google Cloud and feels like doing some deep learning, I recently set it up and it's quite easy to do.

See this github thread for an updated guide. Note that the Microsoft Azure guide currently doesn't work with LeelaZero 0.17, but I'm trying to figure out the solution.

On a single Tesla v100 gpu I am finishing a game every 108 seconds, averaged over 900 games! That means I can expect to make something like 12,000 games (selfplay and matches) before the $300 credit runs out.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #365 Posted: Fri Apr 12, 2019 12:46 am 
Dies in gote

Posts: 49
Liked others: 0
Was liked: 3
Time parity match with statistically significant result :salute: (part I).
LZ0v17 #219 vs Elfv2
2x1080ti, 30s per move.

C:\APPS\l0gpu17\validation.exe -k 219elfv2-30s -s "0:10" -g 6 -n C:\APPS\net\00ff08eb.gz -o "-g --gpu 0 --gpu 1 --noponder -t 24 -q -d --precision single -w" -n C:\APPS\net\05dbca15.gz -o "-g --gpu 0 --gpu 1 --noponder -t 24 -q -d --precision single -w" -- C:\APPS\l0gpu17\leelaz --gtp-command "time_settings 0 30 1" -- C:\APPS\l0gpu17\leelaz --gtp-command "time_settings 0 30 1"

Code:
#219 v elfv2 ( 400 games)
           wins        black       white
#219   175 43.75%   65 41.67%  110 45.08%
elfv2  225 56.25%   91 58.33%  134 54.92%
                   156 39.00%  244 61.00%


Attachments:
219elfv2-30s.zip [347.73 KiB]
Downloaded 19 times
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #366 Posted: Fri Apr 12, 2019 3:13 am 
Dies in gote

Posts: 35
Liked others: 4
Was liked: 19
Cool. I adjusted the PB and PW properties in the SGF files to make it a bit clearer who was who.


Attachments:
219elfv2-30s.zip [338.48 KiB]
Downloaded 16 times
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #367 Posted: Sat Apr 13, 2019 6:10 pm 
Beginner

Posts: 13
Liked others: 0
Was liked: 0
It might provoke an interesting discussion - the folks on GitHub don't feel the time parity matches are a good measure of engine strength, but rather visits. I don't claim to totally understand the reasoning but it might be worth looking into.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #368 Posted: Sat Apr 13, 2019 10:44 pm 
Dies in gote

Posts: 49
Liked others: 0
Was liked: 3
hoa803 wrote:
It might provoke an interesting discussion - the folks on GitHub don't feel the time parity matches are a good measure of engine strength, but rather visits. I don't claim to totally understand the reasoning but it might be worth looking into.

i think these visit's tests are "сферический конь в вакууме", known in the west as a spherical cow. :lol:

P.S. For https://github.com/leela-zero/leela-zero/issues/2330#issuecomment-482729494 wright now:
#219 (-v 1600) vs elfv2 (-v 3200)
10 wins, 40 losses
50 games played.
:o

EDIT 1.

P.S.S For https://github.com/leela-zero/leela-zero/issues/2330#issuecomment-482957781 wright now:

#219 (-v 1600) vs elfv2 (-v 3200)
22 wins, 125 losses
157 games played.
:lol:

to be continued...


Last edited by nbc44 on Sun Apr 14, 2019 5:53 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #369 Posted: Sun Apr 14, 2019 12:09 am 
Dies in gote

Posts: 49
Liked others: 0
Was liked: 3
Time parity match with statistically significant result :salute:
(part II).
LZ0v17 #219 vs Elfv2
2x1080ti, 10s per move.
C:\APPS\l0gpu17\validation.exe -k 219elfv2-10s -s "0:10" -g 6 -n C:\APPS\net\00ff08eb.gz -o "-g --gpu 0 --gpu 1 --noponder -t 24 -q -d --precision single -w" -n C:\APPS\net\05dbca15.gz -o "-g --gpu 0 --gpu 1 --noponder -t 24 -q -d --precision single -w" -- C:\APPS\l0gpu17\leelaz --gtp-command "time_settings 0 10 1" -- C:\APPS\l0gpu17\leelaz --gtp-command "time_settings 0 10 1"

Code:
#219 v elfv2 ( 400 games)
           wins        black       white
#219\  155 38.75%   64 37.21%   91 39.91%
elfv2  245 61.25%  108 62.79%  137 60.09%
                   172 43.00%  228 57.00%

(part III).
LZ0v17 #219 vs Elfv2
2x1080ti, 3s per move.
C:\APPS\l0gpu17\validation.exe -k 219elfv2-3s -s "0:10" -g 6 -n C:\APPS\net\00ff08eb.gz -o "-g --gpu 0 --gpu 1 --noponder -t 24 -q -d --precision single -w" -n C:\APPS\net\05dbca15.gz -o "-g --gpu 0 --gpu 1 --noponder -t 24 -q -d --precision single -w" -- C:\APPS\l0gpu17\leelaz --gtp-command "time_settings 0 3 1" -- C:\APPS\l0gpu17\leelaz --gtp-command "time_settings 0 3 1"

Code:
#219 v elfv2 ( 403 games)
           wins        black       white
#219   153 37.97%   58 35.37%   95 39.75%
elfv2  250 62.03%  106 64.63%  144 60.25%
                   164 40.69%  239 59.31%


Attachments:
219elfv2-3s.zip [326.11 KiB]
Downloaded 11 times
219elfv2-10s.zip [342.38 KiB]
Downloaded 8 times
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #370 Posted: Sun Apr 14, 2019 4:36 am 
Beginner

Posts: 13
Liked others: 0
Was liked: 0
My suspicion is that time parity might be correct with two entirely separate machines, with the same hardware, and using ponder. Basically what seems to have done in the AlphaZero paper? If I recall they used 90 mins main time and 15s/move byo-yomi. I'm not as sure about doing it on a single machine with --noponder, however.

The reason I wonder is because I know that each move the NN makes is not independent of what it calculated on the previous move(s). We also know that the number of visits calculated on each position will vary wildly for a given amount of time. That will definitely add some serious randomness to the performance of an engine throughout a game. What I don't know is, does that even matter? (see: comment)

Ultimate I think it may be like football. Take the best teams in the world, say Manchester City, Barcelona, etc. Now change the rules of the game in some fundamental way. Maybe some other team will now be stronger.

That may be a poor analogy but basically I'm saying that when we declare an engine stronger given test XYZ, basically all we're saying is under those exact conditions only is that a true statement - especially when the engines are very similar in strength, like elf and lz appear to be at this point.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #371 Posted: Mon Apr 15, 2019 7:02 am 
Dies in gote

Posts: 30
Liked others: 2
Was liked: 9
Rank: 3d
of course, just an inconvenient truth

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #372 Posted: Mon Apr 15, 2019 1:51 pm 
Dies in gote

Posts: 49
Liked others: 0
Was liked: 3
hoa803 wrote:
My suspicion is that time parity might be correct with two entirely separate machines, with the same hardware, and using ponder.

What do you think about test with one computer, ponder, one dedicated gpu for each side? I believe that this will be a more or less honest test.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #373 Posted: Mon Apr 15, 2019 4:41 pm 
Dies with sente

Posts: 86
Liked others: 2
Was liked: 12
Rank: KGS 2 D
I think one computer only has one interface bus between the CPU and the GPU. So that part is actually shared and the bot using less interface time will take more advantage.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #374 Posted: Mon Apr 15, 2019 8:08 pm 
Dies in gote

Posts: 49
Liked others: 0
Was liked: 3
splee99 wrote:
I think one computer only has one interface bus between the CPU and the GPU. So that part is actually shared and the bot using less interface time will take more advantage.

For 3200 visits and gpu(!) client? it's funny is not it?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #375 Posted: Mon Apr 15, 2019 11:03 pm 
Dies in gote

Posts: 49
Liked others: 0
Was liked: 3
Visit parity match with statistically significant result :salute:
LZ0v17 #219 (1600 visits) vs Elfv2 (3200 visits) 2x1080ti
C:\APPS\l0gpu17\validation.exe -k 219-elfv2 -s "0:1" -g 6 -n C:\APPS\net\00ff08eb.gz -o "-g -v 1600 --gpu 0 --gpu 1 --noponder -t 24 -q -d --timemanage off --precision single -w" -n C:\APPS\net\05dbca15.gz -o "-g -v 3200 --gpu 0 --gpu 1 --noponder -t 24 -q -d --timemanage off --precision single -w " -- C:\APPS\l0gpu17\leelaz -- C:\APPS\l0gpu17\leelaz

Code:
#219 v elfv2 ( 400 games)
           wins        black       white
#219   129 32.25%   60 31.58%   69 32.86%
elfv2  271 67.75%  130 68.42%  141 67.14%
                   190 47.50%  210 52.50%


In my case, everything is very bad.


Attachments:
219-elfv2.zip [331.93 KiB]
Downloaded 10 times
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #376 Posted: Wed Apr 17, 2019 12:26 am 
Dies in gote

Posts: 49
Liked others: 0
Was liked: 3
Visit parity match
LZ0v17 #219 (1600 visits) vs Elfv2 (3200 visits) 1x1080ti per side + ponder
Part1 - #219 (GPU0) vs Elfv2 (GPU1)
C:\APPS\l0gpu17\validation.exe -k 219-elfv2-1gpu -s "0:1" -g 6 -n C:\APPS\net\00ff08eb.gz -o "-g -v 1600 --gpu 0 -t 24 -q -d --timemanage off --precision single -w" -n C:\APPS\net\05dbca15.gz -o "-g -v 3200 --gpu 1 -t 24 -q -d --timemanage off --precision single -w " -- C:\APPS\l0gpu17\leelaz -- C:\APPS\l0gpu17\leelaz

Code:
#219 v elfv2 ( 208 games)
           wins        black       white
#219    74 35.58%   34 34.69%   40 36.36%
elfv2  134 64.42%   64 65.31%   70 63.64%
                    98 47.12%  110 52.88%

Part2 - #219 (GPU1) vs Elfv2 (GPU0)
C:\APPS\l0gpu17\validation.exe -k 219-elfv2-1gpu -s "0:1" -g 6 -n C:\APPS\net\00ff08eb.gz -o "-g -v 1600 --gpu 1 -t 24 -q -d --timemanage off --precision single -w" -n C:\APPS\net\05dbca15.gz -o "-g -v 3200 --gpu 0 -t 24 -q -d --timemanage off --precision single -w " -- C:\APPS\l0gpu17\leelaz -- C:\APPS\l0gpu17\leelaz

Code:
#219 v elfv2 ( 208 games)
           wins        black       white
#219    81 38.94%   43 39.45%   38 38.38%
elfv2  127 61.06%   66 60.55%   61 61.62%
                   109 52.40%   99 47.60%

Summary:

#219 vs elfv2 (37,26%)
+155-261=0
:clap:


Attachments:
219-elfv2-1gpu-part2.zip [175.14 KiB]
Downloaded 8 times
219-elfv2-1gpu-part1.zip [173.27 KiB]
Downloaded 9 times
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #377 Posted: Wed Apr 17, 2019 2:14 am 
Dies in gote

Posts: 42
Liked others: 3
Was liked: 27
Rank: KGS 2k
Is the difference in speed between the ELF network and the 40b network really 2x for you?
I know that theoretically that could be true, but if ive understood correctly, the difference isnt nearly that large in practise?


If you load the 40b network in leela, and write netbench 50000 and then load the elf network and write netbench 50000,
do you really play those 50.000 playouts in half the time with the ELF network?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #378 Posted: Wed Apr 17, 2019 3:35 am 
Dies in gote

Posts: 49
Liked others: 0
Was liked: 3
Aram wrote:
Is the difference in speed between the ELF network and the 40b network really 2x for you?
I know that theoretically that could be true, but if ive understood correctly, the difference isnt nearly that large in practise?


If you load the 40b network in leela, and write netbench 50000 and then load the elf network and write netbench 50000,
do you really play those 50.000 playouts in half the time with the ELF network?


1). https://lifein19x19.com/viewtopic.php?p=242577#p242577

2).
Code:
c:\apps\l0gpu17\leelaz.exe --precision single -t 24 --gpu 0 --gpu 1  -w C:\APPS\net\00ff08eb.gz

Leela: netbench 50000
50000 evaluations in 58.73 seconds -> 851 n/s

c:\apps\l0gpu17\leelaz.exe --precision single -t 24 --gpu 0 --gpu 1  -w C:\APPS\net\05dbca15.gz

Leela: netbench 50000
50000 evaluations in 29.81 seconds -> 1677 n/s

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #379 Posted: Wed Apr 17, 2019 4:46 pm 
Beginner

Posts: 13
Liked others: 0
Was liked: 0
NBC, if you're using visit parity you shouldn't use ponder. The time to reach that number of visits varies by position. Time parity matches can use ponder on separate hardware though, similar to how Alphago was tested.

There's a thread on GitHub with a visit "parity" (1600 vs 3200) match between 220 and elfv2. The result was inconclusive, seems to indicate they're about the same strength at that visit count.

Early in the match LZ appeared to be stronger with over 95% confidence, but by the end the result evened out.

Edit: a word


Last edited by hoa803 on Tue Apr 23, 2019 4:21 pm, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #380 Posted: Wed Apr 17, 2019 8:20 pm 
Dies with sente

Posts: 86
Liked others: 2
Was liked: 12
Rank: KGS 2 D
My observation is that elfv2 is well trained to make sharp attacks in the early stage of a game. However it does have many blind spots in a complicated life death situations where LZ can take advantage of.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 418 posts ]  Go to page Previous  1 ... 16, 17, 18, 19, 20, 21  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group