LZ's progression

moha · Post by **moha** » Sun Sep 16, 2018 4:29 am

Tests like these could identify the visit points where successive 40b networks overcome the 15b on 4x visits, measuring 40b progress and letting users choose the stronger size for their hardware and time/patience during reviews.

On weak hw this turning point may well remain too high even after substantial 40b training (network strength has less effect during the first bunch of visits, with shallow searches, so the turning point may not decrease too fast in this range), but on 1080ti it seems to be in reach already.

Vargo · Post by **Vargo** » Sun Sep 16, 2018 10:58 am

explo wrote:Based on using lizzie, I need around a minute to get 3200 visits on a 40b network.

On my laptop (gpu : GTX965M), I've just run such a game with twogtp (#157 at 12801 visits v. #177 at 3201 visits)
Total time was 5702s. for 286 moves, around 20s. per move. I don't know why there's such a difference...

explo · Post by **explo** » Sun Sep 16, 2018 11:33 am

Vargo wrote:
explo wrote:Based on using lizzie, I need around a minute to get 3200 visits on a 40b network.
On my laptop (gpu : GTX965M), I've just run such a game with twogtp (#157 at 12801 visits v. #177 at 3201 visits)
Total time was 5702s. for 286 moves, around 20s. per move. I don't know why there's such a difference...
sc.jpg

I wonder if it has to do with lizzie or the way something is set on my computer. I tried it again, setting an easy opening situation on lizzie with pondering off. Then turning pondering on for a minute. I ended with around 2.8 visits over the moves shown.
Earlier this afternoon, I wanted to try it on GRP to get an idea of how long it would take to review a game with lots of visits on a 40b while I'm at work. I used my last game, asked for 10k visits, and it went much faster than expected. I ended the review prematurely but I think it went through 70 moves in around a hour and a half, which is much faster than what I said about lizzie.
When I installed GRP on my laptop (with an old GPU), it changed something with openCL and things got faster on lizzie later. Is there something special I should install or set up?

Vargo · Post by **Vargo** » Sun Sep 16, 2018 9:29 pm

I've tried with Lizzie V0.5 and network #177, on my GTX965M. It's indeed slow, with times from 35s. to 75s. to get 3200 visits.

nbc44 · Post by **nbc44** » Wed Sep 19, 2018 11:17 pm

My test (l0 v15, validation.exe .... --gpu 0 --gpu 1 --noponder -t 12 -q -d -r 5 --timemanage off) #157 vs #176 :

I. by visits

a). 12801:3201 (157-176.zip)

Code: Select all

The first net is better than the second
#157 v #176 ( 227 games)
              wins        black       white
#157  134 59.03%   62 59.62%   72 58.54%
#176   93 40.97%   42 40.38%   51 41.46%
                      104 45.81%  123 54.19%
227 games played.
Status: 2 LLR 2.98898 Lower Bound -2.94444 Upper Bound 2.94444

b). 9601:3201 (157-176-1.zip)

Code: Select all

#157 v #176 ( 54 games)
          wins        black       white
#157   31 57.41%   17 56.67%   14 58.33%
#176   23 42.59%   13 43.33%   10 41.67%
                       30 55.56%   24 44.44%
54 games played.
Status: 0 LLR 0.53734 Lower Bound -2.94444 Upper Bound 2.94444

c). 6401:3201 (157-176-2.zip)

Code: Select all

#157 v #176 ( 60 games)
          wins        black       white
#157   22 36.67%   13 38.24%    9 34.62%
#176   38 63.33%   21 61.76%   17 65.38%
                       34 56.67%   26 43.33%
60 games played.
Status: 0 LLR -1.95476 Lower Bound -2.94444 Upper Bound 2.94444

d). 8001:3201 (157-176-3.zip)

Code: Select all

#157 v #176 ( 68 games)
          wins        black       white
#157   35 51.47%   19 51.35%   16 51.61%
#176   33 48.53%   18 48.65%   15 48.39%
                       37 54.41%   31 45.59%
68 games played.
Status: 0 LLR -0.15023 Lower Bound -2.94444 Upper Bound 2.94444

nbc44 · Post by **nbc44** » Wed Sep 19, 2018 11:18 pm

Last attachment.

P.S. to be continued

nbc44 · Post by **nbc44** » Fri Sep 21, 2018 3:06 am

Another test (l0 v15, validation.exe .... --gpu 0 --gpu 1 --noponder -t 12 -q -d -r 10 --timemanage off) #157 vs #178 :

a). by visits 8001:3201 (157-178.zip)

Code: Select all

The first net is worse than the second
#157 v #178 ( 110 games)
          wins        black       white
#157   43 39.09%   21 38.18%   22 40.00%
#178   67 60.91%   34 61.82%   33 60.00%
                   55 50.00%   55 50.00%
110 games played.
Status: 1 LLR -3.00809 Lower Bound -2.94444 Upper Bound 2.94444

P.S. more better than last test :^)

nbc44 · Post by **nbc44** » Sun Sep 23, 2018 8:52 pm

Yet another test (l0 v15, validation.exe .... --gpu 0 --gpu 1 --noponder -t 12 -q -d -r 5 --timemanage off) #157 vs #178 :

a). by visits 9601:3201 (157-178-5.zip)

Code: Select all

The first net is worse than the second
#157 v #178 ( 241 games)
          wins        black       white
#157  112 46.47%   57 46.34%   55 46.61%
#178  129 53.53%   66 53.66%   63 53.39%
                  123 51.04%  118 48.96%

241 games played.
Status: 1 LLR -2.95061 Lower Bound -2.94444 Upper Bound 2.94444

nbc44 · Post by **nbc44** » Mon Sep 24, 2018 11:51 pm

So-called "time parity" test - 10 min per game.

Code: Select all

gogui-twogtp -black "C:\APPS\l0gpu\leelaz.exe --gtp --weights=C:\APPS\net\d351f06e.gz -t 12 --gpu 0 --gpu 1 --noponder" -white "C:\APPS\l0gpu\leelaz.exe --gtp --weights=C:\APPS\net\58d1222a.gz -t 12 --gpu 0 --gpu 1 --noponder" -games 100 -sgffile 157-178-10m -auto -time 10 -komi 7.5 -verbose

Part1.
#157 (black) vs #178 : +48-52=0

nbc44 · Post by **nbc44** » Wed Sep 26, 2018 12:42 am

Part2.
#157 (white) vs #178 : +60-40=0 (???)

Finally:

#157 vs #178 : +108-92=0

moha · Post by **moha** » Wed Sep 26, 2018 5:50 am

nbc44 wrote:So-called "time parity" test - 10 min per game.

This says almost nothing without mentioning your gpu or average nps.

nbc44 · Post by **nbc44** » Wed Sep 26, 2018 5:55 am

moha wrote:
nbc44 wrote:So-called "time parity" test - 10 min per game.
This says almost nothing without mentioning your gpu or average nps.

2 х 1080ti with Temp. limit 80 C and Power limit 90%.

First two moves (part 1 & 2):

moha · Post by **moha** » Wed Sep 26, 2018 6:43 am

Interesting, there seems quite a visit fluctuation between positions (even for the 1st move on empty board, and even outside tree reuse?). ~~If low-visit move bottlenecks happen randomly throughout the game, that may have an impact on 40b's results.~~

Edit: Never mind, I figured it out.

Btw, what is the actual visit fluctuation, for example between early and late positions? Does it remain within these 25-30%?

nbc44 · Post by **nbc44** » Thu Sep 27, 2018 3:11 am

moha wrote:Btw, what is the actual visit fluctuation, for example between early and late positions? Does it remain within these 25-30%?

FYI, "big data" in action

. Format - "black~white".

1). by visits

nbc44 · Post by **nbc44** » Thu Sep 27, 2018 3:13 am

2). by node/s

Life In 19x19

LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression

Re: LZ's progression