It is currently Thu Mar 28, 2024 2:29 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 418 posts ]  Go to page Previous  1 ... 17, 18, 19, 20, 21  Next
Author Message
Offline
 Post subject: Re: LZ's progression
Post #381 Posted: Thu Apr 25, 2019 2:37 pm 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
The main question is what is more important to us - victory or honesty? :salute:

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #382 Posted: Thu Apr 25, 2019 6:06 pm 
Dies with sente

Posts: 113
Liked others: 11
Was liked: 27
Rank: 1d
Universal go server handle: iopq
Did you set the batch number to half of the threads? You can get better perf.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #383 Posted: Thu Apr 25, 2019 10:21 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Three last benchmarks :

not specifying -t XXX seems to give slightly less n/s
Attachment:
1.gif
1.gif [ 39.75 KiB | Viewed 8764 times ]


iopq wrote:
Did you set the batch number to half of the threads?
I'm not sure it's better... but again, I find the effect of -t XXX rather bizarre, and these benchmarks are maybe flawed, one way or another...
Attachment:
3.gif
3.gif [ 73.69 KiB | Viewed 8764 times ]


--precision half
seems to be about the same as not specifying the precision
Attachment:
2.gif
2.gif [ 57.26 KiB | Viewed 8764 times ]


:scratch: :scratch: :scratch:

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #384 Posted: Fri Apr 26, 2019 12:36 am 
Dies with sente

Posts: 113
Liked others: 11
Was liked: 27
Rank: 1d
Universal go server handle: iopq
Benchmark with batching, it would be faster than just threading

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #385 Posted: Fri Apr 26, 2019 2:20 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
LZ just beat Golaxy in the Fuzhou AI tournament :tmbup:
https://home.yikeweiqi.com/#/live/board/17523


This post by Uberdude was liked by: sorin
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #386 Posted: Fri Apr 26, 2019 4:12 am 
Dies in gote

Posts: 38
Liked others: 4
Was liked: 20
Is there a way to download SGF from that site, and if yes, can someone post it here? :study:

Alright I think I found them...


Attachments:
Leela Zero - Baduki.sgf [2.03 KiB]
Downloaded 1241 times
Leela Zero - YikeBot.sgf [1.23 KiB]
Downloaded 1227 times
Golaxy - Leela Zero.sgf [1.4 KiB]
Downloaded 1243 times
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #387 Posted: Fri Apr 26, 2019 5:44 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
I've previously hacked out the sgf from yike using browser dev tools, don't know if there's an easier way.

Inline sgf players:





Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #388 Posted: Fri Apr 26, 2019 6:38 pm 
Dies with sente

Posts: 101
Liked others: 2
Was liked: 16
Rank: KGS 2 D
iopq wrote:
Benchmark with batching, it would be faster than just threading


Could you please show me the command option for batching? It seems that Sabaki always choose batch size 1 by default, while the autogtp chooses something different.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #389 Posted: Sat Apr 27, 2019 4:45 am 
Dies in gote

Posts: 38
Liked others: 4
Was liked: 20
Leela lost both its semi-final games. I enjoyed watching the second one live, it had a rather drastic semeai, which sadly became 1-eye vs 0-eye...




Attachments:
2019-04-27 LeelaZero vs Golaxy.sgf [1.39 KiB]
Downloaded 1215 times
2019-04-27 Golaxy vs LeelaZero.sgf [1.83 KiB]
Downloaded 1240 times
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #390 Posted: Sat Apr 27, 2019 2:29 pm 
Beginner

Posts: 19
Liked others: 0
Was liked: 2
nbc44 wrote:
hoa803 wrote:
NBC, if you're using visit parity you shouldn't use ponder. The time to reach that number of visits varies by position. Time parity matches can use ponder on separate hardware though, similar to how Alphago was tested.

Why not? I'm using separate GPU for each net.


Think about what you are trying to do in terms of mathematics. LZ has a chance to win a random game, let us call that probability P.

In a match with ponder turned on, you've introduce another variable - the total thinking time permitted for each engine due to use of ponder. On a given game either LZ or Elf is likely to get more overall thinking time. Since we already know that strength is directly related to thinking time, your chance of LZ winning a particular game is now the function P(x), where x is a random variable related to the strength at different thinking times.

That means that the statistical basis being used to evaluate strength is no longer valid, because with fixed visit count and ponder the result is a function of another random variable that we don't know anything about. The function P(x) is most likely Gaussian, but we don't know the standard deviation or anything along those lines. I'm not enough of a mathematician to know what that does to the conclusion over a 400 game match.

Also - you should put your queries about thread and batch count to the actual programmers on GitHub. Again you are introducing variables that you don't understand. I think I've seen some discussion about both batch size and number of threads having an impact on performance. You should definitely ask if you want to understand what is going on. Maybe post your results and see what GCP says about it.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #391 Posted: Sun Apr 28, 2019 10:34 am 
Dies with sente

Posts: 113
Liked others: 11
Was liked: 27
Rank: 1d
Universal go server handle: iopq
splee99 wrote:
iopq wrote:
Benchmark with batching, it would be faster than just threading


Could you please show me the command option for batching? It seems that Sabaki always choose batch size 1 by default, while the autogtp chooses something different.


Something like --batchsize I believe

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #392 Posted: Mon Apr 29, 2019 9:41 am 
Beginner

Posts: 19
Liked others: 0
Was liked: 2
Quote:
Could you please show me the command option for batching? It seems that Sabaki always choose batch size 1 by default, while the autogtp chooses something different.


If you use Leela Zero engine version 0.17 it should choose the optimal thread and batch size for your hardware by default. Note that you cannot have more batches than threads.

Try this command in your LZ directory:

leelaz -h

You will see a list of some things you can do.

In terms of engine strength, this person has done a ton of work:

https://github.com/guitanj/AutoPK/blob/master/README.md

These data show that if we are interested in strength as a function on thinking time, LZ will be noticeably weaker since Elfv2 is only a 20-block network and therefore evaluates more quickly. In terms of pure network strength regardless of thinking time, Leela Zero is generally stronger except at high playouts. I find that very interesting.

Very cool data and a lot of work. This person deserves much credit for doing all these tests.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #393 Posted: Tue May 07, 2019 5:47 am 
Dies with sente

Posts: 113
Liked others: 11
Was liked: 27
Rank: 1d
Universal go server handle: iopq
I believe it will do something like batch size 5 and 10 threads.

Anyone try the new 15b net vs. the 40b at time parity?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #394 Posted: Tue May 07, 2019 7:25 am 
Dies in gote

Posts: 53
Liked others: 3
Was liked: 33
Rank: KGS 2k
There was a new 40b network promoted today, and it was also tested officially against a other 40b network from 10 networks ago.

The result is a 69% winrate against the older 40b network from 10 networks back. Rather impressive!
The 40b network is clearly still growing and learning and getting stronger at a good pace!

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #395 Posted: Wed May 08, 2019 3:52 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Aram wrote:
There was a new 40b network promoted today, and it was also tested officially against a other 40b network from 10 networks ago.

The result is a 69% winrate against the older 40b network from 10 networks back. Rather impressive!
The 40b network is clearly still growing and learning and getting stronger at a good pace!


I wonder what the improvement in winrate of this latest 40b compared to 10 networks ago would be versus a non-LZ bot (for fairest comparison would be time parity but even visit parity I expect same effect). Eg maybe #215 beats Elfv2 at 1600 visit parity 65% of the time (and maybe <50% at time parity), and #225 beats it only 67%. I expect quite a lot less difference than 19% (or equivalent ratio difference).

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #396 Posted: Wed May 08, 2019 6:42 pm 
Dies with sente

Posts: 101
Liked others: 2
Was liked: 16
Rank: KGS 2 D
Here is a game between LZ-225 (B) + LZ0.17 and ELF (W) + LZ0.16 in time parity (20 seconds per move).
The ELFv2 was far ahead with winrate close to 90%, until move 135, when LZ-225 suddenly started a complicated fight. After that ELF still had lots of chance to win the fight, according to LZ-225, but eventually ELF gave up.



Attachment:
225-elf-20s.sgf [3.77 KiB]
Downloaded 2040 times

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #397 Posted: Thu May 09, 2019 12:20 am 
Dies in gote

Posts: 53
Liked others: 3
Was liked: 33
Rank: KGS 2k
One shouldnt use ELF winrates or ELF in general for analysis.

They sharpened the network a rather huge amount to make it stronger in self play and play versus other bots.
(Probably because they plateud and needed to get some more strength out of the relatively small 20-ish block network)
So basically ELFs possibility to grow and explore new and different moves was terminated and it became stronger on a narrow path of moves.

This also sadly led the network to believe that every match is either a 100% win or a 100% loss.

This is quite visible for anyone using both ELF and LZ networks. ELF will always be very black and white and whats good or bad, while the LZ networks provide several moves which are playable.


LZ could do the same and would probably see a boost in strength, but at the same time it probably wouldnt improve anymore and it wouldnt explore either.
Since the LZ network is larger, it still has capacity to grow and improve by searching and exploring, and as such its not needed to try to force it to its full strength right now.

Same thing with learning rates, one could probobly get a small increase in LZ strength by just dropping the learning rate, but that would kill any further increase in playing strength from exploration.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #398 Posted: Thu May 09, 2019 10:09 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
20 game match LZ017#225 v LZ017#215
no pondering, komi 7.5, -v 1600
twogtp 1.5.1 , no error, no duplicate game.

#225 wins 17-3
Wow ! It confirms the 69% , and it's only a 10 networks difference... :clap:

The stats :
Attachment:
225.gif
225.gif [ 46.02 KiB | Viewed 9516 times ]
The games (225 is B in the odd-numbered games) :
Attachment:
225_215_v 1600.rar [15.32 KiB]
Downloaded 363 times

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #399 Posted: Fri May 17, 2019 3:36 am 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
Сity legend check (l0XXX 1600 visits vs elfv2 3200 visits):

2 x 1080ti

[Engine_Master]
enginedir = C:\APPS\l0gpu17\
engine = leelaz.exe -g --gpu 0 --gpu 1 -t 24 --batchsize 12 -r 5 --noponder --timemanage off --precision single
weightdir = C:\APPS\net\
weight = 05dbca15.gz
name = ELFV2

[Engine_01]
enginedir = C:\APPS\l0gpu17\
engine = leelaz.exe -g --gpu 0 --gpu 1 -t 24 --batchsize 12 -r 5 --noponder --timemanage off --precision single
weightdir = C:\APPS\net\
weight = a20c31da.gz
name = LZ226

[Engine_02]
enginedir = C:\APPS\l0gpu17\
engine = leelaz.exe -g --gpu 0 --gpu 1 -t 24 --batchsize 12 -r 5 --noponder --timemanage off --precision single
weightdir = C:\APPS\net\
weight = d42f950a.gz
name = LZ225

[Engine_03]
enginedir = C:\APPS\l0gpu17\
engine = leelaz.exe -g --gpu 0 --gpu 1 -t 24 --batchsize 12 -r 5 --noponder --timemanage off --precision single
weightdir = C:\APPS\net\
weight = f3be3bf8.gz
name = LZ224

Code:
===================================================================================================================
From:May-07-19 07:12:33 to May-08-19 05:08:00. Spend 78927.58483s
                         Game << 400 >>
LZ224                    1600 visits  win(All) = 249  62.25%        win(B) = 115  57.50%        win(W) = 134  67.00%
ELFV2                    1600 visits  win(All) = 151  37.75%        win(B) =  66  33.00%        win(W) =  85  42.50%

===================================================================================================================
From:May-09-19 07:42:53 to May-10-19 06:46:29. Spend 83015.170982s
                         Game << 400 >>
LZ224                    1600 visits  win(All) = 212  53.00%        win(B) = 102  51.00%        win(W) = 110  55.00%
ELFV2                    2000 visits  win(All) = 188  47.00%        win(B) =  90  45.00%        win(W) =  98  49.00%

===================================================================================================================
From:May-08-19 05:08:00 to May-09-19 05:47:02. Spend 88741.358819s
                         Game << 400 >>
LZ224                    1600 visits  win(All) = 180  45.00%        win(B) =  91  45.50%        win(W) =  89  44.50%
ELFV2                    2400 visits  win(All) = 220  55.00%        win(B) = 111  55.50%        win(W) = 109  54.50%

===================================================================================================================
From:May-10-19 07:37:42 to May-11-19 07:16:09. Spend 85106.834626s
                         Game << 400 >>
LZ225                    1600 visits  win(All) = 207  51.75%        win(B) =  94  47.00%        win(W) = 113  56.50%
ELFV2                    2000 visits  win(All) = 193  48.25%        win(B) =  87  43.50%        win(W) = 106  53.00%

===================================================================================================================
From:May-11-19 07:16:09 to May-12-19 07:23:49. Spend 86860.492488s
                         Game << 400 >>
LZ225                    1600 visits  win(All) = 172  43.00%        win(B) =  81  40.50%        win(W) =  91  45.50%
ELFV2                    2200 visits  win(All) = 228  57.00%        win(B) = 109  54.50%        win(W) = 119  59.50%

===================================================================================================================
From:May-14-19 06:55:16 to May-15-19 06:12:54. Spend 83858.237752s
                         Game << 400 >>
LZ226                    1600 visits  win(All) = 212  53.00%        win(B) = 100  50.00%        win(W) = 112  56.00%
ELFV2                    2000 visits  win(All) = 188  47.00%        win(B) =  88  44.00%        win(W) = 100  50.00%

===================================================================================================================
From:May-15-19 06:12:54 to May-16-19 05:59:42. Spend 85607.877284s
                         Game << 400 >>
LZ226                    1600 visits  win(All) = 196  49.00%        win(B) =  94  47.00%        win(W) = 102  51.00%
ELFV2                    2200 visits  win(All) = 204  51.00%        win(B) =  98  49.00%        win(W) = 106  53.00%


P.S. If someone wants the games, I can upload them.


This post by nbc44 was liked by: SHKD
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #400 Posted: Fri May 17, 2019 11:16 am 
Beginner

Posts: 6
Liked others: 0
Was liked: 0
Rank: KGS 3d
KGS: CuTy
Can I have the games, nbc44?
For studying, of course! :study:

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 418 posts ]  Go to page Previous  1 ... 17, 18, 19, 20, 21  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group