It is currently Tue Oct 22, 2019 3:56 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 425 posts ]  Go to page Previous  1 ... 7, 8, 9, 10, 11, 12, 13 ... 22  Next
Author Message
Offline
 Post subject: Re: LZ's progression
Post #181 Posted: Tue Oct 23, 2018 3:26 pm 
Judan

Posts: 6128
Location: Cambridge, UK
Liked others: 351
Was liked: 3299
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Some Elf games are/were (I think they are being thinned/phased out recently) used in the LZ training. I'm not sure if these are real Elf engine+weights self-play games released in some huge dataset by Facebook, the LZ engine with converted Elf weights playing against itself on LZ volunteer machines, or LZ engine with Elf weights vs LZ engine with LZ weights playing on volunteer machines, but suspect it's the middle one. Can anyone confirm? When these Elf games were added to the training pipeline LZ improved very fast initially as it absorbed the Elvish wisdom. You can see latest LZ has a more Elvish style; not so obsessed with 3-3 invasions of 4-4 but approaches them too, the 4 corners then 4 approaches fuseki liked by Elf is now also played by LZ (almost exclusively in the vs elf test matches I looked at, gets kinda boring). Also Elf is somewhat more human in not hating side extensions so much or playing hoshi pincers (kinda retro Japan style, you won't see Koreans do that since the 90s!). Whether LZ would have evolved like this without the influence of the Elf games we can't say, but I think it's fairly safe to say it has speed up that process.


This post by Uberdude was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #182 Posted: Tue Oct 23, 2018 3:38 pm 
Lives with ko

Posts: 247
Liked others: 0
Was liked: 31
Rank: 2d
Bill Spight wrote:
Hmmm. I gather that the training games are all self play. Is it possible to have Elf vs. LZ training games? If so, I expect that both could benefit from that. :)
The nets are not trained on the games themselves but on search results in individual positions. So training on Elf vs LZ games would not be much different to training on some Elf games and some other LZ games (except slightly more variety in the types of positions occurring, and in game results for the value net). What matters is which net was used in the search at each position (whose turn it was).

The Elf games in the training are LZ selfplay with Elf weights AFAIK (the other options are not readily available).

The Elf vs LZ idea sounds most interesting for value net training but also very risky, may introduce some biases: game results would not reliably reflect the position itself anymore but also the strength difference.


This post by moha was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #183 Posted: Tue Oct 30, 2018 4:06 am 
Lives with ko

Posts: 238
Liked others: 4
Was liked: 57
In another thread, there was a very heated discussion about LZ 0.14 being stronger or weaker than LZ 0.15

But now, there's LZ 0.15.9999 (not a gag, you can download it HERE)
Discussion about V0.15.999 and V0.15.9999 HERE

V0.15.9999 is a test for V 0.16, and... it seems really stronger than V0.15

20 game match between
LZ 0.15.9999 network #184 and LZ 0.15 network #184
(3 min per side and per game, no pondering, GPU : 1x1080)
LZ 0.15.9999 wins 14-6
Big win...

Edit : another 40 game match 0.15.9999 v 0.15 (#184, same conditions as before)
LZ 0.15.9999 wins 22-18

So, after 60 games : LZ 0.15.9999 wins 36-24
Not bad ;-)


If someone wants the games...

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #184 Posted: Tue Oct 30, 2018 6:59 am 
Lives in gote

Posts: 428
Liked others: 1
Was liked: 121
Rank: KGS 2k
GD Posts: 100
KGS: Tryss
It's not suprising it is stronger : it's faster.

Look at the difference in playouts you got in your matchs

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #185 Posted: Wed Oct 31, 2018 2:21 pm 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
Vargo wrote:
Edit : another 40 game match 0.15.9999 v 0.15 (#184, same conditions as before)
LZ 0.15.9999 wins 22-18
So, after 60 games : LZ 0.15.9999 wins 36-24
Not bad ;-)
If someone wants the games...

"visit" parity test 0.15.9999 vs 0.15: 193 wins, 191 losses
Code:
C:\APPS\l0gpu15.9999\leelaz.exe -g -v 800 --gpu 1 --gpu 0 --noponder -t 12 -q -d --timemanage off -w C:\APPS\net\2da87ea8.gz
vs
C:\APPS\l0gpu\leelaz.exe -g -v 800 --gpu 0 --gpu 1 --noponder -t 12 -q -d --timemanage off -w C:\APPS\net\2da87ea8.gz

Nothing interesting ... :scratch:
P.S. If someone wants the games...

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #186 Posted: Wed Oct 31, 2018 10:12 pm 
Lives with ko

Posts: 238
Liked others: 4
Was liked: 57
The official V0.16 just came out !
Tryss wrote:
Nothing interesting ... :scratch:
V0.15.9999 or V.0.16 is faster than V0.15, so, it's stronger than V0.15 at TIME PARITY, not at visits parity (your test is at visits parity...)

If you run a match at TIME PARITY, I bet you'll see a big difference !

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #187 Posted: Thu Nov 01, 2018 1:41 am 
Judan

Posts: 6128
Location: Cambridge, UK
Liked others: 351
Was liked: 3299
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
How much faster is the question. I just did a little test on my GTX 1060 and LZ #157 in 10 seconds went from 4k playouts to 5.4k. So that's about 35% speed increase, nice!

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #188 Posted: Thu Nov 01, 2018 4:26 am 
Lives with ko

Posts: 238
Liked others: 4
Was liked: 57
Someone posted this at github about the improvement LZ 0.15 --> LZ 0.15.999

Attachment:
ar.jpg
ar.jpg [ 21.33 KiB | Viewed 3031 times ]


So, it seems you're about right ! :)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #189 Posted: Thu Nov 01, 2018 7:55 am 
Lives in sente

Posts: 935
Liked others: 0
Was liked: 159
Vargo wrote:
If you run a match at TIME PARITY, I bet you'll see a big difference !


And time parity is the correct way to measure.

However ENOUGH time (realistic time) can be critical also. A program that might be better than another at unrealistic short amounts of time but dos not improve as much when given a more realistic amount of time as the second program is not necessarily the stronger at REALISTIC time.

We really should be making some decisions about this. What would be realistic for human blitz games? What would be realistic for fast (but not blitz) human games? What would be realistic for normal speed human games?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #190 Posted: Thu Nov 01, 2018 10:23 am 
Lives with ko

Posts: 238
Liked others: 4
Was liked: 57
New network (185)
20 game match between #157 (best 192x15) and #185 (best 256x40)

LZ 0.16, 5 min per side and game, GPU 1x1080, twogtp 1.4.10

#185 wins 12-8 (6 wins as B, 6 wins as W)

Average length 227 moves , almost exactly 2 sec/move for both sides.

I'll upload the games if someone wants them.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #191 Posted: Thu Nov 01, 2018 9:55 pm 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
Vargo wrote:
If you run a match at TIME PARITY, I bet you'll see a big difference !

"True" time parity test - v16 vs v15 (3s per move, 2x1080ti, #184):
Code:
gogui-twogtp -black "C:\APPS\l0gpu16\leelaz.exe --gtp --weights=C:\APPS\net\2da87ea8.gz -t 12 --gpu 0 --gpu 1 --noponder --precision half" -white "C:\APPS\l0gpu\leelaz.exe --gtp --weights=C:\APPS\net\2da87ea8.gz -t 12 --gpu 0 --gpu 1 --noponder" -games 50 -sgffile v16-v15 -auto -time 1s+4s/1 -komi 7.5 -verbose
gogui-twogtp -white "C:\APPS\l0gpu16\leelaz.exe --gtp --weights=C:\APPS\net\2da87ea8.gz -t 12 --gpu 0 --gpu 1 --noponder --precision half" -black "C:\APPS\l0gpu\leelaz.exe --gtp --weights=C:\APPS\net\2da87ea8.gz -t 12 --gpu 0 --gpu 1 --noponder" -games 50 -sgffile v15-v16 -auto -time 1s+4s/1 -komi 7.5 -verbose

71 : 29

:clap:


Attachments:
v15-v16.zip [92.24 KiB]
Downloaded 30 times
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #192 Posted: Fri Nov 02, 2018 6:29 am 
Lives with ko

Posts: 243
Location: Spain
Liked others: 149
Was liked: 24
Rank: Low
GD Posts: 10
I'm following Leela's progression with delight. Does anyone know what its current rating against human pros on "normal" hardware (let's say one GTX 1080) would be? Is it better than the #1 human on those conditions? If so, by how much?

_________________
Sum ergo non ero.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #193 Posted: Fri Nov 02, 2018 6:55 am 
Lives in sente

Posts: 935
Liked others: 0
Was liked: 159
AND normal time. A human player could do little to no reading at 2 seconds/move.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #194 Posted: Fri Nov 02, 2018 7:32 am 
Beginner

Posts: 12
Liked others: 0
Was liked: 4
luigi wrote:
I'm following Leela's progression with delight. Does anyone know what its current rating against human pros on "normal" hardware (let's say one GTX 1080) would be? Is it better than the #1 human on those conditions? If so, by how much?


Considering that even a few months ago, LeelaZero was beating human professionals with 2-3 handicap stones, which it was not even built for, on consumer hardware. (Since then, there has been various improvements to handicap play)

I would think that LeelaZero is well above any human on a GTX 1080. It is possible that you can find a blind spot with a ladder or similar though, depending on the time settings.


This post by Yakago was liked by: luigi
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #195 Posted: Fri Nov 02, 2018 7:39 pm 
Dies with sente

Posts: 93
Liked others: 2
Was liked: 15
Rank: KGS 2 D
I run two games with equal thinking time (20 seconds per move) between Phoenixgo(W) and Leela zero(B) with weight 186. LZ lost both games.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #196 Posted: Fri Nov 02, 2018 9:02 pm 
Dies with sente

Posts: 93
Liked others: 2
Was liked: 15
Rank: KGS 2 D
LZ weight 185 is better against Phoenixgo.


Attachments:
phoenix-185.sgf [1.95 KiB]
Downloaded 335 times
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #197 Posted: Sat Nov 03, 2018 1:48 am 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
"True" time parity test - #186(black) vs #157a (3s per move, 2x1080ti).

Empire strikes back, part1:
Code:
gogui-twogtp -black "C:\APPS\l0gpu16\leelaz.exe --gtp --weights=C:\APPS\net\f50dc27c.gz -t 12 --gpu 0 --gpu 1 --noponder --precision half" -white "C:\APPS\l0gpu16\leelaz.exe --gtp --weights=C:\APPS\net\fc5e0a50.gz -t 12 --gpu 0 --gpu 1 --noponder --precision half" -games 50 -sgffile 186-157a -auto -time 1s+4s/1 -komi 7.5 -verbose

19:31 :bow:

P.S. Games #22 & #34 are the top of artificial intelligence :lol:.
What/where are we doing/going?

to be continued...


Attachments:
186-157a-34.sgf [1.09 KiB]
Downloaded 50 times
186-157a-22.sgf [1.08 KiB]
Downloaded 41 times


Last edited by nbc44 on Sat Nov 03, 2018 12:57 pm, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #198 Posted: Sat Nov 03, 2018 3:49 am 
Lives with ko

Posts: 238
Liked others: 4
Was liked: 57
splee99 wrote:
I run two games with equal thinking time (20 seconds per move) between Phoenixgo(W) and Leela zero(B) with weight 186..
What is your GPU ?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #199 Posted: Sat Nov 03, 2018 9:08 am 
Dies with sente

Posts: 93
Liked others: 2
Was liked: 15
Rank: KGS 2 D
GTX1050 with 2G memory. In the middle game, 20 s means about 1500 playouts for LZ and more playouts in the end game.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #200 Posted: Sat Nov 03, 2018 12:56 pm 
Dies in gote

Posts: 50
Liked others: 0
Was liked: 3
"True" time parity test - #186 (white) vs #157a (3s per move, 2x1080ti).

New hope, part2:
Code:
gogui-twogtp -white "C:\APPS\l0gpu16\leelaz.exe --gtp --weights=C:\APPS\net\f50dc27c.gz -t 12 --gpu 0 --gpu 1 --noponder --precision half" -black "C:\APPS\l0gpu16\leelaz.exe --gtp --weights=C:\APPS\net\fc5e0a50.gz -t 12 --gpu 0 --gpu 1 --noponder --precision half" -games 50 -sgffile 157a-186 -auto -time 1s+4s/1 -komi 7.5 -verbose

29:21 :scratch:

Finally: #186 vs #157a : +48-52=0

P.S. I can not explain the results of my test. What is wrong with black colour? V16, "--precision half" option, too little games...?


Attachments:
File comment: Part1 & Part2
186-157a.zip [97.33 KiB]
Downloaded 35 times
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 425 posts ]  Go to page Previous  1 ... 7, 8, 9, 10, 11, 12, 13 ... 22  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group