It is currently Sun Apr 21, 2019 7:34 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 380 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7 ... 19  Next
Author Message
Offline
 Post subject: Re: LZ's progression
Post #61 Posted: Sun Jul 08, 2018 3:13 pm 
Dies in gote

Posts: 55
Liked others: 0
Was liked: 3
[admin]
Off-topic link removed.
-JB
[/admin]

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #62 Posted: Mon Jul 09, 2018 2:48 pm 
Dies with sente

Posts: 116
Liked others: 33
Was liked: 13
Rank: KGS 6 dan
jokkebk wrote:
https://www.gokgs.com/gameArchives.jsp?user=petgo3

Hint: KGS archives with color highlight of wins/losses of given user with my Tampermonkey (Chrome plugin) script: http://joonaspihlajamaa.com/data/kgs_graphs.user.js
This script is just excellent, thank you!

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #63 Posted: Wed Jul 11, 2018 12:29 am 
Lives with ko

Posts: 219
Liked others: 3
Was liked: 54
I've run a 100 games twogtp match between regular LZ 0.15, with network #155 and LZ Variable Komi V0.3, with network #155.
Even games, with komi 7.5 , --noponder --visits=3201 -r 10 for both.

Regular LZ wins 52-48 (50 games as W, 50 as B)

It looks like LZVK is as strong as Regular LZ on even games (I should run more games to confirm this...)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #64 Posted: Wed Jul 18, 2018 7:24 am 
Lives with ko

Posts: 219
Liked others: 3
Was liked: 54
Four 10-games matches between LZ variable komi V0.31 (network#156) and gnugo 3.8 at various handicaps (I used twogtp V1.4.10)

komi 7.5, playouts=1000 for LZvk
H6 : LZvk wins 10-0

komi 7.5, playouts=2000 for LZvk
H8 : draw 5-5

komi 7.5, playouts=2000 for LZvk
H9 : gnugo wins 8-2

komi 7.5, playouts=4000 for LZvk
H9 : gnugo wins 9-1 (a bit curious, compared to 8-2 with less playouts)


Two 10-games matches between LZ variable komi(#156) and Leela 0.11.0 OpenCL

komi 7.5, playouts = 1000 for both
H5 : Leela OpenCl wins 6-4

komi 7.5, playouts = 3000 for both
H5 : Leela OpenCl wins 7-3

I didn't use -r 0 , so, in some games, LZvk resigns rather early.
If someone wants to see the games, I can upload them.


This post by Vargo was liked by: Friday9i
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #65 Posted: Wed Jul 18, 2018 9:35 pm 
Lives with ko

Posts: 219
Liked others: 3
Was liked: 54
A while ago, I ran H6, H7 and H8 mini matches between LZvk(#153) and LZ#80.
LZvk won at H6 and H7, but jokkebk's remark is true...

Quote:
Vargo wrote:
Thanks to alreadydone, LZ can now handle high handicap (HERE)
Playing (and winning) H7 and H8 games against 4 Dan... Wow !
Quote:
jokkebk wrote :
Winning against "4d" LZ #80 might not be such an achievement, considering the LZ network is probably very bad playing as black in high handicap.

So, what about a rematch, but with LZvk for both #157 and #80 (they're supposed to be 10.6D and 4D)

visits=3201, komi 7.5, noponder for both, 10 games match with twogtp 1.4.10, at a more reasonable H5.

LZvk with network #157 wins (only) 6-4
If someone wants the games, I'll upload them.

twogtp v1.4.10 with handicap seems to work well with LZvk , with regular Leela OpenCl and with gnugo, but not with gtp4zen, I don't know why.


This post by Vargo was liked by: Friday9i
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #66 Posted: Tue Jul 24, 2018 2:05 am 
Lives with ko

Posts: 219
Liked others: 3
Was liked: 54
In his last witty post (the "#xxx Ten Game Match Report" series at reddit, cbaduk), HERE, Dontbteme writes :
Quote:
Actually, maybe LZ#157 could already give LeelaElf a run for its money with Time Parity... Too bad I didn’t think of setting a match between them before, and REALLY meant it when I said I’d go to bed early.)

He's right, maybe it would show LZ's progress.

So, here are three 20 games matches at 5 minutes (per game and per side), at 10 minutes, and at 15 minutes, no pondering, komi 7.5.
The matches are run with twogtp 1.4.10 (win10, i7, 1x1080)

LZ15#157 vs LZElf :
at 5 minutes , LZElf wins 13-7 (65%) (6 wins as B, and 7 as W)
at 10 minutes, LZElf wins 13-7 (65%) (5 wins as B, and 8 as W)
at 15 minutes, LZElf wins 12-8 (60%) (6 wins as B, and 6 as W)

So, we're not there yet, but it's coming nearer ;-)


Edit : I just saw that Dontbteme has posted an Elf vs #157 match , but with very different result... HERE


Attachments:
File comment: the 60 games
157vElf.rar [47.27 KiB]
Downloaded 21 times

This post by Vargo was liked by 2 people: Uberdude, Waylon
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #67 Posted: Tue Jul 24, 2018 3:14 pm 
Dies in gote

Posts: 40
Liked others: 2
Was liked: 14
Rank: EGF 1 kyu
KGS: finity
Vargo wrote:
LZ15#157 vs LZElf :
at 5 minutes , LZElf wins 13-7 (65%) (6 wins as B, and 7 as W)
at 10 minutes, LZElf wins 13-7 (65%) (5 wins as B, and 8 as W)
at 15 minutes, LZElf wins 12-8 (60%) (6 wins as B, and 6 as W)


Very nice! It means one can likely get already quite on par analysis with Lizzie on Leela Zero #157 as LZElf. If there's still 2-3 promotions with 192x15b net, it might be 50-50 already on time parity, but my bets would be that it'll take too long and Leela will switch to larger net before that.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #68 Posted: Mon Jul 30, 2018 11:12 pm 
Lives with ko

Posts: 219
Liked others: 3
Was liked: 54
There's been some controversy about the passage to 20 block networks.

Three 10 games matches between LZ15 with #157 (192x15) and LZ15 #159 (256x20)

twogtp V 1.4.10 , komi 7.5, no pondering, 5 minutes per side and per game , 5 games as B, 5 games as W. (id with 10 min, and with 15 min.)

All games won or lost by resignation, all 30 games between 125 and 347 moves. Time used is consistently around two thirds of the alloted time.

LZ#157 vs LZ#159 :
at 05 minutes , LZ157 wins 6-4 (4 wins as B, and 2 as W)
at 10 minutes , LZ157 wins 6-4 (3 wins as B, and 3 as W)
at 15 minutes , LZ157 wins 7-3 (5 wins as B, and 2 as W)


Seems like #157 is still stronger at equal time setting...

Edit : hardware : 1xGTX1080, i7, win 10

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #69 Posted: Mon Jul 30, 2018 11:44 pm 
Judan

Posts: 5685
Location: Cambridge, UK
Liked others: 321
Was liked: 3007
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Roughly how many playouts per move are the 15- and 20-block networks getting at these settings?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #70 Posted: Tue Jul 31, 2018 1:20 am 
Lives with ko

Posts: 219
Liked others: 3
Was liked: 54
Same kind of results with #160 at 5' per side and per game

LZ15#157 vs LZ15#160 :
at 5 minutes , LZ157 wins 7-3 (4 wins as B, and 3 as W)

@Uberdude
I don't know. At the end of an automatically played n-game match, twogtp gives reports like this
Attachment:
rep.jpg
rep.jpg [ 32.76 KiB | Viewed 2955 times ]

with : winner, number of duplicate games (hopefully none) , length of game, time for B and for W, number of errors, but nothing about playouts.
I'll run some tests, and I'll let you know.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #71 Posted: Tue Jul 31, 2018 1:47 am 
Lives in gote

Posts: 552
Location: Vienna, Austria
Liked others: 246
Was liked: 285
I'm not very knowledgable about how to interpret the results, but did note that with the 20-block networks there is a wild fluctuation of win rates. For example, yesterday and today some 20-block candidates only had a 30% win rate against the current #160.

So maybe getting a high enough percentage to be promoted is more of a fluke than a steady progression. With the 15-block networks and earlier, there were lots of candidates just below the promotion threshold of 55%.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #72 Posted: Tue Jul 31, 2018 2:00 am 
Lives in gote

Posts: 552
Location: Vienna, Austria
Liked others: 246
Was liked: 285
There's a new ELF network! It's been converted for Leela Zero. https://userscloud.com/4dgfy46eum5p

See the original release at https://github.com/pytorch/ELF/releases

Edit: 1) Lizzie works with ELF-2 as well. 2) I ran only one game with 6000 visits each against ELF-1 and ELF-2 won. Will try to run more games later.


Last edited by Marcel Grünauer on Tue Jul 31, 2018 3:19 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #73 Posted: Tue Jul 31, 2018 2:47 am 
Lives with ko

Posts: 219
Liked others: 3
Was liked: 54
@Uberdude :
#160 at 1600 visits takes a little more time than #157 at 3200 visits, and the matches take about the same time as the 5' games.
cf this :
Attachment:
rep2.jpg
rep2.jpg [ 179.06 KiB | Viewed 2968 times ]


@Marcel Grünauer
I've just downloaded the new ELF network, looking forward to trying it, thanks !


This post by Vargo was liked by: Uberdude
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #74 Posted: Tue Jul 31, 2018 2:53 am 
Judan

Posts: 5685
Location: Cambridge, UK
Liked others: 321
Was liked: 3007
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Marcel Grünauer wrote:
I'm not very knowledgable about how to interpret the results, but did note that with the 20-block networks there is a wild fluctuation of win rates. For example, yesterday and today some 20-block candidates only had a 30% win rate against the current #160.

So maybe getting a high enough percentage to be promoted is more of a fluke than a steady progression. With the 15-block networks and earlier, there were lots of candidates just below the promotion threshold of 55%.


Yes, I noticed this too. There was a test match of #159 (the 2nd 20-block against #157 (best 15-block) and it got 50% win vs 45% for #158. I've not yet seen a test match for the new #160 vs #157. Given all these 20-30% candidates there is more danger you are not progressing but just jumping around about the same level networks that randomly sometimes do well; so these test matches against #157 as a more external benchmark are important. In fact during normal 15-block training I think doing some occasionally e.g. #157 vs #147 would be a good idea to see how inflated the incremental self-improvement Elo is (based on the incremental Elo differences from the promotions 147->148->149 etc it went from 11401 -> 11806 which predicts #157 would beat #147 91% of the time, but I would bet it would be quite a lot lower than that in reality).


This post by Uberdude was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #75 Posted: Tue Jul 31, 2018 5:45 am 
Lives with ko

Posts: 219
Liked others: 3
Was liked: 54
20 games match between ELF1 and ELF2 at 1600 visits.
(twogtp 1.4.10, komi 7.5, noponder, all games by resignation)

ELF2 wins 14:6 (70% , 9 wins as W, 5 wins as B)


If someone wants the games and/or the reports...

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #76 Posted: Tue Jul 31, 2018 6:12 am 
Lives in sente

Posts: 1328
Liked others: 616
Was liked: 412
Rank: AGA 3k KGS 2k
GD Posts: 61
KGS: dfan
There's a test match going on right now between the ELF1 and ELF2 weights, imported into Leela Zero. As I write this ELF2 is up 102 to 51.

Edit: the final result was 265 to 145 (64.6%).


Last edited by dfan on Tue Jul 31, 2018 9:08 am, edited 1 time in total.

This post by dfan was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #77 Posted: Tue Jul 31, 2018 6:42 am 
Judan

Posts: 5685
Location: Cambridge, UK
Liked others: 321
Was liked: 3007
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Instead of win ratios amongst all these super-strong bots, I'd kind of like "average number of moves a strong/weak Tygem/Fox 9d lasts until only 10% winrate" :D

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #78 Posted: Tue Jul 31, 2018 8:16 am 
Lives with ko

Posts: 219
Liked others: 3
Was liked: 54
Last one for today ;-) :

20-game match LZ15#160 v LZ15_ELF2 at time parity.

5 minutes per game (1xGTX1080), noponder, komi 7.5

ELF2 wins 18:2 (90% , 10 wins as W, 8 wins as B)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #79 Posted: Tue Jul 31, 2018 10:51 am 
Dies with sente

Posts: 75
Liked others: 14
Was liked: 18
Rank: EGF 2k
KGS: MKyle
Great stuff. Thanks for sharing.

Almost off topic, but it seems like you have a method that could be useful for generating large numbers of Bot vs Bot games. I'd be interested to play with that. Are you using a freely available tool? Or something you've developed yourself?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #80 Posted: Tue Jul 31, 2018 9:05 pm 
Lives with ko

Posts: 293
Liked others: 241
Was liked: 145
I saw today two games on WBaduk server, between Ichiriki Ryo 8p (one of Japan's top pros) and LeelaZero. Both were played on even.

LeelaZero won both (one playing black, one white).

Does anyone know what was the event?

_________________
Sorin - 361points.com

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 380 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7 ... 19  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: Majestic-12 [Bot] and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group