It is currently Thu Mar 28, 2024 9:20 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 418 posts ]  Go to page Previous  1 ... 12, 13, 14, 15, 16, 17, 18 ... 21  Next
Author Message
Offline
 Post subject: Re: LZ's progression
Post #281 Posted: Tue Jan 29, 2019 12:25 pm 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
MiniGo v15:
https://cloudygo.com/converted_model/v15-19x19/models/

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #282 Posted: Tue Jan 29, 2019 12:29 pm 
Lives with ko

Posts: 198
Liked others: 4
Was liked: 16
Can we use this txt.gz file with lizzie by replacing the weight file?
I heard minigo is very strong.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #283 Posted: Tue Jan 29, 2019 12:36 pm 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
yes, i tried 990, ok

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #284 Posted: Tue Jan 29, 2019 2:39 pm 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
I just used 000513-revenge in a 40 second game on Fox, beat a Fox 9d who I presume was also a bot by half a point (benefiting from being black with 6.5 komi?). viewtopic.php?p=241098#p241098

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #285 Posted: Tue Jan 29, 2019 3:13 pm 
Dies with sente

Posts: 76
Liked others: 10
Was liked: 47
Rank: AGA 4d
Marcel Grünauer wrote:
The latest converted v16 model, 000513-revenge, has a rating of 4202, according to https://cloudygo.com/v16-19x19/eval-graphs .

Now I'm not sure whether these ratings are comparable or whether they are within one run. That is, one model may be rated 4430 according to the v15 scale, and the other model may be rated 4202 according to the v16 scale, but I'm not sure whether these scales are the same.


Hi! Minigo author here :) The ratings are not directly comparable. We'll get some cross-run ratings in a bit when v16 is finished. But I'd love any help folks can provide in figuring out what the best settings to use and how to run it effectively -- we don't really have the charter to make things as strong as possible, so i'm hoping folks can help us there, but that's another story ;)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #286 Posted: Wed Jan 30, 2019 2:57 am 
Dies with sente

Posts: 93
Liked others: 26
Was liked: 13
Rank: Total beginner
Universal go server handle: Satorian
Uberdude wrote:
So that was 5 mins per side per game, these test are 1 minute. 184 is a 40 block so (assuming there isn't an LZ 0.16 fp16 speedup between the two tests, and there may be) 800/5 = 160 so a lot short of the several thousand I like to give the 40 blocks to show their strength. If there is fp16 difference that'll help at most a factor of 2.


Interesting. Thank you! So even with the FP16 best case, we'd be at ~320 playouts. That's a far cry from what I would consider reasonable, especially given your experience and reports on playout effects.

I would prefer to have the engines simply play at 10s or 20s or 30s per move, in pure byo-yomi. No time management, no fancy distractions, and a move speed that is realistic either for online go or longer OTB games.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #287 Posted: Wed Jan 30, 2019 4:10 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
... though if I can get a decent few thousand playouts in 20 seconds per move on my 1060 GPU, Vargo may be able to get the same in 5 seconds on his top-end 1080 Ti.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #288 Posted: Wed Jan 30, 2019 6:31 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
mini 10 game match
Elfv1 v. V16_000513 converted
LZ0.16, komi 7.5, no pondering, 2x1080Ti, -r 10, time parity, twogtp 1.5.0

Result : 10-0 for Elf... :o
I used -alternate, so, Elf-White seems to win 10 times, but in fact, Elf won 5 times as W and 5 times as B.


Something wrong with the parameters ?
(time is several seconds per move)

Attachment:
mgelf.jpg
mgelf.jpg [ 347.8 KiB | Viewed 8837 times ]

I'll try another match, with an other converted network.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #289 Posted: Wed Jan 30, 2019 7:46 am 
Lives in sente

Posts: 1037
Liked others: 0
Was liked: 180
Uberdude wrote:
... though if I can get a decent few thousand playouts in 20 seconds per move on my 1060 GPU, Vargo may be able to get the same in 5 seconds on his top-end 1080 Ti.


THAT is more or less what I have been trying to say. I was NOT saying a particular amount of time but one realistic scaled to what we decide to consider "adequate hardware" << so less actual clock time if tests run on a more powerful machine >>

For example, we might decide "should be a portable machine" so power consumption and cooling requirements limit the gpu to the equivalent of a 1060. Our question would then be "what is the most powerful (go strength( program if run on a powerful, but portable, gaming machine at fast, but playable for humans time control?"

If the actual tests were to be run on a machine X times faster then we would scale the time to 1/x of that.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #290 Posted: Wed Jan 30, 2019 8:38 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
Vargo wrote:
mini 10 game match
Elfv1 v. V16_000513 converted
LZ0.16, komi 7.5, no pondering, 2x1080Ti, -r 10, time parity, twogtp 1.5.0

Result : 10-0 for Elf... :o

v15(20x256) is better:
https://cloudygo.com/v16-19x19/eval-graphs
https://cloudygo.com/all-eval-graphs?bucket=

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #291 Posted: Wed Jan 30, 2019 10:02 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
another mini 10 game match
Elfv1 v. V15-000990-cormorant converted
LZ0.16, komi 7.5, no pondering, 2x1080Ti, -r 5, time parity, twogtp 1.5.0

Result : 6-4 for Elf

time is several seconds per move. 10 games is too few, I'll run more games tomorrow.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #292 Posted: Wed Jan 30, 2019 10:19 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Vargo wrote:
mini 10 game match
Elfv1 v. V16_000513 converted
LZ0.16, komi 7.5, no pondering, 2x1080Ti, -r 10, time parity, twogtp 1.5.0

Result : 10-0 for Elf... :o
I used -alternate, so, Elf-White seems to win 10 times, but in fact, Elf won 5 times as W and 5 times as B.

Something wrong with the parameters ?
(time is several seconds per move)


I expect what we are seeing here is the reduced playouts that 40-block v16_513 minigo gets in this short time parity are really hurting it against the 20-block elfv1. If you ran a test of fewer games but with enough time for minigo to get around 3k playouts per move (so I guess Elf gets about 6-8k, playouts speed isn't quite linear in number of blocks) then you'd find it not so lopsided. (By analogy with 40-block LZ's, I don't have much feel for minigo's behaviour yet)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #293 Posted: Wed Jan 30, 2019 1:23 pm 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
So I manually played a game between elfv1 (black) and minigo v16.513 (white), giving them >30s a move (approx ~3k for minigo and 6k for elf), giving them both extra time on hard moves, watching for where they agreed and differed. Elf as black (which it tends to favour) thought it was winning a lot, but had a blind spot for white p15 so its o17 was a lot worse than it expected. I don't have the time to play it to counting, but they both agree the game is pretty good for white (Elfv1 says 75%, minigo 85%). Here's the sgfs with winrates from each's point of view. Also elf missed q2 and winrate plunged on that, minigo saw it quickly (but probably not quickly enough for Vargo's setup).





And here's their view of the winrate, minigo's is much more stable/consistent. Does this mean it can't detect tiny mistakes as well as Elf can, or is Elf just being erratic and missing moves which cause it to change evaluation? I tend towards the latter.
Attachment:
elf winrate.PNG
elf winrate.PNG [ 287.94 KiB | Viewed 8709 times ]

Attachment:
minigo winrate.PNG
minigo winrate.PNG [ 287.96 KiB | Viewed 8709 times ]


Attachments:
minigo16.513 vs elfv1 minigo view.sgf [15.27 KiB]
Downloaded 934 times
minigo16.513 vs elfv1 elf view.sgf [10.37 KiB]
Downloaded 921 times

This post by Uberdude was liked by: ez4u
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #294 Posted: Wed Jan 30, 2019 10:17 pm 
Dies with sente

Posts: 101
Liked others: 2
Was liked: 16
Rank: KGS 2 D
I have tried minigov15-modeste_converted and minigov15-cormorant_converted. They look indeed stronger than ELFv1. The minigo style is really far from human style. I don't see apparent activities of acquiring solid territory, nor building walls in the center. It just plays rock solid, so the ELF's sharp attack is not effective. However, this also means it is bad at (giving) handicap games because minigo is not good at attacking opponents weakness, in particular, regarding ko fights and capture races.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #295 Posted: Thu Jan 31, 2019 4:59 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
10 game match
Elfv1 v. MG_000513_revenge

LZ0.16, twogtp 1.5.0 , 2x1080Ti, 8000 visits for Elf, 4000 visits for 513 (corresponding roughly to 2.6 and 3s/move)
-r 5 , no pondering
result : Elf wins 8-2

Attachment:
513_elf.jpg
513_elf.jpg [ 388.04 KiB | Viewed 11101 times ]
NB. I forgot to change the default sgf name, so, the games are labelled mg990elf_v4000-x.sgf, but the network used is revenge_000513, as you can see in the command line.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #296 Posted: Sun Feb 03, 2019 10:17 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
The first picture of this post shows contrasted results for 990_cormorant v. Elfv1 (cormorant wins at 10s/move and at 120s, but loses at 60s/move)

Here are two more 10 game matches with settings comparable to his 10s/move.
gpu: 2x1080Ti, constant 5s/move (5.03s effectively used) , twogtp, -alternate, -r 5, no pondering
All games by resignation, no error, no duplicate game.
Average length is 247 moves for the first 10 game match, and 212 moves for the second match.

Result : Elf wins 12-8

If we add the results of the three 10 game matches, we get ... 15-15
My conclusion is that 990 cormorant and Elfv1 are about the same strength at 10s/move with one 1080Ti. ;-)

Stats :
Attachment:
1.jpg
1.jpg [ 294.46 KiB | Viewed 10945 times ]
Attachment:
2.jpg
2.jpg [ 274.02 KiB | Viewed 10945 times ]

I'll upload the games if someone wants them.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #297 Posted: Sun Feb 03, 2019 11:27 pm 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
new network 15x192 60:40 over f438268e:
https://github.com/leela-zero/leela-zero/issues/2192

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #298 Posted: Tue Feb 05, 2019 3:08 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
Vargo wrote:
My conclusion is that 990 cormorant and Elfv1 are about the same strength at 10s/move with one 1080Ti.

"v15-19x19_models_000990-cormorant_converted_quantized.txt" is stronger than "000990-cormorant_converted.txt"
https://github.com/leela-zero/leela-zer ... -459413040

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #299 Posted: Tue Feb 05, 2019 7:40 am 
Lives in gote

Posts: 676
Liked others: 6
Was liked: 31
KGS: 2d
Is there a possiblity to download the last - say 100.000 games - from LZ's self play as a sgf collection?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #300 Posted: Tue Feb 05, 2019 9:47 am 
Lives in gote

Posts: 502
Liked others: 1
Was liked: 153
Rank: KGS 2k
GD Posts: 100
KGS: Tryss
See here :

https://leela.online-go.com/zero/

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 418 posts ]  Go to page Previous  1 ... 12, 13, 14, 15, 16, 17, 18 ... 21  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group