Life In 19x19
http://lifein19x19.com/

LZ's progression
http://lifein19x19.com/viewtopic.php?f=18&t=15718
Page 15 of 21

Author:  And [ Tue Jan 29, 2019 12:25 pm ]
Post subject:  Re: LZ's progression

MiniGo v15:
https://cloudygo.com/converted_model/v15-19x19/models/

Author:  lichigo [ Tue Jan 29, 2019 12:29 pm ]
Post subject:  Re: LZ's progression

Can we use this txt.gz file with lizzie by replacing the weight file?
I heard minigo is very strong.

Author:  And [ Tue Jan 29, 2019 12:36 pm ]
Post subject:  Re: LZ's progression

yes, i tried 990, ok

Author:  Uberdude [ Tue Jan 29, 2019 2:39 pm ]
Post subject:  Re: LZ's progression

I just used 000513-revenge in a 40 second game on Fox, beat a Fox 9d who I presume was also a bot by half a point (benefiting from being black with 6.5 komi?). viewtopic.php?p=241098#p241098

Author:  seigenblues [ Tue Jan 29, 2019 3:13 pm ]
Post subject:  Re: LZ's progression

Marcel GrĂ¼nauer wrote:
The latest converted v16 model, 000513-revenge, has a rating of 4202, according to https://cloudygo.com/v16-19x19/eval-graphs .

Now I'm not sure whether these ratings are comparable or whether they are within one run. That is, one model may be rated 4430 according to the v15 scale, and the other model may be rated 4202 according to the v16 scale, but I'm not sure whether these scales are the same.


Hi! Minigo author here :) The ratings are not directly comparable. We'll get some cross-run ratings in a bit when v16 is finished. But I'd love any help folks can provide in figuring out what the best settings to use and how to run it effectively -- we don't really have the charter to make things as strong as possible, so i'm hoping folks can help us there, but that's another story ;)

Author:  Satorian [ Wed Jan 30, 2019 2:57 am ]
Post subject:  Re: LZ's progression

Uberdude wrote:
So that was 5 mins per side per game, these test are 1 minute. 184 is a 40 block so (assuming there isn't an LZ 0.16 fp16 speedup between the two tests, and there may be) 800/5 = 160 so a lot short of the several thousand I like to give the 40 blocks to show their strength. If there is fp16 difference that'll help at most a factor of 2.


Interesting. Thank you! So even with the FP16 best case, we'd be at ~320 playouts. That's a far cry from what I would consider reasonable, especially given your experience and reports on playout effects.

I would prefer to have the engines simply play at 10s or 20s or 30s per move, in pure byo-yomi. No time management, no fancy distractions, and a move speed that is realistic either for online go or longer OTB games.

Author:  Uberdude [ Wed Jan 30, 2019 4:10 am ]
Post subject:  Re: LZ's progression

... though if I can get a decent few thousand playouts in 20 seconds per move on my 1060 GPU, Vargo may be able to get the same in 5 seconds on his top-end 1080 Ti.

Author:  Vargo [ Wed Jan 30, 2019 6:31 am ]
Post subject:  Re: LZ's progression

mini 10 game match
Elfv1 v. V16_000513 converted
LZ0.16, komi 7.5, no pondering, 2x1080Ti, -r 10, time parity, twogtp 1.5.0

Result : 10-0 for Elf... :o
I used -alternate, so, Elf-White seems to win 10 times, but in fact, Elf won 5 times as W and 5 times as B.


Something wrong with the parameters ?
(time is several seconds per move)

Attachment:
mgelf.jpg
mgelf.jpg [ 347.8 KiB | Viewed 8839 times ]

I'll try another match, with an other converted network.

Author:  Mike Novack [ Wed Jan 30, 2019 7:46 am ]
Post subject:  Re: LZ's progression

Uberdude wrote:
... though if I can get a decent few thousand playouts in 20 seconds per move on my 1060 GPU, Vargo may be able to get the same in 5 seconds on his top-end 1080 Ti.


THAT is more or less what I have been trying to say. I was NOT saying a particular amount of time but one realistic scaled to what we decide to consider "adequate hardware" << so less actual clock time if tests run on a more powerful machine >>

For example, we might decide "should be a portable machine" so power consumption and cooling requirements limit the gpu to the equivalent of a 1060. Our question would then be "what is the most powerful (go strength( program if run on a powerful, but portable, gaming machine at fast, but playable for humans time control?"

If the actual tests were to be run on a machine X times faster then we would scale the time to 1/x of that.

Author:  And [ Wed Jan 30, 2019 8:38 am ]
Post subject:  Re: LZ's progression

Vargo wrote:
mini 10 game match
Elfv1 v. V16_000513 converted
LZ0.16, komi 7.5, no pondering, 2x1080Ti, -r 10, time parity, twogtp 1.5.0

Result : 10-0 for Elf... :o

v15(20x256) is better:
https://cloudygo.com/v16-19x19/eval-graphs
https://cloudygo.com/all-eval-graphs?bucket=

Author:  Vargo [ Wed Jan 30, 2019 10:02 am ]
Post subject:  Re: LZ's progression

another mini 10 game match
Elfv1 v. V15-000990-cormorant converted
LZ0.16, komi 7.5, no pondering, 2x1080Ti, -r 5, time parity, twogtp 1.5.0

Result : 6-4 for Elf

time is several seconds per move. 10 games is too few, I'll run more games tomorrow.

Author:  Uberdude [ Wed Jan 30, 2019 10:19 am ]
Post subject:  Re: LZ's progression

Vargo wrote:
mini 10 game match
Elfv1 v. V16_000513 converted
LZ0.16, komi 7.5, no pondering, 2x1080Ti, -r 10, time parity, twogtp 1.5.0

Result : 10-0 for Elf... :o
I used -alternate, so, Elf-White seems to win 10 times, but in fact, Elf won 5 times as W and 5 times as B.

Something wrong with the parameters ?
(time is several seconds per move)


I expect what we are seeing here is the reduced playouts that 40-block v16_513 minigo gets in this short time parity are really hurting it against the 20-block elfv1. If you ran a test of fewer games but with enough time for minigo to get around 3k playouts per move (so I guess Elf gets about 6-8k, playouts speed isn't quite linear in number of blocks) then you'd find it not so lopsided. (By analogy with 40-block LZ's, I don't have much feel for minigo's behaviour yet)

Author:  Uberdude [ Wed Jan 30, 2019 1:23 pm ]
Post subject:  Re: LZ's progression

So I manually played a game between elfv1 (black) and minigo v16.513 (white), giving them >30s a move (approx ~3k for minigo and 6k for elf), giving them both extra time on hard moves, watching for where they agreed and differed. Elf as black (which it tends to favour) thought it was winning a lot, but had a blind spot for white p15 so its o17 was a lot worse than it expected. I don't have the time to play it to counting, but they both agree the game is pretty good for white (Elfv1 says 75%, minigo 85%). Here's the sgfs with winrates from each's point of view. Also elf missed q2 and winrate plunged on that, minigo saw it quickly (but probably not quickly enough for Vargo's setup).





And here's their view of the winrate, minigo's is much more stable/consistent. Does this mean it can't detect tiny mistakes as well as Elf can, or is Elf just being erratic and missing moves which cause it to change evaluation? I tend towards the latter.
Attachment:
elf winrate.PNG
elf winrate.PNG [ 287.94 KiB | Viewed 8711 times ]

Attachment:
minigo winrate.PNG
minigo winrate.PNG [ 287.96 KiB | Viewed 8711 times ]


Attachments:
minigo16.513 vs elfv1 minigo view.sgf [15.27 KiB]
Downloaded 934 times
minigo16.513 vs elfv1 elf view.sgf [10.37 KiB]
Downloaded 921 times

Author:  splee99 [ Wed Jan 30, 2019 10:17 pm ]
Post subject:  Re: LZ's progression

I have tried minigov15-modeste_converted and minigov15-cormorant_converted. They look indeed stronger than ELFv1. The minigo style is really far from human style. I don't see apparent activities of acquiring solid territory, nor building walls in the center. It just plays rock solid, so the ELF's sharp attack is not effective. However, this also means it is bad at (giving) handicap games because minigo is not good at attacking opponents weakness, in particular, regarding ko fights and capture races.

Author:  Vargo [ Thu Jan 31, 2019 4:59 am ]
Post subject:  Re: LZ's progression

10 game match
Elfv1 v. MG_000513_revenge

LZ0.16, twogtp 1.5.0 , 2x1080Ti, 8000 visits for Elf, 4000 visits for 513 (corresponding roughly to 2.6 and 3s/move)
-r 5 , no pondering
result : Elf wins 8-2

Attachment:
513_elf.jpg
513_elf.jpg [ 388.04 KiB | Viewed 11103 times ]
NB. I forgot to change the default sgf name, so, the games are labelled mg990elf_v4000-x.sgf, but the network used is revenge_000513, as you can see in the command line.

Author:  Vargo [ Sun Feb 03, 2019 10:17 pm ]
Post subject:  Re: LZ's progression

The first picture of this post shows contrasted results for 990_cormorant v. Elfv1 (cormorant wins at 10s/move and at 120s, but loses at 60s/move)

Here are two more 10 game matches with settings comparable to his 10s/move.
gpu: 2x1080Ti, constant 5s/move (5.03s effectively used) , twogtp, -alternate, -r 5, no pondering
All games by resignation, no error, no duplicate game.
Average length is 247 moves for the first 10 game match, and 212 moves for the second match.

Result : Elf wins 12-8

If we add the results of the three 10 game matches, we get ... 15-15
My conclusion is that 990 cormorant and Elfv1 are about the same strength at 10s/move with one 1080Ti. ;-)

Stats :
Attachment:
1.jpg
1.jpg [ 294.46 KiB | Viewed 10947 times ]
Attachment:
2.jpg
2.jpg [ 274.02 KiB | Viewed 10947 times ]

I'll upload the games if someone wants them.

Author:  And [ Sun Feb 03, 2019 11:27 pm ]
Post subject:  Re: LZ's progression

new network 15x192 60:40 over f438268e:
https://github.com/leela-zero/leela-zero/issues/2192

Author:  And [ Tue Feb 05, 2019 3:08 am ]
Post subject:  Re: LZ's progression

Vargo wrote:
My conclusion is that 990 cormorant and Elfv1 are about the same strength at 10s/move with one 1080Ti.

"v15-19x19_models_000990-cormorant_converted_quantized.txt" is stronger than "000990-cormorant_converted.txt"
https://github.com/leela-zero/leela-zer ... -459413040

Author:  Pippen [ Tue Feb 05, 2019 7:40 am ]
Post subject:  Re: LZ's progression

Is there a possiblity to download the last - say 100.000 games - from LZ's self play as a sgf collection?

Author:  Tryss [ Tue Feb 05, 2019 9:47 am ]
Post subject:  Re: LZ's progression

See here :

https://leela.online-go.com/zero/

Page 15 of 21 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/