It is currently Mon Nov 18, 2019 12:13 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 425 posts ]  Go to page Previous  1 ... 11, 12, 13, 14, 15, 16, 17 ... 22  Next
Author Message
Offline
 Post subject: Re: LZ's progression
Post #261 Posted: Sun Jan 20, 2019 2:02 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
Do you have a link to a network file usable with LZ0.16 ?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #262 Posted: Sun Jan 20, 2019 8:01 am 
Lives with ko

Posts: 165
Liked others: 31
Was liked: 10
https://drive.google.com/file/d/11uBKRz ... sp=sharing
when using LZ0.16 and Ms5-192x15 or f438268e sometimes LZ0.16 freezes. better lz0.15

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #263 Posted: Sun Jan 20, 2019 9:23 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
Thanks.
f438268 is OK with LZ016 and twogtp on my computer, but the combination Ms5, LZ(015 or 016), and twogtp is not OK, it crashes every time :sad:

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #264 Posted: Sun Jan 20, 2019 11:12 am 
Lives with ko

Posts: 165
Liked others: 31
Was liked: 10
gogui-twogtp V1.4.9, leela-zero-0.15-cpuonly 20 games ok. I previously unpacked the network. I have a weak video card now, but I'll try to run the gpu version

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #265 Posted: Sun Jan 20, 2019 11:33 am 
Lives with ko

Posts: 165
Liked others: 31
Was liked: 10
I started the gpu version (using sabaki) with f438268e - it plays normally, Ms5 - it works, but it makes stupid moves. It seems that Ms5 has problems with gpu

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #266 Posted: Mon Jan 21, 2019 3:53 am 
Lives with ko

Posts: 165
Liked others: 31
Was liked: 10
40 games f438268e - Ms5 28-12 (10 sec/move, no pondering, LZ0.15-cpuonly, twogtp V1.4.9)

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #267 Posted: Wed Jan 23, 2019 9:01 am 
Lives with ko

Posts: 165
Liked others: 31
Was liked: 10
60 more games, total
100 games f438268e - Ms5 63:37 (10 sec/move, no pondering, LZ0.15-cpuonly, twogtp V1.4.9)
Ms5 disappointed

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #268 Posted: Thu Jan 24, 2019 9:28 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
I heard networks bjiyxo f438268 (192x15) and LM_GX89 (192x15) were very strong. OK, I added two players : LZ_Elfv1 (224x290) and LZ#202 (256x40) to run a 4 players tournament.

Short time parity (2 min/game per player , with gpu:1x1080) no pondering , komi 7.5, every player playing 10 games against every other one.

Results :

202-Elf 3-7
202-f43 5-5
202-x89 2-8
x89-Elf 5-5
x89-f43 7-3
Elf-f43 5-5


I'm very much surprised by the poor result of #202 (too little time for 40b ?) and by the excellent result of GX89.

It's about 3h for the 60 matches. I'll try to run the same thing tomorrow, to see if I get similar results. Maybe (probably) I should let it run for a lot more games.


Complete tournament code :
Code:
rem 202-elf

gogui-twogtp -black "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\202.gz --noponder" -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\elf2.txt --noponder" -games 10 -sgffile 202elf -time 2 -auto -komi 7.5 -alternate

gogui-twogtp -analyze 202elf.dat

rem  202-f43

gogui-twogtp -black "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\202.gz --noponder" -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\f438268.gz --noponder" -games 10 -sgffile 202f43 -time 2 -auto -komi 7.5 -alternate

gogui-twogtp -analyze 202f43.dat

rem 202-X89

gogui-twogtp -black "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\202.gz --noponder" -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\LM_GX89.txt --noponder" -games 10 -sgffile 202x89 -time 2 -auto -komi 7.5 -alternate

gogui-twogtp -analyze 202x89.dat


rem X89-elf

gogui-twogtp -black "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\LM_GX89.txt --noponder" -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\elf2.txt --noponder" -games 10 -sgffile x89elf -time 2 -auto -komi 7.5 -alternate

gogui-twogtp -analyze x89elf.dat

rem X89-f43

gogui-twogtp -black "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\LM_GX89.txt --noponder" -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\f438268.gz --noponder" -games 10 -sgffile x89f43 -time 2 -auto -komi 7.5 -alternate

gogui-twogtp -analyze x89f43.dat

rem elf-f43

gogui-twogtp -black "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\elf2.txt --noponder" -white "C:\Users\jm\gogui150\leela-zero-0.16-win64ok\leelaz.exe --gtp --weights=C:\Users\jm\LZnetworks\f438268.gz --noponder" -games 10 -sgffile elff43 -time 2 -auto -komi 7.5 -alternate

gogui-twogtp -analyze elff43.dat
Attachment:
all the stats.zip [32.08 KiB]
Downloaded 34 times

Attachment:
60 games.zip [49.23 KiB]
Downloaded 37 times

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #269 Posted: Thu Jan 24, 2019 11:56 am 
Lives in sente

Posts: 935
Liked others: 0
Was liked: 159
Vargo wrote:


Short time parity (2 min/game per player , with gpu:1x1080) no pondering , komi 7.5, every player playing 10 games against every other one.

......

I'm very much surprised by the poor result of #202 (too little time for 40b ?) and by the excellent result of GX89.


I think this sort of thing requires REASONABLE time controls. Nothing faster than human blitz play. Saying that this irrelevant, that all that matters is equality, misses an important point.

Think of a race. Which is faster, a human or a race car?

a) Down a quarter mile straightway
b) Ten times around two pylons 50 feet apart

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #270 Posted: Thu Jan 24, 2019 8:49 pm 
Dies with sente

Posts: 94
Liked others: 2
Was liked: 15
Rank: KGS 2 D
Well, a "long game" on my computer (GTX1050) with 20 seconds per move maybe just same as the "2 minutes" game on a GTX1080. #202 lost again to LM-GX89. Sometimes it seems bigger doesn't mean better in the AI world.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #271 Posted: Thu Jan 24, 2019 11:47 pm 
Dies with sente

Posts: 101
Liked others: 0
Was liked: 8
Is it possible to use the gx-89 with lizzie?

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #272 Posted: Fri Jan 25, 2019 2:36 am 
Judan

Posts: 6187
Location: Cambridge, UK
Liked others: 354
Was liked: 3339
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
lichigo wrote:
Is it possible to use the gx-89 with lizzie?

Should be, I've used GX88 with Lizzie.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #273 Posted: Fri Jan 25, 2019 2:55 am 
Dies with sente

Posts: 101
Liked others: 0
Was liked: 8
Oh great, could you share the link. I would like to try it ^^

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #274 Posted: Fri Jan 25, 2019 4:57 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
lichigo wrote:
could you share the link
HERE


Mike Novack wrote:
this sort of thing requires REASONABLE time controls
With 2min/game and good gpu, these networks are already (very) strong, I don't see why it's not reasonable.



I've let it run for 6x20 more games.

After 6x30 games, results are :

202-Elf 12-18
202-f43 17-13
202-x89 12-18
x89-Elf 12-18
x89-f43 14-16
Elf-f43 17-13


I'll let it run up to 6x50 games, results in the evening.
_________________________________________________________

Assigning arbitrarily 4000 Elo to each network at the start of the tournament, after 30 games, I get

1)Elf_ v1 4047
2)LM_GX89 3994
3)f438268 3981
4)LZ_#202 3978


(and yes, I know it's questionable ;-) )

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #275 Posted: Fri Jan 25, 2019 8:55 am 
Lives with ko

Posts: 165
Liked others: 31
Was liked: 10
when using gpu 1080, these networks play very strongly and at 2 min/game. even using cpu only, will many be able to defeat them?
but the results are different if you use only cpu, for example:

100 games f438268e - elf.v1 62:38 (5 sec/move, no pondering, LZ0.15-cpuonly, twogtp V1.4.9)

for an elf there is not enough computational power, it is probably better to take a 15 block network for a CPU only (with little time control)
that is, when the strength of the networks does not differ much, the results of the matches depend not only on the number of games (if there are few of them, then the result is doubtful), but also from time to move + power gpu / cpu

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #276 Posted: Fri Jan 25, 2019 10:44 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
After 6x50 games, results are :

202-Elf 21-29
202-f43 21-29
202-x89 28-22
x89-Elf 22-28
x89-f43 26-24
Elf-f43 27-23


A little oddity : usually, all games are by resignation, but there was one won by 1.5 after... 423 moves :D
Attachment:
gx89_elf.jpg
gx89_elf.jpg [ 143.29 KiB | Viewed 1439 times ]


After 6x50 games, starting with 4000 for each network, I get relative Elos :


1)Elf_ v1 4034
2)LM_GX89 4003
3)f438268 3985
4)LZ_#202 3978

but 6x50 games is too few to be really meaningful.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #277 Posted: Fri Jan 25, 2019 2:15 pm 
Lives in sente

Posts: 935
Liked others: 0
Was liked: 159
Maybe I should make clearer what I am saying.

The whole point of this is determining the question is A stronger than B. It is entirely possible that A runs on less resources than B. In other words, at short enough time control, A "has enough to be strong" while B does not. However while A would be stronger given more time, the rate at which it gets stronger might be less than the rate at which B gets stronger with more time.

That is why am suggesting should be reasonable time controls (no faster than human blitz)

There is ANOTHER question that we could be asking. There is trade off between strength of hardware and time. We MIGHT be asking "what is the strongest program, playing at reasonable time controls, but on weaker hardware". THAT we can model at shorter time controls (like being used here) on this powerful hardware. The "weaker hardware" question IS important to many of us. We just need to be clear about what question we are asking.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #278 Posted: Sat Jan 26, 2019 1:11 am 
Lives with ko

Posts: 242
Liked others: 4
Was liked: 57
Hardware, parameters and time control are all reasonable, and you know it. Other tests with weaker or stronger hardware, with different networks, or different time controls are obviously possible. Feel free to show us your results, it would be interesting.

Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #279 Posted: Sat Jan 26, 2019 4:51 am 
Dies with sente

Posts: 87
Liked others: 26
Was liked: 12
Rank: Total beginner
Universal go server handle: Satorian
Of course everyone is free to do as desired with their own processing power and time, but I agree with Mike that a super short playing/move time has a bias towards smaller/quicker networks by allowing them more playouts within a generally low-playout regime, while those networks would plateau much quicker with longer, more reasonable time controls. I also don't find these super short time controls very interesting. Using an engine for review I would probably give it at least 30-60 minutes total or 10-15s per move, same for playing against it.


Last edited by Satorian on Sat Jan 26, 2019 4:38 pm, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: LZ's progression
Post #280 Posted: Sat Jan 26, 2019 5:50 am 
Judan

Posts: 6187
Location: Cambridge, UK
Liked others: 354
Was liked: 3339
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
How many playouts a move is 40 block LZ getting in these 2 minute test games on a 1080 Ti? If it's at least 3k (what I get in 20 seconds on my 1060, and would take 5 minutes on my crappy laptop) then I think that's about enough for a test where the deeper network can show its strength and not be handicapped by being unable to do basic reading (there was a nice chart on the LZ github showing this I linked a while back).

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 425 posts ]  Go to page Previous  1 ... 11, 12, 13, 14, 15, 16, 17 ... 22  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group