The main question is what is more important to us - victory or honesty?
| Life In 19x19 http://lifein19x19.com/ |
|
| LZ's progression http://lifein19x19.com/viewtopic.php?f=18&t=15718 |
Page 20 of 21 |
| Author: | nbc44 [ Thu Apr 25, 2019 2:37 pm ] |
| Post subject: | Re: LZ's progression |
The main question is what is more important to us - victory or honesty?
|
|
| Author: | iopq [ Thu Apr 25, 2019 6:06 pm ] |
| Post subject: | Re: LZ's progression |
Did you set the batch number to half of the threads? You can get better perf. |
|
| Author: | Vargo [ Thu Apr 25, 2019 10:21 pm ] |
| Post subject: | Re: LZ's progression |
Three last benchmarks : not specifying -t XXX seems to give slightly less n/s iopq wrote: Did you set the batch number to half of the threads? I'm not sure it's better... but again, I find the effect of -t XXX rather bizarre, and these benchmarks are maybe flawed, one way or another...--precision half seems to be about the same as not specifying the precision 
|
|
| Author: | iopq [ Fri Apr 26, 2019 12:36 am ] |
| Post subject: | Re: LZ's progression |
Benchmark with batching, it would be faster than just threading |
|
| Author: | Uberdude [ Fri Apr 26, 2019 2:20 am ] |
| Post subject: | Re: LZ's progression |
LZ just beat Golaxy in the Fuzhou AI tournament  https://home.yikeweiqi.com/#/live/board/17523 |
|
| Author: | Amtiskaw [ Fri Apr 26, 2019 4:12 am ] | ||||
| Post subject: | Re: LZ's progression | ||||
Is there a way to download SGF from that site, and if yes, can someone post it here?  Alright I think I found them...
|
|||||
| Author: | Uberdude [ Fri Apr 26, 2019 5:44 am ] |
| Post subject: | Re: LZ's progression |
I've previously hacked out the sgf from yike using browser dev tools, don't know if there's an easier way. Inline sgf players: |
|
| Author: | splee99 [ Fri Apr 26, 2019 6:38 pm ] |
| Post subject: | Re: LZ's progression |
iopq wrote: Benchmark with batching, it would be faster than just threading Could you please show me the command option for batching? It seems that Sabaki always choose batch size 1 by default, while the autogtp chooses something different. |
|
| Author: | Amtiskaw [ Sat Apr 27, 2019 4:45 am ] | |||
| Post subject: | Re: LZ's progression | |||
Leela lost both its semi-final games. I enjoyed watching the second one live, it had a rather drastic semeai, which sadly became 1-eye vs 0-eye...
|
||||
| Author: | hoa803 [ Sat Apr 27, 2019 2:29 pm ] |
| Post subject: | Re: LZ's progression |
nbc44 wrote: hoa803 wrote: NBC, if you're using visit parity you shouldn't use ponder. The time to reach that number of visits varies by position. Time parity matches can use ponder on separate hardware though, similar to how Alphago was tested. Why not? I'm using separate GPU for each net. Think about what you are trying to do in terms of mathematics. LZ has a chance to win a random game, let us call that probability P. In a match with ponder turned on, you've introduce another variable - the total thinking time permitted for each engine due to use of ponder. On a given game either LZ or Elf is likely to get more overall thinking time. Since we already know that strength is directly related to thinking time, your chance of LZ winning a particular game is now the function P(x), where x is a random variable related to the strength at different thinking times. That means that the statistical basis being used to evaluate strength is no longer valid, because with fixed visit count and ponder the result is a function of another random variable that we don't know anything about. The function P(x) is most likely Gaussian, but we don't know the standard deviation or anything along those lines. I'm not enough of a mathematician to know what that does to the conclusion over a 400 game match. Also - you should put your queries about thread and batch count to the actual programmers on GitHub. Again you are introducing variables that you don't understand. I think I've seen some discussion about both batch size and number of threads having an impact on performance. You should definitely ask if you want to understand what is going on. Maybe post your results and see what GCP says about it. |
|
| Author: | iopq [ Sun Apr 28, 2019 10:34 am ] |
| Post subject: | Re: LZ's progression |
splee99 wrote: iopq wrote: Benchmark with batching, it would be faster than just threading Could you please show me the command option for batching? It seems that Sabaki always choose batch size 1 by default, while the autogtp chooses something different. Something like --batchsize I believe |
|
| Author: | hoa803 [ Mon Apr 29, 2019 9:41 am ] |
| Post subject: | Re: LZ's progression |
Quote: Could you please show me the command option for batching? It seems that Sabaki always choose batch size 1 by default, while the autogtp chooses something different. If you use Leela Zero engine version 0.17 it should choose the optimal thread and batch size for your hardware by default. Note that you cannot have more batches than threads. Try this command in your LZ directory: leelaz -h You will see a list of some things you can do. In terms of engine strength, this person has done a ton of work: https://github.com/guitanj/AutoPK/blob/master/README.md These data show that if we are interested in strength as a function on thinking time, LZ will be noticeably weaker since Elfv2 is only a 20-block network and therefore evaluates more quickly. In terms of pure network strength regardless of thinking time, Leela Zero is generally stronger except at high playouts. I find that very interesting. Very cool data and a lot of work. This person deserves much credit for doing all these tests. |
|
| Author: | iopq [ Tue May 07, 2019 5:47 am ] |
| Post subject: | Re: LZ's progression |
I believe it will do something like batch size 5 and 10 threads. Anyone try the new 15b net vs. the 40b at time parity? |
|
| Author: | Aram [ Tue May 07, 2019 7:25 am ] |
| Post subject: | Re: LZ's progression |
There was a new 40b network promoted today, and it was also tested officially against a other 40b network from 10 networks ago. The result is a 69% winrate against the older 40b network from 10 networks back. Rather impressive! The 40b network is clearly still growing and learning and getting stronger at a good pace! |
|
| Author: | Uberdude [ Wed May 08, 2019 3:52 am ] |
| Post subject: | Re: LZ's progression |
Aram wrote: There was a new 40b network promoted today, and it was also tested officially against a other 40b network from 10 networks ago. The result is a 69% winrate against the older 40b network from 10 networks back. Rather impressive! The 40b network is clearly still growing and learning and getting stronger at a good pace! I wonder what the improvement in winrate of this latest 40b compared to 10 networks ago would be versus a non-LZ bot (for fairest comparison would be time parity but even visit parity I expect same effect). Eg maybe #215 beats Elfv2 at 1600 visit parity 65% of the time (and maybe <50% at time parity), and #225 beats it only 67%. I expect quite a lot less difference than 19% (or equivalent ratio difference). |
|
| Author: | splee99 [ Wed May 08, 2019 6:42 pm ] |
| Post subject: | Re: LZ's progression |
Here is a game between LZ-225 (B) + LZ0.17 and ELF (W) + LZ0.16 in time parity (20 seconds per move). The ELFv2 was far ahead with winrate close to 90%, until move 135, when LZ-225 suddenly started a complicated fight. After that ELF still had lots of chance to win the fight, according to LZ-225, but eventually ELF gave up. Attachment:
|
|
| Author: | Aram [ Thu May 09, 2019 12:20 am ] |
| Post subject: | Re: LZ's progression |
One shouldnt use ELF winrates or ELF in general for analysis. They sharpened the network a rather huge amount to make it stronger in self play and play versus other bots. (Probably because they plateud and needed to get some more strength out of the relatively small 20-ish block network) So basically ELFs possibility to grow and explore new and different moves was terminated and it became stronger on a narrow path of moves. This also sadly led the network to believe that every match is either a 100% win or a 100% loss. This is quite visible for anyone using both ELF and LZ networks. ELF will always be very black and white and whats good or bad, while the LZ networks provide several moves which are playable. LZ could do the same and would probably see a boost in strength, but at the same time it probably wouldnt improve anymore and it wouldnt explore either. Since the LZ network is larger, it still has capacity to grow and improve by searching and exploring, and as such its not needed to try to force it to its full strength right now. Same thing with learning rates, one could probobly get a small increase in LZ strength by just dropping the learning rate, but that would kill any further increase in playing strength from exploration. |
|
| Author: | Vargo [ Thu May 09, 2019 10:09 am ] |
| Post subject: | Re: LZ's progression |
20 game match LZ017#225 v LZ017#215 no pondering, komi 7.5, -v 1600 twogtp 1.5.1 , no error, no duplicate game. #225 wins 17-3 Wow ! It confirms the 69% , and it's only a 10 networks difference...  The stats : |
|
| Author: | nbc44 [ Fri May 17, 2019 3:36 am ] |
| Post subject: | Re: LZ's progression |
Сity legend check (l0XXX 1600 visits vs elfv2 3200 visits): 2 x 1080ti Code: =================================================================================================================== From:May-07-19 07:12:33 to May-08-19 05:08:00. Spend 78927.58483s Game << 400 >> LZ224 1600 visits win(All) = 249 62.25% win(B) = 115 57.50% win(W) = 134 67.00% ELFV2 1600 visits win(All) = 151 37.75% win(B) = 66 33.00% win(W) = 85 42.50% =================================================================================================================== From:May-09-19 07:42:53 to May-10-19 06:46:29. Spend 83015.170982s Game << 400 >> LZ224 1600 visits win(All) = 212 53.00% win(B) = 102 51.00% win(W) = 110 55.00% ELFV2 2000 visits win(All) = 188 47.00% win(B) = 90 45.00% win(W) = 98 49.00% =================================================================================================================== From:May-08-19 05:08:00 to May-09-19 05:47:02. Spend 88741.358819s Game << 400 >> LZ224 1600 visits win(All) = 180 45.00% win(B) = 91 45.50% win(W) = 89 44.50% ELFV2 2400 visits win(All) = 220 55.00% win(B) = 111 55.50% win(W) = 109 54.50% =================================================================================================================== From:May-10-19 07:37:42 to May-11-19 07:16:09. Spend 85106.834626s Game << 400 >> LZ225 1600 visits win(All) = 207 51.75% win(B) = 94 47.00% win(W) = 113 56.50% ELFV2 2000 visits win(All) = 193 48.25% win(B) = 87 43.50% win(W) = 106 53.00% =================================================================================================================== From:May-11-19 07:16:09 to May-12-19 07:23:49. Spend 86860.492488s Game << 400 >> LZ225 1600 visits win(All) = 172 43.00% win(B) = 81 40.50% win(W) = 91 45.50% ELFV2 2200 visits win(All) = 228 57.00% win(B) = 109 54.50% win(W) = 119 59.50% =================================================================================================================== From:May-14-19 06:55:16 to May-15-19 06:12:54. Spend 83858.237752s Game << 400 >> LZ226 1600 visits win(All) = 212 53.00% win(B) = 100 50.00% win(W) = 112 56.00% ELFV2 2000 visits win(All) = 188 47.00% win(B) = 88 44.00% win(W) = 100 50.00% =================================================================================================================== From:May-15-19 06:12:54 to May-16-19 05:59:42. Spend 85607.877284s Game << 400 >> LZ226 1600 visits win(All) = 196 49.00% win(B) = 94 47.00% win(W) = 102 51.00% ELFV2 2200 visits win(All) = 204 51.00% win(B) = 98 49.00% win(W) = 106 53.00% P.S. If someone wants the games, I can upload them. |
|
| Author: | kityanhem [ Fri May 17, 2019 11:16 am ] |
| Post subject: | Re: LZ's progression |
Can I have the games, nbc44? For studying, of course!
|
|
| Page 20 of 21 | All times are UTC - 8 hours [ DST ] |
| Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |
|
