It is currently Thu Mar 28, 2024 4:28 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 51 posts ]  Go to page 1, 2, 3  Next
Author Message
Offline
 Post subject: zen7 30 second : leela-zero 30 second, Zen7 won consistently
Post #1 Posted: Tue Apr 10, 2018 6:18 pm 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
game setting
zen7: uses gtp4zen-0.3.2 for sabaki
arguments: -z 7 -t 4 -T 30 -s 200000
Leeala-zero 13:
arguments: -g -noponder -t 4 -w leelaz-model-swa-4-32000.txt -p 500000
initial commands: time_settings 0 31 1;
(note: the weight text file is dated 4/9/2018, and the 31 is for constant 30 seconds thinking time)
computer: Acer One 10 ; cpu: Intel® Atom™ x5-Z8350 (32 bit, NOT 64 bit)
1.4 GHz; Quad-core; OS: windows 10


Last edited by kyotosato on Tue Apr 10, 2018 8:19 pm, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #2 Posted: Tue Apr 10, 2018 8:11 pm 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
Note1: gtp4zen-0.3.2 is a program designed for Zen7 or Zen6 to be used as a bot in sabaki. I got it by searching the web thoroghly. It is a user-developed program (by someone), which or whom has nothing to do with or no relation to original Zenith author or co.
Note2: weight file matters greatly. I used the file downloaded from http://zero.sjeng.org/ which updated daily.
Note3: The result may be inconclusive due to the weight file updated daily and CPU and GPU grade , the two factors

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #3 Posted: Tue Apr 10, 2018 9:36 pm 
Lives with ko
User avatar

Posts: 284
Liked others: 94
Was liked: 153
Rank: OGS 7 kyu
This is interesting, do you have any idea how much play-outs are performed on that hardware during those 30s thinking time?

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #4 Posted: Tue Apr 10, 2018 9:57 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Very interesting ! I have Zen 6, so, I would be happy to test it against AQ or LZ, but where can I find this gtp4zen ? (I don't speak japanese or chinese)

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #5 Posted: Tue Apr 10, 2018 10:07 pm 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
my note3 said it is inconclusive. at least not in statistical sense of "consistancy," since i posted the game setting, I would like people to be able continuously testing leela-zero 13. I will do the same because, i can set the game over night, and let it run by itself. when time goes by i will be able to say something about these two bots more definitively. right now just a feeling by looking game's winrate hisory of the only two games i experienced. the leelazero once dropped down steadily, then could never recover later on.
by the way, about the arguments of leelazero I might need some folks who have experienced of LZ to give me some advice so i can make adjustment.


Last edited by kyotosato on Tue Apr 10, 2018 11:27 pm, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #6 Posted: Tue Apr 10, 2018 11:17 pm 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
as for gtp4zen-0.3.2.rar:
since it is a shared and password protected zipped file in the baidu cloud, which is a cloud company of china.
Without violating copyright law of any country, i only could give you a website address,
https://tieba.baidu.com/p/5447958378?pi ... 5553152055
in it, third post, dated 2017-11-26 15:35, has a pair of things:
1bo----z,---4
the first is the baidu shared file ID, second is the password when asked.
that is all I could post, other than the above info you might have to find someone knows chinese and how to operate and access baidu cloud disk from US or other area.
I read chinese and japanese by using Google translate to get approximate idea. you may try that way first.
The author is a chinese and he/she keeps on updating the program as far as i can tell.


Last edited by kyotosato on Wed Apr 11, 2018 7:43 am, edited 3 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #7 Posted: Tue Apr 10, 2018 11:30 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Thanks, I'll look into it.
There was THAT,
which works, but has issues with dual GPUs, it tends to crash the system.

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #8 Posted: Tue Apr 10, 2018 11:47 pm 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
it is not the same i believe, because the author in my post is the original author, and the file has been updated to a newer version (it does have 2 prior version 0.2.3 and 0.3.1, the one i used is 0.3.2. i tell by the date, 6/2017 and 11/2017. on the other hand, may be you are right. I cannot talk about crash 'cause I do not have dual GPUs.

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #9 Posted: Wed Apr 11, 2018 1:56 am 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
out of curiosity, i myself did tests to see how strong Zen7 commercial; i registered in wild fox site, and temgen site. Zen7 reaches 9dan in temgen and 8dan in fox with auto/random matching. Both tests took me quite a time, from 3dan default to 8dan and 9dan. if i have time i will try to play with ZenDeep bot in Temgen (only play with 9dans). For wild fox, i have to go further to reach 9dan (which is easy for Zen7) to play with Jue-Yi(name? the china's strongest AI go bot.) and the computer i used is a pentium 4, with Windows XP.
BTW, to use Zen7 (or Zen6) other than dan setting or time setting through out a game, third way is to employ Zen7's "Analysis" feature along with the other two ways to play with an opponent, manually.
The analysis feature is not only good for after-game analysis, but also good for playing a game in realtime. I will explain how to do it later in case you do not yet know how.

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #10 Posted: Wed Apr 11, 2018 3:11 pm 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
To play with an opponent realtime by Zen7 "analysis" feature:
1. setup by choosing view->pane->analysis (1st pic)
2. also choose view->hotspots (1st pic)
3. when on-line play begins (you click for both players, yourself and your opponent), click "analysis" or edit->analysis (2nd pic)
4. play by clicking the suggested the spot (blinking one among hotspots), and afterward it will show your opponent's hotspots (3rd pic)
note1: sometimes you have to wait longer before you choose the suggested spot, if there are a few competitive spots judging from winrates. wait longer the suggested spot may change.
note2: by this way, zen7 is about 8-9dan in wildfox and temgen, and you could save play time for some trivial moves.
Attachment:
screenshot_zen701.JPG
screenshot_zen701.JPG [ 121 KiB | Viewed 15561 times ]

Attachment:
screenshot_zen702.JPG
screenshot_zen702.JPG [ 125.57 KiB | Viewed 15561 times ]

Attachment:
screenshot_zen703.JPG
screenshot_zen703.JPG [ 124.97 KiB | Viewed 15561 times ]

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #11 Posted: Wed Apr 11, 2018 4:19 pm 
Beginner

Posts: 15
Liked others: 0
Was liked: 5
Rank: EGD 1k
kyotosato wrote:
arguments: -g -noponder -t 4 -w leelaz-model-swa-4-32000.txt -p 500000
initial commands: time_settings 0 31 1;
(note: the weight text file is dated 4/9/2018, and the 31 is for constant 30 seconds thinking time)
computer: Acer One 10 ; cpu: Intel® Atom™ x5-Z8350 (32 bit, NOT 64 bit)
1.4 GHz; Quad-core; OS: windows 10


Is this an unofficial network? How many blocks and filters does it have, how strong is it, where is it from?

But the result is not unexpected. Zenith Go 7 (which uses Zen 15, not 7) is very strong even it uses the CPU only, whereas LZ needs a GPU to play strong. With a 15b network I have 850n/s on my GTX 1080, but 75n/s when using the CPU with the 64bit version. The 32bit version is even worse, only 25n/s. :shock:

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #12 Posted: Wed Apr 11, 2018 8:27 pm 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
benchmark evaluation is not my concern, as a go player, and an AI practitioner. i concern about the availability, accessibility, and controlled envi testing of various computer go software. before discussing CPU prone, and/or GPU prone, we do need to know the GPU factor itself weighted how heavy to the baduk/go/weiqi game as a whole. Some "guesses" 1 or 2 stones differences, i cannot judge it without research on it, or statistic data, or benchmark evaluation. it can only be sure that GPU matters. matter how much?
as to temgen deepzen bots, i wonder if it is offered by zenith company. if it is , then that is another story. those bots maybe become test-run for next version Zen8 (or whatever name) commercial.
BTW, any optimization method has overshooting problem. from my own observation, Zen7 may be less overshooting prone than leela-zero, how to tell? from the trend of winrates history of a game. just my own guess.


Last edited by kyotosato on Thu Apr 12, 2018 8:13 am, edited 2 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #13 Posted: Wed Apr 11, 2018 9:02 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
I've finally managed to download gtp4zen, but it doesn't work with zen6...
The initialization phase seems ok, at the end it says zen.dll(zen6) initialize success, and then ... nothing.
Anyway, thanks all the same.

I think there's something missing in gtp4zen's commands : you can't disable pondering. It doesn't seem fair to the other program, as a lot of the CPU could be used by zen's pondering.

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #14 Posted: Wed Apr 11, 2018 10:48 pm 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
I use sabaki; Zen6 uses argument -z 6 differs from zen7's -z 7. I have zen6, and i will try it out, see if it works for zen6. as for -noponder, i will try it out. as for the game shown above, it didn't ponderring by my checking CPU meter from time to time.
might i ask what is your gtp4zen version?

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #15 Posted: Wed Apr 11, 2018 11:13 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
I've tried with the 3 available versions (0.2.3, 0.3.1, 0.3.2)
Sabaki 0.33.4 with parameters : -z 6 -t 4 -T 10 -s 10000

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #16 Posted: Thu Apr 12, 2018 8:01 am 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
I haven't checked it out yet, but i notice that in the web, some users had problem about time setting not working. the author said that the file gtp4zen.lua needs to be taken out of the directory where the gtp4zen.dll is located (or deleted if you don't use it), otherwise whatever you set in arguments or initial commands will be overwritten by the default (as long as the .lua file existed in the directory). my guess -z 6 maybe not work (default is -z 7, zen 7.) maybe i am wrong.
I use sabaki 0.33.2, and did not update it every time asked. it's my habit not to update any software until it is proven to be stabilized (time will tell.) hope this is not the reason.


Last edited by kyotosato on Thu Apr 12, 2018 8:30 am, edited 2 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #17 Posted: Thu Apr 12, 2018 8:25 am 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
pnprog:
Go GridMaster is just what I need for my android devices. thanks to your creation, a decent and nice product. i just wonder why you didn't continue to go further to AI 19x19?

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #18 Posted: Thu Apr 12, 2018 9:44 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Quote:
the author said that the file gtp4zen.lua needs to be taken out of the directory where the gtp4zen.dll is located (or deleted if you don't use it)
Bingo !
That was it, you just have to delete this file.
I've run two games between gtp4zen (zenV6) and AQ2.1.1 (H2, AQ takes white)
One game on a quadcore i7 and a gtx965, and one game on a 12-core i9 and 2x1080Ti
AQ won both at CGOS time settings, AQ took more time per move, so it's not very scientific, but still, it seems to be around 2-3 stones stronger than zen6. Thank you for this software, it's much easier to run matches with gtp than by hand !

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #19 Posted: Thu Apr 12, 2018 4:53 pm 
Dies in gote

Posts: 54
Liked others: 0
Was liked: 0
Rank: KGS 3 dan
would you mind post CGOS time settings in detail? so test may run in sync results will be more meaningful for getting some feeling of the strength for leelaz, AQ, zen6 or zen7.
Zen 6 or 7 if run by setting seconds than by dans will be more accurate i guess. it is not like 30s=5dan as far as i experienced. few months ago i used zen6 using 120 seconds (max strength) to play realtime in yahoo japan with a nihonkiin certified pro 2dan (so he/she said), zen6 won with 8.5 points, it took more than 3 hours. that was zen6 era, now i use zen7 and employ the method i mentioned above to play online. all games were saved, total around ~500 games.

Top
 Profile  
 
Offline
 Post subject: Re: zen7 30 second : leela-zero 30 second, Zen7 won consiste
Post #20 Posted: Thu Apr 12, 2018 6:15 pm 
Dies in gote

Posts: 63
Liked others: 0
Was liked: 3
I have critizied Leela Zero a lot in the past but your test is 100% invalid and wrongly

First, you used only the 32 bit version which means it canot even use more than 4GB RAM.

Typically on my tests AQ uses 9GB ram and LZ uses 30GB ram, so that right there is basically a show stopper for your test.

Then you use a weak CPU with no GPU... which come on, is not a fair test... everybody has at least a gtx 1080ti these days. Any test or benchmark that doesn't include a >= gtx 1080 isn't a real world test imho

Run it again on 4X titan and lets see whether Zen7 can beat Leela Zero.


btw your system had no gpu, what is the point of seting playouts to half a million yet giving it only 30 seconds thinking time? must be a joke right?! it wouldn't even hit the limit even if you gave it 30 hours thinking time per move

as an "AI practitioner" you should know this is a rigged test in Zen7's favor. I'm willing to bet if they put AlphaGoZero on a Pentium I against Zen7, Zen7 would also win. proves nothing tho

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 51 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group