Engine Tournament
-
q30
- Lives with ko
- Posts: 145
- Joined: Sat Aug 13, 2016 8:23 am
- Rank: 30 kyu
- GD Posts: 0
- Has thanked: 1 time
- Been thanked: 1 time
Re: Engine Tournament
Alex, can You specify: when You run gogui-twogtp, the Hiratuka is connected as gtp engine?
Because when I had run Hiratuka19x19.exe from Hiratuka10_38B-No5 under wine64, there was Go board with abracadabra (because Japan hieroglyphs) interface, and no command-line help with '-?', '-h' or '--help'...
Thanks in advance.
Because when I had run Hiratuka19x19.exe from Hiratuka10_38B-No5 under wine64, there was Go board with abracadabra (because Japan hieroglyphs) interface, and no command-line help with '-?', '-h' or '--help'...
Thanks in advance.
-
as0770
- Lives with ko
- Posts: 180
- Joined: Sun Jun 26, 2016 8:07 am
- Rank: Beginner
- GD Posts: 0
- Has thanked: 15 times
- Been thanked: 23 times
Re: Engine Tournament
q30 wrote:Alex, can You specify: when You run gogui-twogtp, the Hiratuka is connected as gtp engine?
Because when I had run Hiratuka19x19.exe from Hiratuka10_38B-No5 under wine64, there was Go board with abracadabra (because Japan hieroglyphs) interface, and no command-line help with '-?', '-h' or '--help'...
Thanks in advance.
Yes, it is a gtp engine. Hira opens its own board, you have to leave the window opened, but v10.38B doesn't work here,
-
as0770
- Lives with ko
- Posts: 180
- Joined: Sun Jun 26, 2016 8:07 am
- Rank: Beginner
- GD Posts: 0
- Has thanked: 15 times
- Been thanked: 23 times
Re: Engine Tournament
Hi,
finally I got Oakfoam to run, and it did quite well. It ran on one CPU core and the GPU while Leela is the CPU only version. On my system the GPU version of Leela seems not to be faster. Because the tournament usually runs in 3 instances it might happen that 3 versions of Oakfoam share the GPU, but in that case the speed loss is around 10% so I can live with that.
Later I will try the GPU version of Leela to see if it performes better.
2nd new entry is deltaGo (http://home.q00.itscom.net/otsuki/delta.html). I have to admit it is quite a while on my harddisc and I didn't run the test because it doesn't resign or pass in lost games but crashes after many many useless moves. This is quite annoying because you have to save every game manually but beside that the engine is very interesting because it is supposed to be a copy of AlphaGo's policy network. It plays nearly instantly and a quite nice game of Go.
Leela vs. Oakfoam:
League A:
League B:
League C:
All engines are set up to play 1h/game with pondering off, one thread and Memory around 1GB.
Configuration:
Links:
Best,
Alex
finally I got Oakfoam to run, and it did quite well. It ran on one CPU core and the GPU while Leela is the CPU only version. On my system the GPU version of Leela seems not to be faster. Because the tournament usually runs in 3 instances it might happen that 3 versions of Oakfoam share the GPU, but in that case the speed loss is around 10% so I can live with that.
Later I will try the GPU version of Leela to see if it performes better.
2nd new entry is deltaGo (http://home.q00.itscom.net/otsuki/delta.html). I have to admit it is quite a while on my harddisc and I didn't run the test because it doesn't resign or pass in lost games but crashes after many many useless moves. This is quite annoying because you have to save every game manually but beside that the engine is very interesting because it is supposed to be a copy of AlphaGo's policy network. It plays nearly instantly and a quite nice game of Go.
Leela vs. Oakfoam:
Code: Select all
1. Leela 0.10.0 9/16
2. Oakfoam 0.2.1 NG-06 7/16League A:
Code: Select all
1. Leela 0.10.0 35/36
2. Oakfoam 0.2.1 NG-06 32/36
3. Hiratuka 10.37B 27/36
4. PachiUCT 11.99 23/36
5. Ray 8.0.1 19/36
6. MoGo 4.86 15/36
7. deltaGo 1.0.0 12/36
8. Fuego 1.1 11/36
9. Orego 7.08 5/36
10. GNU Go 3.8 1/36League B:
Code: Select all
1. GNU Go 3.8 28/32
2. Michi-C 26/32
3. Hara 0.9 19/32
4. Dariush 3.1.5.7 17/32
5. Indigo 2009 15/32
6. Matilda 1.24 15/32
7. Aya 6.34 11/32
8. Fudo Go 3.0 11/32
9. JrefBot 081016-2022 2/32League C:
Code: Select all
1. JrefBot 081016-2022 20/20
2. AmiGo 1.8 14/20
3. Crazy Patterns 0008-13 13/20
4. Stop 0.9-005 6/20
5. CopyBot 0.1 4/20
6. Brown 1.0 3/20All engines are set up to play 1h/game with pondering off, one thread and Memory around 1GB.
Configuration:
Links:
Best,
Alex
Last edited by as0770 on Mon Jun 12, 2017 10:01 pm, edited 6 times in total.
-
q30
- Lives with ko
- Posts: 145
- Joined: Sat Aug 13, 2016 8:23 am
- Rank: 30 kyu
- GD Posts: 0
- Has thanked: 1 time
- Been thanked: 1 time
Re: Engine Tournament
as0770 wrote:Yes, it is a gtp engine. Hira opens its own board, you have to leave the window opened, but v10.38B doesn't work here,
Hi, Alex!
Can You specify (because I don't know, how to run its command line or GUI English help), how to run Hiratuka10_37B (if it runs as GTP engine) with time set per move and (how it's in real games) - with pondering.
Thanks in advance.
-
as0770
- Lives with ko
- Posts: 180
- Joined: Sun Jun 26, 2016 8:07 am
- Rank: Beginner
- GD Posts: 0
- Has thanked: 15 times
- Been thanked: 23 times
Re: Engine Tournament
q30 wrote:as0770 wrote:Yes, it is a gtp engine. Hira opens its own board, you have to leave the window opened, but v10.38B doesn't work here,
Hi, Alex!
Can You specify (because I don't know, how to run its command line or GUI English help), how to run Hiratuka10_37B (if it runs as GTP engine) with time set per move and (how it's in real games) - with pondering.
Thanks in advance.
Hi,
I am not aware of any other command line options than -po. It seems that Hira doesn't support any time commands nor pondering. So the only way is to find out what -po option fits to your timecontrol on your hardware and to live without pondering.
-
as0770
- Lives with ko
- Posts: 180
- Joined: Sun Jun 26, 2016 8:07 am
- Rank: Beginner
- GD Posts: 0
- Has thanked: 15 times
- Been thanked: 23 times
Re: Engine Tournament
as0770 wrote:Leela vs. Oakfoam:Code: Select all
1. Leela 0.10.0 9/16
2. Oakfoam 0.2.1 NG-06 7/16
After the close result between Leela CPU and Oakfoam GPU I repeated this with the GPU version of Leela:
Code: Select all
1. Leela 0.10.0 GPU 14/16
2. Oakfoam 0.2.1 NG-06 2/16So from now on I'll use the GPU version of Leela because the opponents get stronger. That means also that I can only run one instance of twogtp because there is only one GPU. 2nd new entry in League A is Rayon which I got finally to work. Looks like a close race between Leela and Ray and I'll run more games.
Be aware that Rayon uses all available threads, running a ponder on match will penalize the opponent. I avoided this problem with the taskset command.
There are also new entries in League B, Michi C-2 replaced Michi-C, and in League D, Iomrascálaí and Marcos Go,
League A:
Code: Select all
1. Leela 0.10.0 14/16
2. Rayon 4.2.5 14/16
3. Oakfoam 0.2.1 NG-06 7/16
4. Hiratuka 10.37B (CPU) 4/16
5. Pachi 11.99 1/16League B:
Code: Select all
1. Pachi 11.99 23/24
2. MoGo 4.86 19/24
3. deltaGo 1.0.0 14/24
4. Fuego 1.1 13/24
5. Michi C-2 1.4.2 8/24
6. Orego 7.08 5/24
7. GNU Go 3.8 2/24League C:
Code: Select all
1. GNU Go 3.8 24/28
2. Hara 0.9 18/28
3. Dariush 3.1.5.7 16/28
4. Indigo 2009 15/28
5. Matilda 1.24 15/28
6. Aya 6.34 11/28
7. Fudo Go 3.0 11/28
8. JrefBot 081016-2022 2/28League D:
Code: Select all
1. JrefBot 081016-2022 25/28
2. Iomrascálaí 0.3.2 21/28
3. Crazy Patterns 0008-13 19/28
4. Marcos Go 1.0 18/28
5. AmiGo 1.8 16/28
6. Stop 0.9-005 6/28
7. CopyBot 0.1 4/28
8. Brown 1.0 3/28All engines are set up to play 1h/game with pondering off, one thread and memory usage around 1GB on a Acer Aspire VN 7 with 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
Configuration:
Links:
Best,
Alex
Last edited by as0770 on Sat Jun 17, 2017 8:10 am, edited 1 time in total.
-
q30
- Lives with ko
- Posts: 145
- Joined: Sat Aug 13, 2016 8:23 am
- Rank: 30 kyu
- GD Posts: 0
- Has thanked: 1 time
- Been thanked: 1 time
Re: Engine Tournament
Oakfoam 0.2.1-dev needs for compilation new OS versions and is a weak engine: it lost all 12 games to Ray, Pachi and MoGo (but it has many parameters: see dsmic-oakfoam-e0b81cfb75de/src/engine/Parameters.h - if someone will recommend the strengthening game set of them, for example, the code of nicego-cnn-06.gtp, I'll realize a new sparring).
In spent sparring Oakfoam used book.dat from Pachi (i.e. corrected from FueGo) and configuration file:
I have discovered, that Hiratuka10_37B has time limit 1 min / move. So to release this time setting it's possible to set a big value for -po parameter (for existing and near future PCs will be enough 7777777, for example, on my old PC in game beginning it's a 1000po/sec/core).
I'll sparring Hiratuka10_37B and present the results next time...
About Michi-c2...
Apparently from the carried-out sparrings, the results of synthetic and real play tests may differ. But I'll sparring so weak in synthetic tests Michi-c2 only if someone confirms, that with upgrading run parameters to real playing sets this engine strength increases very strong.
Alex, can You specify, Rayon in Your rating is something different, than Ray (that is mentioned in Your links), and what is the goal to run it with taskset and --thread parameter together?
Thanks in advance.
In spent sparring Oakfoam used book.dat from Pachi (i.e. corrected from FueGo) and configuration file:
Code: Select all
param book_use 1
param thread_count 2
param memory_usage_max 6000
param playouts_per_move 7777777
param time_move_max 120
param uct_keep_subtree 1
param pondering_enabled 1
param uct_virtual_loss 1
param tree resign_ratio_threshold 0.25
param uct_terminal_handling 1
param debug_on 1
param debug_only_best_cs_playout 1
param other auto_save_sgf 0
param dt_output_mm 1
I have discovered, that Hiratuka10_37B has time limit 1 min / move. So to release this time setting it's possible to set a big value for -po parameter (for existing and near future PCs will be enough 7777777, for example, on my old PC in game beginning it's a 1000po/sec/core).
I'll sparring Hiratuka10_37B and present the results next time...
About Michi-c2...
Apparently from the carried-out sparrings, the results of synthetic and real play tests may differ. But I'll sparring so weak in synthetic tests Michi-c2 only if someone confirms, that with upgrading run parameters to real playing sets this engine strength increases very strong.
Alex, can You specify, Rayon in Your rating is something different, than Ray (that is mentioned in Your links), and what is the goal to run it with taskset and --thread parameter together?
Thanks in advance.
-
as0770
- Lives with ko
- Posts: 180
- Joined: Sun Jun 26, 2016 8:07 am
- Rank: Beginner
- GD Posts: 0
- Has thanked: 15 times
- Been thanked: 23 times
Re: Engine Tournament
q30 wrote:Oakfoam 0.2.1-dev needs for compilation new OS versions and is a weak engine: it lost all 12 games to Ray, Pachi and MoGo (but it has many parameters: see dsmic-oakfoam-e0b81cfb75de/src/engine/Parameters.h - if someone will recommend the strengthening game set of them, for example, the code of nicego-cnn-06.gtp, I'll realize a new sparring).
nicego-cnn-06.gtp is available in the oakfoam download. But it won't make Oakfoam stronger as long as you can't compile it for GPU. I tried Oakfoam CPU with the same settings as the GPU version and it lost even against GNU Go.
q30 wrote:Alex, can You specify, Rayon in Your rating is something different, than Ray (that is mentioned in Your links), and what is the goal to run it with taskset and --thread parameter together?
Thanks in advance.
Ray is just the old name. I changed the name in my links, thanks for the hint.
First I tried to run it with the thread parameter and it used all cores, so I added taskset. I left the thread option because it don't hurt and because I am a very lazy person
-
as0770
- Lives with ko
- Posts: 180
- Joined: Sun Jun 26, 2016 8:07 am
- Rank: Beginner
- GD Posts: 0
- Has thanked: 15 times
- Been thanked: 23 times
Re: Engine Tournament
New entry in League A is DarkForest. Unfortunately it is not possible to run it with fair conditions in my tournament because it always uses system resources of CPU and GPU even without pondering. On the other hand it is designed for high end systems, when I set it up to use one thread another process additionally uses 2 threads for the GPU management. Forcing it to use one thread it can only do 750 rollouts / move to play a game in 1h.
League A:
League B:
League C:
League D:
All engines are set up to play 1h/game with pondering off, one thread and memory usage around 1GB on a Acer Aspire VN 7 with 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
Configuration:
Links:
Best,
Alex
League A:
Code: Select all
1. Leela 0.10.0 18/20
2. Rayon 4.2.5 18/20
3. Oakfoam 0.2.1 NG-06 11/20
4. DarkForest v2 MCTS 1.0 6/20
5. Hiratuka 10.37B (CPU) 5/20
6. Pachi 11.99 2/20League B:
Code: Select all
1. Pachi 11.99 23/24
2. MoGo 4.86 19/24
3. deltaGo 1.0.0 14/24
4. Fuego 1.1 13/24
5. Michi C-2 1.4.2 8/24
6. Orego 7.08 5/24
7. GNU Go 3.8 2/24League C:
Code: Select all
1. GNU Go 3.8 24/28
2. Hara 0.9 18/28
3. Dariush 3.1.5.7 16/28
4. Indigo 2009 15/28
5. Matilda 1.24 15/28
6. Aya 6.34 11/28
7. Fudo Go 3.0 11/28
8. JrefBot 081016-2022 2/28League D:
Code: Select all
1. JrefBot 081016-2022 25/28
2. Iomrascálaí 0.3.2 21/28
3. Crazy Patterns 0008-13 19/28
4. Marcos Go 1.0 18/28
5. AmiGo 1.8 16/28
6. Stop 0.9-005 6/28
7. CopyBot 0.1 4/28
8. Brown 1.0 3/28All engines are set up to play 1h/game with pondering off, one thread and memory usage around 1GB on a Acer Aspire VN 7 with 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
Configuration:
Links:
Best,
Alex
-
q30
- Lives with ko
- Posts: 145
- Joined: Sat Aug 13, 2016 8:23 am
- Rank: 30 kyu
- GD Posts: 0
- Has thanked: 1 time
- Been thanked: 1 time
Re: Engine Tournament
as0770 wrote:But it won't make Oakfoam stronger as long as you can't compile it for GPU. I tried Oakfoam CPU with the same settings as the GPU version and it lost even against GNU Go.
Ray is just the old name. I changed the name in my links, thanks for the hint.
First I tried to run it with the thread parameter and it used all cores, so I added taskset. I left the thread option because it don't hurt and because I am a very lazy person :-)
I have old PC with Nvidia 9500GT video card. And, I think, there is no a reason to try compiling for GPU support. But it's something strange, if only additional GPU using makes the engine so stronger. I think, in non synthetic tests it would not be so...
What did You mean under "old name"? From official github (that is updated continuously and has, for example, latest commit at this moment 3 days old) engine has name "Ray" still. And it uses number of cores, that is specified in thread parameter...
Thanks in advance.
-
q30
- Lives with ko
- Posts: 145
- Joined: Sat Aug 13, 2016 8:23 am
- Rank: 30 kyu
- GD Posts: 0
- Has thanked: 1 time
- Been thanked: 1 time
Re: Engine Tournament
I have sparred Hiratuka_V10_37B-No180-root (details).
Current rating of Go engines:
League A
1. Leela
League B
2. Hiratuka
3. Ray
4. Pachi
5. MoGo
Go engines non default compiling and start up parameters.
Leela start up parameters: -g -t 2 -b 0 (with gogui-twogtp.jar -time 1s+2/1).
Hiratuka start up parameter: -po 7777777
In future LeelaZero start up parameters: -g -t 2 -r 25 -w {NeroNet file} (with gogui-twogtp.jar -time 1s+2/1).
Current rating of Go engines:
League A
1. Leela
League B
2. Hiratuka
3. Ray
4. Pachi
5. MoGo
Go engines non default compiling and start up parameters.
Leela start up parameters: -g -t 2 -b 0 (with gogui-twogtp.jar -time 1s+2/1).
Hiratuka start up parameter: -po 7777777
In future LeelaZero start up parameters: -g -t 2 -r 25 -w {NeroNet file} (with gogui-twogtp.jar -time 1s+2/1).
Last edited by q30 on Sat Jun 30, 2018 2:03 am, edited 1 time in total.
-
q30
- Lives with ko
- Posts: 145
- Joined: Sat Aug 13, 2016 8:23 am
- Rank: 30 kyu
- GD Posts: 0
- Has thanked: 1 time
- Been thanked: 1 time
Re: Engine Tournament
DarkforestGo compiles, but doesn't run on my PC...
With "th":
Without "th":
I think, that it isn't a standalone program and requires something not included in its distribution. So DarkforestGo isn't ready for usual users yet...
With "th":
Code: Select all
If 'th' is not a typo you can use command-not-found to lookup the package that contains it, like this:
cnf thWithout "th":
Code: Select all
./cnnPlayerMCTSV2.lua: line 1: --: команда не найдена
./cnnPlayerMCTSV2.lua: line 2: syntax error near unexpected token `('
./cnnPlayerMCTSV2.lua: line 2: `-- Copyright (c) 2016-present, Facebook, Inc.'I think, that it isn't a standalone program and requires something not included in its distribution. So DarkforestGo isn't ready for usual users yet...
-
as0770
- Lives with ko
- Posts: 180
- Joined: Sun Jun 26, 2016 8:07 am
- Rank: Beginner
- GD Posts: 0
- Has thanked: 15 times
- Been thanked: 23 times
Re: Engine Tournament
q30 wrote:What did You mean under "old name"? From official github (that is updated continuously and has, for example, latest commit at this moment 3 days old) engine has name "Ray" still.
You can see which version I used by the links.
q30 wrote:Leela start up parameters: -g -t 2 -b 0 (with gogui-twogtp.jar -time 1s+2/1).
Hiratuka start up parameter: -po 7777777
You seriously test 1s/Move?
q30 wrote:I think, that it isn't a standalone program and requires something not included in its distribution. So DarkforestGo isn't ready for usual users yet...
Just follow the readme:
Code: Select all
Build
Dependencies:
Install torch7.
Install CUDA / CuDNN
Install a few packages
luarocks install class
luarocks install image
luarocks install tds
luarocks install cudnn
Last edited by as0770 on Fri Jul 07, 2017 11:09 pm, edited 1 time in total.
-
q30
- Lives with ko
- Posts: 145
- Joined: Sat Aug 13, 2016 8:23 am
- Rank: 30 kyu
- GD Posts: 0
- Has thanked: 1 time
- Been thanked: 1 time
Re: Engine Tournament
as0770 wrote:You can see which version I used by the links.
You seriously test 1s/Move?q30 wrote:I think, that it isn't a standalone program and requires something not included in its distribution. So DarkforestGo isn't ready for usual users yet...
Just follow the readme:Code: Select all
Build
Dependencies:
Install torch7.
Install CUDA / CuDNN
Install a few packages
luarocks install class
luarocks install image
luarocks install tds
luarocks install cudnn
Yes, I see. But "Ray" is newer, than "Ray-on"...
No, Leela has up to 2 min/move (see twogtp.jar parameters) and Hiratuka, how I had written, has 1 min/move...
For building (compilation) it wasn't necessary: it had compiled without problem. But it doesn't run (on standard distribution) ...