Engine Tournament

q30 · **#61**

Alex, can You specify: when You run gogui-twogtp, the Hiratuka is connected as gtp engine?
Because when I had run Hiratuka19x19.exe from Hiratuka10_38B-No5 under wine64, there was Go board with abracadabra (because Japan hieroglyphs) interface, and no command-line help with '-?', '-h' or '--help'...

Thanks in advance.

as0770 · **#62**

q30 wrote:

Alex, can You specify: when You run gogui-twogtp, the Hiratuka is connected as gtp engine?
Because when I had run Hiratuka19x19.exe from Hiratuka10_38B-No5 under wine64, there was Go board with abracadabra (because Japan hieroglyphs) interface, and no command-line help with '-?', '-h' or '--help'...

Thanks in advance.

Yes, it is a gtp engine. Hira opens its own board, you have to leave the window opened, but v10.38B doesn't work here,

as0770 · **#63**

Hi,

finally I got Oakfoam to run, and it did quite well. It ran on one CPU core and the GPU while Leela is the CPU only version. On my system the GPU version of Leela seems not to be faster. Because the tournament usually runs in 3 instances it might happen that 3 versions of Oakfoam share the GPU, but in that case the speed loss is around 10% so I can live with that.

Later I will try the GPU version of Leela to see if it performes better.

2nd new entry is deltaGo (http://home.q00.itscom.net/otsuki/delta.html). I have to admit it is quite a while on my harddisc and I didn't run the test because it doesn't resign or pass in lost games but crashes after many many useless moves. This is quite annoying because you have to save every game manually but beside that the engine is very interesting because it is supposed to be a copy of AlphaGo's policy network. It plays nearly instantly and a quite nice game of Go.

Leela vs. Oakfoam:

Code:

    1. Leela 0.10.0                  9/16
    2. Oakfoam 0.2.1 NG-06           7/16

League A:

Code:

    1. Leela 0.10.0                 35/36
    2. Oakfoam 0.2.1 NG-06          32/36
    3. Hiratuka 10.37B              27/36
    4. PachiUCT 11.99               23/36
    5. Ray 8.0.1                    19/36
    6. MoGo 4.86                    15/36
    7. deltaGo 1.0.0                12/36
    8. Fuego 1.1                    11/36
    9. Orego 7.08                    5/36
   10. GNU Go 3.8                    1/36

League B:

Code:

    1. GNU Go 3.8                   28/32
    2. Michi-C                      26/32
    3. Hara 0.9                     19/32
    4. Dariush 3.1.5.7              17/32
    5. Indigo 2009                  15/32
    6. Matilda 1.24                 15/32
    7. Aya 6.34                     11/32
    8. Fudo Go 3.0                  11/32
    9. JrefBot 081016-2022           2/32

League C:

Code:

    1. JrefBot 081016-2022          20/20
    2. AmiGo 1.8                    14/20
    3. Crazy Patterns 0008-13       13/20
    4. Stop 0.9-005                  6/20
    5. CopyBot 0.1                   4/20
    6. Brown 1.0                     3/20

All engines are set up to play 1h/game with pondering off, one thread and Memory around 1GB.

Configuration:

Links:

Best,
Alex

q30 · **#64**

as0770 wrote:

Yes, it is a gtp engine. Hira opens its own board, you have to leave the window opened, but v10.38B doesn't work here,

Hi, Alex!

Can You specify (because I don't know, how to run its command line or GUI English help), how to run Hiratuka10_37B (if it runs as GTP engine) with time set per move and (how it's in real games) - with pondering.

Thanks in advance.

as0770 · **#65**

q30 wrote:

as0770 wrote:

Yes, it is a gtp engine. Hira opens its own board, you have to leave the window opened, but v10.38B doesn't work here,

Hi, Alex!

Can You specify (because I don't know, how to run its command line or GUI English help), how to run Hiratuka10_37B (if it runs as GTP engine) with time set per move and (how it's in real games) - with pondering.

Thanks in advance.

Hi,

I am not aware of any other command line options than -po. It seems that Hira doesn't support any time commands nor pondering. So the only way is to find out what -po option fits to your timecontrol on your hardware and to live without pondering.

Hane · **#66**

Michi-c2 updated! new version: 1.4.2
https://github.com/db3108/michi-c2

as0770 · **#67**

as0770 wrote:

Leela vs. Oakfoam:

Code:

    1. Leela 0.10.0                  9/16
    2. Oakfoam 0.2.1 NG-06           7/16

After the close result between Leela CPU and Oakfoam GPU I repeated this with the GPU version of Leela:

Code:

    1. Leela 0.10.0 GPU              14/16
    2. Oakfoam 0.2.1 NG-06           2/16

So from now on I'll use the GPU version of Leela because the opponents get stronger. That means also that I can only run one instance of twogtp because there is only one GPU. 2nd new entry in League A is Rayon which I got finally to work. Looks like a close race between Leela and Ray and I'll run more games.

Be aware that Rayon uses all available threads, running a ponder on match will penalize the opponent. I avoided this problem with the taskset command.

There are also new entries in League B, Michi C-2 replaced Michi-C, and in League D, Iomrascálaí and Marcos Go,

League A:

Code:

    1. Leela 0.10.0                 14/16
    2. Rayon 4.2.5                  14/16
    3. Oakfoam 0.2.1 NG-06           7/16
    4. Hiratuka 10.37B (CPU)         4/16
    5. Pachi 11.99                   1/16

League B:

Code:

    1. Pachi 11.99                  23/24
    2. MoGo 4.86                    19/24
    3. deltaGo 1.0.0                14/24
    4. Fuego 1.1                    13/24
    5. Michi C-2 1.4.2               8/24
    6. Orego 7.08                    5/24
    7. GNU Go 3.8                    2/24

League C:

Code:

    1. GNU Go 3.8                   24/28
    2. Hara 0.9                     18/28
    3. Dariush 3.1.5.7              16/28
    4. Indigo 2009                  15/28
    5. Matilda 1.24                 15/28
    6. Aya 6.34                     11/28
    7. Fudo Go 3.0                  11/28
    8. JrefBot 081016-2022           2/28

League D:

Code:

    1. JrefBot 081016-2022          25/28
    2. Iomrascálaí 0.3.2            21/28
    3. Crazy Patterns 0008-13       19/28
    4. Marcos Go 1.0                18/28
    5. AmiGo 1.8                    16/28
    6. Stop 0.9-005                  6/28
    7. CopyBot 0.1                   4/28
    8. Brown 1.0                     3/28

All engines are set up to play 1h/game with pondering off, one thread and memory usage around 1GB on a Acer Aspire VN 7 with 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

Configuration:

Links:

Best,
Alex

q30 · **#68**

Oakfoam 0.2.1-dev needs for compilation new OS versions and is a weak engine: it lost all 12 games to Ray, Pachi and MoGo (but it has many parameters: see dsmic-oakfoam-e0b81cfb75de/src/engine/Parameters.h - if someone will recommend the strengthening game set of them, for example, the code of nicego-cnn-06.gtp, I'll realize a new sparring).
In spent sparring Oakfoam used book.dat from Pachi (i.e. corrected from FueGo) and configuration file:

Code:

param book_use 1
param thread_count 2
param memory_usage_max 6000
param playouts_per_move 7777777
param time_move_max 120
param uct_keep_subtree 1
param pondering_enabled 1
param uct_virtual_loss 1
param tree resign_ratio_threshold 0.25
param uct_terminal_handling 1
param debug_on 1
param debug_only_best_cs_playout 1
param other auto_save_sgf 0
param dt_output_mm 1

I have discovered, that Hiratuka10_37B has time limit 1 min / move. So to release this time setting it's possible to set a big value for -po parameter (for existing and near future PCs will be enough 7777777, for example, on my old PC in game beginning it's a 1000po/sec/core).
I'll sparring Hiratuka10_37B and present the results next time...

About Michi-c2...
Apparently from the carried-out sparrings, the results of synthetic and real play tests may differ. But I'll sparring so weak in synthetic tests Michi-c2 only if someone confirms, that with upgrading run parameters to real playing sets this engine strength increases very strong.

Alex, can You specify, Rayon in Your rating is something different, than Ray (that is mentioned in Your links), and what is the goal to run it with taskset and --thread parameter together?
Thanks in advance.

as0770 · **#69**

q30 wrote:

Oakfoam 0.2.1-dev needs for compilation new OS versions and is a weak engine: it lost all 12 games to Ray, Pachi and MoGo (but it has many parameters: see dsmic-oakfoam-e0b81cfb75de/src/engine/Parameters.h - if someone will recommend the strengthening game set of them, for example, the code of nicego-cnn-06.gtp, I'll realize a new sparring).

nicego-cnn-06.gtp is available in the oakfoam download. But it won't make Oakfoam stronger as long as you can't compile it for GPU. I tried Oakfoam CPU with the same settings as the GPU version and it lost even against GNU Go.

q30 wrote:

Alex, can You specify, Rayon in Your rating is something different, than Ray (that is mentioned in Your links), and what is the goal to run it with taskset and --thread parameter together?
Thanks in advance.

Ray is just the old name. I changed the name in my links, thanks for the hint.

First I tried to run it with the thread parameter and it used all cores, so I added taskset. I left the thread option because it don't hurt and because I am a very lazy person :-)

as0770 · **#70**

New entry in League A is DarkForest. Unfortunately it is not possible to run it with fair conditions in my tournament because it always uses system resources of CPU and GPU even without pondering. On the other hand it is designed for high end systems, when I set it up to use one thread another process additionally uses 2 threads for the GPU management. Forcing it to use one thread it can only do 750 rollouts / move to play a game in 1h.

League A:

Code:

    1. Leela 0.10.0                 18/20
    2. Rayon 4.2.5                  18/20
    3. Oakfoam 0.2.1 NG-06          11/20
    4. DarkForest v2 MCTS 1.0        6/20
    5. Hiratuka 10.37B (CPU)         5/20
    6. Pachi 11.99                   2/20

League B:

Code:

    1. Pachi 11.99                  23/24
    2. MoGo 4.86                    19/24
    3. deltaGo 1.0.0                14/24
    4. Fuego 1.1                    13/24
    5. Michi C-2 1.4.2               8/24
    6. Orego 7.08                    5/24
    7. GNU Go 3.8                    2/24

League C:

Code:

    1. GNU Go 3.8                   24/28
    2. Hara 0.9                     18/28
    3. Dariush 3.1.5.7              16/28
    4. Indigo 2009                  15/28
    5. Matilda 1.24                 15/28
    6. Aya 6.34                     11/28
    7. Fudo Go 3.0                  11/28
    8. JrefBot 081016-2022           2/28

League D:

Code:

    1. JrefBot 081016-2022          25/28
    2. Iomrascálaí 0.3.2            21/28
    3. Crazy Patterns 0008-13       19/28
    4. Marcos Go 1.0                18/28
    5. AmiGo 1.8                    16/28
    6. Stop 0.9-005                  6/28
    7. CopyBot 0.1                   4/28
    8. Brown 1.0                     3/28

All engines are set up to play 1h/game with pondering off, one thread and memory usage around 1GB on a Acer Aspire VN 7 with 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

Configuration:

Links:

Best,
Alex

q30 · **#71**

as0770 wrote:

But it won't make Oakfoam stronger as long as you can't compile it for GPU. I tried Oakfoam CPU with the same settings as the GPU version and it lost even against GNU Go.

Ray is just the old name. I changed the name in my links, thanks for the hint.
First I tried to run it with the thread parameter and it used all cores, so I added taskset. I left the thread option because it don't hurt and because I am a very lazy person :-)

I have old PC with Nvidia 9500GT video card. And, I think, there is no a reason to try compiling for GPU support. But it's something strange, if only additional GPU using makes the engine so stronger. I think, in non synthetic tests it would not be so...

What did You mean under "old name"? From official github (that is updated continuously and has, for example, latest commit at this moment 3 days old) engine has name "Ray" still. And it uses number of cores, that is specified in thread parameter...

Thanks in advance.

q30 · **#72**

I have sparred Hiratuka_V10_37B-No180-root (details).
Current rating of Go engines:

League A
1. Leela

League B
2. Hiratuka
3. Ray
4. Pachi
5. MoGo

Go engines non default compiling and start up parameters.
Leela start up parameters: -g -t 2 -b 0 (with gogui-twogtp.jar -time 1s+2/1).
Hiratuka start up parameter: -po 7777777

In future LeelaZero start up parameters: -g -t 2 -r 25 -w {NeroNet file} (with gogui-twogtp.jar -time 1s+2/1).

q30 · **#73**

DarkforestGo compiles, but doesn't run on my PC...
With "th":

Code:

If 'th' is not a typo you can use command-not-found to lookup the package that contains it, like this:
    cnf th

Without "th":

Code:

./cnnPlayerMCTSV2.lua: line 1: --: команда не найдена
./cnnPlayerMCTSV2.lua: line 2: syntax error near unexpected token `('
./cnnPlayerMCTSV2.lua: line 2: `-- Copyright (c) 2016-present, Facebook, Inc.'

I think, that it isn't a standalone program and requires something not included in its distribution. So DarkforestGo isn't ready for usual users yet...

as0770 · **#74**

q30 wrote:

What did You mean under "old name"? From official github (that is updated continuously and has, for example, latest commit at this moment 3 days old) engine has name "Ray" still.

You can see which version I used by the links.

q30 wrote:

Leela start up parameters: -g -t 2 -b 0 (with gogui-twogtp.jar -time 1s+2/1).
Hiratuka start up parameter: -po 7777777

You seriously test 1s/Move?

q30 wrote:

I think, that it isn't a standalone program and requires something not included in its distribution. So DarkforestGo isn't ready for usual users yet...

Just follow the readme:

Code:

Build

Dependencies:

    Install torch7.
    Install CUDA / CuDNN
    Install a few packages

luarocks install class
luarocks install image
luarocks install tds
luarocks install cudnn

q30 · **#75**

as0770 wrote:

You can see which version I used by the links.

You seriously test 1s/Move?

q30 wrote:

I think, that it isn't a standalone program and requires something not included in its distribution. So DarkforestGo isn't ready for usual users yet...

Just follow the readme:

Code:

Build

Dependencies:

    Install torch7.
    Install CUDA / CuDNN
    Install a few packages

luarocks install class
luarocks install image
luarocks install tds
luarocks install cudnn

Yes, I see. But "Ray" is newer, than "Ray-on"...

No, Leela has up to 2 min/move (see twogtp.jar parameters) and Hiratuka, how I had written, has 1 min/move...

For building (compilation) it wasn't necessary: it had compiled without problem. But it doesn't run (on standard distribution) ...

as0770 · **#76**

q30 wrote:

No, Leela has up to 2 min/move (see twogtp.jar parameters) and Hiratuka, how I had written, has 1 min/move...

That's why I'm asking: "gogui-twogtp.jar -time 1s+2/1"

q30 · **#77**

Leela makes a few moves in first second (how and Hiratuka nevertheless of gogui parameters - I think, because embedded opening book) and then thinks up to 2 min/move (Hiratuka thinks with enough -po parameter value 1 min/move because of embedded limits). You can try run with mentioned parameters and see all yourself...

q30 · **#78**

Ray became a bit stronger (not against Hiratuka: details), but the rate is the same.
Distribution is renewed.

gamesorry · **#79**

The OGS Alan Turing Main Title Tournament 2017 (https://online-go.com/tournament/27492) will begin in a week on July 22, 2017. 150 players, including strong bots like RoyalLeela and Valkyria, have signed up so far.

It would be great if some of the engines here could join the tournament and play with other engines/human players!

as0770 · **#80**

Update League A with Rayon 4.6.0, now 2h/game and 4 threads.

League A:

Code:

    1. Leela 0.10.0                 18/20
    2. Rayon 4.6.0                  15/20
    3. Oakfoam 0.2.1 NG-06          14/20
    4. DarkForest v2 MCTS 1.0        7/20
    5. Hiratuka 10.37B (CPU)         6/20
    6. Pachi 11.99                   0/20

Configuration:

League B:

Code:

    1. Pachi 11.99                  23/24
    2. MoGo 4.86                    19/24
    3. deltaGo 1.0.0                14/24
    4. Fuego 1.1                    13/24
    5. Michi C-2 1.4.2               8/24
    6. Orego 7.08                    5/24
    7. GNU Go 3.8                    2/24

Configuration:

League C:

Code:

    1. GNU Go 3.8                   24/28
    2. Hara 0.9                     18/28
    3. Dariush 3.1.5.7              16/28
    4. Indigo 2009                  15/28
    5. Matilda 1.24                 15/28
    6. Aya 6.34                     11/28
    7. Fudo Go 3.0                  11/28
    8. JrefBot 081016-2022           2/28

Configuration:

League D:

Code:

    1. JrefBot 081016-2022          25/28
    2. Iomrascálaí 0.3.2            21/28
    3. Crazy Patterns 0008-13       19/28
    4. Marcos Go 1.0                18/28
    5. AmiGo 1.8                    16/28
    6. Stop 0.9-005                  6/28
    7. CopyBot 0.1                   4/28
    8. Brown 1.0                     3/28

Configuration:

Links:

Best,
Alex

Engine Tournament

Who is online