Life In 19x19

Life in 19x19. Go, Weiqi, Baduk... Thats the life.
https://lifein19x19.com/

Engine Tournament

https://lifein19x19.com/viewtopic.php?t=13322

Page 13 of 27

Re: Engine Tournament

Posted: Sat Mar 31, 2018 1:34 am
by q30
lightvector wrote:Yep. I had exactly this issue in mind when I posted this topic:
forum/viewtopic.php?f=45&t=15314

As should be obvious from the discussion, there might indeed be a way, or indeed several ways, but it's not a trivial issue.
What is the lack of next trivial way: to train such engines with komi = 7.5, when playing black, and with komi = 6.5, when playing white?

Re: Engine Tournament

Posted: Sun Apr 01, 2018 10:16 pm
by as0770
New entries: V-Run, which is little more than a random player, in League F and OnePunchGo in League D.

Leela vs. AQ

Code: Select all

    1. AQ 2.1.1                         5/8
    2. Leela Zero 0.13 CPU 85c6f2ad     3/8
League A:

Code: Select all

    1. Leela Zero 0.13 CPU 85c6f2ad    18/20
    2. Leela 0.11.0                    15/20
    3. Oakfoam 0.2.1 NG-06             11/20
    4. Rayon 4.6.0                     11/20
    5. Hiratuka 10.37B (CPU)            4/20
    6. DreamGo 0.5.0                    1/20
League B:

Code: Select all

    1. DreamGo 0.5.0                   15/20
    2. DarkForrest MCTS 1.0            12/20
    3. Pachi 11.99                     12/20
    4. DarkGo 1.0                      10/20
    5. Ray 9.0.1                        9/20
    6. Mogo 4.86                        2/20
League C:

Code: Select all

    1. MoGo 4.86                       18/20
    2. deltaGo 1.0.0                   14/20
    3. Fuego 1.1                       13/20
    4. Michi C-2 1.4.2                  8/20
    5. Orego 7.08                       5/20
    6. GNU Go 3.8                       2/20
League D:

Code: Select all

    1. GNU Go 3.8                      29/32
    2. Hara 0.9                        22/32
    3. Matilda 1.25                    19/32
    4. Indigo 2009                     19/32
    5. Dariush 3.1.5.7                 17/32
    6. Aya 6.34                        14/32
    7. OnePunchGo 1.0                  12/32
    8. Fudo Go 3.0                      9/32
    9. JrefBot 081016-2022              2/32
League E:

Code: Select all

    1. JrefBot 081016-2022             16/20             
    2. Iomrascálaí 0.3.2               12/20
    3. SimpleGo 0.4.3                  11/20
    4. Crazy Patterns 0008-13           7/20
    5. Marcos Go 1.0                    7/20
    6. AmiGo 1.8                        7/20
League F:

Code: Select all

    1. AmiGo 1.8                       23/24
    2. Beancounter 0.1                 19/24
    3. Stop 0.9-005                    14/24
    4. GoTraxx 1.4.2                   11/24
    5. CopyBot 0.1                      7/24
    6. V-Run 1                          7/24
    7. Brown 1.0                        3/24
Configuration:
Leela Zero vs AQ: 2h/game, pondering off, 4 threads, 4GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 120 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League A: 1h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League B-F: 1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

Amigo: amigogtp
AQ: AQ

Code: Select all

aq_config.txt:
-main　time[sec] =3600
-time controll =true
-japanese rule =true
Aya: Aya.exe –mode gtp –level max
Beancounter: beancounter
Brown: brown.exe
Copybot: python /path/to/__main__.py
CrazyPatterns: CrazyPatterns.exe
Dariush: DarGTP.exe –level 10
DarkForest: taskset -c 0 bash cnn_evaluator.sh 1 /data/local/go and taskset -c 0 th cnnPlayerMCTSV2.lua --num_gpu 1 --num_tree_thread 1 --rollout 750 --win_rate_thres 0.1
DarkGo: darknet go engine cfg/go.test.cfg go.weights
deltaGo: deltaGo.exe
Dream Go: export NUM_ITER=1375 and dream_go
Fudo Go: taskset -c 0 fudo –boardsize=19 –komi=6.5
Fuego: fuego.exe –config fuego.cfg

Code: Select all

fuego.cfg:
uct_param_search number_threads 1
uct_param_search lock_free 0
uct_max_memory 1024000000
uct_param_player reuse_subtree 1
uct_param_player ponder 0
uct_param_player early_pass 1
GnuGo: gnugo --mode gtp --level 10 --resign-allowed
GoTraxx: GoTraxx.exe
Hara: hara
Hiratuka: Hiratuka-19×19.exe -po 75000
IndiGo: Indigo.exe -gtp
Iomrascálaí: taskset -c 0,1 iomrascalai
JrefBot: java -jar jrefgo.jar 10000
Leela: leela_gtp_opencl –gtp –threads 4 –noponder
Leela Zero: leelaz --gtp --threads 4 --w /path/to/Leelaz_best-network_yyyy_mm_dd --noponder
Matilda: matilda

Code: Select all

matilda.h:
#define BOARD_SIZ 19
#define DEFAULT_UCT_MEMORY 1000
#define DEFAULT_NUM_THREADS 1
Marcos Go: marcos_go --patterns /path/to/patterns.txt --cycles_mcts 10000 --threads_mcts 1
Michi C-2: michi gtp

Code: Select all

ui.c:
init_large_patterns("patterns2.prob", "patterns2.spat"); // Michis pattern files renamed because they have the same name as Pachis files.
MoGo: mogo
Oakfoam: oakfoam -c nicego-cnn-06.gtp

Code: Select all

nicego-cnn-06.gtp:
param playouts_per_move_max 40000
param thread_count 4
OnePunchGo: taskset -c 0 OPG

Code: Select all

TimeInfo.h (Modified to play 15sec/move because OPG requires time_left command which is not supported by twogtp and Sabaki)
timeAvailable = 15;
Orego: java -jar /path/to/orego-7.08.jar threads=1 grace
Pachi: pachidcnn -f pachibook.dat threads=1,max_tree_size=1024,pondering=0
Ray: ray --time 3600 --thread 1 --no-debug
Rayon: rayon --thread 4 --no-debug
Simple Go: python /path/to/play_gtp.py --node_limit=100
Stop: /usr/bin/java -ea -jar /path/to/stop-09-005.jar --mode gtp
V-Run: python3 /path/to/V-Run
Links:
Amigo: https://sourceforge.net/projects/amigogtp/
AQ: https://github.com/ymgaq/AQ
Aya: http://www.yss-aya.com/
Brown: http://ricoh51.free.fr/go/engineeng.htm
Beancounter: Private
Copybot: https://github.com/sirtango/ICopyMoves
CrazyPatterns: https://www.remi-coulom.fr/Amsterdam2007/
Dariush: http://ricoh51.free.fr/go/engineeng.htm
DarkForest: https://github.com/facebookresearch/darkforestGo
DarkGo: https://pjreddie.com/darknet/darkgo-go-in-darknet/
deltaGo: http://home.q00.itscom.net/otsuki/delta.html
Dream Go: https://github.com/Chicoryn/dream-go
Fudo Go: http://www.geocities.jp/hideki_katoh/
Fuego: http://fuego.sourceforge.net/
GnuGo: https://www.gnu.org/software/gnugo/devel.html
GoTraxx http://gotraxx.codeplex.com/
Hara: https://github.com/antoniogarro/Hara
Hiratuka: Non GPU version (10.37B): http://www.vector.co.jp/download/file/w ... 73259.html / GPU version (10.38B): http://www.vector.co.jp/download/file/w ... 88349.html
IndiGo: http://www.math-info.univ-paris5.fr/~bouzy/INDIGO.html
Iomrascálaí: https://github.com/ujh/iomrascalai
JrefBot: http://ricoh51.free.fr/go/engineeng.htm
Leela: https://sjeng.org/leela.html
Leela Zero: [urlhttp://zero.sjeng.org/[/url]
Marcos Go: https://github.com/MarcosPividori/Go-player
Matilda: https://github.com/gonmf/matilda
Michi C-2 https://github.com/db3108/michi-c2
MoGo: forum/viewtopic.php?p=211091#p211091
Oakfoam: https://bitbucket.org/dsmic/oakfoam
OnePunchGo https://github.com/AlexKent3141/OnePunchGo
Orego: https://sites.google.com/a/lclark.edu/d ... arch/orego
Pachi: http://pachi.or.cz/
Rayon: https://github.com/zakki/Ray
Ray: https://github.com/kobanium/Ray
Simple Go: https://sourceforge.net/projects/londerings/
Stop: https://www.vanheusden.com/stop/
V-Run: https://github.com/Fioelkais/MCTS
Best,
Alex

Re: Engine Tournament

Posted: Sat Apr 14, 2018 9:31 am
by q30
I had compiled last Pachi version from 28.02.18. It became a bit stronger details.
So the rating of generally available Go engines (i.e. without GPU support) is next:

League A
1. Leela

League B
2. Hiratuka
3. Pachi
4. Ray
5. MoGo

Re: Engine Tournament

Posted: Tue Apr 17, 2018 5:22 am
by pnprog
I noticed this bot that was mentioned on Leela Zero github page: https://github.com/johnhuang-cn/FancyBing

You might give it a try :)

Re: Engine Tournament

Posted: Tue Apr 17, 2018 7:15 am
by as0770
pnprog wrote:I noticed this bot that was mentioned on Leela Zero github page: https://github.com/johnhuang-cn/FancyBing

You might give it a try :)
Thanks! I'll try it after finishing tests of Pachi and Zen.

Re: Engine Tournament

Posted: Thu Apr 19, 2018 9:30 am
by as0770
New entry: Zen 7 via gtp4zen. Update: Pachi 12 and Leela Zero with the best 128x10 Network. In the future I will test the best Network of each size. Also there are some changes in the leagues, because of new entries I reduced the size.

League A:

Code: Select all

    1. Zen 7                           16/24
    2. AQ 2.1.1                        13/24
    3. Leela 0.11                      10/14
    4. Leela Zero 0.13 128x10 CPU       9/24
League B:

Code: Select all

    1. Leela Zero 0.13 128x10 CPU      18/20
    2. Leela 0.11.0                    15/20
    3. Oakfoam 0.2.1 NG-06             11/20
    4. Rayon 4.6.0                     11/20
    5. Hiratuka 10.37B (CPU)            4/20
    6. DreamGo 0.5.0                    1/20
League C:

Code: Select all

    1. DreamGo 0.5.0                   15/20
    2. Pachi 12                        14/20
    3. DarkForrest MCTS 1.0            12/20
    4. Ray 9.0.1                        9/20
    5. DarkGo 1.0                       8/20
    6. Mogo 4.86                        2/20
League D:

Code: Select all

    1. MoGo 4.86                       18/20
    2. deltaGo 1.0.0                   14/20
    3. Fuego 1.1                       13/20
    4. Michi C-2 1.4.2                  8/20
    5. Orego 7.08                       5/20
    6. GNU Go 3.8                       2/20
League E:

Code: Select all

    1. GNU Go 3.8                      18/20
    2. Hara 0.9                        11/20
    3. Dariush 3.1.5.7                  9/20
    4. Indigo 2009                      9/20
    5. Matilda 1.25                     9/20
    6. OnePunchGo 1.0                   4/20
League F:

Code: Select all

    1. OnePunchGo 1.0                  17/20
    2. Aya 6.34                        16/20  
    3. JrefBot 081016-2022             12/20  
    4. Fudo Go 3.0                     10/20
    5. Iomrascálaí 0.3.2                4/20
    6. SimpleGo 0.4.3                   1/20
League G:

Code: Select all

    1. SimpleGo 0.4.3                  17/20
    2. AmiGo 1.8                       15/20
    3. Crazy Patterns 0008-13          13/20
    4. Marcos Go 1.0                   11/20
    5. Beancounter 0.1                  4/20
    6. Stop 0.9-005                     0/20
League H:

Code: Select all

    1. Stop 0.9-005                    14/16
    2. GoTraxx 1.4.2                   10/16
    3. V-Run 1                          7/16
    4. CopyBot 0.1                      6/16
    5. Brown 1.0                        3/16

Configuration:
League A-B: 1h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League C-H: 1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

Amigo: amigogtp
AQ: AQ

Code: Select all

aq_config.txt:
-main　time[sec] =3600
-time controll =true
-japanese rule =true
Aya: Aya.exe –mode gtp –level max
Beancounter: beancounter
Brown: brown.exe
Copybot: python /path/to/__main__.py
CrazyPatterns: CrazyPatterns.exe
Dariush: DarGTP.exe –level 10
DarkForest: taskset -c 0 bash cnn_evaluator.sh 1 /data/local/go and taskset -c 0 th cnnPlayerMCTSV2.lua --num_gpu 1 --num_tree_thread 1 --rollout 750 --win_rate_thres 0.1
DarkGo: darknet go engine cfg/go.test.cfg go.weights
deltaGo: deltaGo.exe
Dream Go: export NUM_ITER=1375 and dream_go
Fudo Go: taskset -c 0 fudo –boardsize=19 –komi=6.5
Fuego: fuego.exe –config fuego.cfg

Code: Select all

fuego.cfg:
uct_param_search number_threads 1
uct_param_search lock_free 0
uct_max_memory 1024000000
uct_param_player reuse_subtree 1
uct_param_player ponder 0
uct_param_player early_pass 1
GnuGo: gnugo --mode gtp --level 10 --resign-allowed
GoTraxx: GoTraxx.exe
Hara: hara
Hiratuka: Hiratuka-19×19.exe -po 50000
IndiGo: Indigo.exe -gtp
Iomrascálaí: taskset -c 0,1 iomrascalai
JrefBot: java -jar jrefgo.jar 50000
Leela: leela_gtp_opencl –gtp –threads 4 –noponder
Leela Zero: leelaz --gtp --threads 4 --w /path/to/Leelaz_best-network_yyyy_mm_dd --noponder
Matilda: matilda

Code: Select all

matilda.h:
#define BOARD_SIZ 19
#define DEFAULT_UCT_MEMORY 1000
#define DEFAULT_NUM_THREADS 1
Marcos Go: marcos_go --patterns /path/to/patterns.txt --cycles_mcts 10000 --threads_mcts 1
Michi C-2: michi gtp

Code: Select all

ui.c:
init_large_patterns("patterns2.prob", "patterns2.spat"); // Michis pattern files renamed because they have the same name as Pachis files.
MoGo: mogo
Oakfoam: oakfoam -c nicego-cnn-06.gtp

Code: Select all

nicego-cnn-06.gtp:
param playouts_per_move_max 40000
param thread_count 4
OnePunchGo: taskset -c 0 OPG

Code: Select all

TimeInfo.h (Modified to play 15sec/move because OPG requires time_left command which is not supported by twogtp and Sabaki)
timeAvailable = 15;
Orego: java -jar /path/to/orego-7.08.jar threads=1 grace
Pachi: pachidcnn -f pachibook.dat threads=1,max_tree_size=1024,pondering=0
Ray: ray --time 3600 --thread 1 --no-debug
Rayon: rayon --thread 4 --no-debug
Simple Go: python /path/to/play_gtp.py --node_limit=100
Stop: /usr/bin/java -ea -jar /path/to/stop-09-005.jar --mode gtp
V-Run: python3 /path/to/V-Run
Zen: gtp4zen.exe -z 7 -t 4 -T 30 -s 10000 -i 1000
Links:
Amigo: https://sourceforge.net/projects/amigogtp/
AQ: https://github.com/ymgaq/AQ
Aya: http://www.yss-aya.com/
Brown: http://ricoh51.free.fr/go/engineeng.htm
Beancounter: Private
Copybot: https://github.com/sirtango/ICopyMoves
CrazyPatterns: https://www.remi-coulom.fr/Amsterdam2007/
Dariush: http://ricoh51.free.fr/go/engineeng.htm
DarkForest: https://github.com/facebookresearch/darkforestGo
DarkGo: https://pjreddie.com/darknet/darkgo-go-in-darknet/
deltaGo: http://home.q00.itscom.net/otsuki/delta.html
Dream Go: https://github.com/Chicoryn/dream-go
Fudo Go: http://www.geocities.jp/hideki_katoh/
Fuego: http://fuego.sourceforge.net/
GnuGo: https://www.gnu.org/software/gnugo/devel.html
GoTraxx http://gotraxx.codeplex.com/
Hara: https://github.com/antoniogarro/Hara
Hiratuka: Non GPU version (10.37B): http://www.vector.co.jp/download/file/w ... 73259.html / GPU version (10.38B): http://www.vector.co.jp/download/file/w ... 88349.html
IndiGo: http://www.math-info.univ-paris5.fr/~bouzy/INDIGO.html
Iomrascálaí: https://github.com/ujh/iomrascalai
JrefBot: http://ricoh51.free.fr/go/engineeng.htm
Leela: https://sjeng.org/leela.html
Leela Zero: [urlhttp://zero.sjeng.org/[/url]
Marcos Go: https://github.com/MarcosPividori/Go-player
Matilda: https://github.com/gonmf/matilda
Michi C-2 https://github.com/db3108/michi-c2
MoGo: forum/viewtopic.php?p=211091#p211091
Oakfoam: https://bitbucket.org/dsmic/oakfoam
OnePunchGo https://github.com/AlexKent3141/OnePunchGo
Orego: https://sites.google.com/a/lclark.edu/d ... arch/orego
Pachi: https://github.com/pasky/pachi
Rayon: https://github.com/zakki/Ray
Ray: https://github.com/kobanium/Ray
Simple Go: https://sourceforge.net/projects/londerings/
Stop: https://www.vanheusden.com/stop/
V-Run: https://github.com/Fioelkais/MCTS
Zen via gtp4zen (info): https://www.lifein19x19.com/forum/viewt ... 18&t=15599
Best,
Alex

Re: Engine Tournament

Posted: Sat Apr 21, 2018 9:47 am
by as0770
And finally Zen 7 vs. Leela Zero GPU.

Final:

Code: Select all

    1. Leela Zero 0.13 128x10 GPU       5/8
    2. Zen 7                            3/8
League A:

Code: Select all

    1. Zen 7                           16/24
    2. AQ 2.1.1                        13/24
    3. Leela 0.11                      10/24
    4. Leela Zero 0.13 128x10 CPU       9/24
League B:

Code: Select all

    1. Leela Zero 0.13 128x10 CPU      18/20
    2. Leela 0.11.0                    15/20
    3. Oakfoam 0.2.1 NG-06             11/20
    4. Rayon 4.6.0                     11/20
    5. Hiratuka 10.37B (CPU)            4/20
    6. DreamGo 0.5.0                    1/20
League C:

Code: Select all

    1. DreamGo 0.5.0                   15/20
    2. Pachi 12                        14/20
    3. DarkForrest MCTS 1.0            12/20
    4. Ray 9.0.1                        9/20
    5. DarkGo 1.0                       8/20
    6. Mogo 4.86                        2/20
League D:

Code: Select all

    1. MoGo 4.86                       18/20
    2. deltaGo 1.0.0                   14/20
    3. Fuego 1.1                       13/20
    4. Michi C-2 1.4.2                  8/20
    5. Orego 7.08                       5/20
    6. GNU Go 3.8                       2/20
League E:

Code: Select all

    1. GNU Go 3.8                      18/20
    2. Hara 0.9                        11/20
    3. Dariush 3.1.5.7                  9/20
    4. Indigo 2009                      9/20
    5. Matilda 1.25                     9/20
    6. OnePunchGo 1.0                   4/20
League F:

Code: Select all

    1. OnePunchGo 1.0                  17/20
    2. Aya 6.34                        16/20  
    3. JrefBot 081016-2022             12/20  
    4. Fudo Go 3.0                     10/20
    5. Iomrascálaí 0.3.2                4/20
    6. SimpleGo 0.4.3                   1/20
League G:

Code: Select all

    1. SimpleGo 0.4.3                  17/20
    2. AmiGo 1.8                       15/20
    3. Crazy Patterns 0008-13          13/20
    4. Marcos Go 1.0                   11/20
    5. Beancounter 0.1                  4/20
    6. Stop 0.9-005                     0/20
League H:

Code: Select all

    1. Stop 0.9-005                    14/16
    2. GoTraxx 1.4.2                   10/16
    3. V-Run 1                          7/16
    4. CopyBot 0.1                      6/16
    5. Brown 1.0                        3/16

Configuration:
Final: 2h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 120 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League A-B: 1h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League C-H: 1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

Amigo: amigogtp
AQ: AQ

Code: Select all

aq_config.txt:
-main　time[sec] =3600
-time controll =true
-japanese rule =true
Aya: Aya.exe –mode gtp –level max
Beancounter: beancounter
Brown: brown.exe
Copybot: python /path/to/__main__.py
CrazyPatterns: CrazyPatterns.exe
Dariush: DarGTP.exe –level 10
DarkForest: taskset -c 0 bash cnn_evaluator.sh 1 /data/local/go and taskset -c 0 th cnnPlayerMCTSV2.lua --num_gpu 1 --num_tree_thread 1 --rollout 750 --win_rate_thres 0.1
DarkGo: darknet go engine cfg/go.test.cfg go.weights
deltaGo: deltaGo.exe
Dream Go: export NUM_ITER=1375 and dream_go
Fudo Go: taskset -c 0 fudo –boardsize=19 –komi=6.5
Fuego: fuego.exe –config fuego.cfg

Code: Select all

fuego.cfg:
uct_param_search number_threads 1
uct_param_search lock_free 0
uct_max_memory 1024000000
uct_param_player reuse_subtree 1
uct_param_player ponder 0
uct_param_player early_pass 1
GnuGo: gnugo --mode gtp --level 10 --resign-allowed
GoTraxx: GoTraxx.exe
Hara: hara
Hiratuka: Hiratuka-19×19.exe -po 50000
IndiGo: Indigo.exe -gtp
Iomrascálaí: taskset -c 0,1 iomrascalai
JrefBot: java -jar jrefgo.jar 50000
Leela: leela_gtp_opencl –gtp –threads 4 –noponder
Leela Zero: leelaz --gtp --threads 4 --w /path/to/Leelaz_best-network_yyyy_mm_dd --noponder
Matilda: matilda

Code: Select all

matilda.h:
#define BOARD_SIZ 19
#define DEFAULT_UCT_MEMORY 1000
#define DEFAULT_NUM_THREADS 1
Marcos Go: marcos_go --patterns /path/to/patterns.txt --cycles_mcts 10000 --threads_mcts 1
Michi C-2: michi gtp

Code: Select all

ui.c:
init_large_patterns("patterns2.prob", "patterns2.spat"); // Michis pattern files renamed because they have the same name as Pachis files.
MoGo: mogo
Oakfoam: oakfoam -c nicego-cnn-06.gtp

Code: Select all

nicego-cnn-06.gtp:
param playouts_per_move_max 40000
param thread_count 4
OnePunchGo: taskset -c 0 OPG

Code: Select all

TimeInfo.h (Modified to play 15sec/move because OPG requires time_left command which is not supported by twogtp and Sabaki)
timeAvailable = 15;
Orego: java -jar /path/to/orego-7.08.jar threads=1 grace
Pachi: pachidcnn -f pachibook.dat threads=1,max_tree_size=1024,pondering=0
Ray: ray --time 3600 --thread 1 --no-debug
Rayon: rayon --thread 4 --no-debug
Simple Go: python /path/to/play_gtp.py --node_limit=100
Stop: /usr/bin/java -ea -jar /path/to/stop-09-005.jar --mode gtp
V-Run: python3 /path/to/V-Run
Zen: gtp4zen.exe -z 7 -t 4 -T 30 -s 10000 -i 1000
Links:
Amigo: https://sourceforge.net/projects/amigogtp/
AQ: https://github.com/ymgaq/AQ
Aya: http://www.yss-aya.com/
Brown: http://ricoh51.free.fr/go/engineeng.htm
Beancounter: Private
Copybot: https://github.com/sirtango/ICopyMoves
CrazyPatterns: https://www.remi-coulom.fr/Amsterdam2007/
Dariush: http://ricoh51.free.fr/go/engineeng.htm
DarkForest: https://github.com/facebookresearch/darkforestGo
DarkGo: https://pjreddie.com/darknet/darkgo-go-in-darknet/
deltaGo: http://home.q00.itscom.net/otsuki/delta.html
Dream Go: https://github.com/Chicoryn/dream-go
Fudo Go: http://www.geocities.jp/hideki_katoh/
Fuego: http://fuego.sourceforge.net/
GnuGo: https://www.gnu.org/software/gnugo/devel.html
GoTraxx http://gotraxx.codeplex.com/
Hara: https://github.com/antoniogarro/Hara
Hiratuka: Non GPU version (10.37B): http://www.vector.co.jp/download/file/w ... 73259.html / GPU version (10.38B): http://www.vector.co.jp/download/file/w ... 88349.html
IndiGo: http://www.math-info.univ-paris5.fr/~bouzy/INDIGO.html
Iomrascálaí: https://github.com/ujh/iomrascalai
JrefBot: http://ricoh51.free.fr/go/engineeng.htm
Leela: https://sjeng.org/leela.html
Leela Zero: [urlhttp://zero.sjeng.org/[/url]
Marcos Go: https://github.com/MarcosPividori/Go-player
Matilda: https://github.com/gonmf/matilda
Michi C-2 https://github.com/db3108/michi-c2
MoGo: forum/viewtopic.php?p=211091#p211091
Oakfoam: https://bitbucket.org/dsmic/oakfoam
OnePunchGo https://github.com/AlexKent3141/OnePunchGo
Orego: https://sites.google.com/a/lclark.edu/d ... arch/orego
Pachi: https://github.com/pasky/pachi
Rayon: https://github.com/zakki/Ray
Ray: https://github.com/kobanium/Ray
Simple Go: https://sourceforge.net/projects/londerings/
Stop: https://www.vanheusden.com/stop/
V-Run: https://github.com/Fioelkais/MCTS
Zen via gtp4zen (info): https://www.lifein19x19.com/forum/viewt ... 18&t=15599
Best,
Alex

Re: Engine Tournament

Posted: Sat May 05, 2018 2:48 am
by q30
I had compiled Ray (new version) and fought it with other engines details.
The rating of Go engines with GTP and without GPU support is next:

League A
1. Leela

League B
2. Hiratuka
3&4.Pachi & Ray
5. MoGo

Re: Engine Tournament

Posted: Sun May 06, 2018 11:15 pm
by as0770
Updates:
-Leela Zero 128x6 CPU in League B, Leela Zero 64x5 CPU in League C and Leela Zero 32x4 CPU in League H.
-Leela Zero with ELF OpenGo Network in League A as CPU version with around 5 playouts/sec and with full strength and around 30 playouts/sec in the "Champions League".

The performance of ELF OpenGo GPU is not quite as expected. It lost 2 games by 0.5 vs. Zen and one by a stupid ladder. Maybe I'll repeat the tournament with a komi of 7.5 but honestly I think a bot should be able to play also with a komi of 6.5. Zen can handle this and thus gets 2 points extra. Leela Zero won one game without komi or ladder problems, ELF resigned.

The CPU version lost one because of a ladder but none because of the komi.

Champions League:

Code: Select all

    1. ELF OpenGo v0                   12/16
    2. Leela Zero 0.13 128x10           6/16
    2. Zen 7                            6/16
League A:

Code: Select all

    1. Zen 7                           11/16
    2. ELF OpenGo v0 (CPU)             11/16
    3. AQ 2.1.1                         8/16
    4. Leela 0.11                       6/16
    5. Leela Zero 0.13 128x10 (CPU)     4/16
League B:

Code: Select all

    1. Leela Zero 0.13 128x10 (CPU)    22/24
    2. Leela 0.11.0                    19/24
    3. Rayon 4.6.0                     13/24
    4. Oakfoam 0.2.1 NG-06             12/24
    5. Leela Zero 128x6 (CPU)          12/24
    5. Hiratuka 10.37B (CPU)            4/24
    6. DreamGo 0.5.0                    2/24
League C:

Code: Select all

    1. DreamGo 0.5.0                   18/24
    2. DarkForrest MCTS 1.0            16/24
    3. Pachi 12                        16/24
    4. Ray 9.0.1                       11/24
    5. DarkGo 1.0                      10/24
    6. Leela Zero 64x5 (CPU)           10/24
    7. Mogo 4.86                        3/24
League D:

Code: Select all

    1. MoGo 4.86                       18/20
    2. deltaGo 1.0.0                   14/20
    3. Fuego 1.1                       13/20
    4. Michi C-2 1.4.2                  8/20
    5. Orego 7.08                       5/20
    6. GNU Go 3.8                       2/20
League E:

Code: Select all

    1. GNU Go 3.8                      18/20
    2. Hara 0.9                        11/20
    3. Dariush 3.1.5.7                  9/20
    4. Indigo 2009                      9/20
    5. Matilda 1.25                     9/20
    6. OnePunchGo 1.0                   4/20
League F:

Code: Select all

    1. OnePunchGo 1.0                  17/20
    2. Aya 6.34                        16/20  
    3. JrefBot 081016-2022             12/20  
    4. Fudo Go 3.0                     10/20
    5. Iomrascálaí 0.3.2                4/20
    6. SimpleGo 0.4.3                   1/20
League G:

Code: Select all

    1. SimpleGo 0.4.3                  17/20
    2. AmiGo 1.8                       15/20
    3. Crazy Patterns 0008-13          13/20
    4. Marcos Go 1.0                   11/20
    5. Beancounter 0.1                  4/20
    6. Stop 0.9-005                     0/20
League H:

Code: Select all

    1. Stop 0.9-005                    18/20
    2. GoTraxx 1.4.2                   14/20
    3. V-Run 1                          9/20
    4. CopyBot 0.1                      8/20
    5. Leela Zero 32x4 (CPU)            7/20
    5. Brown 1.0                        4/20

Configuration:
Champions League: 2h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 120 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League A-B: 1h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

League C-H: 1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2
TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto

Amigo: amigogtp
AQ: AQ

Code: Select all

aq_config.txt:
-main　time[sec] =3600
-time controll =true
-japanese rule =true
Aya: Aya.exe –mode gtp –level max
Beancounter: beancounter
Brown: brown.exe
Copybot: python /path/to/__main__.py
CrazyPatterns: CrazyPatterns.exe
Dariush: DarGTP.exe –level 10
DarkForest: taskset -c 0 bash cnn_evaluator.sh 1 /data/local/go and taskset -c 0 th cnnPlayerMCTSV2.lua --num_gpu 1 --num_tree_thread 1 --rollout 750 --win_rate_thres 0.1
DarkGo: darknet go engine cfg/go.test.cfg go.weights
deltaGo: deltaGo.exe
Dream Go: export NUM_ITER=1375 and dream_go
ELF OpenGo: leelaz_elf --gtp --threads 4 --weights /path/to/network_elf --noponder
Fudo Go: taskset -c 0 fudo –boardsize=19 –komi=6.5
Fuego: fuego.exe –config fuego.cfg

Code: Select all

fuego.cfg:
uct_param_search number_threads 1
uct_param_search lock_free 0
uct_max_memory 1024000000
uct_param_player reuse_subtree 1
uct_param_player ponder 0
uct_param_player early_pass 1
GnuGo: gnugo --mode gtp --level 10 --resign-allowed
GoTraxx: GoTraxx.exe
Hara: hara
Hiratuka: Hiratuka-19×19.exe -po 50000
IndiGo: Indigo.exe -gtp
Iomrascálaí: taskset -c 0,1 iomrascalai
JrefBot: java -jar jrefgo.jar 50000
Leela: leela_gtp_opencl –gtp –threads 4 –noponder
Leela Zero: leelaz --gtp --threads 4 --w /path/to/Leelaz_best-network_yyyy_mm_dd --noponder
Matilda: matilda

Code: Select all

matilda.h:
#define BOARD_SIZ 19
#define DEFAULT_UCT_MEMORY 1000
#define DEFAULT_NUM_THREADS 1
Marcos Go: marcos_go --patterns /path/to/patterns.txt --cycles_mcts 10000 --threads_mcts 1
Michi C-2: michi gtp

Code: Select all

ui.c:
init_large_patterns("patterns2.prob", "patterns2.spat"); // Michis pattern files renamed because they have the same name as Pachis files.
MoGo: mogo
Oakfoam: oakfoam -c nicego-cnn-06.gtp

Code: Select all

nicego-cnn-06.gtp:
param playouts_per_move_max 40000
param thread_count 4
OnePunchGo: taskset -c 0 OPG

Code: Select all

TimeInfo.h (Modified to play 15sec/move because OPG requires time_left command which is not supported by twogtp and Sabaki)
timeAvailable = 15;
Orego: java -jar /path/to/orego-7.08.jar threads=1 grace
Pachi: pachidcnn -f pachibook.dat threads=1,max_tree_size=1024,pondering=0
Ray: ray --time 3600 --thread 1 --no-debug
Rayon: rayon --thread 4 --no-debug
Simple Go: python /path/to/play_gtp.py --node_limit=100
Stop: /usr/bin/java -ea -jar /path/to/stop-09-005.jar --mode gtp
V-Run: python3 /path/to/V-Run
Zen: gtp4zen.exe -z 7 -t 4 -T 30 -s 10000 -i 1000
Links:
Amigo: https://sourceforge.net/projects/amigogtp/
AQ: https://github.com/ymgaq/AQ
Aya: http://www.yss-aya.com/
Brown: http://ricoh51.free.fr/go/engineeng.htm
Beancounter: Private
Copybot: https://github.com/sirtango/ICopyMoves
CrazyPatterns: https://www.remi-coulom.fr/Amsterdam2007/
Dariush: http://ricoh51.free.fr/go/engineeng.htm
DarkForest: https://github.com/facebookresearch/darkforestGo
DarkGo: https://pjreddie.com/darknet/darkgo-go-in-darknet/
deltaGo: http://home.q00.itscom.net/otsuki/delta.html
Dream Go: https://github.com/Chicoryn/dream-go
ELF OpenGo: https://github.com/gcp/leela-zero
Fudo Go: http://www.geocities.jp/hideki_katoh/
Fuego: http://fuego.sourceforge.net/
GnuGo: https://www.gnu.org/software/gnugo/devel.html
GoTraxx http://gotraxx.codeplex.com/
Hara: https://github.com/antoniogarro/Hara
Hiratuka: Non GPU version (10.37B): http://www.vector.co.jp/download/file/w ... 73259.html / GPU version (10.38B): http://www.vector.co.jp/download/file/w ... 88349.html
IndiGo: http://www.math-info.univ-paris5.fr/~bouzy/INDIGO.html
Iomrascálaí: https://github.com/ujh/iomrascalai
JrefBot: http://ricoh51.free.fr/go/engineeng.htm
Leela: https://sjeng.org/leela.html
Leela Zero: http://zero.sjeng.org/
Marcos Go: https://github.com/MarcosPividori/Go-player
Matilda: https://github.com/gonmf/matilda
Michi C-2 https://github.com/db3108/michi-c2
MoGo: forum/viewtopic.php?p=211091#p211091
Oakfoam: https://bitbucket.org/dsmic/oakfoam
OnePunchGo https://github.com/AlexKent3141/OnePunchGo
Orego: https://sites.google.com/a/lclark.edu/d ... arch/orego
Pachi: https://github.com/pasky/pachi
Rayon: https://github.com/zakki/Ray
Ray: https://github.com/kobanium/Ray
Simple Go: https://sourceforge.net/projects/londerings/
Stop: https://www.vanheusden.com/stop/
V-Run: https://github.com/Fioelkais/MCTS
Zen via gtp4zen (info): https://www.lifein19x19.com/forum/viewt ... 18&t=15599
Best,
Alex

Re: Engine Tournament

Posted: Mon May 07, 2018 6:14 am
by luigi
Why would you use 6.5 komi anyway? Aren't you using area scoring?

Re: Engine Tournament

Posted: Mon May 07, 2018 6:51 am
by Vargo
I've tried various settings for LZ and the ELF weights.
On my computer, it's very nearly unbeatable, with caveats :

1) It MUST play even games
2) Komi MUST be 7.5
3) I think there could be a kind of threshold for the playouts, a minimum of 20000 (?) to get this level.

Even if LZ (latest network) is given quadruple thinking time per move, LZ_ELF wins every time...( time_settings 0 61 1 for LZ and time_settings 0 16 1 for LZ_ELF) I've run maybe 10-12 such games, and LZ hasn't won a single one. It's really uncanny.

GPU 2x1080Ti, CPU i9-7920X

PS. New best networks for LZ seem to pop up like crazy today (already 3 new ones !) is it related to the ELF weights ?
LZ's strength graph is getting almost vertical

Re: Engine Tournament

Posted: Mon May 07, 2018 10:35 am
by pnprog
as0770 wrote:Updates:
-Leela Zero 128x6 CPU in League B, Leela Zero 64x5 CPU in League C and Leela Zero 32x4 CPU in League H.
-Leela Zero with ELF OpenGo Network in League A as CPU version with around 5 playouts/sec and with full strength and around 30 playouts/sec in the "Champions League".

The performance of ELF OpenGo GPU is not quite as expected. It lost 2 games by 0.5 vs. Zen and one by a stupid ladder. Maybe I'll repeat the tournament with a komi of 7.5 but honestly I think a bot should be able to play also with a komi of 6.5. Zen can handle this and thus gets 2 points extra. Leela Zero won one game without komi or ladder problems, ELF resigned.

The CPU version lost one because of a ladder but none because of the komi.
I think it would be more reasonable to wait for the ELF team to release a windows binary. ELF's network might be stronger, but there is also the fact that the original software run faster than Leela Zero from what I read (more play-outs per second than LeelaZero).

Re: Engine Tournament

Posted: Mon May 07, 2018 11:27 am
by jlt
Vargo wrote: PS. New best networks for LZ seem to pop up like crazy today (already 3 new ones !) is it related to the ELF weights ?
LZ's strength graph is getting almost vertical
Yes, probably. See here and there.

Re: Engine Tournament

Posted: Sat May 12, 2018 4:08 am
by q30
as0770 wrote:...
ELF OpenGo: https://github.com/gcp/leela-zero
...
Is this right, or must be that one:
https://github.com/pytorch/ELF

Re: Engine Tournament

Posted: Sat May 12, 2018 5:08 am
by q30
I had compiled last versions of Pachi_DCNN (required for its start additional soft is included: see link below). It's a bit stronger, than Pachi (details):

League A
1.  Leela

League B
2.Hiratuka
3.Pachi_DCNN
4&5.Pachi & Ray
6.MoGo
All times are UTC-07:00
Page 13 of 27

Powered by phpBB® Forum Software © phpBB Limited