It is currently Thu Mar 28, 2024 10:15 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 390 posts ]  Go to page Previous  1 ... 4, 5, 6, 7, 8, 9, 10 ... 20  Next
Author Message
Offline
 Post subject: Re: Engine Tournament
Post #121 Posted: Thu Dec 07, 2017 3:56 pm 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
New entries in League D are Leela Zero with the netfork file from 2017-12-03 and Beancounter, my own attempt to write a go engine.

Leela vs. AQ
Code:
    1. AQ 2.0.3                     12/16
    2. Leela 0.11.0 Beta 11          4/16

Configuration:
2h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

AQ: AQ
Code:
aq_config.txt:
-main time[sec] =7200
-time controll =true
-japanese rule =true

Leela: leela_gtp_opencl –gtp –threads 4

twogtp: gogui-twogtp -auto -white "leela_gtp_opencl --gtp --threads 4" -black "AQ" -games 4 -size 19 -time 120 -verbose -sgffile xxxx


League A:
Code:
    1. Leela 0.10.0                 22/24
    2. Rayon 4.6.0                  19/24
    3. Oakfoam 0.2.1 NG-06          18/24
    4. Hiratuka 10.37B (CPU)         9/24
    5. DarkForest v2 MCTS 1.0        7/24
    6  DarkGo 1.0                    5/24
    7. Pachi DCNN 11.99              4/24

Configuration:
2h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

DarkForest: taskset -c 0 bash cnn_evaluator.sh 1 /data/local/go and taskset -c 0 th cnnPlayerMCTSV2.lua --num_gpu 1 --num_tree_thread 1 --rollout 2500 --win_rate_thres 0.1
DarkGo: darknet go engine cfg/go.test.cfg go.weights
Hiratuka: Hiratuka-19×19.exe -po 175000
Leela: leela_gtp_opencl –gtp –threads 4 –noponder
Oakfoam: oakfoam -c nicego-cnn-06.gtp
Code:
nicego-cnn-06.gtp:
param playouts_per_move_max 40000
param thread_count 4

Pachi: pachidcnn -f pachibook.dat threads=4,max_tree_size=2048,pondering=0
Rayon: rayon --thread 4 --no-debug

twogtp: TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 120 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto


League B:
Code:
    1. Pachi DCNN 11.99             29/32
    2. Ray 9.0.1                    27/32
    3. MoGo 4.86                    21/32
    4. deltaGo 1.0.0                17/32
    5. Fuego 1.1                    17/32
    6. Leela Zero 0.1               15/32   
    7. Michi C-2 1.4.2               9/32
    8. Orego 7.08                    7/32
    9. GNU Go 3.8                    2/32

Configuration:
1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

deltaGo: deltaGo.exe
Fuego: fuego.exe –config fuego.cfg
Code:
fuego.cfg:
uct_param_search number_threads 1
uct_param_search lock_free 0
uct_max_memory 1024000000
uct_param_player reuse_subtree 1
uct_param_player ponder 0
uct_param_player early_pass 1

GnuGo: gnugo --mode gtp --level 10 --resign-allowed
Leela Zero: leelaz --gtp --w /path/to/weights.txt --noponder
Michi C-2: michi gtp
Code:
ui.c:
init_large_patterns("patterns2.prob", "patterns2.spat"); // Michis pattern files renamed because they have the same name as Pachis files.

MoGo: mogo
Orego: java -jar /path/to/orego-7.08.jar threads=1 grace
Pachi: pachi -f pachibook.dat threads=1,max_tree_size=1024,pondering=0
Ray: ray --time 3600 --thread 1 --no-debug
twogtp: TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto


League C:
Code:
    1. GNU Go 3.8                   24/28
    2. Hara 0.9                     18/28
    3. Dariush 3.1.5.7              16/28
    4. Indigo 2009                  15/28
    5. Matilda 1.24                 15/28
    6. Aya 6.34                     11/28
    7. Fudo Go 3.0                  11/28
    8. JrefBot 081016-2022           2/28

Configuration:
1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

Aya: Aya.exe –mode gtp –level max
Dariush: DarGTP.exe –level 10
Fudo Go: taskset -c 0 fudo –boardsize=19 –komi=6.5
GnuGo: gnugo --mode gtp --level 10 --resign-allowed
Hara: hara
IndiGo: Indigo.exe -gtp
JrefBot: java -jar jrefgo.jar 10000
Matilda: matilda
Code:
matilda.h:
#define BOARD_SIZ 19
#define DEFAULT_UCT_MEMORY 1000
#define DEFAULT_NUM_THREADS 1


twogtp: TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto


League D:
Code:
    1. JrefBot 081016-2022          18/20             
    2. Iomrascálaí 0.3.2            17/20
    3. Crazy Patterns 0008-13       15/20
    4. Marcos Go 1.0                15/20
    5. AmiGo 1.8                    15/20
    6. Beancounter 0.1              10/20
    7. Leela Zero 0.6 (2017-12-03)   7/20
    8. Stop 0.9-005                  5/20
    9. GoTraxx 1.4.2                 4/20
   10. CopyBot 0.1                   2/20
   11. Brown 1.0                     2/20

Configuration:
1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

Amigo: amigogtp
Beancounter: beancounter
Brown: brown.exe
Copybot: python /path/to/__main__.py
CrazyPatterns: CrazyPatterns.exe
GoTraxx: GoTraxx.exe
Iomrascálaí: taskset -c 0,1 iomrascalai
JrefBot: java -jar jrefgo.jar 10000
Leela Zero: leelaz --gtp --w /path/to/best-network-2017-12-03 --noponder
Marcos Go: marcos_go --patterns /path/to/patterns.txt --cycles_mcts 10000 --threads_mcts 1
Stop: /usr/bin/java -ea -jar /path/to/stop-09-005.jar --mode gtp

twogtp: TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto



Links:
Amigo: https://sourceforge.net/projects/amigogtp/
AQ: https://github.com/ymgaq/AQ
Aya: http://www.yss-aya.com/
Brown: http://ricoh51.free.fr/go/engineeng.htm
Copybot: https://github.com/sirtango/ICopyMoves
CrazyPatterns: https://www.remi-coulom.fr/Amsterdam2007/
Dariush: http://ricoh51.free.fr/go/engineeng.htm
DarkForest: https://github.com/facebookresearch/darkforestGo
DarkGo: https://pjreddie.com/darknet/darkgo-go-in-darknet/
deltaGo: http://home.q00.itscom.net/otsuki/delta.html
Fudo Go: http://www.geocities.jp/hideki_katoh/
Fuego: http://fuego.sourceforge.net/
GnuGo: https://www.gnu.org/software/gnugo/devel.html
GoTraxx http://gotraxx.codeplex.com/
Hara: https://github.com/antoniogarro/Hara
Hiratuka: Non GPU version (10.37B): http://www.vector.co.jp/download/file/winnt/game/fh673259.html / GPU version (10.38B): http://www.vector.co.jp/download/file/winnt/game/fh688349.html
IndiGo: http://www.math-info.univ-paris5.fr/~bouzy/INDIGO.html
Iomrascálaí: https://github.com/ujh/iomrascalai
JrefBot: http://ricoh51.free.fr/go/engineeng.htm
Leela: https://sjeng.org/leela.html
Leela Zero: https://github.com/gcp/leela-zero
Marcos Go: https://github.com/MarcosPividori/Go-player
Matilda: https://github.com/gonmf/matilda
Michi C-2 https://github.com/db3108/michi-c2
MoGo: https://lifein19x19.com/forum/viewtopic.php?p=211091#p211091
Oakfoam: https://bitbucket.org/dsmic/oakfoam
Orego: https://sites.google.com/a/lclark.edu/drake/research/orego
Pachi: http://pachi.or.cz/
Rayon: https://github.com/zakki/Ray
Ray: https://github.com/kobanium/Ray
Stop: https://www.vanheusden.com/stop/

Best,
Alex


Last edited by as0770 on Tue Jan 02, 2018 12:42 pm, edited 2 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #122 Posted: Fri Dec 08, 2017 12:16 pm 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
as0770 wrote:
Code:
    1. JrefBot 081016-2022          18/20             
    2. Iomrascálaí 0.3.2            17/20
    3. Crazy Patterns 0008-13       15/20
    4. Marcos Go 1.0                15/20
    5. AmiGo 1.8                    15/20
    6. Beancounter 0.1              10/20
    7. Leela Zero 0.6 (2017-12-03)   7/20
    8. Stop 0.9-005                  5/20
    9. GoTraxx 1.4.2                 4/20
   10. CopyBot 0.1                   2/20
   11. Brown 1.0                     2/20



Four more days of training for Leela Zero and this is the result:

Code:
    1. JrefBot 081016-2022          18/20             
    2. Iomrascálaí 0.3.2            17/20
    3. Crazy Patterns 0008-13       15/20
    4. AmiGo 1.8                    14/20
    5. Marcos Go 1.0                13/20
    6. Leela Zero 0.8 (2017-12-07)  12/20
    7. Beancounter 0.1               8/20
    8. Stop 0.9-005                  5/20
    9. GoTraxx 1.4.2                 4/20
   10. CopyBot 0.1                   2/20
   11. Brown 1.0                     2/20

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #123 Posted: Sat Dec 09, 2017 5:32 am 
Lives with ko

Posts: 135
Liked others: 1
Was liked: 1
Rank: 30 kyu
as0770 wrote:
...The difference between ponder on and ponder off matches is nearly irrelevant compared to all the other influences and for sure interfered by the statistical fluctuation when rating the engines with less than 100 games.

For same strength engines sparring ponder is significant parameter...
But most significant (for the game strength) difference of these synthetic tests and real prof. games is in the time control parameter values.

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #124 Posted: Sun Dec 10, 2017 9:29 am 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
q30 wrote:
For same strength engines sparring ponder is significant parameter...


Based on what tests? I played thousands of computer AI games, even if one engine is pondering, and one not, you need hundrets of games to see the difference.

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #125 Posted: Sat Dec 16, 2017 1:25 am 
Lives with ko

Posts: 135
Liked others: 1
Was liked: 1
Rank: 30 kyu
as0770 wrote:
Based on what tests? ...

On tests with close to real game parameters (2'' on move, for example).

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #126 Posted: Sun Dec 17, 2017 11:56 pm 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
q30 wrote:
as0770 wrote:
Based on what tests? ...

On tests with close to real game parameters (2'' on move, for example).


In chess doubling the calculating time will make Engines stronger by 60 ELO Points. Engines of the same strength have about 50% ponderhits. So the difference of a pondering Engine to a non pondering Engine is about 30 ELO. You need more than 1000 games to measure an ELO difference of 30 ELO. And we are talking about ponder vs. no ponder. In Go the difference is even smaller because there are less ponderhits. Also the ELO gap between engines and their pondering ELO gain is much less 30 ELO. So after all it is simple impossible to meassure a difference in the ELO gain with pomdering with such a small amount of games.

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #127 Posted: Sat Dec 23, 2017 12:51 am 
Lives with ko

Posts: 135
Liked others: 1
Was liked: 1
Rank: 30 kyu
as0770 wrote:
In chess doubling the calculating time will make Engines stronger by 60 ELO Points...

It's not absolutely linear strength(time) dependency. So strength increasing value by doubling time will depend on absolute time value.
Pondering may not affect only on simple MC engines, such MoGo, where, for example, increasing by "--earlyCut 0" thinking time doesn't make the engine game stronger.
I'll test pondering effect on MoGo, Pachi, Ray and Leela soon.


This post by q30 was liked by: Waylon
Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #128 Posted: Tue Dec 26, 2017 8:34 am 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
q30 wrote:
as0770 wrote:
In chess doubling the calculating time will make Engines stronger by 60 ELO Points...

It's not absolutely linear strength(time) dependency. So strength increasing value by doubling time will depend on absolute time value


30 years of statistics in computerchess say something different.

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #129 Posted: Wed Dec 27, 2017 9:37 am 
Dies in gote

Posts: 34
Liked others: 2
Was liked: 5
Rank: 3k
Leela Zero is much stronger now, can you test it again please?

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #130 Posted: Wed Dec 27, 2017 10:06 am 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
Cyan wrote:
Leela Zero is much stronger now, can you test it again please?


I'd love to do so, but I am on vacations :) I'll be back in 1-2 weeks.

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #131 Posted: Wed Dec 27, 2017 1:03 pm 
Lives in sente

Posts: 757
Liked others: 114
Was liked: 916
Rank: maybe 2d
as0770 wrote:
q30 wrote:
as0770 wrote:
In chess doubling the calculating time will make Engines stronger by 60 ELO Points...

It's not absolutely linear strength(time) dependency. So strength increasing value by doubling time will depend on absolute time value


30 years of statistics in computerchess say something different.


Are you sure? I'm pretty sure I recall more than one case of computer chess statistics, one informally posted on a forum, and one from some published paper, indicating slightly sublinear elo gains with log(time). Possibly other attempts I didn't see got more linear results, perhaps it depends a little on the engine and perhaps you only see it if you test a wide enough range.

The order of magnitude differences were like a +35 elo difference for a given time multiplication factor becoming a +25 elo difference for that time multiplication factor between the ends of a range that was 5 or 6 orders of magnitude wide, or something like that (those numbers are all made up, I'm just trying to convey the rough scale of things that I fuzzily recall). So, not a big difference, but still a bit nonlinear. Unless I just made up those memories. :D

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #132 Posted: Thu Dec 28, 2017 10:12 am 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
lightvector wrote:
Are you sure? I'm pretty sure I recall more than one case of computer chess statistics, one informally posted on a forum, and one from some published paper, indicating slightly sublinear elo gains with log(time). Possibly other attempts I didn't see got more linear results, perhaps it depends a little on the engine and perhaps you only see it if you test a wide enough range.

The order of magnitude differences were like a +35 elo difference for a given time multiplication factor becoming a +25 elo difference for that time multiplication factor between the ends of a range that was 5 or 6 orders of magnitude wide, or something like that (those numbers are all made up, I'm just trying to convey the rough scale of things that I fuzzily recall). So, not a big difference, but still a bit nonlinear. Unless I just made up those memories. :D


Indeed this is true. I defalcated that with faster hardware or longer timecontrol there is a slight decrase in the ELO gain. But we are talking about a decrase from a 70 ELO gain on an 286 30 years ago to a 60, maybe 50 ELO gain nowadays. This has something to do with the increasing amount of draws with nearly perfect play.

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #133 Posted: Fri Dec 29, 2017 8:15 am 
Lives with ko

Posts: 135
Liked others: 1
Was liked: 1
Rank: 30 kyu
The results (with pondering - without pondering):
MoGo 3 - 1;
Pachi 3 - 1;
Ray 3 - 1;
Leela 3 - 1;
in all 12 - 4 (details).
I don't know, what about quantitatively results (in ELO), but definitely there is qualitative effect, and in sparrings of equivalent strength Go engines the same with pondering may pass in rating engine without pondering.

_________________
Go board with attached strong engines on few strength levels from random to "God-like" playing (instructions for automated attachment in Russian) & sparring games of Go engines

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #134 Posted: Mon Jan 01, 2018 4:29 pm 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
q30 wrote:
... and in sparrings of equivalent strength Go engines the same with pondering may pass in rating engine without pondering.


So we agree that the question is only relevant in matches between engines where one is able to ponder and the other engine is not? Fine. In your ratinglist Hiratuka is the only engine that does not ponder. Once you had DarkGo which moves instandly, Hira is limited to one minute. So for both the question of the absolute timecontrol and CPU power is much more relevant than the question of running it in ponder on or ponder off matches. And still our results are similar to equal. So where is your point always claiming others as "synthetic" results? Your testing does not become more precious by depreciating others. For me it does not make any sense to test engines made for GPU support without GPU, so I have to play ponder off to get realistic results.

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #135 Posted: Wed Jan 03, 2018 6:48 am 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
Finally a Leela Zero update. v0.9 with the network file from 2018.1.1 makes it into League B and is now stronger than the human trained version 0.1 which was placed 6th in League B with 15 points against the same opponents.

Leela vs. AQ
Code:
    1. AQ 2.0.3                     12/16
    2. Leela 0.11.0 Beta 11          4/16

Configuration:
2h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

AQ: AQ
Code:
aq_config.txt:
-main time[sec] =7200
-time controll =true
-japanese rule =true

Leela: leela_gtp_opencl –gtp –threads 4 –noponder

twogtp: gogui-twogtp -auto -white "leela_gtp_opencl --gtp --threads 4 --noponder" -black "AQ" -games 4 -size 19 -time 120 -verbose -sgffile xxxx


League A:
Code:
    1. Leela 0.10.0                 22/24
    2. Rayon 4.6.0                  19/24
    3. Oakfoam 0.2.1 NG-06          18/24
    4. Hiratuka 10.37B (CPU)         9/24
    5. DarkForest v2 MCTS 1.0        7/24
    6  DarkGo 1.0                    5/24
    7. Pachi DCNN 11.99              4/24

Configuration:
2h/game, pondering off, 4 threads, 2GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

DarkForest: taskset -c 0 bash cnn_evaluator.sh 1 /data/local/go and taskset -c 0 th cnnPlayerMCTSV2.lua --num_gpu 1 --num_tree_thread 1 --rollout 2500 --win_rate_thres 0.1
DarkGo: darknet go engine cfg/go.test.cfg go.weights
Hiratuka: Hiratuka-19×19.exe -po 175000
Leela: leela_gtp_opencl –gtp –threads 4 –noponder
Oakfoam: oakfoam -c nicego-cnn-06.gtp
Code:
nicego-cnn-06.gtp:
param playouts_per_move_max 40000
param thread_count 4

Pachi: pachidcnn -f pachibook.dat threads=4,max_tree_size=2048,pondering=0
Rayon: rayon --thread 4 --no-debug

twogtp: TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 120 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto


League B:
Code:
    1. Ray 9.0.1                    29/32
    2. Pachi DCNN 11.99             28/32
    3. Leela Zero 0.9 (2018.01.01)  19/32
    4. MoGo 4.86                    18/32
    5. deltaGo 1.0.0                17/32
    6. Fuego 1.1                    15/32
    7. Michi C-2 1.4.2               8/32
    8. Orego 7.08                    8/32
    9. GNU Go 3.8                    2/32

Configuration:
1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

deltaGo: deltaGo.exe
Fuego: fuego.exe –config fuego.cfg
Code:
fuego.cfg:
uct_param_search number_threads 1
uct_param_search lock_free 0
uct_max_memory 1024000000
uct_param_player reuse_subtree 1
uct_param_player ponder 0
uct_param_player early_pass 1

GnuGo: gnugo --mode gtp --level 10 --resign-allowed
Leela Zero: leelaz --gtp --threads 1 --w /path/to/Leelaz_best-network_2018_01_01 --noponder
Michi C-2: michi gtp
Code:
ui.c:
init_large_patterns("patterns2.prob", "patterns2.spat"); // Michis pattern files renamed because they have the same name as Pachis files.

MoGo: mogo
Orego: java -jar /path/to/orego-7.08.jar threads=1 grace
Pachi: pachi -f pachibook.dat threads=1,max_tree_size=1024,pondering=0
Ray: ray --time 3600 --thread 1 --no-debug
twogtp: TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto


League C:
Code:
    1. GNU Go 3.8                   24/28
    2. Hara 0.9                     18/28
    3. Dariush 3.1.5.7              16/28
    4. Indigo 2009                  15/28
    5. Matilda 1.24                 15/28
    6. Aya 6.34                     11/28
    7. Fudo Go 3.0                  11/28
    8. JrefBot 081016-2022           2/28

Configuration:
1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

Aya: Aya.exe –mode gtp –level max
Dariush: DarGTP.exe –level 10
Fudo Go: taskset -c 0 fudo –boardsize=19 –komi=6.5
GnuGo: gnugo --mode gtp --level 10 --resign-allowed
Hara: hara
IndiGo: Indigo.exe -gtp
JrefBot: java -jar jrefgo.jar 10000
Matilda: matilda
Code:
matilda.h:
#define BOARD_SIZ 19
#define DEFAULT_UCT_MEMORY 1000
#define DEFAULT_NUM_THREADS 1


twogtp: TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto


League D:
Code:
    1. JrefBot 081016-2022          16/18             
    2. Iomrascálaí 0.3.2            15/18
    3. Crazy Patterns 0008-13       13/18
    4. Marcos Go 1.0                13/18
    5. AmiGo 1.8                    13/18
    6. Beancounter 0.1               8/18
    7. Stop 0.9-005                  5/18
    8. GoTraxx 1.4.2                 3/18
    0. CopyBot 0.1                   2/18
   10. Brown 1.0                     2/18

Configuration:
1h/game, pondering off, 1 thread, 1GB on 4 x Intel® Core™ i5-4210H CPU @ 2.90GHz, 8 GiB Ram and GeForce 840M/PCIe/SSE2

Amigo: amigogtp
Beancounter: beancounter
Brown: brown.exe
Copybot: python /path/to/__main__.py
CrazyPatterns: CrazyPatterns.exe
GoTraxx: GoTraxx.exe
Iomrascálaí: taskset -c 0,1 iomrascalai
JrefBot: java -jar jrefgo.jar 10000
Leela Zero: leelaz --gtp --w /path/to/best-network-2017-12-03 --noponder
Marcos Go: marcos_go --patterns /path/to/patterns.txt --cycles_mcts 10000 --threads_mcts 1
Stop: /usr/bin/java -ea -jar /path/to/stop-09-005.jar --mode gtp

twogtp: TWOGTP=”gogui-twogtp -black \”$BLACK\” -white \”$WHITE\” -games 2 -size 19 -time 60 -sgffile xxxx”
gogui -size 19 -program “$TWOGTP” -computer-both -auto



Links:
Amigo: https://sourceforge.net/projects/amigogtp/
AQ: https://github.com/ymgaq/AQ
Aya: http://www.yss-aya.com/
Brown: http://ricoh51.free.fr/go/engineeng.htm
Copybot: https://github.com/sirtango/ICopyMoves
CrazyPatterns: https://www.remi-coulom.fr/Amsterdam2007/
Dariush: http://ricoh51.free.fr/go/engineeng.htm
DarkForest: https://github.com/facebookresearch/darkforestGo
DarkGo: https://pjreddie.com/darknet/darkgo-go-in-darknet/
deltaGo: http://home.q00.itscom.net/otsuki/delta.html
Fudo Go: http://www.geocities.jp/hideki_katoh/
Fuego: http://fuego.sourceforge.net/
GnuGo: https://www.gnu.org/software/gnugo/devel.html
GoTraxx http://gotraxx.codeplex.com/
Hara: https://github.com/antoniogarro/Hara
Hiratuka: Non GPU version (10.37B): http://www.vector.co.jp/download/file/winnt/game/fh673259.html / GPU version (10.38B): http://www.vector.co.jp/download/file/winnt/game/fh688349.html
IndiGo: http://www.math-info.univ-paris5.fr/~bouzy/INDIGO.html
Iomrascálaí: https://github.com/ujh/iomrascalai
JrefBot: http://ricoh51.free.fr/go/engineeng.htm
Leela: https://sjeng.org/leela.html
Leela Zero: https://github.com/gcp/leela-zero
Marcos Go: https://github.com/MarcosPividori/Go-player
Matilda: https://github.com/gonmf/matilda
Michi C-2 https://github.com/db3108/michi-c2
MoGo: https://lifein19x19.com/forum/viewtopic.php?p=211091#p211091
Oakfoam: https://bitbucket.org/dsmic/oakfoam
Orego: https://sites.google.com/a/lclark.edu/drake/research/orego
Pachi: http://pachi.or.cz/
Rayon: https://github.com/zakki/Ray
Ray: https://github.com/kobanium/Ray
Stop: https://www.vanheusden.com/stop/

Best,
Alex


Last edited by as0770 on Wed Jan 10, 2018 11:02 pm, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #136 Posted: Fri Jan 05, 2018 3:05 pm 
Beginner

Posts: 1
Liked others: 3
Was liked: 0
Any idea how much time per move and how many playouts were being used by LZ?

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #137 Posted: Sat Jan 06, 2018 1:11 am 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
LetterRip wrote:
Any idea how much time per move and how many playouts were being used by LZ?


1h/game on one thread. Starting with 50sec/move what is around 7000 playouts.


This post by as0770 was liked by: LetterRip
Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #138 Posted: Sat Jan 06, 2018 7:32 am 
Lives with ko

Posts: 180
Liked others: 15
Was liked: 23
Rank: Beginner
q30 wrote:
The results (with pondering - without pondering):
MoGo 3 - 1;
Pachi 3 - 1;
Ray 3 - 1;
Leela 3 - 1;
in all 12 - 4 (details).
I don't know, what about quantitatively results (in ELO), but definitely there is qualitative effect, and in sparrings of equivalent strength Go engines the same with pondering may pass in rating engine without pondering.


One more thing about this test: If you play with one engine against itself the ponder hits are close to 100%, That means the pondering side will benefit a lot more than against other engines.

I did also some testing:

In my League A there are two engines that do not ponder: Hiratuka and DarkGo. DarkGo plays instandly, so it doesn't matter if the opponent is pondering because he don't get any time to ponder. So the only Engine that may be affected is Hiratuka, so I testet Hiratuka against Pachi in 30min games with ponder on and ponder off.

First Round:

Ponder Off: Pachi vs. Hiratuka 5:11
Ponder On: Pachi vs. Hiratuka 8:8

This could indeed be a hint that Pachi becomes significantly stronger with pondering. Because I don't believe this I did the same match again with the same conditions. Now I got:

Ponder Off: Pachi vs. Hiratuka 6:10
Ponder On: Pachi vs. Hiratuka 2:14

That is exactly what I expected: The statistical fluctuation when playing matches between engines with similar strength is very high, just like rolling a dice. The difference between pondering and not pondering is simply not meassurable with such a small amount of games.


This post by as0770 was liked by 2 people: LetterRip, Waylon
Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #139 Posted: Sat Jan 20, 2018 8:18 am 
Lives with ko

Posts: 135
Liked others: 1
Was liked: 1
Rank: 30 kyu
as0770 wrote:
q30 wrote:
... and in sparrings of equivalent strength Go engines the same with pondering may pass in rating engine without pondering.


So we agree that the question is only relevant in matches between engines where one is able to ponder and the other engine is not? ...
So for both the question of the absolute timecontrol and CPU power is much more relevant than the question of running it in ponder on or ponder off matches...
So where is your point always claiming others as "synthetic" results? ...


If both can ponder too, because one may ponder more effectively, than other...
Yes.
Only on results, obtained from tests with non realistic parameters, for example, time control. On engines with no big difference in strength the results may vary from real control parameters tests.

Top
 Profile  
 
Offline
 Post subject: Re: Engine Tournament
Post #140 Posted: Sat Jan 20, 2018 8:29 am 
Lives with ko

Posts: 135
Liked others: 1
Was liked: 1
Rank: 30 kyu
as0770 wrote:
...

That is exactly what I expected: The statistical fluctuation when playing matches between engines with similar strength is very high, just like rolling a dice. The difference between pondering and not pondering is simply not meassurable with such a small amount of games.

Try to use time control equivalent to 2 min per move. In this case fluctuations will be much smaller and difference between pondering and not pondering will be measurable...

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 390 posts ]  Go to page Previous  1 ... 4, 5, 6, 7, 8, 9, 10 ... 20  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group