It is currently Thu Apr 25, 2019 7:06 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 1 post ] 
Author Message
Offline
 Post subject: Performance comparison between Leela Zero 0.16 and 0.17
Post #1 Posted: Sat Mar 23, 2019 11:41 am 
Lives in gote

Posts: 553
Location: Vienna, Austria
Liked others: 247
Was liked: 285
I ran benchmarks for Leela Zero 0.16 - the current version - and 0.17, which is what the 'next' branch will become once it's released. The benchmarks span various networks and thread settings.

Hardware: MacBook Pro 2017, 16 GB RAM, Quad-core i7, Radeon Pro 555

Command:
Code:
leelaz -w <network.gz> -t <number-of-threads> --benchmark


The "15b-trained-on-40b" network is the latest 15-block network trained on 40-block games, as described in https://github.com/leela-zero/leela-zero/issues/2192

Network: 214
Code:
version   no -t arg   1 thread   2 threads   3 threads   4 threads   5 threads   6 threads   7 threads   8 threads   9 threads   10 threads   11 threads   12 threads
0.16      34 n/s      34 n/s     36 n/s      36 n/s      36 n/s      36 n/s      37 n/s      36 n/s      36 n/s      37 n/s      37 n/s       37 n/s       37 n/s
0.17      63 n/s      35 n/s     37 n/s      45 n/s      46 n/s      48 n/s      49 n/s      50 n/s      53 n/s      63 n/s      59 n/s       60 n/s       59 n/s


Network: ELFv1
Code:
version   no -t arg   1 thread   2 threads   3 threads   4 threads   5 threads   6 threads   7 threads   8 threads   9 threads   10 threads   11 threads   12 threads
0.16       85 n/s     85 n/s     95 n/s       98 n/s     105 n/s     106 n/s     104 n/s     101 n/s     101 n/s     110 n/s     103 n/s      106 n/s      104 n/s
0.17      160 n/s     87 n/s     95 n/s      106 n/s     112 n/s     123 n/s     126 n/s     133 n/s     129 n/s     163 n/s     161 n/s      158 n/s      159 n/s


Network: ELFv2
Code:
version   no -t arg   1 thread   2 threads   3 threads   4 threads   5 threads   6 threads   7 threads   8 threads   9 threads   10 threads   11 threads   12 threads
0.16       56 n/s     56 n/s     61 n/s      62 n/s      65 n/s      65 n/s      64 n/s      65 n/s      64 n/s       64 n/s      65 n/s      65 n/s        66 n/s
0.17      104 n/s     58 n/s     65 n/s      72 n/s      76 n/s      76 n/s      79 n/s      89 n/s      91 n/s      108 n/s     109 n/s      99 n/s       106 n/s


Network: 15b-trained-on-40b
Code:
version   no -t arg   1 thread   2 threads   3 threads   4 threads   5 threads   6 threads   7 threads   8 threads   9 threads   10 threads   11 threads   12 threads
0.16      117 n/s     117 n/s    132 n/s     142 n/s     139 n/s     148 n/s     148 n/s     150 n/s     147 n/s     149 n/s     148 n/s      146 n/s      151 n/s
0.17      236 n/s     120 n/s    139 n/s     154 n/s     173 n/s     174 n/s     181 n/s     200 n/s     206 n/s     223 n/s     238 n/s      236 n/s      240 n/s


Here are my conclusions, which may only be appropriate for this particular OS and GPU.

1) Version 0.17 is much better at scaling with threads. Especially with the 40-block #214 version 0.16 doesn't scale at all.

2) With 0.17, it doesn't seem necessary to specify the number of threads on the command line; it finds the optimum - in my case, 9 or 10 threads - by itself. But version 0.16 only seems to use 1 thread by default.

3) At optimum thread settings, version 0.17 manages between 45% and 70% more playouts than version 0.16.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 1 post ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: Bing [Bot] and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group