How to run KataGo in Lizzie v0.7.2?

[yoyoma]

GPUS AND RAM
-I have set as you can see 60 GB, is this good?

By default, KataGo will cache up to about 3GB of positions in memory (RAM), in addition to whatever the current search is using.

I would keep the default 3GB. When doing long searches, the majority of RAM will be in the current search tree. You don't want to reserve too much for cache, leaving not enough room for the current search tree.

lightvector, I wonder if 3GB default is a little high if many users have 8GB RAM, and Windows + a few apps will leave only 4-5GB free?

[goame]

I just realized based on your output that the genconfig command has a misleading description - it will only be configured to use one of your GPUs. In the option where it asks you to select the devices to use, you should NOT use the default here, instead actually specify both devices as described, and let it re-tune based on that.

Sorry about that! I pushed a fix to master branch that clarifies the explanation here which will go out next release.

Additionally, I agree with you 700 is a bit small. It leans towards making it a bit friendly to users in terms of not taking too long, but maybe this is too much. You can manually specify a larger number when it asks you next time, when it says "Specify number of visits to use test/tune performance with".

To use it in Lizzie, you should take the "gtp" command it tells you at the end below "DONE" and tell Lizzie that this is the engine command. Although, you might need to adjust the paths if Lizzie is in a different directory, such that it will "see" your file system from a different than you were in when you tuned KataGo.

I'm surprised that you were getting 40K visits per second with LZ, unless you were using a smaller network with LZ, or using even more GPUs, or LZ was "cheating" by counted 8x as many visits on the opening empty board due to symmetries, which it would not be able to sustain once a few moves were played breaking the symmetry, or something else like that.

Ok then GPU 0,1 should fix it.

If I understand correctly, tuning with more visits leads to more accuracy but tuning takes more time?

Do you mean I should copy and paste this?:
LG0\Lizzie\katago\katago.exe gtp -model '\LG0\Lizzie\katago\g170-b30c320x2-s1287
828224-d525929064.bin.gz' -config 'gtp_custom.cfg'
How to tell Lizzie this is the engine command? Lizzie.jar, setting, engine, and past it there?

The 40K visits are ordinary at the beginning and going fast down, when "all playouts" becomes bigger. Maybe something like 1500 visits per second, after maybe 5-10 minutes, when all playouts(all visits) are above 3000000.

Should I tune for 3 GB RAM or 30 GB RAM?
I have 64 GB RAM.
What are the pros and cons?

[goame]

Tuning with 50000 visits:

Z:\>LG0\Lizzie\katago\katago.exe genconfig -model \LG0\Lizzie\katago\g170-b30c32
0x2-s1287828224-d525929064.bin.gz -output gtp_custom.cfg

=========================================================================
RULES

What rules should KataGo use by default for play and analysis?
(chinese, japanese, korean, tromp-taylor, aga, chinese-ogs, new-zealand, bga, st
one-scoring, aga-button):
japanese

=========================================================================
SEARCH LIMITS

When playing games, KataGo will always obey the time controls given by the GUI/t
ournament/match/online server.
But you can specify an additional limit to make KataGo move much faster. This do
es NOT affect analysis/review,
only affects playing games. Add a limit? (y/n) (default n):
n

NOTE: No limits configured for KataGo. KataGo will obey time controls provided b
y the GUI or server or match script
but if they don't specify any, when playing games KataGo may think forever witho
ut moving. (press enter to continue)


When playing games, KataGo can optionally ponder during the opponent's turn. Thi
s gives faster/stronger play
in real games but should NOT be enabled if you are running tests with fixed limi
ts (pondering may exceed those
limits), or to avoid stealing the opponent's compute time when testing two bots
on the same machine.
Enable pondering? (y/n, default n):y

Specify max num seconds KataGo should ponder during the opponent's turn. Leave b
lank for no limit:


=========================================================================
GPUS AND RAM

Finding available GPU-like devices...
Found CUDA device 0: GeForce RTX 2080 Ti
Found CUDA device 1: GeForce RTX 2080 Ti

Specify devices/GPUs to use (for example "0,1,2" to use devices 0, 1, and 2). Le
ave blank for good default:
"0,1"
could not parse int: "0
Specify devices/GPUs to use (for example "0,1,2" to use devices 0, 1, and 2). Le
ave blank for good default:
0,1

By default, KataGo will cache up to about 3GB of positions in memory (RAM), in a
ddition to
whatever the current search is using. Specify a max in GB or leave blank for def
ault:
60

=========================================================================
PERFORMANCE TUNING

Specify number of visits to use test/tune performance with, leave blank for defa
ult based on GPU speed.
Use large number for more accurate results, small if your GPU is old and this is
taking forever:
50000

Specify number of seconds/move to optimize performance for (default 5), leave bl
ank for default:

2020-03-12 22:55:26+0100: Loading model and initializing benchmark...

=========================================================================
TUNING NOW
Tuning using 50000 visits.
Automatically trying different numbers of threads to home in on the best:

2020-03-12 22:55:26+0100: nnRandSeed0 = 2369906978592220054
2020-03-12 22:55:26+0100: After dedups: nnModelFile0 = \LG0\Lizzie\katago\g170-b
30c320x2-s1287828224-d525929064.bin.gz useFP16 auto useNHWC auto
2020-03-12 22:55:28+0100: Cuda backend: Found GPU GeForce RTX 2080 Ti memory 118
11160064 compute capability major 7 minor 5
2020-03-12 22:55:28+0100: Cuda backend: Found GPU GeForce RTX 2080 Ti memory 118
11160064 compute capability major 7 minor 5
2020-03-12 22:55:28+0100: Cuda backend: Model version 8 useFP16 = true useNHWC =
true
2020-03-12 22:55:28+0100: Cuda backend: Model name: g170-b30c320x2-s1287828224-d
525929064
2020-03-12 22:55:28+0100: Cuda backend: Model version 8 useFP16 = true useNHWC =
true
2020-03-12 22:55:28+0100: Cuda backend: Model name: g170-b30c320x2-s1287828224-d
525929064


Possible numbers of threads to test: 1, 2, 3, 4, 5, 6, 8, 10, 12, 16, 20, 24, 32
,

numSearchThreads = 5: 10 / 10 positions, visits/s = 533.10 nnEvals/s = 350.16 n
nBatches/s = 213.88 avgBatchSize = 1.64 (938.0 secs)
numSearchThreads = 12: 10 / 10 positions, visits/s = 1131.75 nnEvals/s = 769.38
nnBatches/s = 198.99 avgBatchSize = 3.87 (441.9 secs)
numSearchThreads = 10: 10 / 10 positions, visits/s = 964.41 nnEvals/s = 649.12 n
nBatches/s = 204.31 avgBatchSize = 3.18 (518.5 secs)
numSearchThreads = 20: 10 / 10 positions, visits/s = 1520.41 nnEvals/s = 1003.61
nnBatches/s = 152.46 avgBatchSize = 6.58 (329.0 secs)
numSearchThreads = 16: 10 / 10 positions, visits/s = 1387.92 nnEvals/s = 932.16
nnBatches/s = 178.77 avgBatchSize = 5.21 (360.4 secs)
numSearchThreads = 24: 10 / 10 positions, visits/s = 1624.20 nnEvals/s = 1089.80
nnBatches/s = 136.46 avgBatchSize = 7.99 (308.0 secs)
numSearchThreads = 32: 10 / 10 positions, visits/s = 1796.26 nnEvals/s = 1201.35
nnBatches/s = 113.86 avgBatchSize = 10.55 (278.5 secs)


Optimal number of threads is fairly high, tripling the search limit and trying a
gain.

2020-03-12 23:49:10+0100: nnRandSeed0 = 6506758374797114957
2020-03-12 23:49:10+0100: After dedups: nnModelFile0 = \LG0\Lizzie\katago\g170-b
30c320x2-s1287828224-d525929064.bin.gz useFP16 auto useNHWC auto
2020-03-12 23:49:13+0100: Cuda backend: Found GPU GeForce RTX 2080 Ti memory 118
11160064 compute capability major 7 minor 5
2020-03-12 23:49:13+0100: Cuda backend: Found GPU GeForce RTX 2080 Ti memory 118
11160064 compute capability major 7 minor 5
2020-03-12 23:49:13+0100: Cuda backend: Model version 8 useFP16 = true useNHWC =
true
2020-03-12 23:49:13+0100: Cuda backend: Model name: g170-b30c320x2-s1287828224-d
525929064
2020-03-12 23:49:13+0100: Cuda backend: Model version 8 useFP16 = true useNHWC =
true
2020-03-12 23:49:13+0100: Cuda backend: Model name: g170-b30c320x2-s1287828224-d
525929064


Possible numbers of threads to test: 1, 2, 3, 4, 5, 6, 8, 10, 12, 16, 20, 24, 32
, 40, 48, 64, 80, 96,

numSearchThreads = 6: 10 / 10 positions, visits/s = 626.73 nnEvals/s = 407.14 n
nBatches/s = 209.06 avgBatchSize = 1.95 (797.9 secs)
numSearchThreads = 48: 10 / 10 positions, visits/s = 2214.93 nnEvals/s = 1421.03
nnBatches/s = 93.34 avgBatchSize = 15.22 (226.0 secs)
numSearchThreads = 64: 10 / 10 positions, visits/s = 2301.42 nnEvals/s = 1500.58
nnBatches/s = 77.43 avgBatchSize = 19.38 (217.5 secs)
numSearchThreads = 80: 10 / 10 positions, visits/s = 2322.34 nnEvals/s = 1543.88
nnBatches/s = 65.55 avgBatchSize = 23.55 (215.6 secs)
numSearchThreads = 40: 10 / 10 positions, visits/s = 1983.09 nnEvals/s = 1353.57
nnBatches/s = 104.84 avgBatchSize = 12.91 (252.3 secs)


Ordered summary of results:

numSearchThreads = 5: 10 / 10 positions, visits/s = 533.10 nnEvals/s = 350.16 n
nBatches/s = 213.88 avgBatchSize = 1.64 (938.0 secs) (EloDiff baseline)
numSearchThreads = 6: 10 / 10 positions, visits/s = 626.73 nnEvals/s = 407.14 n
nBatches/s = 209.06 avgBatchSize = 1.95 (797.9 secs) (EloDiff +57)
numSearchThreads = 10: 10 / 10 positions, visits/s = 964.41 nnEvals/s = 649.12 n
nBatches/s = 204.31 avgBatchSize = 3.18 (518.5 secs) (EloDiff +208)
numSearchThreads = 12: 10 / 10 positions, visits/s = 1131.75 nnEvals/s = 769.38
nnBatches/s = 198.99 avgBatchSize = 3.87 (441.9 secs) (EloDiff +264)
numSearchThreads = 16: 10 / 10 positions, visits/s = 1387.92 nnEvals/s = 932.16
nnBatches/s = 178.77 avgBatchSize = 5.21 (360.4 secs) (EloDiff +334)
numSearchThreads = 20: 10 / 10 positions, visits/s = 1520.41 nnEvals/s = 1003.61
nnBatches/s = 152.46 avgBatchSize = 6.58 (329.0 secs) (EloDiff +362)
numSearchThreads = 24: 10 / 10 positions, visits/s = 1624.20 nnEvals/s = 1089.80
nnBatches/s = 136.46 avgBatchSize = 7.99 (308.0 secs) (EloDiff +381)
numSearchThreads = 32: 10 / 10 positions, visits/s = 1796.26 nnEvals/s = 1201.35
nnBatches/s = 113.86 avgBatchSize = 10.55 (278.5 secs) (EloDiff +408)
numSearchThreads = 40: 10 / 10 positions, visits/s = 1983.09 nnEvals/s = 1353.57
nnBatches/s = 104.84 avgBatchSize = 12.91 (252.3 secs) (EloDiff +436)
numSearchThreads = 48: 10 / 10 positions, visits/s = 2214.93 nnEvals/s = 1421.03
nnBatches/s = 93.34 avgBatchSize = 15.22 (226.0 secs) (EloDiff +471)
numSearchThreads = 64: 10 / 10 positions, visits/s = 2301.42 nnEvals/s = 1500.58
nnBatches/s = 77.43 avgBatchSize = 19.38 (217.5 secs) (EloDiff +467)
numSearchThreads = 80: 10 / 10 positions, visits/s = 2322.34 nnEvals/s = 1543.88
nnBatches/s = 65.55 avgBatchSize = 23.55 (215.6 secs) (EloDiff +451)


Based on some test data, each speed doubling gains perhaps ~250 Elo by searching
deeper.
Based on some test data, each thread costs perhaps 7 Elo if using 800 visits, an
d 2 Elo if using 5000 visits (by making MCTS worse).
So APPROXIMATELY based on this benchmark, if you intend to do a 5 second search:

numSearchThreads = 5: (baseline)
numSearchThreads = 6: +57 Elo
numSearchThreads = 10: +208 Elo
numSearchThreads = 12: +264 Elo
numSearchThreads = 16: +334 Elo
numSearchThreads = 20: +362 Elo
numSearchThreads = 24: +381 Elo
numSearchThreads = 32: +408 Elo
numSearchThreads = 40: +436 Elo
numSearchThreads = 48: +471 Elo (recommended)
numSearchThreads = 64: +467 Elo
numSearchThreads = 80: +451 Elo

Using 48 numSearchThreads!

=========================================================================
DONE

Writing new config file to gtp_custom.cfg
You should be now able to run KataGo with this config via something like:
LG0\Lizzie\katago\katago.exe gtp -model '\LG0\Lizzie\katago\g170-b30c320x2-s1287
828224-d525929064.bin.gz' -config 'gtp_custom.cfg'

Feel free to look at and edit the above config file further by hand in a txt edi
tor.
For more detailed notes about performance and what options in the config do, see
:
https://github.com/lightvector/KataGo/b ... xample.cfg

[goame]

I copy and paste this:
LG0\Lizzie\katago\katago.exe gtp -model '\LG0\Lizzie\katago\g170-b30c320x2-s1287
828224-d525929064.bin.gz' -config 'gtp_custom.cfg'

into:
lizzie.jar, settings, engine config

but something is still wrong and KataGo is not analysing.

[Javaness2]

Is it better to just use this?
https://github.com/kaorahi/lizgoban

[ez4u]

I copy and paste this:
LG0\Lizzie\katago\katago.exe gtp -model '\LG0\Lizzie\katago\g170-b30c320x2-s1287
828224-d525929064.bin.gz' -config 'gtp_custom.cfg'

into:
lizzie.jar, settings, engine config

but something is still wrong and KataGo is not analysing.

This is strange. You should be editing lizzie's config.txt not the jar file.

If you want to start with katago, edit the engine command in config.txt, for example mine says:

Code: Select all

"engine-command": "./katago133open/katago gtp -model ./katago133open/g170e-15b-s1672.bin.gz -config ./katago133open/gtp_jprules_15b.cfg",

Otherwise (and) add the same string and any other flavors to

Code: Select all

"engine-command-list": [
"./leelazero/leelaz --gtp --lagbuffer 0 --weights lznetwork268.gz",
"./leelazero/leelaz-minvisits --gtp --lagbuffer 0 --weights lznetwork268.gz",
"./katago133open/katago gtp -model ./katago133open/g170e-10b-s1141.bin.gz -config ./katago133open/gtp_jprules_10b.cfg",
"./katago133open/katago gtp -model ./katago133open/g170e-15b-s1672.bin.gz -config ./katago133open/gtp_jprules_15b.cfg",
"./katago133open/katago gtp -model ./katago133open/g170e-20b-s2430.bin.gz -config ./katago133open/gtp_jprules_20b.cfg",
""
],

Run lizzie from a command window. If things are not working in lizzie, check the command window. If you see "Engine process ended.", then in my experience there is most likely something wrong with your command string for katago.

In your case, is it correct that you have these three flavors:
(no back slash)LGO\...\katago.exe
'\LGO\...\g170-b30c320x2-s1287828224-d525929064.bin.gz'
'gtp_custom.cfg'
Are your model, your config file, and katago.exe in different directories?

[yoyoma]

Specify devices/GPUs to use (for example "0,1,2" to use devices 0, 1, and 2). Le
ave blank for good default:
"0,1"
could not parse int: "0

For GPUs you need to type 0,1 without the quotes around it.

[goame]

To fix the problem, I have deleted all things and I have done a new install of Lizzie and Katago.
I'm using the gtp_custom.cfg which I created last time.
I changed the name into katanetwork.bin.gz

= Now it works fine.

Some questions:
1. When I look at the board I see something like:
43,1
56k
-1,1 (-> what is the meaning of this one?)

2. How to avoid preload of Leela Zero?
I want to use KataGo, running from the beginning.

3. Why KataGo is doing only a tenth of visits per second, compared to Leela Zero?
g170-b30c320x2-s1287828224-d525929064.bin.gz
Leela Zero 40x256 nets

4. Which things should or could I change in Lizzie to improve the (analysis).

5. What can I change or try in the config.txt file?

{
"leelaz": {
"engine-preload-list": [
true,
false,
false,
false,
false,
false,
false,
false,
false
],
"engine-command-list": [
"./katago/katago gtp -model katanetwork.bin.gz -config gtp_custom.cfg",
"",
"",
"",
"",
"",
"",
"",
""
],
"max-analyze-time-minutes": 99999,
"analyze-update-interval-centisec": 10,
"network-file": "lznetwork.gz",
"_comment": "note, network-file is obselete in Lizzie 0.7+, ignore network-file, kept for compatibility",
"max-game-thinking-time-seconds": 2,
"engine-start-location": ".",
"avoid-keep-variations": 15,
"engine-command": "./leelazero/leelaz --gtp --lagbuffer 0 --weights lznetwork.gz",
"print-comms": false,
"show-lcb-winrate": false
},
"ui": {
"comment-font-size": 0,
"hold-bestmoves-to-sgf": true,
"shadow-size": 100,
"show-winrate": true,
"autosave-interval-seconds": -1,
"limit-best-move-num": 0,
"stone-indicator-type": 1,
"win-rate-always-black": false,
"board-width": 19,
"show-border": false,
"show-move-number": false,
"winrate-stroke-width": 3,
"show-next-moves": true,
"comment-node-color": [
0,
0,
255
],
"show-comment": true,
"show-leelaz-variation": true,
"show-bestmoves-by-hold": true,
"min-playout-ratio-for-stats": 0.1,
"fancy-stones": true,
"resume-previous-game": false,
"comment-font-color": [
255,
255,
255
],
"show-coordinates": false,
"shadows-enabled": true,
"show-katago-estimate-onsubboard": false,
"show-variation-graph": true,
"show-dynamic-komi": true,
"gtp-console-style": "body {background:#000000; color:#d0d0d0; font-family:Consolas, Menlo, Monaco, 'Ubuntu Mono', monospace; margin:4px;} .command {color:#ffffff;font-weight:bold;} .winrate {color:#ffffff;font-weight:bold;} .coord {color:#ffffff;font-weight:bold;}",
"katago-scoremean-alwaysblack": false,
"katago-notshow-winrate": false,
"minimum-blunder-bar-width": 3,
"large-winrate": false,
"confirm-exit": false,
"show-katago-estimate-onmainboard": true,
"scoremean-line-color": [
255,
0,
255
],
"show-katago-estimate": true,
"show-best-moves": true,
"board-color": [
217,
152,
77
],
"append-winrate-to-comment": false,
"fancy-board": true,
"color-by-winrate-instead-of-visits": false,
"show-captured": true,
"replay-branch-interval-seconds": 1,
"panel-ui": false,
"blunder-bar-color": [
255,
0,
0,
150
],
"weighted-blunder-bar-height": false,
"katago-estimate-mode": "small+dead",
"theme": "Default",
"show-winrate-in-suggestion": true,
"show-scoremean-in-suggestion": true,
"new-move-number-in-branch": true,
"winrate-line-color": [
0,
255,
0
],
"blunder-node-colors": [],
"minPlayoutRatioForStats": 0.1,
"show-katago-boardscoremean": false,
"show-playouts-in-suggestion": true,
"limit-branch-length": 0,
"blunder-winrate-thresholds": [],
"board-position-proportion": 4,
"show-blunder-bar": false,
"only-last-move-number": 0,
"board-height": 19,
"winrate-miss-line-color": [
0,
0,
178
],
"show-status": true,
"handicap-instead-of-winrate": false,
"large-subboard": false,
"comment-background-color": [
0,
0,
0,
200
],
"dynamic-winrate-graph-width": false,
"show-subboard": true,
"show-katago-scoremean": true,
"show-comment-node-color": false,
"board-size": 19
}
}

[lightvector]

Glad you got it working. It sounds like you're new to some ways of using computers. All of your earlier troubles seem to be related to how to specify file paths, which it sounds like you don't understand. The main idea is when you input a file path, you need to say what the file path is from the perspective of your current directory or the directory of the program using it. So if you had directory A containing B, and B containing both C and D, and C contains the executable you're trying to run, and D contains a text file you want to provide to the executable inside C, then IF you are in directory A, you need to do:

B\C\the_executable_name.exe B\D\the_text_file.txt

But if you were already in directory B, the above would not work and instead you would need:

C\the_executable_name.exe D\the_text_file.txt

And suppose a program like Lizzie were inside directory D, so it would see things from directory D's perspective. Then if you wanted that program to know the right paths, you would need:

..\C\the_executable_name.exe the_text_file.txt

(".." means "go back out the directory that contains this one", so since we're already inside D, to find C, we need to go back OUT to B, then go down into C.)

Does that make sense? (This applies generally to any computer usage, not just Go-related).

Your questions:

1. The meaning of that is the lead in points. So in your case, black (or the player to move, depending on your settings) is behind slightly, by about 1 point.

2. In the config, swap KataGo and Leela Zero in the engine-command-list versus the engine-command. Be very careful to get all the quotations and punctuation correct, otherwise the config will not parse and Lizzie might not load or work at at all. (make a backup of your config file before editing if necessary)

3. How are you measuring Leela Zero's visits per second? KataGo is slower than Leela Zero not because the network is bigger or worse, but simply because it has less expertise on the GPU code (if an expert wanted to help improve the code, it might immediately become faster) - but should not be 10x slower. For example, you are looking at the empty board position, then Leela Zero is inflated by 8x compared to almost all of the rest of the game because it is treating board symmetries as interchangeable. As soon as a real game starts and the board stops being symmetrical, this will stop being the case.

4,5. Entirely up to you. Experiment with the settings to see what you like. Probably you should do so using Lizzie's GUI if you are afraid of making a typo and breaking your config file.

[goame]

Is it better to just use this?
https://github.com/kaorahi/lizgoban

For me this one is much more difficult than Lizzie.

[goame]

Glad you got it working. It sounds like you're new to some ways of using computers. All of your earlier troubles seem to be related to how to specify file paths, which it sounds like you don't understand. The main idea is when you input a file path, you need to say what the file path is from the perspective of your current directory or the directory of the program using it. So if you had directory A containing B, and B containing both C and D, and C contains the executable you're trying to run, and D contains a text file you want to provide to the executable inside C, then IF you are in directory A, you need to do:

B\C\the_executable_name.exe B\D\the_text_file.txt

But if you were already in directory B, the above would not work and instead you would need:

C\the_executable_name.exe D\the_text_file.txt

And suppose a program like Lizzie were inside directory D, so it would see things from directory D's perspective. Then if you wanted that program to know the right paths, you would need:

..\C\the_executable_name.exe the_text_file.txt

(".." means "go back out the directory that contains this one", so since we're already inside D, to find C, we need to go back OUT to B, then go down into C.)

Does that make sense? (This applies generally to any computer usage, not just Go-related).

Your questions:

1. The meaning of that is the lead in points. So in your case, black (or the player to move, depending on your settings) is behind slightly, by about 1 point.

2. In the config, swap KataGo and Leela Zero in the engine-command-list versus the engine-command. Be very careful to get all the quotations and punctuation correct, otherwise the config will not parse and Lizzie might not load or work at at all. (make a backup of your config file before editing if necessary)

3. How are you measuring Leela Zero's visits per second? KataGo is slower than Leela Zero not because the network is bigger or worse, but simply because it has less expertise on the GPU code (if an expert wanted to help improve the code, it might immediately become faster) - but should not be 10x slower. For example, you are looking at the empty board position, then Leela Zero is inflated by 8x compared to almost all of the rest of the game because it is treating board symmetries as interchangeable. As soon as a real game starts and the board stops being symmetrical, this will stop being the case.

4,5. Entirely up to you. Experiment with the settings to see what you like. Probably you should do so using Lizzie's GUI if you are afraid of making a typo and breaking your config file.

Thx I have learned something new.
I'm not using the command line 99.99% of my time.

1. From 43.3% to 56.7% we have a difference of 13.4%.
Does that mean If we are 13.4% behind our opponent, we are only 1 point behind?

2. I will let it as it is now.

3. I run Leela Zero and can see the visits per second at the top.
Maybe that's because I'm using two RTX 2080 Ti and maybe with other hardware it's only 8 times faster.
Of course they are very high at the beginning but on my very strong/fast hardware, the 64 GB RAM is filled also very fast and the visits per second goes also down very fast.
I think that after the speed improvements are done up to the level of Leela Zeros speed code, every KataGo net would play at least 162.5 Elo stronger.

You should do two things:
1. Open a "new topic" and ask for help about improving code to increase the speed of KataGo.
If you don't do this, I think 99.99% will not know that you need help, so they can't help you.
2. Also ask Gian-Carlo Pascutto for help.

[lightvector]

3. I run Leela Zero and can see the visits per second at the top.
Maybe that's because I'm using two RTX 2080 Ti and maybe with other hardware it's only 8 times faster.
Of course they are very high at the beginning but on my very strong/fast hardware, the 64 GB RAM is filled also very fast and the visits per second goes also down very fast.
I think that after the speed improvements are done up to the level of Leela Zeros speed code, every KataGo net would play at least 162.5 Elo stronger.

I think you misunderstand. If you are using the empty board to evaluate visits/second, then for Leela Zero, the reason its visits/second is so large has nothing to do with a good GPU implementation. It's because Leela Zero is inflated by 8x due to board symmetries. This 8x will help Leela Zero search deeper for... choosing which corner to play, choosing 3-4 or 4-4, and maybe for choosing the second corner and the first 3-3 invasion (for the latter two, it will be at best 2x, not 8x). But a bot with fewer visits can easily make those same moves anyways, deep search isn't helping for the first 3 moves. So there is a negligible effect on strength on these moves, and the total gain over the game might be at best a couple percent of time saved due to tree reuse.

Precisely because it's only a couple percent gain, I never bothered to implement this in KataGo. It's extra complexity for only slight benefit, it's not a "true" 8x improvement.

If you want to compare fairly, you need to manually divide Leela Zero's number by about 8. Or you need to benchmark on a position 5-10 moves in that is not symmetrical instead of the empty board.
(if you are already doing these things, then I'm surprised and maybe it's worth investigating.)

[Bill Spight]

But a bot with fewer visits can easily make those same moves anyways, deep search isn't helping for the first 3 moves.

I guess you mean the top three choices?

[lightvector]

No, I mean the first 3 moves. It's 8x on the first move, and then after a corner is occupied, depending on 4-4 or 3-4 it will be 2x or immediately 1x. It may stay 2x if the opponent takes the diagonally opposing 4-4, and also if the game develops to all 4-4 points. Thereafter, probably symmetry is broken for that game forever.

So about for the first three moves that, say, Black, will make in a game, you can have a speedup, and for two of them, it's at best 2x (only for 4-4 points), not 8x.

Deep search isn't helping isn't really helping the bot to make good choices here. The differences between opening corner moves are extremely slight, even for bots, and a bot with shallow search will already typically make the same reasonable choices. There aren't any tactics yet where evaluations will swing a lot or search will massively override the policy prior about moves. The main gain is the percent or two saving of time for the later game where search really will matter.

[goame]

3. I run Leela Zero and can see the visits per second at the top.
Maybe that's because I'm using two RTX 2080 Ti and maybe with other hardware it's only 8 times faster.
Of course they are very high at the beginning but on my very strong/fast hardware, the 64 GB RAM is filled also very fast and the visits per second goes also down very fast.
I think that after the speed improvements are done up to the level of Leela Zeros speed code, every KataGo net would play at least 162.5 Elo stronger.

I think you misunderstand. If you are using the empty board to evaluate visits/second, then for Leela Zero, the reason its visits/second is so large has nothing to do with a good GPU implementation. It's because Leela Zero is inflated by 8x due to board symmetries. This 8x will help Leela Zero search deeper for... choosing which corner to play, choosing 3-4 or 4-4, and maybe for choosing the second corner and the first 3-3 invasion (for the latter two, it will be at best 2x, not 8x). But a bot with fewer visits can easily make those same moves anyways, deep search isn't helping for the first 3 moves. So there is a negligible effect on strength on these moves, and the total gain over the game might be at best a couple percent of time saved due to tree reuse.

Precisely because it's only a couple percent gain, I never bothered to implement this in KataGo. It's extra complexity for only slight benefit, it's not a "true" 8x improvement.

If you want to compare fairly, you need to manually divide Leela Zero's number by about 8. Or you need to benchmark on a position 5-10 moves in that is not symmetrical instead of the empty board.
(if you are already doing these things, then I'm surprised and maybe it's worth investigating.)

I checked it with a deeper position and without symmetries.
Leela Zero is 2x faster.
That means to improve the speed to Leela Zeros level will give all KataGo nets an improvement of 50 Elo.
Even more are possible, because I don't think that Leela Zeros speed is perfect.

I think the 8x speed gain at the beginning is also very important, because the depth is much higher = better moves. Even if the opponent can find this moves too, he will have later a lower depth and the chance that he overlooks the best moves is much higher.
This 8x speed improvement gains the main benefit not in the beginning but some moves later.