Run lizzie from a command window. If things are not working in lizzie, check the command window. If you see "Engine process ended.", then in my experience there is most likely something wrong with your command string for katago.
In your case, is it correct that you have these three flavors:
(no back slash)LGO\...\katago.exe
'\LGO\...\g170-b30c320x2-s1287828224-d525929064.bin.gz'
'gtp_custom.cfg'
Are your model, your config file, and katago.exe in different directories?
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21
To fix the problem, I have deleted all things and I have done a new install of Lizzie and Katago.
I'm using the gtp_custom.cfg which I created last time.
I changed the name into katanetwork.bin.gz
= Now it works fine.
Some questions:
1. When I look at the board I see something like:
43,1
56k
-1,1 (-> what is the meaning of this one?)
2. How to avoid preload of Leela Zero?
I want to use KataGo, running from the beginning.
3. Why KataGo is doing only a tenth of visits per second, compared to Leela Zero?
g170-b30c320x2-s1287828224-d525929064.bin.gz
Leela Zero 40x256 nets
4. Which things should or could I change in Lizzie to improve the (analysis).
5. What can I change or try in the config.txt file?
Glad you got it working. It sounds like you're new to some ways of using computers. All of your earlier troubles seem to be related to how to specify file paths, which it sounds like you don't understand. The main idea is when you input a file path, you need to say what the file path is from the perspective of your current directory or the directory of the program using it. So if you had directory A containing B, and B containing both C and D, and C contains the executable you're trying to run, and D contains a text file you want to provide to the executable inside C, then IF you are in directory A, you need to do:
B\C\the_executable_name.exe B\D\the_text_file.txt
But if you were already in directory B, the above would not work and instead you would need:
C\the_executable_name.exe D\the_text_file.txt
And suppose a program like Lizzie were inside directory D, so it would see things from directory D's perspective. Then if you wanted that program to know the right paths, you would need:
..\C\the_executable_name.exe the_text_file.txt
(".." means "go back out the directory that contains this one", so since we're already inside D, to find C, we need to go back OUT to B, then go down into C.)
Does that make sense? (This applies generally to any computer usage, not just Go-related).
Your questions:
1. The meaning of that is the lead in points. So in your case, black (or the player to move, depending on your settings) is behind slightly, by about 1 point.
2. In the config, swap KataGo and Leela Zero in the engine-command-list versus the engine-command. Be very careful to get all the quotations and punctuation correct, otherwise the config will not parse and Lizzie might not load or work at at all. (make a backup of your config file before editing if necessary)
3. How are you measuring Leela Zero's visits per second? KataGo is slower than Leela Zero not because the network is bigger or worse, but simply because it has less expertise on the GPU code (if an expert wanted to help improve the code, it might immediately become faster) - but should not be 10x slower. For example, you are looking at the empty board position, then Leela Zero is inflated by 8x compared to almost all of the rest of the game because it is treating board symmetries as interchangeable. As soon as a real game starts and the board stops being symmetrical, this will stop being the case.
4,5. Entirely up to you. Experiment with the settings to see what you like. Probably you should do so using Lizzie's GUI if you are afraid of making a typo and breaking your config file.
lightvector wrote:Glad you got it working. It sounds like you're new to some ways of using computers. All of your earlier troubles seem to be related to how to specify file paths, which it sounds like you don't understand. The main idea is when you input a file path, you need to say what the file path is from the perspective of your current directory or the directory of the program using it. So if you had directory A containing B, and B containing both C and D, and C contains the executable you're trying to run, and D contains a text file you want to provide to the executable inside C, then IF you are in directory A, you need to do:
B\C\the_executable_name.exe B\D\the_text_file.txt
But if you were already in directory B, the above would not work and instead you would need:
C\the_executable_name.exe D\the_text_file.txt
And suppose a program like Lizzie were inside directory D, so it would see things from directory D's perspective. Then if you wanted that program to know the right paths, you would need:
..\C\the_executable_name.exe the_text_file.txt
(".." means "go back out the directory that contains this one", so since we're already inside D, to find C, we need to go back OUT to B, then go down into C.)
Does that make sense? (This applies generally to any computer usage, not just Go-related).
Your questions:
1. The meaning of that is the lead in points. So in your case, black (or the player to move, depending on your settings) is behind slightly, by about 1 point.
2. In the config, swap KataGo and Leela Zero in the engine-command-list versus the engine-command. Be very careful to get all the quotations and punctuation correct, otherwise the config will not parse and Lizzie might not load or work at at all. (make a backup of your config file before editing if necessary)
3. How are you measuring Leela Zero's visits per second? KataGo is slower than Leela Zero not because the network is bigger or worse, but simply because it has less expertise on the GPU code (if an expert wanted to help improve the code, it might immediately become faster) - but should not be 10x slower. For example, you are looking at the empty board position, then Leela Zero is inflated by 8x compared to almost all of the rest of the game because it is treating board symmetries as interchangeable. As soon as a real game starts and the board stops being symmetrical, this will stop being the case.
4,5. Entirely up to you. Experiment with the settings to see what you like. Probably you should do so using Lizzie's GUI if you are afraid of making a typo and breaking your config file.
Thx I have learned something new.
I'm not using the command line 99.99% of my time.
1. From 43.3% to 56.7% we have a difference of 13.4%.
Does that mean If we are 13.4% behind our opponent, we are only 1 point behind?
2. I will let it as it is now.
3. I run Leela Zero and can see the visits per second at the top.
Maybe that's because I'm using two RTX 2080 Ti and maybe with other hardware it's only 8 times faster.
Of course they are very high at the beginning but on my very strong/fast hardware, the 64 GB RAM is filled also very fast and the visits per second goes also down very fast.
I think that after the speed improvements are done up to the level of Leela Zeros speed code, every KataGo net would play at least 162.5 Elo stronger.
You should do two things:
1. Open a "new topic" and ask for help about improving code to increase the speed of KataGo.
If you don't do this, I think 99.99% will not know that you need help, so they can't help you.
2. Also ask Gian-Carlo Pascutto for help.
3. I run Leela Zero and can see the visits per second at the top.
Maybe that's because I'm using two RTX 2080 Ti and maybe with other hardware it's only 8 times faster.
Of course they are very high at the beginning but on my very strong/fast hardware, the 64 GB RAM is filled also very fast and the visits per second goes also down very fast.
I think that after the speed improvements are done up to the level of Leela Zeros speed code, every KataGo net would play at least 162.5 Elo stronger.
I think you misunderstand. If you are using the empty board to evaluate visits/second, then for Leela Zero, the reason its visits/second is so large has nothing to do with a good GPU implementation. It's because Leela Zero is inflated by 8x due to board symmetries. This 8x will help Leela Zero search deeper for... choosing which corner to play, choosing 3-4 or 4-4, and maybe for choosing the second corner and the first 3-3 invasion (for the latter two, it will be at best 2x, not 8x). But a bot with fewer visits can easily make those same moves anyways, deep search isn't helping for the first 3 moves. So there is a negligible effect on strength on these moves, and the total gain over the game might be at best a couple percent of time saved due to tree reuse.
Precisely because it's only a couple percent gain, I never bothered to implement this in KataGo. It's extra complexity for only slight benefit, it's not a "true" 8x improvement.
If you want to compare fairly, you need to manually divide Leela Zero's number by about 8. Or you need to benchmark on a position 5-10 moves in that is not symmetrical instead of the empty board.
(if you are already doing these things, then I'm surprised and maybe it's worth investigating.)
No, I mean the first 3 moves. It's 8x on the first move, and then after a corner is occupied, depending on 4-4 or 3-4 it will be 2x or immediately 1x. It may stay 2x if the opponent takes the diagonally opposing 4-4, and also if the game develops to all 4-4 points. Thereafter, probably symmetry is broken for that game forever.
So about for the first three moves that, say, Black, will make in a game, you can have a speedup, and for two of them, it's at best 2x (only for 4-4 points), not 8x.
Deep search isn't helping isn't really helping the bot to make good choices here. The differences between opening corner moves are extremely slight, even for bots, and a bot with shallow search will already typically make the same reasonable choices. There aren't any tactics yet where evaluations will swing a lot or search will massively override the policy prior about moves. The main gain is the percent or two saving of time for the later game where search really will matter.
3. I run Leela Zero and can see the visits per second at the top.
Maybe that's because I'm using two RTX 2080 Ti and maybe with other hardware it's only 8 times faster.
Of course they are very high at the beginning but on my very strong/fast hardware, the 64 GB RAM is filled also very fast and the visits per second goes also down very fast.
I think that after the speed improvements are done up to the level of Leela Zeros speed code, every KataGo net would play at least 162.5 Elo stronger.
I think you misunderstand. If you are using the empty board to evaluate visits/second, then for Leela Zero, the reason its visits/second is so large has nothing to do with a good GPU implementation. It's because Leela Zero is inflated by 8x due to board symmetries. This 8x will help Leela Zero search deeper for... choosing which corner to play, choosing 3-4 or 4-4, and maybe for choosing the second corner and the first 3-3 invasion (for the latter two, it will be at best 2x, not 8x). But a bot with fewer visits can easily make those same moves anyways, deep search isn't helping for the first 3 moves. So there is a negligible effect on strength on these moves, and the total gain over the game might be at best a couple percent of time saved due to tree reuse.
Precisely because it's only a couple percent gain, I never bothered to implement this in KataGo. It's extra complexity for only slight benefit, it's not a "true" 8x improvement.
If you want to compare fairly, you need to manually divide Leela Zero's number by about 8. Or you need to benchmark on a position 5-10 moves in that is not symmetrical instead of the empty board.
(if you are already doing these things, then I'm surprised and maybe it's worth investigating.)
I checked it with a deeper position and without symmetries.
Leela Zero is 2x faster.
That means to improve the speed to Leela Zeros level will give all KataGo nets an improvement of 50 Elo.
Even more are possible, because I don't think that Leela Zeros speed is perfect.
I think the 8x speed gain at the beginning is also very important, because the depth is much higher = better moves. Even if the opponent can find this moves too, he will have later a lower depth and the chance that he overlooks the best moves is much higher.
This 8x speed improvement gains the main benefit not in the beginning but some moves later.
goame wrote:
I checked it with a deeper position and without symmetries.
Leela Zero is 2x faster.
That means to improve the speed to Leela Zeros level will give all KataGo nets an improvement of 50 Elo.
Even more are possible, because I don't think that Leela Zeros speed is perfect.
I think the 8x speed gain at the beginning is also very important, because the depth is much higher = better moves. Even if the opponent can find this moves too, he will have later a lower depth and the chance that he overlooks the best moves is much higher.
This 8x speed improvement gains the main benefit not in the beginning but some moves later.
Thanks, yes, 2x sounds more reasonable, about in line with how much one might imagine optimizing the GPU implementation in the future. I think you're mostly mistaken about how much the 8x matters. As I mentioned before, it should be equivalent to maybe being able to save 1% to 2% on one's time control due to tree reuse, or equivalently, 1% to 2% speedup. Nice, but not a big deal right now, particularly if it causes users to be confused about the performance of the bot sometimes.
lightvector wrote:No, I mean the first 3 moves. It's 8x on the first move, and then after a corner is occupied, depending on 4-4 or 3-4 it will be 2x or immediately 1x. It may stay 2x if the opponent takes the diagonally opposing 4-4, and also if the game develops to all 4-4 points. Thereafter, probably symmetry is broken for that game forever.
So about for the first three moves that, say, Black, will make in a game, you can have a speedup, and for two of them, it's at best 2x (only for 4-4 points), not 8x.
Symmetry breaking is one reason behind my question about focused (or restricted) search. E. g., at move one, restrict search to the right-top-right section, Aside from possible speedup, doing so during training makes an asymmetric neural network expected. (To train the bot to make move one in that section it is probably best to allow search anywhere, but to penalize choices outside it.) Whether that matters is another question. I didn't think that 2x symmetry at move two or later would make much of a difference.
The Adkins Principle: At some point, doesn't thinking have to go on?
— Winona Adkins
