KataGo V1.3

[Tryss]

At how many playing games (approximate numbers) did the 10b capped out?

[lightvector]

I'd guess somewhere around 4-5 million games is when I stopped training it. It's a little annoying to check more precisely, since all my data is recorded in terms of number data rows, not games. Only about half of these games were 19x19, mind you. The rest are on smaller board sizes.

[lightvector]

v1.3.2 is now released. If you're using the OpenCL version, you should get a noticeable performance boost. And along with stronger nets - new net gains another 100ish Elo over the old one from a week ago, at least in self-play testing, and there is a very slightly better 15-block net too.

If you were finding KataGo to be roughly evenly matched with LZ before, then with the speed boost and the better nets here hopefully this puts KataGo more clearly ahead.

As usual, find things here:
https://github.com/lightvector/KataGo/releases

If you find a bug or something silly that I messed up in this release that isn't working, poke me and I will fix it. Enjoy!

[explo]

v1.3.2 is now released. If you're using the OpenCL version, you should get a noticeable performance boost.

I have a GTX 1660 Ti GPU but I have issues when trying to install the CUDA version. I didn't report it because the OpenCL version is working fine and I have no experience with compiling so helping me would have been a hassle. That being said, I wondered if I was missing out on performance.
I can confirm the boost in performance. With the 1.3.1 katago and 15 blocks, Lizzie reports about 500 visits per second. With the 1.3.2 katago and new 15 blocks, Lizzie shows around 800 visits per second.

Edit: Spent some more time to explore the CUDA version. Here is what I've done to set up katago so far:
- I downloaded the lastest version of Lizzie not long ago.
- I donwloaded the new version(s) of katago and put them in new folders
- I added new engines in Lizzie, pointing to the correct katago folder and net
This has been working fine with the OpenCL versions so far.
I had done the same with the CUDA versions but in Lizzie the engine was loading forever. Trying to execute the katago file would prompt a missing dll file. I finally (today) found that dll file and copied it in the katago folder. The engine loaded But it was super slow (about 100 visits per second). I guess I need to tweak the parameters but I don't know how. It seems it was done automatically with OpenCL.

[lightvector]

I'm not exactly sure why it would be slow. There aren't actually that many parameters you can tune that would have that big of an effect, just one mainly: have you run the benchmark command in a console/command window to select a number of numSearchThreads to use? (See the readme, and/or the main page https://github.com/lightvector/KataGo#how-to-use for the command).

Also, is it automatically using FP16?

Edit: Oh, interesting, looks like according to this: https://www.tomshardware.com/reviews/nv ... ,6002.html GTX 1660 Ti doesn't have any tensor cores, despite being on a high compute capability past where tensor cores were introduced. If that's indeed true and I'm not misreading things, then that would explain it. So you can also try going into your config and specifying:

cudaUseFP16 = false
cudaUseNHWC = false

to explicitly disable FP16 and the attempt to use tensor cores, but I suspect the OpenCL will be better for you, as the big reason for the CUDA on top-end cards is the tensor cores.

[explo]

I'm not exactly sure why it would be slow. There aren't actually that many parameters you can tune that would have that big of an effect, just one mainly: have you run the benchmark command in a console/command window to select a number of numSearchThreads to use? (See the readme, and/or the main page https://github.com/lightvector/KataGo#how-to-use for the command).

Also, is it automatically using FP16?

Edit: Oh, interesting, looks like according to this: https://www.tomshardware.com/reviews/nv ... ,6002.html GTX 1660 Ti doesn't have any tensor cores, despite being on a high compute capability past where tensor cores were introduced. If that's indeed true and I'm not misreading things, then that would explain it. So you can also try going into your config and specifying:

cudaUseFP16 = false
cudaUseNHWC = false

to explicitly disable FP16 and the attempt to use tensor cores, but I suspect the OpenCL will be better for you, as the big reason for the CUDA on top-end cards is the tensor cores.

I had not run the benchmark command. It's done now (correctly I hope). It suggested numSearchThreads=16 so I changed accordingly.
The katago-gtp10.cfg file has cudaUseFP16 = true and cudaUseNHWC = true. I changed them to false, but saw no difference in Lizzie.

I can confirm my card doesn't have tensor cores. The GTX 1660 cards are based on the same architecture as the RTX cards so the fp16 thing should also work, but they don't support Ray Tracing and DLSS which are the gaming features using the tensor cores.

I'll stick to the OpenCL version for the moment. If someone has a RTX 2060 I'm curious how fast it is compared my my graphic card.

[lightvector]

A bug has been now fixed in the v1.3.2 release where KataGo could crash if you have pondering enabled and the GUI you are using then tries to ask it to score the game while it's pondering.

The release and all the downloadable executables have been updated at the same place as before:
https://github.com/lightvector/KataGo/releases/

(thanks to the LZ discord user who reported it).

Additionally, KataGo will now tolerate interleaving white passes with black moves on an otherwise empty board as a way of "setting up" a handicap game - which should work around some issues with Lizzie and free handicap placement (although hopefully Lizzie gets fixed eventually too).

[lightvector]

A bit more detail about the Lizzie handicap thing:

Due to KataGo not detecting such a handicap placement method before, if you tried to manually place stones on the board to play a handicap game within Lizzie, then KataGo would treat it as an even game, and therefore quickly attempt to resign, because if you are behind by that much in an even game at high levels of play, there is no hope.

Lizzie has a bug where if the bot attempts to resign in such a situation (or maybe attempts to resign ever in any situation?), Lizzie treats it as a pass - which might be very very confusing if you're trying to play the game. Additionally, Lizzie then sends a nonsensical command to the bot after that but ignores the response, which causes Lizzie GUI and the bot to be desync'ed in their idea of what the board position.

This bug still exists in Lizzie as far as I can tell, but you're less likely to hit it now because KataGo will not resign so early any more if you use this method of setting up an initial handicap.

[Vargo]

40 game test at time parity, 1600 visits for LZ :

Katago v1.3.2 (20b_s19_d43) v LZ017 (#261)

Katago wins 30-10 (75%)

Gogui-twogtp 1.5.1, all games by resignation, no error, no duplicate game.
LZ : --gtp -w 261_0a054a.gz -v 1601 --noponder -r 10
Katago : gtp -model g170-b20c256x2-s1913382912-d435450331.txt.gz -config gtp.cfg
LZ takes slightly more time per game (LZ average : 239s, Katago : 232s)

If you were finding KataGo to be roughly evenly matched with LZ before, then with the speed boost and the better nets here hopefully this puts KataGo more clearly ahead.

Yes indeed !

Games details : (LZ always apear as B, because of the command -alternate)

k1.jpg (66.67 KiB) Viewed 10940 times

Details :

k2.jpg (188.05 KiB) Viewed 10940 times

If someone wants the games, I'll upload them

[go4thewin]

Thanks! Can you do a similar test between the new 15b net and Lz-Elfv2?

[Vargo]

Thanks! Can you do a similar test between the new 15b net and Lz-Elfv2?

40 game test at visits parity (not far from time parity on this PC : average 245s/game for ELFv2, 275s for KataGo)

KataGo v1.3.2 : g170e-b15c192-s1672170752-d466197061, maxVisits = 3200
v
LZ017 : leelaz.exe --gtp -w ELFv2 -v 3201 --noponder -r 10

Gogui-twogtp 1.5.1, all games by resignation, no error, but 3 duplicate games (=37 games for the results)

Katago wins 24:13 (~65%)

Details (KataGo always appears as W, because of the command -alternate)

k1b.jpg (75.58 KiB) Viewed 10803 times

k2b.jpg (210.51 KiB) Viewed 10803 times

[Vargo]

Another 40 game test at near time parity (ELF takes ~57s/game KataGo 51s)

KataGo v1.3.2 : g170e-b15c192-s1672170752-d466197061, maxVisits = 700
v
LZ017 : leelaz.exe --gtp -w ELFv2 -v 801 --noponder -r 10

Gogui-twogtp 1.5.1, all games by resignation, no error, no duplicate games

Katago wins 32:8 (80%)

Details : (KataGo always appears as W, because of the command -alternate)

k1c.jpg (82.19 KiB) Viewed 10793 times

k2c.jpg (206.59 KiB) Viewed 10793 times

[ez4u]

40 game test at time parity, 1600 visits for LZ :

Katago v1.3.2 (20b_s19_d43) v LZ017 (#261)

Katago wins 30-10 (75%)

Gogui-twogtp 1.5.1, all games by resignation, no error, no duplicate game.
LZ : --gtp -w 261_0a054a.gz -v 1601 --noponder -r 10
Katago : gtp -model g170-b20c256x2-s1913382912-d435450331.txt.gz -config gtp.cfg
LZ takes slightly more time per game (LZ average : 239s, Katago : 232s)

...

How many visits did this mean for Katago?

[Vargo]

How many visits did this mean for Katago?

maxVisits=1600 (or maybe 1500, not 100% sure, because I changed the config since)

[go4thewin]

Thanks! Can you do a similar test between the new 15b net and Lz-Elfv2?

40 game test at visits parity (not far from time parity on this PC : average 245s/game for ELFv2, 275s for KataGo)

KataGo v1.3.2 : g170e-b15c192-s1672170752-d466197061, maxVisits = 3200
v
LZ017 : leelaz.exe --gtp -w ELFv2 -v 3201 --noponder -r 10

Gogui-twogtp 1.5.1, all games by resignation, no error, but 3 duplicate games (=37 games for the results)



Details (KataGo always appears as W, because of the command -alternate)

k1b.jpg

k2b.jpg

wow thank you!
for anyone else interested, cgos now has a kata bot 191 playing with 800 playouts http://www.yss-aya.com/cgos/19x19/cross ... _p800.html
with 1 playout http://www.yss-aya.com/cgos/19x19/cross ... t1_p1.html