LeelaSabaki - Leela integration with Sabaki

[yishn]

Several people have requested some type of Leela integration for Sabaki, but it's not Sabaki's philosophy to prefer one engine over another. So I've decided to draft some sort of engine integration protocol for Sabaki as a GTP extension, so that any engine can tell Sabaki what to do (in a limited way). It's still in the works, but I've implemented something in the Sabaki v0.32.0 beta version.

Together with the new Sabaki beta, I've also written LeelaSabaki, a laconically named proxy to Leela, which extends Leela with the aforementioned engine integration protocol. Currently, LeelaSabaki only does one thing: Whenever a move is generated, LeelaSabaki will add other variations Leela considered to the game tree in Sabaki, along with winning statistics. It also works with Leela Zero.

Leela integration with Sabaki

screenshot.png (489.7 KiB) Viewed 24732 times

I don't want to say any more about the engine integration protocol, as I don't want other engines to implement it just yet, because it might change drastically soon. But I'd be happy if you give LeelaSabaki a try; just follow the instructions and make sure you're using Sabaki v0.32.0 beta. Furthermore, I'm interested in hearing how you think Leela (or any engine in general) can integrate itself more with Sabaki.

[yoyoma]

This is great, I'll have a look later. I wanted to mention I think I saw an issue with Sabaki and LeelaZero where if you click somewhere in the tree and ask LeelaZero to make a move, Sabaki does not replay the game to LeelaZero correctly. It replays all black moves and then all white moves. LeelaZero depends on the correct move order, so this confuses it. Can you look into this?

[yishn]

I wanted to mention I think I saw an issue with Sabaki and LeelaZero where if you click somewhere in the tree and ask LeelaZero to make a move, Sabaki does not replay the game to LeelaZero correctly. It replays all black moves and then all white moves.

I've explained this behavior here: https://github.com/yishn/Sabaki/pull/294

But I see why replaying with correct move order is important. I'm open to solutions.

[yoyoma]

I see you point out that in general SGF files can do almost anything. But in trying to handle all SGF file types, Sabaki doesn't support LeelaZero (or AQ and others like it). I propose you parse the SGF and determine if it is a normal game with simple moves (no position setup, removing/overwriting etc). In that case play back the moves to the engine as normal. At least this will allow it to work with LeelaZero.

ETA: Maybe someone (I nominate you? ) can define some GTP command extensions. Something like explain_move or dump_analysis, and it returns an SGF string containing moves the engine analyzed. If it's SGF the engine would be free to annotate those variations with whatever text it likes describing winrates etc.

ETA2: Also a command like winrate that returns the engine's position evaluation.

[qianyilong]

But I see why replaying with correct move order is important. I'm open to solutions.

My two cents on a simple solution. Perhaps when you configure a new engine you could have a simple toggle that configures strict ordering. You can enable strict ordering for engines that need it and leave it off for those that don't.

I personally hate the proliferation of switches and toggles from a software dev perspective because they often introduce more spots for bugs and problems but sometimes it is the lesser of two evils.

[Dontbtme]

Hi!
I use Leela and LeelaZero on Sabaki a lot with no trouble. I love Sabaki and really was looking forward to use LeelaSabaki as well, but somehow I can't seem to find the way to run it
I've tried to type many things, like:
--flat --help C:\Users\Dontbtme\Downloads\Sauvegarde\Jeu de go\Logiciels\leelasabaki-v0.2.0\Leela0110GTP_OpenCL.exe --gtp --noponder
I've tried with the <> and [] used in the explanations, or with and without the --gtp, etc.
Anyway, even though I suspect the answer might be obvious, I'm becoming more and more desperate
What am I doing wrong?

[yishn]

Hi!
I use Leela and LeelaZero on Sabaki a lot with no trouble. I love Sabaki and really was looking forward to use LeelaSabaki as well, but somehow I can't seem to find the way to run it
I've tried to type many things, like:
--flat --help C:\Users\Ra\Downloads\Sauvegarde\Jeu de go\Logiciels\leelasabaki-v0.2.0\Leela0110GTP_OpenCL.exe --gtp --noponder
I've tried with the <> and [] used in the explanations, or with and without the --gtp, etc.
Anyway, even though I suspect the answer might be obvious, I'm becoming more and more desperate
What am I doing wrong?

You should remove the '--help' option, as it will only print a help note and therefore not enter GTP mode.

Did you try to put the path to Leela in quotation marks? It seems like you have spaces in your path. Also, you can try running LeelaSabaki in the command line to see what error messages it writes.

[Dontbtme]

I tried all the above and... that's it!
For the curious and the helpless like me, here's the working line (in my case):
--flat "C:\Users\Dontbtme\Downloads\Sauvegarde\Jeu de go\Logiciels\leelasabaki-v0.2.0\Leela0110GTP_OpenCL.exe" --noponder
I just tried it a bit, but I can already tell I'll use LeelaSabaki a lot
Oh, and being able to flatten or not the variations is an awesome idea, depending on how we intend to use the sgf file later on
Thanks a lot for this great tool!

[yishn]

I've just released a new version of LeelaSabaki (together with a new version of Sabaki) which lets you visualize network probabilities from Leela in Sabaki as heatmaps.

LeelaSabaki heatmap

leelasabaki-heatmap.png (433.84 KiB) Viewed 23285 times

[pnprog]

I've just released a new version of LeelaSabaki (together with a new version of Sabaki) which lets you visualize network probabilities from Leela in Sabaki as heatmaps.

The result looks nice!

[djhbrown]

first time trying out Sabaki tree navigation, i found that right arrow didn't do what i expected; i wanted to hop to the next branch from the red node, but instead it jumped back and then across, taking me into a different variation.

I personally hate the proliferation of switches and toggles

second that; it's hard to provide functionality and obviousness at one and the same time, but in general i think its better to be obvious than not. like, who the heck would expect that Alt is a magic button to see a menu? menus dont take up that much space, so should be always visible.

PS. policy net probabilities or win%? of the two, the only one that interests me is win%

[Dontbtme]

So, thanks to LeelaSabaki I had a lot of fun reviewing my games from LeelaZero's point of view
Here's my thoughts as to this particular use: When I ask LeelaZero what it would play, it usually considers a lot of candidate moves, showing numerous variations as a result... The thing is: I intend on keeping a neat review to look at again in the future, so I end up trimming off each time any variations BUT the best one. On the other hand, if there was some command I could type in "Manage Engine" ordering LeelaSabaki to show only a given number of variations (1 in my case), I wouldn't have to trim off anything.
That was my two cents, but anyway: LeelaSabaki (and Sabaki for that matter) is a really cool tool already, so thanks a lot for making and sharing it!

[johnsmith]

I have one Sabaki question, so maybe it's a bit off topic. But is it possible to have the screen automatically maximized when Sabaki opens up?

[sleepyEDB]

So much great software available for training and reviewing. I already really liked sabaki as an SGF editor so I'll make sure to try this out. Thank you!


sleepy

[00Noles]

Nice project! I like Sabaki very much and would like to use it with Leela to review games (as the common Leela GUI doesn't let you do so). Unfortunately I'm having some trouble, as Leela does not seem to respond.

At first I just generated a new game and attached Leela to white. After I play a move with black you can see her recognizing it in the console and the "beach ball"-cursor indicates the program is working but even after 5-10min it doesn't play a move. I then changed the --playouts parameter to 10 but it didn't work either. I tried with and without the --gtp but nothing works (I'll attach a screenshot below).

The same goes for reviewing games. If i attach Leela to let's say the color i played, i can skip through the game but i don't any variations or suggested moves. If i type the "heatmap" command into the console still nothing happens. If i generate a move or attach the engine to both players the result is the same as if i start a game against Leela.

Am i missing something important while setting it up?