Life In 19x19

Could somebody please tell me if it's possible to download a ready-to-use Go AI, without having to go through any complicated steps involving programming or compiling? I'm looking for a practise tool that I could use "off the peg".

Thanks a lot!

If you are on Windows Lizzie as the interface for LeelaZero should be easy: https://github.com/featurecat/lizzie/releases. You need Java (which most computers have) and might need the VC++ redistributables if you haven't got those from installing other things: https://www.microsoft.com/en-us/downloa ... x?id=48145

Thanks a lot, Überdude! I now have my very own Lizzie up and running, ready to instruct me.

If I may, I'd like to ask a couple of questions which I expect may cross Tami's mind, too.

I've got LeelaZero/Lizzie. I've spent very little time with it, but as far as I can see it's working fine. My questions are to do with interpretation.

1. Is there a non-geek guide to interpreting LeelaZero's output somewhere?

2. Absent the above, why does the winrate in the starting position (i.e. Black to move) show as something 46% or 47% and at move 2 show as 53%/54%? I'm guessing this is because it thinks 7.5 komi favours White. That would not surprise me, but (assuming my guess is right) what does surprise me is that so little seems to have been made in discussions of such a wide gap. I'd have expected calls for komi to be changed and/or discussions whether Black's play has to be reckless (with all that implies for reliability of evaluations) to overcome such a big discrepancy.

3. In the position Black d16 + q16, White q4, I see LZ displaying two moves (c4 and d4) as being pondered. Does this really mean it has pruned the list of candidate moves to two? Even if these two are much better than others, is there no easy way of seeing other moves and what they score?

John Fairbairn wrote:1. Is there a non-geek guide to interpreting LeelaZero's output somewhere?

The top number in a circle is the winning percentage from the point of view of the player to move (by default; see answer 2 below). The bottom number is the number of times Leela Zero has looked at that move. The way Leela Zero works when it is playing on its own is to play the move that it has visited the most, which is not necessarily the move that currently has the best winning percentage.

The circle is color-coded according to how good that move is compared to the other ones. Greener is better and redder is worse. The best move is colored cyan. If there is a move with a higher winning percentage than the most-visited move, it gets a blue outline to call it out.

Moves that exist in the game tree get a white or black outline (depending on the player). The main-line moves get a thicker outline.

2. Absent the above, why does the winrate in the starting position (i.e. Black to move) show as something 46% or 47% and at move 2 show as 53%/54%? I'm guessing this is because it thinks 7.5 komi favours White. That would not surprise me, but (assuming my guess is right) what does surprise me is that so little seems to have been made in discussions of such a wide gap. I'd have expected calls for komi to be changed and/or discussions whether Black's play has to be reckless (with all that implies for reliability of evaluations) to overcome such a big discrepancy.

The win percentage is by default from the point of view of the player to move. I find it annoying to see the winrate keep bouncing back and forth between 62% and 38% (say) so I added an option, which you can take advantage of as well, to always display the winrate from Black's point of view, by setting the "win-rate-always-black" property in config.txt to true.

Leela Zero (and all the AlphaGo variants) think that White is favored in the initial position with 7.5 komi (I believe that ELF thinks the game is closer to even). There has indeed been plenty of discussion of this. I believe there is agreement that 5.5 komi would be a bigger advantage for Black than 7.5 is for White. In any case, I think the consensus is that it's close enough that Black shouldn't feel the need to play recklessly (as White may in a handicap game).

In the position Black d16 + q16, White q4, I see LZ displaying two moves (c4 and d4) as being pondered. Does this really mean it has pruned the list of candidate moves to two? Even if these two are much better than others, is there no easy way of seeing other moves and what they score?

Correct. As the networks improve, they tend to consider fewer moves. You can always find out what it thinks of other moves by actually executing them on the board. Of course you will have to mentally flip the winrate numbers afterwards unless you change the option I mentioned above.

Ninja by dfan but here's my version:

1. There's some details on https://github.com/featurecat/lizzie/releases/tag/0.5. Tami's question and various other people asking how to use Lizzie to review (here and on reddit) have made me think I should make a video "How to review with Lizzie". It requires a certain amount of Go skill to interpret her answers, but knowing how and what questions to ask helps a lot. For starters what the UI means (in lizzie 0.5).
- Coloured filled-in circles are the moves LZ is thinking about
- If reviewing a game record black hollow circle is game move black played, white circle is white's game move.
- Blue-filled circle is LZ's best move, the one she would play now (if no more time to think). Green to red colours are good to bad moves (though of course bad is still pretty strong)
- Top number is the win% of that move for person to play, so bigger is better
- Bottom number is number of playouts investigating that move, essentially how much reading has been devoted to it.
- MCTS bots actually choose the move with the most playouts rather than highest win%. This is because a move with high win% but low playouts still has a lot of uncertainty, maybe it is great but maybe not. The bot will then spend some playouts investigating that move as well as the current favourite, if it does turn out to be good then it will get more playouts and become the #1 blue move. A blue outline circle highlights such promising moves with a win% higher than the current solid blue #1 move. This means it's a good idea to let LeelaZero analyse longer to decide if that really is a better move.
- Current black/white win% chart at top left, with change in win% of last move below.
- Win graph on the middle left, white wins at bottom, black at top (hint in the border)
- Mini board bottom left shows principal variation: #1 choice for subsequent moves.
- You can see principal variation for each choice displayed on the main board by hovering over it.

2. Yes, LZ thinks white is better with 7.5 komi, but the scale of the win% is not well-defined and differs between bots. For example FineArt gives black 41%, according to a comment from Ohashi Hirofumi 6p. The first version of Facebook's Elf network thought white was better like pretty much every other bot these days, but their most recent version gives black initially 51.5% which is intriguing (and hopefully not a bug).

3. Yes, LZ can be quite narrow minded in sticking to the moves it like. "Even if these two are much better than others, is there no easy way of seeing other moves and what they score?" Yes, simply play that move for white on the board. It will then show black's suggested next moves with their win%, and to save you doing a little mental arithmetic to see how good/bad the last move was you can check the top left. For example if I play 3-3 then it says "Last move: -0.9%".

Thanks dfan/uberdude - great answers, especially as they were also kind enough allow me the delusion that I hadn't made a complete prat of myself by asking those basic questions

have made me think I should make a video "How to review with Lizzie".

First-rate idea (and so long you make the sound loud enough I'll queue up to watch it).

dfan wrote:

John Fairbairn wrote:In the position Black d16 + q16, White q4, I see LZ displaying two moves (c4 and d4) as being pondered. Does this really mean it has pruned the list of candidate moves to two?

Correct. As the networks improve, they tend to consider fewer moves.

Is that desirable? Currently Leela Zero and Elf have blind spots that even amateur dan players can detect. Don't they have a parameter that controls exploration?

Edit: For review, if not for play, my inclination would be to have a relatively high degree of exploration, if for nothing else, to see a direct comparison between a number of plays.

But for review, that's not a problem : even if LZ didn't consider your move, when you play it, it will be evaluated and you can see the difference between the board evaluation before you played your move, and after you played it.

An exemple, before the move, LZ don't consider it at all : And after, LZ evaluate the new situation : I don't see how more exploration could be more usefull in this case.

Bill Spight wrote:

dfan wrote:As the networks improve, they tend to consider fewer moves.

Is that desirable?

It's hard to say for sure, but if their performance improves over time as their exploration decreases (and their policy becomes more sophisticated), I think the burden of proof is on us to show that the decreasing exploration is harmful. Of course it is very easy to come up with individual cases (lots of them!) where it does the wrong thing, but that doesn't mean it is a bad idea in aggregate.

Currently Leela Zero and Elf have blind spots that even amateur dan players can detect. Don't they have a parameter that controls exploration?

Yes, they do. Tuning it is a delicate matter. My understanding is that when people have tweaked Leela Zero to perform more exploration, it performs worse overall when playing. This is despite the fact that increasing exploration makes it easier to understand ladders, for example. Don't forget that exploitation is pretty valuable, too!

Edit: For review, if not for play, my inclination would be to have a relatively high degree of exploration, if for nothing else, to see a direct comparison between a number of plays.

I agree that more exploration would be nice, but it's a tough balance to strike, if it actually decreases the strength of the engine overall. Also, the policy of the network has been learned in a symbiotic context with the tree search, which has a certain exploration/exploitation setting, so it's not obvious to me that changing one without the other (by turning up the exploration parameter when analyzing) won't have some unintended side effects.

Tryss wrote:But for review, that's not a problem : even if LZ didn't consider your move, when you play it, it will be evaluated and you can see the difference between the board evaluation before you played your move, and after you played it.

An exemple, before the move, LZ don't consider it at all: [...]

And after, LZ evaluate the new situation: [...]

I don't see how more exploration could be more usefull in this case.

It would be useful if you could assume that, after some number of total visits, all reasonable moves for the current player are displayed, so you don't have to continually wonder whether there are other reasonable moves lurking around. It certainly is true that for any individual move you have a question about, it's easy enough to ask it what it thinks.

There's a separate issue where the engine doesn't understand a ladder a few ply into a variation (due to not examining it), so it totally misevaluates the current position, which is not so easily fixable just by trying out moves by hand.

dfan wrote:

Bill Spight wrote:

dfan wrote:As the networks improve, they tend to consider fewer moves.

Is that desirable?

It's hard to say for sure, but if their performance improves over time as their exploration decreases (and their policy becomes more sophisticated), I think the burden of proof is on us to show that the decreasing exploration is harmful. Of course it is very easy to come up with individual cases (lots of them!) where it does the wrong thing, but that doesn't mean it is a bad idea in aggregate.

Currently Leela Zero and Elf have blind spots that even amateur dan players can detect. Don't they have a parameter that controls exploration?

Yes, they do. Tuning it is a delicate matter. My understanding is that when people have tweaked Leela Zero to perform more exploration, it performs worse overall when playing.

Well, that's evidence. But, IMO, not the right approach. (More below.)

This is despite the fact that increasing exploration makes it easier to understand ladders, for example. Don't forget that exploitation is pretty valuable, too!

Fer shure.

Bill Spight wrote:Edit: For review, if not for play, my inclination would be to have a relatively high degree of exploration, if for nothing else, to see a direct comparison between a number of plays.

dfan wrote:I agree that more exploration would be nice, but it's a tough balance to strike, if it actually decreases the strength of the engine overall. Also, the policy of the network has been learned in a symbiotic context with the tree search, which has a certain exploration/exploitation setting, so it's not obvious to me that changing one without the other (by turning up the exploration parameter when analyzing) won't have some unintended side effects.

Indeed. Which is why I say that I don't think that human tweaking of the exploration parameter is the right approach. I would have more confidence in the degree of exploration of Leela Zero and Elf if that parameter were trained, as well as the rest of the bot.

Bill Spight wrote:Indeed. Which is why I say that I don't think that human tweaking of the exploration parameter is the right approach. I would have more confidence in the degree of exploration of Leela Zero and Elf if that parameter were trained, as well as the rest of the bot.

You will not be surprised to hear that people are working on that too (DeepMind paper, Leela Zero discussion #1, Leela Zero discussion #2).

Uberdude wrote:If you are on Windows

What about those who are on MacOS?

tj86430 wrote:

Uberdude wrote:If you are on Windows

What about those who are on MacOS?

Lizzie / Leela Zero works on Mac too, but you need to compile it yourself (which Tami requested not needing to do), there are instructions in the mac release on the page I linked. Quite why there can't be a pre-built executable I don't know, is each Mac really so different? "Mac: It just works"