About the komi advantage at the start of the game...

Usually, my goal in using LZ is not to see who is leading - it’s to see where I made mistakes in my game, and to see other strong options.

From this perspective, it is irrelevant who is leading at the start of the game, because it doesn’t give information about a mistake I have made - I haven’t played any moves, yet! What matters to me is interpreting later parts of the game where I can potentially find mistakes.

_________________
be immersed

(You can always change the komi in the sgf file before analyzing if you like to.)

Sure, but I would find it pleasing if the game started with a 50-50 chance. It would increase my confidence in the differential analysis.

With KataGo (easy using LZ 0.7) set komi to 7.0, and it will say the game starts at close to 50-50.

i think of it like a game theory problem where the expected payoff is negative.

for example, let's say you are going to play tic-tac-toe with your friend. if you win, you have to pay him $5. if you tie, you have to pay him $10, and if you lose you have to pay him $20. all situations are bad for you, but there is still an optimal way to play - if your goal is to end up with the most amount of money, you should aim to have minimal loss. (unless you say that the optimal strategy is to not play the game - in that case, just don't play go as black :-p)

i suppose that go is a zero-sum game, so you can argue that if it's a losing position, it doesn't matter what you play. but from that perspective, we don't really need win rates - we just need a binary value from LZ that says, "black is winning" or "white is winning". i think it's useful to have a more descriptive quantification of the amount of gain/loss.

i guess my conclusion is that if we accept that non-binary evaluation of board state in the form of percentages to be a useful form of analysis, looking for move recommendations that minimize loss, even from a losing position, is useful for identifying mistakes.

_________________
be immersed

Could you explain this? If 7.5 is too much (very likely) then W winrate will approach 100% as the level increase, won't it?

I was not assuming that 7.5 komi is too much. If that is given, then indeed the White winrate will equal 100% with perfect play with 7.5 komi.



Assuming that both players are evenly matched, in general it is Black who is better able to make use of the advantage of the first move, so that the statistical komi generally increases with their skill level. Statistical komi might be 3.5 for DDKs, for instance. For Black to give 5.5 komi at that level may favor White. But 5.5 komi favors Black at the pro level. And if 7.5 komi favors White at the level of today's top bots, it surely favors White at the human level. That's what I should have said, instead of predicting what will happen as bots get stronger. That was speculation.

But there is no guarantee that, as bots get stronger, 7.5 komi will continue to favor White.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

These numbers are pretty much what I'd expect if seven points is the "correct" komi:

• 44.7% of playouts end up with black 8 or more points ahead on the board, so a win for black with either komi.
• 45.7% of playouts end up with black no more than 6 points ahead on the board, so a win for white with either komi.
• 9.6% of playouts end up with black exactly 7 points ahead on the board, so could go either way depending on komi.

The third number being so small tells us that perfect play is still some way off.

I'm skeptical.

Does it really help to analyze every legal top-level move but not every legal response to those moves? You clearly don't want to force analysis of every response to every move at every level of the tree, because then you're just doing a brute-force search and you don't have anywhere close to the computing power you would need for that. But with only 10 playouts for each legal move, do you really get any kind of accurate estimate of the value of those moves?

But, of course, it is worth trying anyway just to see what happens.

Here is a screenshot of LZ 247 network with the normal LZ engine pondering on the Takemiya reduction position form John's original thread after 100k playouts. Note that in Lizzie 0.7 I changed the default setting to show stats on all moves considered rather than the default threshold of only showing if they have at least 0.1 of the total playouts (n which case you get just a dull red circle). It is very single-minded and focuses almost all of its resources on further exploring the shoulder hit.



And here is the same position using the LZ review mode engine which is modified to force LZ to consider more moves (from https://github.com/AncalagonX/leela-zer ... v0.16-next)


It didn't actually look at as many alternatives as I'd hoped: it worked better with older weaker networks that weren't so single minded. The 4 options reflects a hardcoded parameter of the modified engine, that could be increased if desired. But at least they have more playouts so have a better winrate estimate.

Also for completeness here is LZ247's view after I play Takemiya's gentler reduction, -6% over the shoulder hit. Interesting to note that even after the gentle reduction, it still ends up wanting to go in deep as black when white clamps the attachment of 2 in the principal variation (which only happens because it thinks white's knight response at o10 is a little better than Takemiya's opponent's block at o9, that seems to be because it thinks o10 has a good follow up at n8 if black tenuki, whilst o11 is easier to tenuki and leaves n11 as a good forcing move before Takemiya's m7).

It might seem like a reasonable thing to say, but it's not.

I've trained a 9x9 bot and the white advantage with 7.5 on the first move only grows as the bot gets stronger. I can say fairly certainly that on 9x9 boards the correct komi is 7. We are much less certain about the correct komi on 19x19, but it's far more likely to be 7 than any other value (assuming Chinese counting). The real winrates of white are going up in Leela Zero as White is more able to keep the point advantage. Similarly, with 5.5 komi (if that was the one used in self-play) Black would have a growing winrate.

KataGo has a close to 50% win rate using 7 komi.

There's a lot of evidence pointing to 7 komi on 19x19 and none that may suggest otherwise. There's no guarantee the correct komi is 7, but that's the same as saying there is no guarantee the sun will shine tomorrow. Where is the proof otherwise?

My reasoning is that you don't need an accurate estimate, as the search works off error bounds rather than exact winrates. Consider:

• Move A: the one that the bot would pick as the best move with a winrate of 52%
• Move B: the "pro move" that the bot ignored, which if it were analysed would get a winrate of 51.5%
• Move C: pretty hopeless

On ten visits for every move, you might find:

• Move A: winrate looks like it's somewhere between 30% and 70%
• Move B: winrate looks like it's between 28% and 71%
• Move C: winrate looks like it's between 2% and 40%

Then moves A and B will both get a few extra playouts to refine the winrate estimate, but C won't. You end up being able to see an estimate for move B without the trouble of explicitly asking for it.


Done (for LZ). I feel like the apocryphal plumber: the code changes only took a few minutes, but it was more than an hour's work to figure out that I needed to change the UCTNode::uct_select_child function. This is a dodgy first cut: the number 10 is hard-coded. If I get some more spare time, I'll add a command line option to change the number of playouts. Of course this is all open source, so no reason why someone else couldn't pick it up and run with it.

Initial results look promising but not spectacular. Using LZ network 242, in the medium term (5,000-10,000 playouts) you see a wider range of moves, but then it converges on to what the unmodified LZ-242 would have done anyway. Using the "bubblesId" network packaged with Lizzie 0.7, there seems to be more variety. Over the weekend I'll play with some different positions and different networks, and post some examples here unless I get distracted by the next shiny thing.

Analysis, particularly if the analyst focuses on only a few key moves instead of trying to analyze the whole game at once, can be too slow for actually playing a game. I would be willing, in certain cases, to spend 36k playouts on the first pass (100 on each of 360 moves) to discover playable moves, even possibly a "best" move that the bot would otherwise have overlooked.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

OK, I've modified my code so that you can use different numbers of playouts. And I've had a go at making a Windows binary so that other people can more easily play with this -- although I'm not sure if this will work, as I'm used to doing this sort of thing on linux instead. Download from here. During the next day or three I might start a new thread to post some screen shots of results so far.