Bill, I wouldn't put much weight on the LZ percentages at such low playouts: I just have it pondering as I input the game.

Does this 5% difference reflect Golaxy's error as a player, or LZ's error as an analyst at fewer than 1k playouts? (Or both? The two are not mutually exclusive, OC. )

I only glanced at your thread about Winrate errors, but as lightvector said vs what? vs perfect play is just itself or 100% - self. vs the bot's best understanding of the position where we say that is what it says at near infinite playouts is more tractable comparison and can be studied. So we can ask what is the error in the Winrate at 100 or 1000 thousand playouts when we use that to estimate what that bot would think at a billion playouts. maybe a billion takes too long so we could take 10 million as "a lot" as that's what deepmind used for the AG teaching tool.

RE 72, LZ's view doesn't change much with about 100k playouts. Hanging connection gives white 56.4% (104k), hane lets black get 49.1% (91k) with the cut ie white 50.9 so 5% difference.

The way to see what LZ thinks of the actual 72 hane with 100k playouts is to play it and wait for 100k playouts to happen, as shown in the 2nd picture. To wait for that 580 to turn into 100k on the first position would likely mean the 1st choice move has also gone up by a factor of 200, which is 20 million which would take ages.

The way to see what LZ thinks of the actual 72 hane with 100k playouts is to play it and wait for 100k playouts to happen, as shown in the 2nd picture. To wait for that 580 to turn into 100k on the first position would likely mean the 1st choice move has also gone up by a factor of 200, which is 20 million which would take ages.

Oh, I haven't been taking the second picture and making a direct comparison. What I have been doing with Deep Leela is this. After playing as in the game and generating the second picture, then backing up and playing again. That way DL compares the options for directly, utilizing the altered search tree which focuses more on the actual move in the game than the original tree. Generating the second picture alters the winrate estimate and number of playouts for the first picture. At least, that happens with DL. My guess is that it works that way with LZ as well.

This will sometimes work, but in a chaotic way. The internals go like this: After going to it will build a tree and analyze positions, and as a side effect, cache those positions. Then when you go back to , it will reset the tree search part, and start searching again. The search part will go normally, it does not know the result of the deep search. But all searches starting with will be in the cache. These will take a shortcut, bypassing the GPU. So even if normally starts off bad and only becomes good later, this shortcut might allow it to get enough visits to it to see that it's actually a good move.

So you can't really rely on this method, it's more reliable to compare moves by clicking into each one and noting the winrates.

Black's reply to the hanging connection has a winrate estimate of 47.9%.

Black's reply to FineArt's has a winrate estimate of 43.7%.

So, FWIW, DL prefers FineArt's actual play over the hanging connection by more than 4%, even though the actual play was not on its radar after , and still is not. Right?

Yes I think we're on the same page now. Before actually playing move 72, LZ doesn't spend enough time considering FineArt's play to realize it's good. Also LZ code isn't smart enough to reuse results from a forced deeper search of 72 in the way you tried -- going forward and then back.

I just realized you're talking about the original Leela with deep learning, not Leela Zero? Are those screenshots from original Leela's built in GUI? I was talking about Leela Zero, but I think the same logic applies to both.