100 challenging problems checked with AI-not what you think!

[Knotwilg]

https://senseis.xmp.net/?100Challenging ... aysOfStudy

I found this book in our club's library which recently returned from the previous librarian's home. You can check the page to see what its content and structure. I'm adding a page per problem type where I review one problem using KataGo. I'm doing and then reviewing all 100 problems but I won't publish all of it because of copyright violation - I think 8 problems, sampling the categories, is ok. I'm also not copying the book's advice, rather giving the key message and recreate it using KataGo.

What might be interesting for this forum is that my experience has been different from professional game commentaries previously checked with AI. Like those, the book's opening and middle game problems often show a solution which AI doesn't endorse. What's different is that the book's solution and the accompanying commentary is making more sense to me than the AI solution. The book's "suji" are instructive and reusable, the AI recommended sequence is often more mundane. Even if there are partial refutations of the solution proposed, there's often a flow to those that makes them worthy of exercising.

Things take another turn in the tesuji chapter ("perception" in the book, a funny term coined by translator Bob Terry). Here, the solution path is rather narrow and either needs a huge amount of playouts or force feeding for AI to appreciate. This confirms what I think and often see misinterpreted by others: AI is NOT about finding one narrow path of success but about maximizing probability of success.

Here's a problem from each category, checked with KataGo

https://senseis.xmp.net/?100Challenging ... y/Problem1 - Opening
https://senseis.xmp.net/?100Challenging ... /Problem14 - Middle game
https://senseis.xmp.net/?100Challenging ... /Problem32 - Tesuji

[John Fairbairn]

I say this with great diffidence, but did you set the komi at 5.5? That was the norm when the book was published.

I have no idea how much difference that would make in katago terms, but the change to 6.5 (in 2003) was made precisely because it was thought that Black had a signifi8cant advantage (in pro play) at 5.5. This was supposedly confirmed by before-and-after surveys. The change reduced Black's win rate by two percentage points to something close to parity.

That change in turn elicited a change in opening strategy. The answers in the book may therefore be not so much wrong as just not updated.

[xela]

When KataGo first came out, I looked for examples of choosing a different move in a no-komi game versus the same position with komi. They were surprisingly hard to come by! Of course that doesn't mean the human pros are wrong. The move which gives you the best practical chance in a no-komi game between humans could well be different from the best chance in an AI-vs-AI game. But from a pure game theory perspective, it makes sense: the move which maximises your score on the board is the same regardless of komi, and the closer you get to perfect play, the less difference there is between the score-maximising move and the best winning chance.

That's a long way of saying: I'll be surprised if changing komi between 5.5 and 6.5 changes KataGo's move choices. (But also happy to be proven wrong. That would be an interesting conversation.)

[Gérard TAILLE]

Interestiong. It seems to me, as a human, difficult to understand what means "estimate score" for katago.
I tested katago with an empty board (I do not use millions playouts) and komi equal to 0.5, 1.5, 2.5, ... up to 10.5 and I discovevered that the estimate score is not very consistent.
In addition I made the same serie of tests with japanese rule and AGA rule. I expected +0.5 points for black when using AGA rule but it apperas not so simple.
What is your own observation?
Do you have any explanation?