It is currently Sat Apr 27, 2024 11:21 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 4 posts ] 
Author Message
Offline
 Post subject: 100 challenging problems checked with AI-not what you think!
Post #1 Posted: Thu Feb 29, 2024 9:43 am 
Oza
User avatar

Posts: 2411
Location: Ghent, Belgium
Liked others: 359
Was liked: 1019
Rank: KGS 2d OGS 1d Fox 4d
KGS: Artevelde
OGS: Knotwilg
Online playing schedule: UTC 18:00 - 22:00
https://senseis.xmp.net/?100Challenging ... aysOfStudy

I found this book in our club's library which recently returned from the previous librarian's home. You can check the page to see what its content and structure. I'm adding a page per problem type where I review one problem using KataGo. I'm doing and then reviewing all 100 problems but I won't publish all of it because of copyright violation - I think 8 problems, sampling the categories, is ok. I'm also not copying the book's advice, rather giving the key message and recreate it using KataGo.

What might be interesting for this forum is that my experience has been different from professional game commentaries previously checked with AI. Like those, the book's opening and middle game problems often show a solution which AI doesn't endorse. What's different is that the book's solution and the accompanying commentary is making more sense to me than the AI solution. The book's "suji" are instructive and reusable, the AI recommended sequence is often more mundane. Even if there are partial refutations of the solution proposed, there's often a flow to those that makes them worthy of exercising.

Things take another turn in the tesuji chapter ("perception" in the book, a funny term coined by translator Bob Terry). Here, the solution path is rather narrow and either needs a huge amount of playouts or force feeding for AI to appreciate. This confirms what I think and often see misinterpreted by others: AI is NOT about finding one narrow path of success but about maximizing probability of success.

Here's a problem from each category, checked with KataGo

https://senseis.xmp.net/?100Challenging ... y/Problem1 - Opening
https://senseis.xmp.net/?100Challenging ... /Problem14 - Middle game
https://senseis.xmp.net/?100Challenging ... /Problem32 - Tesuji


This post by Knotwilg was liked by 2 people: dust, Harleqin
Top
 Profile  
 
Offline
 Post subject: Re: 100 challenging problems checked with AI-not what you th
Post #2 Posted: Thu Feb 29, 2024 2:01 pm 
Oza

Posts: 3658
Liked others: 20
Was liked: 4633
I say this with great diffidence, but did you set the komi at 5.5? That was the norm when the book was published.

I have no idea how much difference that would make in katago terms, but the change to 6.5 (in 2003) was made precisely because it was thought that Black had a signifi8cant advantage (in pro play) at 5.5. This was supposedly confirmed by before-and-after surveys. The change reduced Black's win rate by two percentage points to something close to parity.

That change in turn elicited a change in opening strategy. The answers in the book may therefore be not so much wrong as just not updated.

Top
 Profile  
 
Offline
 Post subject: Re: 100 challenging problems checked with AI-not what you th
Post #3 Posted: Thu Feb 29, 2024 3:20 pm 
Lives in gote

Posts: 586
Location: Adelaide, South Australia
Liked others: 208
Was liked: 265
Rank: Australian 2 dan
GD Posts: 200
When KataGo first came out, I looked for examples of choosing a different move in a no-komi game versus the same position with komi. They were surprisingly hard to come by! Of course that doesn't mean the human pros are wrong. The move which gives you the best practical chance in a no-komi game between humans could well be different from the best chance in an AI-vs-AI game. But from a pure game theory perspective, it makes sense: the move which maximises your score on the board is the same regardless of komi, and the closer you get to perfect play, the less difference there is between the score-maximising move and the best winning chance.

That's a long way of saying: I'll be surprised if changing komi between 5.5 and 6.5 changes KataGo's move choices. (But also happy to be proven wrong. That would be an interesting conversation.)

Top
 Profile  
 
Offline
 Post subject: Re: 100 challenging problems checked with AI-not what you th
Post #4 Posted: Fri Mar 01, 2024 5:56 am 
Lives in sente

Posts: 1297
Liked others: 21
Was liked: 57
Rank: 1d
Interestiong. It seems to me, as a human, difficult to understand what means "estimate score" for katago.
I tested katago with an empty board (I do not use millions playouts) and komi equal to 0.5, 1.5, 2.5, ... up to 10.5 and I discovevered that the estimate score is not very consistent.
In addition I made the same serie of tests with japanese rule and AGA rule. I expected +0.5 points for black when using AGA rule but it apperas not so simple.
What is your own observation?
Do you have any explanation?

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group