The facilities of ZBaduk allowed me to study what LeelaZero and KataGo currently might be "thinking" of Igo Hatsuyoron 120.
According to Joachim Meinhardt (from our solution team), these two programs' behaviour is similar to what he encountered by using "Ah Q Go" a few months ago.
The results seem to be independent of komi (7.5 points used by the programs vs. no komi in the problem), most likely because any mistake in the solution sequence costs much more than that.
By the way: I am looking forward to trying "no komi" with KataGo some months in the future.
Those who want to try it on their own may use the SGF attached.
Please note that the first 142 moves contain the problem's setup.
In LifeIn19x19, our solution (as of 2011) is commented here.
AI & Igo Hatsuyoron 120
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
AI & Igo Hatsuyoron 120
- Attachments
-
- 2017_OurNewSolution4AI.sgf
- (2.23 KiB) Downloaded 691 times
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
Re: AI & Igo Hatsuyoron 120
Good understanding of the seki in the lower right corner.
The creation process of the seki in the lower right corner is handled correctly, ...
... starting with the solid connection at Q5, ...
... sometimes including changes in the order of moves, as here with P8 instead of Q7.
Even my second throw-in at O9 is choosen.
The creation process of the seki in the lower right corner is handled correctly, ...
... starting with the solid connection at Q5, ...
... sometimes including changes in the order of moves, as here with P8 instead of Q7.
Even my second throw-in at O9 is choosen.
Last edited by Cassandra on Thu Sep 12, 2019 8:33 am, edited 1 time in total.
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
Re: AI & Igo Hatsuyoron 120
Poor understanding of the hanezeki.
Everything is fine with the growth of the hanezeki's tail, until ...
... Black misses the important kikashi at G2 and plays at J4, instead, continuing with the tail.
Without this kikashi (it must be played before the Crosscut Sequence) Black will not get enough territory in the lower left corner to win the problem.
The tail must not be blocked, but ...
... KataGo does so with move W 186 (= 44) at J7, a fatal mistake.
White should have connected at H6, instead.
Both programs are not immune against this mistake during the further course of the tail building sequence. Further details will be shown later.
Everything is fine with the growth of the hanezeki's tail, until ...
... Black misses the important kikashi at G2 and plays at J4, instead, continuing with the tail.
Without this kikashi (it must be played before the Crosscut Sequence) Black will not get enough territory in the lower left corner to win the problem.
The tail must not be blocked, but ...
... KataGo does so with move W 186 (= 44) at J7, a fatal mistake.
White should have connected at H6, instead.
Both programs are not immune against this mistake during the further course of the tail building sequence. Further details will be shown later.
Last edited by Cassandra on Thu Sep 12, 2019 9:10 am, edited 1 time in total.
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
Re: AI & Igo Hatsuyoron 120
Black has a fatal inclination to R2 (/ R1) in the hanezeki.
Both programs' unfortunate passion for playing in the hanezeki first breaks out with KataGo.
Black's move at R2 (or R1) is a fatal mistake.
If we let KataGo explicitely consider how to move next with White, ...
... the correct reply at J9 (aka L9 here) is preferred, having a win rate of (only) 43 %.
Black's large group in the upper right is dead, so the game is over hereafter.
= = = = = = = = = = = = = = = = = = = =
The above mentioned behaviour can also be encountered with LeelaZero:
We already know that blocking the path of the tail (with W J11 here) is a fatal mistake.
LeelaZero's preferred continuation ...
... begins with the forcing moves of B2 (by the way: should be at 3, instead) and B4.
Nothing very serious has happened with these forcing moves, although these moves should be played later in the solution sequence, at this moment avoiding the unnecessary loss of some ko threats.
But unfortunately, Black continues with the move at R2 (aka T2) in the hanezeki.
LeelaZero gives a winrate for Black of 57 % for Black's final move.
For White, ...
... LeelaZero also prefers the correct reply at H10 (aka K10).
In the candidate's moves list, this move can be understood to be the "only" move (other candidates have only a few playouts).
LeeleZero gives a win rate about 50 % for White's move here.
Both programs' unfortunate passion for playing in the hanezeki first breaks out with KataGo.
Black's move at R2 (or R1) is a fatal mistake.
If we let KataGo explicitely consider how to move next with White, ...
... the correct reply at J9 (aka L9 here) is preferred, having a win rate of (only) 43 %.
Black's large group in the upper right is dead, so the game is over hereafter.
= = = = = = = = = = = = = = = = = = = =
The above mentioned behaviour can also be encountered with LeelaZero:
We already know that blocking the path of the tail (with W J11 here) is a fatal mistake.
LeelaZero's preferred continuation ...
... begins with the forcing moves of B2 (by the way: should be at 3, instead) and B4.
Nothing very serious has happened with these forcing moves, although these moves should be played later in the solution sequence, at this moment avoiding the unnecessary loss of some ko threats.
But unfortunately, Black continues with the move at R2 (aka T2) in the hanezeki.
LeelaZero gives a winrate for Black of 57 % for Black's final move.
For White, ...
... LeelaZero also prefers the correct reply at H10 (aka K10).
In the candidate's moves list, this move can be understood to be the "only" move (other candidates have only a few playouts).
LeeleZero gives a win rate about 50 % for White's move here.
Last edited by Cassandra on Sun Sep 15, 2019 4:16 am, edited 1 time in total.
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)
-
Bill Spight
- Honinbo
- Posts: 10905
- Joined: Wed Apr 21, 2010 1:24 pm
- Has thanked: 3651 times
- Been thanked: 3373 times
Re: AI & Igo Hatsuyoron 120
Very interesting. Many thanks. 
However, there are a couple of things about the bots' mistakes. First, the main virtue of today's top bots lies in evaluation, not the calculation of variations. My first thought when I saw the title of this thread was, "Did they actually solve it?"
Alas, no.
Second, komi may be a factor in some of the errors. For instance, why should Black play the peep at G-02 if it leads to a loss, like everything else? (Actually, today's bots seem to like to play kikashi early, so I was surprised that it did not make that play.
) It might be interesting to try out KataGo's no komi option sooner rather than later.
Still, an interesting thread, especially as today's bots were not built to handle such positions.
However, there are a couple of things about the bots' mistakes. First, the main virtue of today's top bots lies in evaluation, not the calculation of variations. My first thought when I saw the title of this thread was, "Did they actually solve it?"
Alas, no.Second, komi may be a factor in some of the errors. For instance, why should Black play the peep at G-02 if it leads to a loss, like everything else? (Actually, today's bots seem to like to play kikashi early, so I was surprised that it did not make that play.
) It might be interesting to try out KataGo's no komi option sooner rather than later. Still, an interesting thread, especially as today's bots were not built to handle such positions.
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins
Visualize whirled peas.
Everything with love. Stay safe.
At some point, doesn't thinking have to go on?
— Winona Adkins
Visualize whirled peas.
Everything with love. Stay safe.
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
Re: AI & Igo Hatsuyoron 120
Dear Bill, thank you very much for your kind comment.
I will first continue with posting several further observations, before trying to give some speculations what might be the reasoning for the AI's behaviour. For doing so, I will assume human thinking, being well aware of the fact that this does not match AI-thinking.
However, I was very surprised that now the AI found the initial sequence for creating the hanezeki, what Fujisawa's study group took so long to do.
Following your statement above, I still doubt Fan Hui's remark that (the game playing version of) AlphaGo would have been able to really solve Igo Hatsuyoron 120.
I will first continue with posting several further observations, before trying to give some speculations what might be the reasoning for the AI's behaviour. For doing so, I will assume human thinking, being well aware of the fact that this does not match AI-thinking.
Inoue Dosetsu Inseki knew that over 300 years agoBill Spight wrote:Still, an interesting thread, especially as today's bots were not built to handle such positions.
However, I was very surprised that now the AI found the initial sequence for creating the hanezeki, what Fujisawa's study group took so long to do.
Following your statement above, I still doubt Fan Hui's remark that (the game playing version of) AlphaGo would have been able to really solve Igo Hatsuyoron 120.
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)
-
Bill Spight
- Honinbo
- Posts: 10905
- Joined: Wed Apr 21, 2010 1:24 pm
- Has thanked: 3651 times
- Been thanked: 3373 times
Re: AI & Igo Hatsuyoron 120
Very interesting.Cassandra wrote:I was very surprised that now the AI found the initial sequence for creating the hanezeki, what Fujisawa's study group took so long to do.
He was a cheerleader for AlphaGo.Following your statement above, I still doubt Fan Hui's remark that (the game playing version of) AlphaGo would have been able to really solve Igo Hatsuyoron 120.
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins
Visualize whirled peas.
Everything with love. Stay safe.
At some point, doesn't thinking have to go on?
— Winona Adkins
Visualize whirled peas.
Everything with love. Stay safe.
-
Uberdude
- Judan
- Posts: 6727
- Joined: Thu Nov 24, 2011 11:35 am
- Rank: UK 4 dan
- GD Posts: 0
- KGS: Uberdude 4d
- OGS: Uberdude 7d
- Location: Cambridge, UK
- Has thanked: 436 times
- Been thanked: 3718 times
Re: AI & Igo Hatsuyoron 120
Are you guiding these bots by playing out sequences and asking for their views after the initial position, or just showing the initial position and letting them read deeply from there?
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
Re: AI & Igo Hatsuyoron 120
I started from the position after move 142 of the modified (";B[]" and ";W[]" instead AB[] and AW[]) SGF that contains the sequence of our solution by using the option "Analyze this" in ZBaduk.Uberdude wrote:Are you guiding these bots by playing out sequences and asking for their views after the initial position, or just showing the initial position and letting them read deeply from there?
This will make the AI evaluate the next move.
When I got the feeling that the result of this evaluation was quite "stable" (e.g. there was a move with x.000 playouts, and all other candidates shown had only y00 playouts, or even less / one candidate move had much more playouts than all others), I advanced to the next move (of our solution sequence, not the AI's preferred move) by clicking ">".
For the variations shown, I clicked the first move of it on the board, thereafter following each of the AI's preferred moves until the end of the variation.
All results of the analysis were still available (unless I closed the browser), so I made screenshots for every move analyzed so far. There were several sessions, each starting with the (first) position of the solution sequence not yet analyzed.
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
Re: AI & Igo Hatsuyoron 120
Black has a fatal inclination to O3 in the hanezeki (also often after inserting the R11 / S10 exchange).
The programs' passion for mistakenly connecting the hanezeki's tail with a move at O3 (therewith capturing three White stones) is shown here with LeelaZero at move B 199 = B 57 (orinal solution sequence).
This move is a fatal mistake. Black should have continued with K13.
When entering the variation, White is occupying the vital point of the three-point nakade.
The sequence of first forcing with 1 and 3, thereafter playing at R1 / R2 (aka T1 / T2) in the corner is already known to us.
As Bill pointed out, the AI is very fond of EARLY forcing moves. Capturing three Black stones with W 6 in the lower right corner is a mistake, however.
Black and White continue in the corner.
Finally, White chooses the move at K13 (aka M13) that she should have played much earlier, instead of capturing with 6.
After her mistake in the corner, White is unable to capture Black's tail group now.
White's move is the "only" move in the candidate moves' list, but has a win rate of about 33 % only.
Please remember that Black's large group in the upper right is unable to get two eyes.
The programs' passion for mistakenly connecting the hanezeki's tail with a move at O3 (therewith capturing three White stones) is shown here with LeelaZero at move B 199 = B 57 (orinal solution sequence).
This move is a fatal mistake. Black should have continued with K13.
When entering the variation, White is occupying the vital point of the three-point nakade.
The sequence of first forcing with 1 and 3, thereafter playing at R1 / R2 (aka T1 / T2) in the corner is already known to us.
As Bill pointed out, the AI is very fond of EARLY forcing moves. Capturing three Black stones with W 6 in the lower right corner is a mistake, however.
Black and White continue in the corner.
Finally, White chooses the move at K13 (aka M13) that she should have played much earlier, instead of capturing with 6.
After her mistake in the corner, White is unable to capture Black's tail group now.
White's move is the "only" move in the candidate moves' list, but has a win rate of about 33 % only.
Please remember that Black's large group in the upper right is unable to get two eyes.
Last edited by Cassandra on Sun Sep 15, 2019 12:23 pm, edited 1 time in total.
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
Re: AI & Igo Hatsuyoron 120
LeelaZero: "Need support! Don't know what to do!"
White 202 = W 60 is the first move where LeelaZero examines a wild bunch of candidate moves that are located in a small win rate corridor.
Each of the candidate moves from a to s has a win rate in the range 43.5 % - 43.0 %.
These candidate moves may differ from analysis to analysis, but the priciple effect remains stable.
KataGo does not have a similar problem here. It prefers the correct move W H12.
White 202 = W 60 is the first move where LeelaZero examines a wild bunch of candidate moves that are located in a small win rate corridor.
Each of the candidate moves from a to s has a win rate in the range 43.5 % - 43.0 %.
These candidate moves may differ from analysis to analysis, but the priciple effect remains stable.
KataGo does not have a similar problem here. It prefers the correct move W H12.
Last edited by Cassandra on Mon Sep 16, 2019 12:30 pm, edited 1 time in total.
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
Re: AI & Igo Hatsuyoron 120
Confusion in the analysis.
With move 65 of the solution sequence, both programs prefer to follow the classical solution by initiating the Crosscut Sequence with G14. Let's call this decision "semi-correct".
Win rates:
55 % LeelaZero
30 % KataGo
However, after the six moves of the forcing sequence in the top right corner have been inserted (Black Q16 to White P18), both programs switch to the suicidal Black move at R14, instead of playing my guzumi at S13 or at least following the classical solution with a mvoe at c again. Black will lose decisive liberties by creating an eye for his large group in the top right.
Win rates:
75 % LeelaZero
35 % KataGo
With move 65 of the solution sequence, both programs prefer to follow the classical solution by initiating the Crosscut Sequence with G14. Let's call this decision "semi-correct".
Win rates:
55 % LeelaZero
30 % KataGo
However, after the six moves of the forcing sequence in the top right corner have been inserted (Black Q16 to White P18), both programs switch to the suicidal Black move at R14, instead of playing my guzumi at S13 or at least following the classical solution with a mvoe at c again. Black will lose decisive liberties by creating an eye for his large group in the top right.
Win rates:
75 % LeelaZero
35 % KataGo
Last edited by Cassandra on Tue Sep 17, 2019 1:19 am, edited 1 time in total.
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
Re: AI & Igo Hatsuyoron 120
LeelaZero: "Need support! Don't know what to do!"
White 214 = W 72 (i. e. after Black's guzumi) is another move where LeelaZero examines a wild bunch of candidate moves that are located in a small win rate corridor.
The win rate of these candidate moves is 35.5 % - 35.0 %.
= = = = = = = = = = = = = = = = = = = =
KatoGo is much more willing to make decisions, but nevertheless fatally mistaken.
Another fatal inclination to a point in the hanezeki.
KataGo's preferred move might be meant as a forcing move, as Black answers in the lower right corner, following his own inclination.
Finally, White blocks the tail with a move at G14, but preventing two Black eyes at the right with a move at U14 (aka S14) would have been correct.
White 214 = W 72 (i. e. after Black's guzumi) is another move where LeelaZero examines a wild bunch of candidate moves that are located in a small win rate corridor.
The win rate of these candidate moves is 35.5 % - 35.0 %.
= = = = = = = = = = = = = = = = = = = =
KatoGo is much more willing to make decisions, but nevertheless fatally mistaken.
Another fatal inclination to a point in the hanezeki.
KataGo's preferred move might be meant as a forcing move, as Black answers in the lower right corner, following his own inclination.
Finally, White blocks the tail with a move at G14, but preventing two Black eyes at the right with a move at U14 (aka S14) would have been correct.
Last edited by Cassandra on Wed Sep 25, 2019 2:25 am, edited 1 time in total.
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)
-
RobertJasiek
- Judan
- Posts: 6273
- Joined: Tue Apr 27, 2010 8:54 pm
- GD Posts: 0
- Been thanked: 797 times
- Contact:
Re: AI & Igo Hatsuyoron 120
Just a quick first question: does your analysis mean that AI is too weak to play well in H120?
-
Cassandra
- Lives in sente
- Posts: 1326
- Joined: Wed Apr 28, 2010 11:33 am
- Rank: German 1 Kyu
- GD Posts: 0
- Has thanked: 14 times
- Been thanked: 153 times
Re: AI & Igo Hatsuyoron 120
I would like to say it with Bill ...RobertJasiek wrote:Just a quick first question: does your analysis mean that AI is too weak to play well in H120?
Bill Spight wrote:... the main virtue of today's top bots lies in evaluation, not the calculation of variations.
As far as I know, AI can become superhuman ONLY in the very small field of tasks it has been trained for.Bill Spight wrote:... today's bots were not built to handle such positions.
The Go AI has been trained to PLAY GAMES of Go, so it's a pity, but not very surprising that the Go AI does not perform very well when the task is to continue and win an ARTIFICIAL board position.
I am very sure that the AIs never encountered e.g. a hanezeki, a nine-point nakade, one large semeai across the board, or any of the other (sub-problem) shapes masterfully created by Inoue Dosetsu Inseki during their training processes. The programs' inclinations to (locally) gote moves in the hanezeki are evidence of that.
I think that this is comparable with the (also) whole board shape that Lee Sedol managed to create, therewith successfully beating AlphaGo. In these cases, it seems to me that the programs cannot resort to their superhuman knowledge, and fail (and in doing so seemingly playing at amateur level).
Especially with Igo Hatsuyoron 120, the reading depth of the programs may be too low to "understand" the implications of this whole board position.
= = = = = = = = = = = = = = = = = = = = =
tl;dr
The AI has still a long way to go ...
The really most difficult Go problem ever: https://igohatsuyoron120.de/index.htm
Igo Hatsuyōron #120 (really solved by KataGo)
Igo Hatsuyōron #120 (really solved by KataGo)