Leela Zero analysis of 'Making good shape' and other books.

[zermelo]

My new hobby is to use LZ to analyse problems from books. Mostly of the whole board, shape, haengma type rather than l&d. It is quite easy to find problems where LZ has the opposite opinion from the book answers.

I'll show here a few from 'Making Good Shape' by van Zeijst and Bozulich. (Incidentally, I always found the book difficult, and took it to mean I have a poor brain for shape. At some point I tried to memorise all the answers from the book, but many of them would just not stick to my brain.)

The book problems are not whole board problems, so I tried to put a minimal amount of reasonably placed stones on the board that makes LZ interested in the problem area. We can discuss of course how much the rest of the board matters.

So far I looked at the first 5 problems in the book. On the first 3 LZ agreed, but on problems 4 and 5 it already disagreed.

Problem 4:


Here the original problem is for black to move for stones in the top right corner of the board. Book answer is the circle, and LZ thinks it is -4.5%. LZ answer depends on the rest of the board but I could not make it agree with the book move on any board. My intuition was the angle move that has 58% in the above image, so LZ somewhat agrees.

Continuation if B plays the book move:


Book continuation for W is the white circle, inducing B nobi (incidentally, LZ would then jump, not nobi).
LZ as W continues with the kosumi. Now B could theoretically cut but that would create bad aji in the corner it seems.

Then problem 5, again B to move, with top right stones shown in the problem:


Here book answer is the kosumi that LZ finds slightly inferior. LZ wants to descend, which the book calls a 'failure', because white descent next threatens the cut. Looking at the continuation, LZ does not worry about the cut but would e.g. sacrifice the 2 stones.

Instead LZ worries about this peep:


Of course W could peep after the earlier descent too, but according to LZ it would be less helpful then.

I will maybe post more cases in this or another thread, maybe from other books too.

[Edit: Tried to improve some wording, and added the last sentence]

[Uberdude]

Problem 4 is already known to be dubious, Mingjiu Jiang 7p disagreed with the book answer (didn't suggest LZ's attach but the 56.5% kosumi from the corner), see validity section here: https://senseis.xmp.net/?MakingGoodShape. But yes, a nice exercise, I've been doing some similar things myself.

[zermelo]

Continuing the exercise I find it more and more fascinating. I decided to go through the first 9 problems of the book, which are on the first problem page. Now that I did it, I'll put the analysis in the next posts, and then give some summary afterwards.

So on problems 1-2 LZ agrees with the book. Problems 4-5 I already went through.
I actually claimed that LZ agrees on problems 1-3 with the book. I now need to confess I actually had only checked problems 1-2, and thought problem 3 is so obvious I can skip it. But fortunately I decided to check it with LZ anyway, because the result was very interesting. I promise from now on I will have actually checked all problems I mention.


So, problem 3:


I added the black stone in the lower left corner so that there is a consistent number of B and W stones and the win rates are reasonable.

Here the problem description asks to connect the black stones. The book answer is the well known double keima tesuji. But how is it possible that LZ likes the hane as good or better?

The book calls the hane a failure, because white can cut first at 3-3 and then separate the black stones, ending like this:


But notice the win rate for black has now increased! It seems that the issue is sente, i.e. white can only cut in gote. Also the white wall has the bad defect. It seems this is why LZ does not worry about the cut much at all. The issue is not only because the board is so empty. Even after adding many moves, LZ does not find the cut necessarily very interesting.

Actually after the initial black hane white can also try to cut by wedging at 5-3. Whether that works, depends on a ladder. That also seems to affect how LZ evaluates the book 'failure'.

In conclusion, I would not say that the book is wrong, because it asks to connect the stones. Still, if the book 'failure' is on a real board often better than the correct answer, I think the problem is not very well constructed. If the goal is to teach the double keima connection, that could be done in a better context.

[zermelo]

On problem 6, LZ seemed to agree with the book unambiguously.

Then problem 7:


Again, the circled intersection is the book move. On this board, LZ clearly prefers the kosumi-tsuke, which the book calls a failure.

Here is the book continuation for the 'correct' move:


Here is the book continuation for the 'failure':


According to the book, now B "stones are not as thick, and White has more room to maneuver". But e.g. on this board LZ sees things the other way round. Notice that the 'failure' has the lower win rate for W, i.e. is better for B in LZ evaluation.

You can notice that the board is quite empty. Maybe one issue is who gets to play in the empty corner. After playing around I found that on some boards with B stones in upper left corner LZ likes the book answer. It still seems to me it likes the failure more often.

I see that the book answer has value, because it shows a way for black to wrap around the W stones (in gote). However calling the other option a failure seems to be just wrong.

[Calvin Clark]

The reckoning begins...

This is going to be ugly.

There is a BIBA video where Blackie plays BensonDarr and loses. At one point Diana Kosegi, who is sitting beside him commenting, says:

Shape is not important. Sente is important.

I have to find that point in the video. It's awesome.

[zermelo]

Problem 8 was really interesting:



I had to add quite many stones elsewhere to make the problem area 'hot' enough.

The problem asked to separate the white stones, so the sequence started by the hane and then inside cut is an obvious way to do it.

LZ consistently wanted to play the tsuke instead, as in the image above.

Look at the variations here:



Variation 1 shows what LZ wants to do, building a higher and stronger wall and cutting a stone on the center side.

Variation 2 shows the book answer with one LZ continuation. B cuts some stones, but at the end it seems B shape is not so great and W stones look light to me.

In this case I again would not say that the book answer is wrong, but it seems that Leela Zero found a more interesting way to play. So if the point is to teach the hane-inside-cut sequence, the problem could be better constructed. And if the reader is left wondering if there are more interesting moves, he or she could be right...

[Edit: corrected some black/white mistakes]

[zermelo]

Then problem 9:


Here the problem asks where B should play to "preserve his shape".

It was again surprisingly difficult to make LZ interested to play in the problem area as B at all. As white it wanted to hane, but then black can live on the side and white cannot wrap black solidly without losing sente. When B wanted to play, it wanted to jump rather than nobi, which is the book answer. The image does not show the win rate for the nobi, but it was considered a mistake of a couple of percentage points.

The book tells us that the hane is a mistake, because white will cut and black cannot capture the cutting stone. LZ agrees with that, but suggests a better move than the book.

[EdLee]

Hi Calvin,

I enjoy to read zermelo's studies.

Shape is not important. Sente is important.

Proverbs (and glibs, in general) are a very double edged sword, at least for me.

She was referring to a particular board position,
just like zermelo is demonstrating.

Yes, it's a trap to focus only on a local shape and miss the global context.
But it's also bad to focus exclusively on sente ( or any other aspects of Go, really ) at the expense of the most important thing on the whole board at the moment, which could well be spending a gote to fix a local defect ( that has big global implications! ) It's why Go is so fun and frustrating.

[zermelo]

My summary on the Leela Zero analysis of the 9 first problems of 'Making good shape':

3/9 problems: LZ agrees clearly. (Problems 1, 2, 6)
3/9 problems: LZ disagrees clearly, finding the ‘failure’ better on most boards. (Problems 4, 5, 7)
2/9 problems: LZ agrees that the 'failure' is wrong, but finds a better local move than book solution. (Problems 8, 9)
1/9 problem: LZ thinks doing what the problem asks is often just wrong. (Problem 3)

Of course it was just a few problems. I now regret that I did not sample problems randomly, but started from the beginning. If I find time, I'll look at a few random problems more.

[dfan]

Thanks, this was fascinating. I did a similar experiment with some problems from Test Your Go Strength (and those are whole-board problems, which is more fair to Leela Zero, though maybe the komi is off by 1 or 2), and found some interesting results, although I doubt I will find the energy to write them up as nicely as you did here.

[Bill Spight]

I really appreciate this thread. Much food for thought.

As a composer of problems I am familiar with the following phenomenon. You have a certain idea in mind when you compose a problem and, after some effort, you succeed in making up a problem to illustrate or embody that idea. Later you realize (hopefully) that there is some aspect of your problem that you completely ignored, but is important. This happens a lot, at least to me. The thing is, even if your solution is still superior to other plays, you really ought to address the other aspect, or redo the problem. Anyway, I sympathize with composers to whom this happens.

Problem 3 is not really a composition by the authors, even if they did not copy it from somewhere else. It is a standard position that illustrates the tesuji to connect underneath. If the problem is presented as such: How does Black connect? Which was how I learned it, that's fine. The hane does not connect, so it is a failure. If the composer adds that connecting may not be best, that's good. If not, I don't particularly see that as a problem. OTOH, I learned that tesuji as an SDK, and I have never considered not connecting, but playing the hane instead and taking sente. So that's a blind spot for me. I blame you, Sakata!

So I sympathize with the composers, but as the saying goes, it's not what you don't know, it's what you know that ain't so. This is a very important study. Many thanks, zermelo!

[zermelo]

So, I checked 5 more problems. Now I chose them randomly with a software RNG.
Problems were 107, 164, 36, 211, and 186. Of these, on 36 and 186 LZ agreed perfectly (though I'd say 36 is more of a tsumego than shape problem).

Problem 107 is pretty interesting:


I admit I thought the answer must be obvious. But no! LZ thinks that the 'failure' is at least 5 percentage points better than the book solution.

See variations here:


Variation 1 shows how LZ would continue the book solution.

Variation 2 shows how LZ would continue the 'failure' variation, which it judges better for white.

Again, the problem is supposed to illustrate how white can stay connected (though that goal is not mentioned in the problem setting). It just fails to consider that getting cut can actually give a better result.

Then problem 164:



Here LZ thinks it's move is 3.5% better than the book solution (the 2nd line hane). It agrees that the book failure (descent to 2nd line) is much worse. If white plays the hane, LZ does not block but plays the keima on the center side. However, even if B responds as in the book, LZ thinks it is worse for white than starting with the 5th line tsuke above.

Last problem, 211:



The book asks for the vital point of white's shape, and suggests the angle play. LZ is not impressed, but wants to play one move further from white stones. I checked quite many board configurations, and LZ never seemed to like the 'vital point' best. Sometimes it wanted to block from the center side. The 'failure' given by the book is the 4th line move that LZ does not like either.

See some variations below. The win rate differences are not very big:


So a summary on these 5 problems:
2/5: LZ agrees with the book
1/5: LZ has the opposite opinion of success and failure
2/5: LZ agrees on 'failures', but sees a better local move than the book solution

[EdLee]

Hi Bill,

it's not what you don't know, it's what you know that ain't so.

Glad someone carved out the extra box...
( However, the proportions may be in question... )

IMG_0128.PNG (3.43 KiB) Viewed 20296 times

[mitsun]

"It's not what you don't know, it's what you know that ain't so."
I know this quote originally came from Mark Twain

[Uberdude]

Very interesting stuff zermelo. About the double knight connection of problem 3, I think having empty corners still available is too distorting in increasing the value of sente, though the lesson that it it ok not to connect but let the opponent cut in gote is indeed very valuable. So I think more care needs to be taken in creating a realistic whole board setting for the problem. One idea is to pro pattern search and stick the whole pro game in. I find only one such game when you include the details of the white wall (Cho Sonjin 6p vs Asano Hideaki 6p in the Ryusei from 1990). In this position LZ #157 started off liking the hane (with the double knight connect 1% behind) but then decided tenuki was better (if black descends at top right just answer). If hane and black cut and then descend in gote black loses 20%, this makes sense as white already has a fairly strong group on the right side and the peep of cutting point is nice (at least atari for ko instead of descend for black). After hane it expects black to wedge and then there is the question of the ladder and that seems to confuse LZ somewhat: when black wedges the expected sequence is wedge, descend, split, clamp, descend, extend/peep outside, connect, live 2-2. But when you play out the sequence LZ realizes extend/peep can be cut instead of black doesn't have ladder (he can squeeze though). After even more playouts after the initial hane and wedge LZ would atari from underneath and fight the ko.

MGS p3 pro.PNG (694.27 KiB) Viewed 20267 times

Or here is an old OGS game of mine in which this double knight connection didn't happen but it was a possible sequence I had read and would have been happy to play (it's a kind of san-ren-sei invasion joseki), let's see what LZ thinks:

MGS p3 uber ogs.PNG (687.75 KiB) Viewed 20267 times

Nope, doesn't like it, hane is 9% better! If black does cut then the ko happens, white has some threats lower left but it says fine to ignore black f2 to win ko, but I don't like that wall having to run. What does white gain by winning the ko and spending 1 extra local move vs knight connection: a thicker shape and a few more points but I find it hard to prefer that when the cost is ignoring a black ko threat.

Now here is my attempt to create a vaguely realistic position in which the book answer is correct, without first checking it in LZ:

MGS p3 connect good.PNG (694.74 KiB) Viewed 20267 times

I was successful! I think this is because I gave black those 2 stones on the right side so that it is important and large for white to connect to the r14 stone, and the p13 stone takes care of the ladder with the clamp sequence after hane and wedge. Playing around with the position, just p13 and no p10 is enough to make it strongly prefer knight connection, but with just p10 it prefers hane (realizes ladder good for W). With p10, no p13, and black near tengen to make the ladder it then prefers the knight connection. It actually seems pretty quick to realize the importance of the ladder, for example taking off quite a few stones and then plopping a random ladder-making stone at g7 (vs l4 say) is enough to make it switch from hane to knight connect within 1000 playouts.

MGS p3 ladder.PNG (687.21 KiB) Viewed 20266 times