reviewing SL articles using LZ and criticism

Knotwilg · #1

Recently I have been reviewing some of SL's material using LZ, notably whole board positions and claims by either amateurs or pros recited by amateurs, that a play is good, biggest, urgent ... For example, at https://senseis.xmp.net/?GetTheLastBigMoveInFuseki one diagram is called "good" because it "allows taking the last big point" but another is bad because "it allows the opponent to do so". By LZ's evaluation, these diagrams differ 1% and both differ 10% from LZ's preferred sequence. This led me to question the content on the page and suggesting to rephrase and even rename the page or remove it altogether.

While adapting the material is more positive than removing it, one SL deshi has the following criticism on my review of the page:

Quote:

how do we know that LZ suggestion isnt refuted by Golaxy suggestion, which might be refuted by FineArt, which might be overturn by AGZ which might be wrong according to Dosaku? I agree with Xela, we need human way of thinking until we can utilize super strong bots suggestions into heuristics available for humans.Remember that SL is rather for players up to strong dan, not pros.

This criticism surprises me because it suggests we should continue to rely on pro advice even if it is refuted today by LZ, because we are allegedly unable to understand LZ (or bots in general). I would say that we have equal tentative understanding of Dosaku, Shusaku, Go Seigen or AlphaGo, while on the other hand we aren't totally void of Go insights, so we can do the interpretation of either with varying degrees of confidence. Although the shin fuseki was annotated by Go Seigen, still a large part of our understanding of it comes through our own mental process, as it should, and therefore, we should not refrain of making similar attempts to understand unannotated bot analysis.

Also, first experiments show that bots have a tendency to think alike and there are no major differences in policy. Elf may have a more radical probability evaluation than others, but I've learned to treat a 10% probability difference in LZ as significant. Significant enough to make a critical review of traditional evaluation as in the mentioned page.

The final message that "SL is for players up to strong dan, not pros" strikes me as a non sequitur.

What do you think?

Bill Spight · #2

I say Bravo! :salute:

Bill Spight · #3

Knotwilg wrote:

For example, at https://senseis.xmp.net/?GetTheLastBigMoveInFuseki one diagram is called "good" because it "allows taking the last big point" but another is bad because "it allows the opponent to do so". By LZ's evaluation, these diagrams differ 1% and both differ 10% from LZ's preferred sequence.

I find the page confusing in its present state, but I am sure that it will get cleaned up soon. However, looking back to version 13, before the recent edits, I think it was wrong, anyway. How can the joseki (now maybe obsolete) in the top left corner get the last big point in the opening when the right side is wide open?

Edit: Looking again at the original page, it looks like the play Black actually made on the bottom side he is supposed to get later. But it seems to me that it is more of an attacking move than a big point. Anyway, rather than saying it was wrong, I now think that it was incoherent.

But anyway, I cannot tell from the Elf commentaries that the bots care much, if at all, about either the last big play in the opening or direction of play. I have always been a student of the opening, and I have to say that much of what I learned is simply wrong. It needs a rethink.

Kirby · #4

Hmm, interesting situation.

On one hand, there is the question of accuracy. What’s said to be the “best move” may not be correct. On the other hand, there is value in understanding a principle or model of thinking, even if it doesn’t always give the right answer. It’s a way for humans to think about the game, even if bots sometimes disagree.

Given that the purpose of that page is to illustrate the principle of getting the last big play, I would advocate trying to illustrate that principle. Maybe it’d be more accurate to find game positions for which the principle more accurately applies. But it’s good to illustrate the principle in some way. I’d in fact favor illustration of the principle to accuracy if one must be sacrificed (ideally you can have both).

All that being said, I think the argument about skill level of SL readers is bogus.

Note: If it can be shown that the principle is NEVER good, maybe we should rethink having the page at all.

Knotwilg · #5

Kirby wrote:

Hmm, interesting situation.
On one hand, there is the question of accuracy. What’s said to be the “best move” may not be correct. On the other hand, there is value in understanding a principle or model of thinking, even if it doesn’t always give the right answer. It’s a way for humans to think about the game, even if bots sometimes disagree.

I think the following example falls into that category:

Click Here To Show Diagram Code: [go]$$B Mimic go "punished" $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . 1 . . . . . . . . . . . . c . . | $$ | . . . , . . . . . 9 . . . . . 3 . . . | $$ | . . . 5 . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . 8 . . . . . a . . . . . 7 . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . 6 . . . | $$ | . . . 4 . . . . . 0 . . . . . , . . . | $$ | . . b . . . . . . . . . . . . 2 . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

To many players, the idea of playing A will be visually appealing, although the bots think Black should invade at B and if he plays A, then White should invade at C.

So the point is nicely illustrated but the bots advise against it. Should we still teach a principle of which the most telling example is wrong by AI standards?

John Fairbairn · #6

Three points:

1. You say 10% LZ is what you regard as significant. Not sure what you mean by that, but if it means something like "anything less than 10% does not matter very much" then, in the mimic go example, a Black move at the centre point seems to lose ~5% - so does that mean it's "insignificant."

2. Many examples in the traditional canon come from books and games in eras when there was no komi, 4.5 komi or 5.5 komi. Presumably the bots can't take that into account; humans can. That said, SL does not often give sources or authorities and so we can't know whether komi differences have been/can be factored into what is said there.

I went through some books I bought on my last trip to Japan to see whether the bots agreed with the pro authors - I chose that sample in the belief that they had possibly incorporated AI results. It seems they didn't. I reported some results here, but can't remember the detail. However, I do recall shock at some books where around 50% of a book's content was flawed according to bot analysis (and because these were recent books, they can't use the excuse of komi differences). But were these really pro opinions, or those of amateur ghosts? I don't know, but strongly suspect the latter. And even if a pro is involved, I further suspect he is giving the answer he thinks the amateur reader is most likely to understand and get some use from - not what he really thinks is the best move for himself.

All of which brings me to my third point:

3. Be careful not to throw the baby out with the bathwater.

John Fairbairn · #7

Here is a specific example to go with what I said above. I think it shows the problems are bigger than we can imagine.

This is from a 2015 (pre AI) book by Yamada Kimio (a titleholder) on how to lay waste to an opponent's prospective area by the techniques of arashi. It is White to play.

Yamada (or his ghost) has already explained that where the opponent has a star point position flanked by extensions, the correct point to invade is usually the 3-3 point (and bots seem to agree). But where there is no extension (the maximum is a five-spacer) on one side, the correct point is likely to be an approach move. His explanations are easily understandable and seem very useful to me - I can do the example problems he gives later.

But LZ does not agree here. LZ wants to play A here. That I can't really understand. It seems to offer Black an almost cost-free squeeze which just strengthen the area (sphere of influence in the lower right quadrant) that Black would surely love to strengthen.

But what LZ actually is expects after A is Black B, White C!!!!!! Sure, I can understand the bots like 3-3 point invasions, but why on earth waste A and offer Black an alleged 60-point tortoise-shell capture? In the LZ analysis, play continues in the lower right (though it looks reasonable enough for Black to me) then White returns to the tortoiseshell and lets Black finish off the squeeze. Beyond surmising that A was a probe, I am completely bewildered by this sort of play. And I believe even pros would be bewildered by it, on the grounds that I don't recall ever seeing any similar play or any analysis of a similar position. I even suspect a possible AI bug, to do with ladders.

The move chosen by Yamada was D. I found this easy to predict and easy to understand (White has room to settle himself easily when there is no extension) and easy (I think) to evaluate. But LZ says A gives White win rate of around 66% whereas D gives 57%. That seems a big difference. This does not change materially when you do the Spight analysis and check the win rate after playing the human move.

Of course the AI move is not without interest. But in practical terms for us humans it is worse than useless. This seems to apply in very many other examples, too.

Knotwilg · #8

John Fairbairn wrote:

Three points:

1. You say 10% LZ is what you regard as significant. Not sure what you mean by that, but if it means something like "anything less than 10% does not matter very much" then, in the mimic go example, a Black move at the centre point seems to lose ~5% - so does that mean it's "insignificant"

That I consider 10% significant, does not logically imply I consider 5% insignificant. I've taken that threshold from discussions here. Let's say that I won't try to understand why a bot makes a 1% difference but 10% probably means there's a meaningful, intelligible and applicable evaluation by human amateur standards. How about 5%? I may try understanding where it comes from, but I won't be frustrated if I can't.

Quote:

I reported some results here, but can't remember the detail. However, I do recall shock at some books where around 50% of a book's content was flawed according to bot analysis (and because these were recent books, they can't use the excuse of komi differences). But were these really pro opinions, or those of amateur ghosts? I don't know, but strongly suspect the latter.

On SL, I'd allow myself to critically review amateur interpretations and color professional advice with significant deviations reported by LZ.

Quote:

And even if a pro is involved, I further suspect he is giving the answer he thinks the amateur reader is most likely to understand and get some use from - not what he really thinks is the best move for himself.

Again, the move a pro would choose or a move the bot will choose comes from marginal differences in evaluation. If a bot makes a 10% difference, that's not marginal for me but meaningful, even at the amateur level. If a pro recommends a move to an amateur, because it is within their grasp, but the bot says it's 10% worse than something else, I think we should try understanding where that evaluation comes from, which means we may find out the pro's positional judgment was off.

Not trying to do so, because of the risk of committing sacrilege, is not what human knowledge is made of.

Quote:

All of which brings me to my third point:

3. Be careful not to throw the baby out with the bathwater.

I won't. Thanks for the comments.

Bill Spight · #9

Kirby wrote:

Given that the purpose of that page is to illustrate the principle of getting the last big play, I would advocate trying to illustrate that principle. Maybe it’d be more accurate to find game positions for which the principle more accurately applies. But it’s good to illustrate the principle in some way. I’d in fact favor illustration of the principle to accuracy if one must be sacrificed (ideally you can have both).

A couple of things. First, I am not at all sure that that page in particular illustrates getting the last big play. Second, it expresses a strong opinion, probably second hand from a pro, that one play is good and the other is bad. Then LZ tells Knotwilg that both plays are bad. That is useful information, and deserves to be on that page.

I took a look at the 1950s, the heyday of Takagawa and GoSeigen, for examples of oba no tedomari, and not just by them. My impression is that Elf usually regarded such plays as minor errors in the 5-7½% range, and sometimes well within the margin of error. By contrast, I noted a few months ago that Elf often regarded oba, even Takagawa's, as definite errors, losing 10% or more. So maybe the principle, from Elf's point of view, is that the last oba is not as bad as the first. :lol:

Bill Spight · **#10**

John Fairbairn wrote:

LZ wants to play A here. That I can't really understand. It seems to offer Black an almost cost-free squeeze which just strengthen the area (sphere of influence in the lower right quadrant) that Black would surely love to strengthen.

But what LZ actually is expects after A is Black B, White C!!!!!! Sure, I can understand the bots like 3-3 point invasions, but why on earth waste A and offer Black an alleged 60-point tortoise-shell capture? In the LZ analysis, play continues in the lower right (though it looks reasonable enough for Black to me) then White returns to the tortoiseshell and lets Black finish off the squeeze. Beyond surmising that A was a probe, I am completely bewildered by this sort of play. And I believe even pros would be bewildered by it, on the grounds that I don't recall ever seeing any similar play or any analysis of a similar position. I even suspect a possible AI bug, to do with ladders.

That's my guess. Maybe LZ does not realize that the White D-03 stone is not a ladder breaker. But it also has learned that Black can play any number of ladder makers in the center fight that may well come. (Edit: Or even without a fight.) So before that happens it plays A, expecting B, so that Black cannot play such a ladder maker anymore. With that accomplished, it switches to its beloved 3-3 invasion.

Uberdude · **#11**

Regarding John's example, I'm not going to speculate on the ladder escape without doing some LZ analysis myself, though a reminder that what one version of LZ thinks with some number of playouts can be very different to another LZ or another number of playouts.

But one candidate move that I have learned from AI to strongly consider in this sort of position where you might approach on the side but the kick is a good response is the r4 attachment. If they hane outside you get a comfier corner than direct 3-3 (because if they block at Q3 you made them block the 3-3 on the wrong side as you get into the valuable right side, so solid connect is normal and then you can Knight move slide), and if they hane inside them you get more eyespace on the side than with the approach (but at cost of losing 3-3 aji, but if your side group is going to be weak will you ever get a reasonable time to use it?). Antti Tormanen mentioned this was one of the important things he learnt from AI. Maybe it used to be played very occasionally as a special purpose move, but we should add it to our standard repertoire of ways to make a group in opponents framework from a 4-4 (without a 2nd move, attach against 4-4 shimari was a standard idea).

Knotwilg · **#12**

Whenever I suspect a ladder is at play in weird LZ suggestions, I check with KataGo, which is known to be much better at ladders.

Indeed, when feeding this game into Kata's engine, it understands the ladder very well and doesn't consider the preposterous move suggested by LeelaZero. Even more, it gives the ladder a prominent place in its evaluations and instructively so. Who among us would consider the ladder breaker - moyo builder?

Kirby · **#13**

How about a simpler example? Bots like the early 3-3 invasion, which is against traditional go theory. But 3-3 invasion can still be bad in some cases.

If we had an SL page talking about how early 3-3 invasion is bad, should we just get rid of it? Should we adjust it with an updated model of what seems to be more correct? I'd think it'd be good to do the latter - utilize AI to identify the conditions under which the old model is correct (i.e. when it's bad to invade 3-3 relatively early) and when it's not correct (when it's good). If we can find a good pattern, then it's worth keeping the page.

If the old idea seems totally wrong (e.g. it's always good to invade 3-3 early), then maybe it's worth getting rid of the page(?).

Principles are useful if we can identify principles that can help us. I'd like to aim to do that if we can... But if there's no underlying principle to be found from an old proverb... Maybe it's best forgotten.

Bantari · **#14**

<rant>
I am always suspicious when confronted with this kinds of arguments. LZ says something is 10% better... so what? What does it mean? WHY is it 10% better, what is the IDEA behind making it so? Is it only better because LZ can prune the tree (or whatever LZ does) in a way humans will never be able to do? Or. is there an actual reason?

Same goes for human pro evaluations, or even human ama evaluations. Many times different players have different opinions about moves... It is not clear to me that we should always accept the opinion of player A as gospel - just because player A can beat player B and player C. Maybe this time player C is right?... Or maybe there is no spoon?...

My point here is that I wish such evaluations were done in terms of 'ideas' rather than relative weights, since ideas you can translate into a game plan and this can help you play better subsequent moves. Sure, knowing that LZ finds something 10% better might make you think about your ideas, but maybe not. And until you you get into the realm of ideas, it is all pretty meaningless.

Having said that - the SL article from the original post - at least it tries to give you an indication WHY the author thinks one move is better than the other. You can do something with that... agree, disagree, come up with a better idea, something... It is USEFUL.

What if LZ can never be translated into human thinking? Its like designing a road because it s good for a race car... but is it also good for a truck or a train or a donkey?

Sometimes I think the idea of AI 'advice' for human play is even worse than a pro 'advice' for a beginner - saying 'this move is better' does not mean much at all, and can even be counter productive. And its funny to watch people on the servers trying to play like an AI... and a little sad. Like that time when everybody wanted to be Sai or Hikaru... what is happening to this game I used to like?

Ok, I feel better now. Thanks, everybody.

</rant>

xela · **#15**

Sorry for the brevity of my comments on SL! I'm travelling this week, with limited time for go discussion. My main point is don't be too hasty to delete anything. Will have more to say on Saturday. Great to see such a lively debate!

Bantari · **#16**

Kirby wrote:

If the old idea seems totally wrong (e.g. it's always good to invade 3-3 early), then maybe it's worth getting rid of the page(?).

I would be careful here. My idea is to absolutely keep the page, but maybe provide a disclaimer with a link to the page that refutes the idea.

What if "LZ version 100.0" - or some other AI - will decide that the idea is good after all, and it will beat all the other AIs with that?

If nothing else, it will be interesting for historical reasons to see what were the outdated ideas, and what it took to abandon them. And what replaced them and why. This by itself can be a good learning tool... Like going through phases.

Bill Spight · **#17**

Knotwilg wrote:

Even more, {Kata} gives the ladder a prominent place in its evaluations and instructively so. Who among us would consider the ladder breaker - moyo builder?

Go Seigen, fer shure.

He showed such light reductions in his 21st century go books, and all pros are aware of ladders. Probably O Meien, Yoda, and others, as well.

Knotwilg · **#18**

Bantari wrote:

<rant>
I am always suspicious when confronted with this kinds of arguments. LZ says something is 10% better... so what? What does it mean? WHY is it 10% better, what is the IDEA behind making it so? Is it only better because LZ can prune the tree (or whatever LZ does) in a way humans will never be able to do? Or. is there an actual reason?
</rant>

Indeed LZ won't tell us. But as I've often said, pros don't talk a lot either during analysis. They show sequences to each other.
And while LZ doesn't tell us, we can still try understand why. If you look at the original page, you'll see that I'm trying to articulate what LZ shows us. This is of course my fallible low dan interpretation, but it's unfair to state that all we do in there is dismissing pro analyses with mere LZ percentages.

Your other argument, that maybe one day a superbot will tell us the pro was right after all. OK, that, or the superbot tells us 1-1 is best. We can't work with these hypotheses. We must trust in today's strongest evaluators and try learning from them, whether by developing our intuition and seeing moves in certain situations, or by abstracting and articulating concepts.

gowan · **#19**

When bot evaluations are based in high numbers of playouts presumably we are looking at results of "reading" deeper than humans can do. Therefore, it seems to me that we humans might not understand why the move is best or even how to handle the ramifications.

Bill Spight · **#20**

gowan wrote:

When bot evaluations are based in high numbers of playouts presumably we are looking at results of "reading" deeper than humans can do. Therefore, it seems to me that we humans might not understand why the move is best or even how to handle the ramifications.

Except that today's bots read the whole board with a variant of breadth first search. That's why humans, who tend to use depth first search, can still read more deeply than they can in certain situations. Like reading out a ladder better than LZ can.

I am really quite optimistic about human understanding. Look how easy it has been to play the early 3-3 invasion of the 4-4 and to realize that you want to avoid the second line hane-tsugi in most cases. Why do the bots love the 3-3 invasion, but not so much the 3-3 opening? Quien sabe? But I don't have to know to play better.

reviewing SL articles using LZ and criticism

Who is online