Life In 19x19

I discovered recently KataGo has a little bit of a rules bug. Japanese rules are hard to program.

https://lightvector.github.io/KataGo/rules.html

An unblock-ko-recapture action consists of a player choosing a ko-move for that player that would capture a region containing a point marked as ko-recapture-blocked, and removing that mark.

So here's the issue, with an example.

Click Here To Show Diagram Code

[go]$$Bc Example double ko death
$$ ------------------
$$ | X O . O X . . . .
$$ | . X O O X . . . .
$$ | X O O X X . . . .
$$ | O . O X . . . . .
$$ | O O O X . . . . .
$$ | X X X X . . . . .
$$ | . . . . . . . . .[/go]

Ideally, white should be able to score all the territory here and count Black's stones as dead, *without* owing another move. Given the way that KataGo's postgame cleanup mechanically works, during the appropriate part of cleanup, White must hold this position stable *without* spending more net board plays than Black, as part of asserting their "claim" that they do not owe a move here, since being forced to play would fill and lose 1 point of territory. White needs to carry out this holding under so-called "pass-for-ko" rules, termed in KataGo's rules document as "unblocking" and "blocking" of ko recaptures, to disassociate it with actual passing. Here's how it plays out right now:

Click Here To Show Diagram Code

[go]$$Bc Example double ko death
$$ ------------------
$$ | X . 1 O X . . . .
$$ | 2 X O O X . . . .
$$ | . O O X X . . . .
$$ | O . O X . . . . .
$$ | O O O X . . . . .
$$ | X X X X . . . . .
$$ | . . . . . . . . .[/go]

Next spends a turn to unmark from being ko-recapture-blocked. Now, white would like to spend a turn to unmark from being ko-recapture-blocked. The issue is, technically at this moment B19 is not a ko-move. There is no ko mouth there at this moment, a White play there captures three stones, not one. However, White does NOT want to capture three stones. Doing so loses a point since White would end up making 2 board plays to Black's 1. White would much prefer to merely unmark without a board play yet, in preparation to be able to recapture what will be a ko mouth at B19 on the next turn after black recaptures at A17. With the current rule, since B19 is not a ko-move right now, White cannot unmark anything, so White will be forced to lose a point.

This quirk of the rules was definitely unintentional, and I'd like to fix it if possible. I think the following fix should work:

Old rule: An unblock-ko-recapture action consists of a player choosing a ko-move for that player that would capture a region containing a point marked as ko-recapture-blocked, and removing that mark.
New rule: An unblock-ko-recapture action consists of a player choosing a single-point region of the opposing color that is in atari and marked as ko-recapture-blocked, and removing that mark.

Can anyone sanity-check if they also think this modification works, and most importantly: can anyone think of any oddities or pathologies that would newly arise from this modification?

As from earlier discussions on this board, we already know that KataGo's rules don't exactly match Japanese practice in rare cases, and I'm okay with that, but I'd like for double-ko-death to not be one of those mismatches. Right now I am NOT particularly looking for side discussions on J89 or how ambiguous or complex Japanese rules may or may not be, or advice on whether some completely different ruleset formulation is called for... just any help and sanity checking for this one specific detail would be wonderful and greatly appreciated, because if it will work, I would like to put this rules patch into practice very soon for the now-recently-started training run.

lightvector wrote:I discovered recently KataGo has a little bit of a rules bug. Japanese rules are hard to program.

https://lightvector.github.io/KataGo/rules.html

An unblock-ko-recapture action consists of a player choosing a ko-move for that player that would capture a region containing a point marked as ko-recapture-blocked, and removing that mark.

So here's the issue, with an example.

$$Bc Example double ko death
$$ ------------------
$$ | X O . O X . . . .
$$ | . X O O X . . . .
$$ | X O O X X . . . .
$$ | O . O X . . . . .
$$ | O O O X . . . . .
$$ | X X X X . . . . .
$$ | . . . . . . . . .

Click Here To Show Diagram Code

[go]$$Bc Example double ko death
$$ ------------------
$$ | X O . O X . . . .
$$ | . X O O X . . . .
$$ | X O O X X . . . .
$$ | O . O X . . . . .
$$ | O O O X . . . . .
$$ | X X X X . . . . .
$$ | . . . . . . . . .[/go]

Ideally, white should be able to score all the territory here and count Black's stones as dead, *without* owing another move. Given the way that KataGo's postgame cleanup mechanically works, during the appropriate part of cleanup, White must hold this position stable *without* spending more net board plays than Black, as part of asserting their "claim" that they do not owe a move here, since being forced to play would fill and lose 1 point of territory. White needs to carry out this holding under so-called "pass-for-ko" rules, termed in KataGo's rules document as "unblocking" and "blocking" of ko recaptures, to disassociate it with actual passing. Here's how it plays out right now:

$$Bc Example double ko death
$$ ------------------
$$ | X . 1 O X . . . .
$$ | 2 X O O X . . . .
$$ | . O O X X . . . .
$$ | O . O X . . . . .
$$ | O O O X . . . . .
$$ | X X X X . . . . .
$$ | . . . . . . . . .

Click Here To Show Diagram Code

[go]$$Bc Example double ko death
$$ ------------------
$$ | X . 1 O X . . . .
$$ | 2 X O O X . . . .
$$ | . O O X X . . . .
$$ | O . O X . . . . .
$$ | O O O X . . . . .
$$ | X X X X . . . . .
$$ | . . . . . . . . .[/go]

Next spends a turn to unmark from being ko-recapture-blocked. Now, white would like to spend a turn to unmark from being ko-recapture-blocked. The issue is, technically at this moment B19 is not a ko-move. There is no ko mouth there at this moment, a White play there captures three stones, not one. However, White does NOT want to capture three stones. Doing so loses a point since White would end up making 2 board plays to Black's 1. White would much prefer to merely unmark without a board play yet, in preparation to be able to recapture what will be a ko mouth at B19 on the next turn after black recaptures at A17. With the current rule, since B19 is not a ko-move right now, White cannot unmark anything, so White will be forced to lose a point.

Under J89 rules the play is hypothetical, so making the extra board play does not matter. The capture proves that the White stones are alive.

If you consider this part of an encore, then it takes place at temperature -1. In that case you can penalize the pass or reward each board play. Also, for encores my Japanese style rules ( https://senseis.xmp.net/?SpightJapaneseStyleRules#toc7 ) use the once only rule.

2. A player may take a specific ko in a particular position of the whole board only once.

With the once only rule the play could go this way.

takes ko one
takes ko two
pass
pass
takes ko two back
takes ko one back

The original position is repeated, but Black cannot pass for ko one, because she has already taken it. Play stops with the Black stones dead.

The J1989 pass-for-a-particular-ko-rule does not produce the behaviour intended by the J1989 Rules authors in all ko positions. Therefore, you cannot maintain the concept of pass-for-a-particular-ko, modify its rule(s) and succeed.

Read my J1989 commentary:

http://home.snafu.de/jasiek/j1989c.html

You learn that disambiguating the J1989 pass-for-a-particular-ko-rule is tedious.

To produce the behaviour intended by the J1989 Rules in all ko positions, you need the generic ko pass rule of my Japanese 2003 Rules

http://home.snafu.de/jasiek/j2003.html

, but you have declared that you do not want other rules. Therefore, accept the reality: YOU DO NOT PRODUCE THE INTENDED BEHAVIOUR.

Since you do not do so anyway, you need not care for the behaviour arcane ko positions.

"An unblock-ko-recapture action consists of a player choosing a single-point region of the opposing color that is in atari and marked as ko-recapture-blocked, and removing that mark."

Since you want to stick closely to J1989, why don't you actually do so? J1989 has a pass-for-a-particular-ko-rule - it does not have an unblock-ko-recapture rule. Such is interpretation rather than the rule itself. Decide: what do you want? J1989 itself, some model rule for programs, or compliance to intended behaviour?

I dislike (un)blocking a stone in atari because play is on an adjacent empty intersection; you should (un)mark the latter.

Bill has already mentioned that the rule is for hypothetical play only.

J1989 ko-pass, J003 ko-pass, your old rule and your new rule all have the same consequence on the played HYPOTHETICAL, alternating sequence for the life of the white stones:

ko capture
ko capture
ko-pass / ko-pass-for-particular-ko / particular-ko-unmarking
dissolve ko
pass (the alternative tenuki is irrelevant, see my commentaries for local-2)
pass

That is, the problem of ko-passing for a currently not existing ko is irrelevant in your example. (But relevant in much rarer examples.)

The HYPOTHETICAL, alternating sequence for the death of the black stones in the corner is:

capture
pass
pass

EDIT: sequences

Two questions:

- I guess that the “net moves” requirement comes from a different use case. Which one?
- What is the definition of “stable”?

I have defined "stable" state of a disturbing ko somewhere directly but cannot find it quickly. It does, however, follow from the definition of double disturbing death here:

http://home.snafu.de/jasiek/ko_types.pdf

So stable is when we have the closed opponent, who then moving first cannot virtual-force uncapturable life of his on the ko.

Unstable is when he as the open opponent can.

Under J89 rules the play is hypothetical, so making the extra board play does not matter. The capture proves that the White stones are alive.

If you consider this part of an encore, then it takes place at temperature -1. In that case you can penalize the pass or reward each board play. Also, for encores my Japanese style rules ( https://senseis.xmp.net/?SpightJapaneseStyleRules#toc7 ) use the once only rule.

Therefore, accept the reality: YOU DO NOT PRODUCE THE INTENDED BEHAVIOUR. Since you do not do so anyway, you need not care for the behaviour arcane ko positions.

Bill has already mentioned that the rule is for hypothetical play only.

Hey everyone, again, I am *not* looking for commentary about J89 rules or ways of entirely overhauling KataGo's rules implementation, or things like that. I am asking a help with a very specific question - does the change proposed, given the rest of https://lightvector.github.io/KataGo/rules.html, result in White keeping full points in double-ko-death, without breaking any other "common" situation?.

Let me give some more context then:

Robert - yes J2003 is impressive work, I actually enjoyed reading through that formulation a lot . But I don't know how I would computationally implement it in a way efficient enough for self-play. And it is precisely the kind of overhaul I am NOT asking for right now and is doubly useless to me because of the constraint I am working under (namely, there are existing neural nets and training data that I need to maintain compatibility with). KataGo's cleanup phase is unfortunately not hypothetical, it functions as an actual play phase, as Bill mentioned, an "encore". I have already said that I accept that the match is not perfect, I am simply asking whether, locally, this patch fixes this one case or not - if you are only interested in suggesting completely alternative methods or claiming that my current approach is hopeless, that is not helpful to me, but thanks anyways.

Harleqin wrote: - I guess that the “net moves” requirement comes from a different use case. Which one?
- What is the definition of “stable”?

@Bill @Harleqin - thanks, yes KataGo has two "encores", where the first one introduces altered ko rules, and the second introduces compensation by 1 point per move, so the second encore behaves a lot like Spight-style rules, where players can actually carry out the captures of the stones they would claim as dead, without loss. The need for the first encore is to avoid the pass fight in this position that is present in Spight-style rules:

Click Here To Show Diagram Code

[go]$$c Does white need to protect at a?
$$ -----------------------------
$$ | . O . O . X a . X O . . . |
$$ | O O O X X X X X X O . . . |
$$ | X X X O O O O O O O . . . |
$$ | . . X X X X . . . . . . . |
$$ | . . . . . . . . . . . . . |
$$[/go]

So anyways, returning to the double ko death case:

Click Here To Show Diagram Code

[go]$$Bc Example double ko death
$$ ------------------
$$ | X O . O X . . . .
$$ | . X O O X . . . .
$$ | X O O X X . . . .
$$ | O . O X . . . . .
$$ | O O O X . . . . .
$$ | X X X X . . . . .
$$ | . . . . . . . . .[/go]

White needs only preserve the double ko death as-is without making net extra plays relative to Black, until the game enters the second "encore". Then in the second encore, White can actually finish off the capture without loss, because of the 1-point compensation per move. White's goal is just to make it through the first encore - with altered ko rules - without dying or having to make an extra play, and the rules change I made seeks to enable White to do this.

A player may take a specific ko in a particular position of the whole board only once.

Yes, KataGo employs this rule, and this is part of the key that I think makes my rules change work. Because with the change I am asking about, we can have: B1 capture, W2 capture, B3 unblock ko, W4 unblock ko, B5 take, W6 take, and then black is stopped from re-cycling without making point-losing plays (for simplicity, suppose the whole rest of the board is settled and dameless and ko-threatless), so must move to the second encore, and at that point, white *can* clean up the position without point loss. Pretty much exactly as you said in your post just now.

The entire issue is whether "W4 unblock ko" is possible. The problem is that "W4 unblock ko" needs to happen at a time when the ko that white needs to unblock is not a ko at all, it is a 3-stone capture. By the current text and computer-implementation (which rigorously follows the text), this is not possible. So the patch I am making to the rules is an attempt to make "W4 unblock ko" possible at that moment.

But just to reiterate for people in general: I am not looking for massive overhauls of things or entirely new rules or discussion about how KataGo mechanically does or doesn't match J89 or should do this or that. What I am looking for is if anyone can help confirm if (A) this specific change works as intended for all "basic" "common" double-ko-deaths and (B) there aren't problematic pathologies newly introduced by it.

In some earlier threads, there were multiple people generous enough to help me debug KataGo's rules as they actually are (https://lightvector.github.io/KataGo/rules.html) , and I'm grateful for that help, and seeking it again here, if possible.

(edit: some revisions and elaborations of for clarity and fixes and elaboration of stuff)

After

B1 ko capture
W2 ko capture
B3 ko-pass / ko-unmark

why do you or Bill need

W4 ko-pass / ko-unmark?!

Again, the simplest next move is

W4 dissolve ko!

Therefore, it is academic whether White may ko-pass at that moment.


Your rule is like J1989 ko-pass so has the same wrong arcane position treatments as J1989 ko-pass. Although you do not want to read it, you would find such wrong positions in my J1989 commentary. They are infrequent but common enough to bother you given the infrequency of the other positions you study. Therefore, sorry, J1989 ko-pass may be implemented but that does not make it correct.

Your major problem though is not to use hypothetical play but continue regular alternation. Therefore, while you pretend J1989, you are actually closer to the Simplified Japanese Rules!

Robert - because W4 dissolve ko, as I explained in the original post, given KataGo's rules as they actually are in https://lightvector.github.io/KataGo/rules.html, causes white to lose a point if done during the first encore phase, because white makes net one more board play than black (and because it is not hypothetical play, it is an actual encore). White needs to survive through the first encore to the second one, before dissolving everything. And yes, I would say that KataGo's rules are most resembling Spight-style Japanese rules, https://senseis.xmp.net/?SpightJapaneseStyleRules#toc7 as opposed to J89 but share enough of the mechanics to reproduce a lot of the same results.

If your next suggestion should be that this whole approach is flawed, or that I should implement rules some entirely different way, then again, thanks, but that doesn't help me. I'm looking for someone to engage with me on the rules as they are right now, and whether this one local patch works as intended without bad side effects.

(I also respect that you might not have time to analyze this, as you have often expressed in the past regarding these things - but if someone else does have the time, that would be wonderful!)

A check for all known suspect positions takes one or two weeks, time I don't have indeed.

Yes, I must say that your 3 rules phases (regular, ko encore, tax encore) approach is bad. Such creates more complications than it solves. Besides, tax without equal number of moves results in pass fights or button anomalies for positions more frequent than double disturbing death at the end. Such are not too bad, but you must understand that you do not model Japanese rules but rather something close to World Mind Sports Games Rules.

I understand your current desire for a quick fix rather than a fundamental correction, but do not expect my approval. Especially none of confirming conformity to Japanese rules design. The sanity check is: your current "Japanese" rules are not Japanese style.

As a mere patch, it is good enough. Whenever you should have time for a greater change, ignore ko arcana and first address fundamental rules design. Again, for the latter the Simplified Japanese Rules (1 temporary encore continues the regular alternation, perfect play is not required, superko) work the best (maybe add the obvious seki exception) and would be as close to J1989 in practice as your way too complicated rules. I would be surprised if retraining nets would be required, unless you can't just tell them to ignore different ko bans.

RobertJasiek wrote: Yes, I must say that your 3 rules phases (regular, ko encore, tax encore) approach is bad. Such creates more complications than it solves. Besides, tax without equal number of moves results in pass fights or button anomalies for positions more frequent than double disturbing death at the end. Such are not too bad, but you must understand that you do not model Japanese rules but rather something close to World Mind Sports Games Rules.

I understand your current desire for a quick fix rather than a fundamental correction, but do not expect my approval. Especially none of confirming conformity to Japanese rules design. The sanity check is: your current "Japanese" rules are not Japanese style.

As a mere patch, it is good enough. Whenever you should have time for a greater change, ignore ko arcana and first address fundamental rules design. Again, for the latter the Simplified Japanese Rules (1 temporary encore continues the regular alternation, perfect play is not required, superko) work the best (maybe add the obvious seki exception) and would be as close to J1989 in practice as your way too complicated rules. I would be surprised if retraining nets would be required, unless you can't just tell them to ignore different ko bans.

Thanks, but I would hope that KataGo already achieves a much closer match to Japanese players' intuitions about how positions should be scored than Simplified Japanese rules even with a seki exception. With the patch I mentioned above and assuming players play skillfully, I *think* we should have all of the following points of agreement with Japanese players' intuition below. Many of which are also achieved by https://senseis.xmp.net/?SpightJapaneseStyleRules (and not by simplified Japanese rules, as far as I can tell), and which again, is the ruleset that KataGo's rules by far most-resembles, with several Spight-Japanese-Style elements being an inspiration for the design of the rules.

1. Simple ko: A player cannot gain an extra point by trying to leave the ko mouth open, even if they have many ko threats on the board.
2. Bent 4 in corner: Dead even if there are unremovable ko threats on the board.
3. Regions surrounded by one side only in seki: Do not count as territory.
4. Seki one-sided dame: One-sided dame will not result in gains to the player able to fill them.
5. Two-step ko: Defender does not need to lose an extra point defending if they have enough ko threats. (Life-and-Death Example 10 in https://www.cs.cmu.edu/~wjh/go/rules/Japanese.html)
6. Double ko seki: Should resolve as a seki, through all cleanup phases.
7. Moonshine life: The false eye group will die, regardless of the presence of the double ko seki.
8. Double ko death: Should resolve as a death, without the player having to lose a point for defending an extra time. (*** this is what we're trying to address with the patch ***)
9. Matti's double-ko+chained-ko (viewtopic.php?p=251373#p251373) Should resolve as a seki.
10. Matti's triple-chained-ko-death (viewtopic.php?p=251327#p251327) Should resolve as a death, without the player having to lose a point for defending an extra time. (*** and this ***)
11. Life-and-Death Example 14 in https://www.cs.cmu.edu/~wjh/go/rules/Japanese.html - white must lose the point for an extra play, in agreement with the commentary, and there is no way to avoid this with a pass fight.
12. A lot of "compound" positions resolve the way that Japanese pro intuition wants them to resolve, for example Life-and-Death Example 16,17,18,... in https://www.cs.cmu.edu/~wjh/go/rules/Japanese.html are all in agreement as far as I can tell.
13. I am not aware of any pass fights at all right now. Maybe one exists, but the mechanism I know of that could normally cause it (such as the one I mentioned in a post above) doesn't seem to pose a problem here.

It is possible I am mistaken on one of the above points, but to the best of my knowledge all of the above are in agreement if we have this patch, assuming this patch works and has no adverse effects. I think it is surprising how large a number of things *can* be reproduced without having to resort to hypothetical play - I was not expecting initially to be able to do so. Even though I am very aware there are still some fundamental limitations of alternating play that make it impossible to be exactly perfect without going to hypothetical play and having something like J2003.

And to list a few known areas of difference or potential areas of difference with Japanese practice (far from exhaustive, of course):
14. Molasses ko is unstable during the cleanup phase and collapses.
15. Torazu sanmoku is three points, instead of seki.
16. The anomalous position B here: viewtopic.php?p=251494#p251494, and in general any situation where a double-ko-death starts during cleanup via throw-in but isn't formed yet prior to cleanup may require an extra defense. (I'd guess this category is probably the most common difference for real games out of all systematic differences).
17. Two dead, only one capturable - http://denisfeldmann.fr/rules.htm#p4 figure 2 - I'm pretty sure this resolves differently in KataGo rules versus actual Japanese practice, especially once you start varying the sizes of some of the groups.

Anyways, back to the topic of double-ko-death: if anyone else has thoughts (or questions) about the situation, or there is a corner case that comes to mind that might be adversely affected by the patch to KataGo rules that I described above, that would be great. Thanks!

@Bill, @Harleqin - let me know if you have any further thoughts or questions.

You try to fulfil mostly the rare shapes but what should worry you are the (more) frequent shapes.

Ko-pass-like rules are only needed to alter a very few rare shapes, such as 1) an ordinary teire ko and elsewhere a double ko death and 2) triple ko with one external ko; for all other shapes, the basic ko rule produces the same result.

That playout in continued alternation leads to the same result compared to hypothetical analysis play in almost all cases we already know from the similarity of area and territory scoring.

The Simplified Japanese Rules are not designed to reproduce the rarities. But the following "frequent" shapes create non-traditional behaviour:

Unfilled basic endgame ko, player's turn:

Strategy 1

pass
ko capture
pass
ko filling
pass
pass

Therefore, the ko stone is not independently alive. Its region is adjacent to both players' independently alive strings so remains on the board. Its empty region is adjacent to both players' strings so not territory.

Strategy 2

pass
ko capture
ko threat
... // player wins ko fight
ko filling
ko threat execution
...
pass
pass

The players would have fought this ko fight before the analysis so it is no problem that the analysis could disturb the final position of the [regular] alternation.

Strategy 3 // correct

ko filling
... // player creates two-eye-formation
pass
pass

Therefore, the ko stone is independently alive and remains on the board. Its empty region is adjacent to only the player's strings so is territory.

Hence you might be right that the Simplified Japanese Rules modified by no territory in sekis is farther from Japanese rules "intuition" than KataGo Rules, provided the latter do not fail in more frequent shapes (such as double threats to kill or ko exchange positions before ko-pass rules apply or 10000-year ko), for which I lack time to check.

Life-and-Death Example 14, White's turn:

play at A // creates seki shape; Black cannot approach to create ko to remove White
... // optional removal of the stone at A
pass
pass

Therefore, the black and white strings are not independently alive and there is no territory. Unlike traditional understanding.

(In the case of Black's turn, Black removes White but a ko exchange can occur. Similar with area scoring, where throwing in to defend does not alter the result though compared to the initial shape.)


EDITs

Do you have a test suite with that list, lightvector?

I don't have anything useful to add re this case, but just want to say I think it's great we have KataGo supporting as close to Japanese rules, whatever they exactly are, as it does already

我觉得，
1、黑白棋子必须成对出现，也就是：偶数终局；
2、如果一方PASS，必须交给另一方一颗棋子作为俘虏；
3、应避免首先下子的一方，收最后单官而导致的单官获利问题；

如果满足以上条件，那么日本数目法还是比较逻辑自洽的。

当然，上述并没有提到日本规则篡改唐宋规则的问题。

ENGLISH：

I think,

1. Black and white Stones must appear in pairs, that is: even end;

2. If one side passes, it must give the other side a stone as a prisoner;

3. We should avoid the problem of last dame's profit caused by 1

If the above conditions are met, then the Japanese rule is more logical and self consistent.

Of course, the above did not mention the Japanese rules tampering with the rules of the Tang and Song dynasties.

pgwq wrote:3. We should avoid the problem of last dame's profit caused by 1

Why do you perceive it as a problem?

If the above conditions are met, then the Japanese rule is more logical and self consistent.

How to meet above conditions without introducing pass-fights?

If rulesets A and B are logical, then one of them is not more logical than the other. Therefore, even if made logical, Japanese rules cannot be MORE logical than other rules.

Self-consistency of go rules needs context. What shall we perceive as self-consistent? Under which axioms? One possible characteristic of self-consistency is absence of exceptions. To start with the most fundamental exception in Japanese rulesets, they have different rules for moves before and after the game end. Therefore, they can never be self-consistent. Sakai Takeshi, J1989 co-author, has understood this so defends them having flaws as fulfilling Yin and Yang;) Japanese rules may conform to Japanese tradition but such consistent, sic, culture is not self-consistency of the rules themselves.