Personal library of mistakes

[xela]

Perhaps the representations produced by the value and policy networks can be concatenated to produce a feature vector on which we can build a model to classify the mistakes?

I've already had a go at classifying mistakes in a very crude way. I think this can be greatly improved if someone has time to invest in it. Certainly the policy network values are very useful. Other ingredients would be to compare policy values from weaker and stronger nets (go to https://neuralnetgoproblems.com and click "about"...) and a shape recognition engine similar to Sabaki's auto-generated move names.

Then we just need a large and non-copyrighted collection of labelled examples to train the model on.

[thirdfogie]

Jaeja: thanks for your kind words about the Key to All Mythologies^W^W^W^W Catalogue of Calamities. Apologies for the late reply: a birthday and a kitchen fire took priority.

Diagram production for the catalogue is largely automated, but you probably don't want to copy my methods. Here are the reasons.

1. I use Debian Linux: you probably do not.
2. If I was starting over, I would do it differently.
3. It all took much longer than expected: I started the catalogue in June 2017,
and it is still incomplete. Of course, I haven't been working on it full time.

The "Typesetting guidelines" section of the following article on the BGA website might be a better guide for you.

http://britgo.org/bgj/guidelines

Returning to my work-flow, instructions for working on the catalogue form two pages of that document.
They are hidden below.

First page.

00-editing1.png (237.57 KiB) Viewed 19423 times

Second page.

00-editing2.png (284.02 KiB) Viewed 19423 times

Section 7.1 of page 2 omitted to mention a C program I wrote to remove captured stones from starting diagrams, and to note stones played at a previously occupied point. The notes provide hints for annotating ko and other recaptures below the relevant diagram. I couldn't find a way to make sgf2dg do those jobs, but I may not have tried hard enough.

Finally, the next hidden section shows a simple BASH script used to prepare the .png files for these posts. I will reply to John Tilley separately.

Code: Select all

#!/bin/bash
# do-conv - Convert PDF file to png for publishing on Lifein19x19.

# The pdftk program is PDF ToolKit.  Its burst command splits the input file
# into its constituent pages, resulting in files 0001.pdf to 0035.pdf in this case.
ptftk avoid5k.pdf burst

# The convert command is from ImageMagick.  The cropping minimises the
# size of the upload.  The "00-" prefixes make the output files sort to the
# beginning of the directory for ease of selection.
convert -density 216 -trim pg_0001.pdf -flatten \
        -crop 1520x2010+160+190 +repage -resize 40% +repage 00-contents.png
convert -density 216 -trim pg_0003.pdf -flatten \
        -crop 1670x1950+60+180 +repage -resize 40% +repage 00-calamities.png
convert -density 216 -trim pg_0009.pdf -flatten \
        -crop 1715x1910+55+185 +repage -resize 40% +repage 00-glass.png
convert -density 216 -trim pg_0017.pdf -flatten \
        -crop 1735x1910+40+185 +repage -resize 40% +repage 00-push.png
convert -density 216 -trim pg_0032.pdf -flatten \
        -crop 1445x1360+190+185 +repage -resize 40% +repage 00-coordinates.png
convert -density 216 -trim pg_0030.pdf -flatten \
        -crop 1445x2020+190+185 +repage -resize 40% +repage 00-editing1.png
convert -density 216 -trim pg_0031.pdf -flatten \
        -crop 1445x2020+190+185 +repage -resize 40% +repage 00-editing2.png

[xela]

I use Debian Linux: you probably do not.

I use Ubuntu Linux and LaTeX. You are not alone (if you admit Ubuntu as an acceptable substitute).

I couldn't find a way to make sgf2dg do those jobs, but I may not have tried hard enough.

No, you tried the right amount. If you try hard enough, you can make LaTeX macros do anything, but a higher level language is probably a good idea in this case.

[thirdfogie]

A few more thoughts about my Catalogue of Calamities.

Xela wrote:

I'm really enjoying the titles and the overall style of the presentation! Where can I buy a copy of your book?

Surely you jest. Anyway, it will never be published, except for occasional
excerpts on this forum.

John Tilley wrote.

I have to say that after a gap of some 30 years in playing at go clubs, I found
that SDK players were making the same mistakes that were being made in the
1970s..... The hardest thing is taking these mistakes seriously enough.

I want to disagree with this slightly. To adapt Tolstoy, every weak player is weak
in his or her own way. At least to some extent.

Here is the justification. Since November 2019, I have been lucky enough to
find a regular opponent of similar strength: perhaps half to three-quarters
of a stone weaker. The 40 games played so far tend to fall into
a regular pattern: I get ahead in the opening, followed by chaotic fighting
where my opponent usually does better. If I haven't already resigned, I usually gain
many points during the last part of the game.

In several recent games, Leela Zero thinks my winning percentage has been up to 93% after
only 14 moves. The reason is obvious: the opponent does not know any joseki or many
principles of fuseki.

My relative weakness in fighting is probably down to the fact that he is much fitter than
I am, though of the same age.

The only obvious difference in the later parts of these games is that I have better
internalised the idea that it is often more efficient to puncture
the other player's territory than to defend one's own.

None of these relative weaknesses seem easily amenable to change.

The two of us have learnt some things from one another.

• At first, I would often catch some of his stones in a net: he no longer lets that happen.
  
  He no longer lets me get all the (supposed) double-sente points.
  
  He has a tactic for breaking into my potential area by attaching to the side of one of
  my stones. When I answer with a hane, he crosscuts and then forces a way
  in with a series of threats to capture. I have slowly and painfully developed
  a new personal proverb to cover these cases: To preserve an area, don't hane
  but back off to thwart the cross-cut. All I need to do now is to apply it sensibly.

Here is an anecdote about John Tilley. The only time we have been in the same
room was at the old London Go Centre, so more than 40 years ago. Four or five
people were watching a game in its late stages. The bottom of the board looked
something like this. I don't recall how it came about.

$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . X X . . . . . . . . . |
$$ | . . . O . . X X X O X X X . . , . . . |
$$ | . . . . . X X . . . . . . X . X . . . |
$$ | . . . . . . . X X O X X X . . . . . . |
$$ | . . . . . . . . . X . . . . . . . . . |
$$ ---------------------------------------

Click Here To Show Diagram Code

[go]$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . X X . . . . . . . . . |
$$ | . . . O . . X X X O X X X . . , . . . |
$$ | . . . . . X X . . . . . . X . X . . . |
$$ | . . . . . . . X X O X X X . . . . . . |
$$ | . . . . . . . . . X . . . . . . . . . |
$$ ---------------------------------------[/go]

JT breezed into the room, took one glance at the board and immediately said "Ikken tobi
is never a bad move". It was clear that a superior intellect was at work.

Lastly, here are some plans for improving the Compendium of Catastrophes.
It should be more positive. In particular, the sections of pure despair
should somehow be expanded into helpful psychological advice. Knotwilg's
frustration equation will have a place of honour. I also want to plunder
Richard Hunter's excellent books. I only understand about the first 10% of
each book, so the plan is to distill the bits that I do understand into pithy
statements that will prompt my memory. Incidentally, this is one reason for
not publishing the whole Catalogue: it shamelessly incorporates other peoples' work.
This is OK for private study, but not for publication. (The extracts given
in this forum are mostly my own work, leavened with an occasional quotation.)

I have also been compiling material concerning the common situation shown
below.

00-jump-ahead-2.png (215.06 KiB) Viewed 19327 times

The analysis which follows in the Catalogue is still a mess. For what it's worth, here is a shortened version.
1. Assume any forcing moves by the opponent have already been played before counting liberties.
2. If the white stones have fewer than 5 liberties, jumping ahead will result in at least
one white stone being captured if Black pushes in.
3. If the white stones have exactly 5 liberties, Black will be able to push through and
squeeze.
4. Above 5 liberties, much depends on how much space there is at the other end of the
white stones, whether that space has any value to White, and whether either side can grab it.

Any suggestions?

Edited on 10 March 2020 to remove reference to quarantine, caused by a misremembered post.

[xela]

Xela wrote:

I'm really enjoying the titles and the overall style of the presentation! Where can I buy a copy of your book?

Surely you jest.

No jest. I really think you're on to something good here. I'd love to see more.

Regarding your page on jumping ahead on the third line: for me, this is the way joseki should be taught! Group together a bunch of joseki with similar shapes or similar themes, and look at what's the same and what's different. Why do all the mainstream joseki books just list them in dictionary order? You don't learn a language by starting with all the 'a' words, then the 'b' words... Where's the go player's equivalent to The Ideas Behind the Chess Openings? Is it possible that thirdfogie is writing it for us right now?

[Uberdude]

Regarding your page on jumping ahead on the third line: for me, this is the way joseki should be taught! Group together a bunch of joseki with similar shapes or similar themes, and look at what's the same and what's different.

I didn't look at other shapes with jumping ahead on 3rd line, but when teaching the the enter corner after pincer joseki I do explain why you play the hane connect to get enough liberties to jump. https://youtu.be/csIMlfMzA_k?t=237

[John Fairbairn]

The analysis which follows in the Catalogue is still a mess. For what it's worth, here is a shortened version.
1. Assume any forcing moves by the opponent have already been played before counting liberties.
2. If the white stones have fewer than 5 liberties, jumping ahead will result in at least
one white stone being captured if Black pushes in.
3. If the white stones have exactly 5 liberties, Black will be able to push through and
squeeze.
4. Above 5 liberties, much depends on how much space there is at the other end of the
white stones, whether that space has any value to White, and whether either side can grab it.

Any suggestions?

1. The five liberties aspect: this is part of something I first came across in Korea and called it "five alive" - in local situations your groups needs (or can make) at least five liberties otherwise you shouldn't fight with it as is. Victor Guang Chow also told me a range of similar shape concepts was taught in Chinese classes (at least in Guilin), and I think Clive Hunt may have collected (and published?) examples when he and Victor were both living in South Africa. I have seen nothing like this in Japanese, Korean of Chinese books but someone may be able to provide a bit from more oral sources (e.g. the Guilin Qiyuan or Myongji University?).

2. Your examples are binary: A or B (jump or crawl). But often real life is ternary. A knight's move slide C is often the best move - it combines imlied extra liberties with extra distance. And kosumi is very occasionally a tesuji in such cases.

[Jæja]

1. The five liberties aspect: this is part of something I first came across in Korea and called it "five alive" - in local situations your groups needs (or can make) at least five liberties otherwise you shouldn't fight with it as is. Victor Guang Chow also told me a range of similar shape concepts was taught in Chinese classes (at least in Guilin), and I think Clive Hunt may have collected (and published?) examples when he and Victor were both living in South Africa. I have seen nothing like this in Japanese, Korean of Chinese books but someone may be able to provide a bit from more oral sources (e.g. the Guilin Qiyuan or Myongji University?).

Bruce Wilcox uses the same theory in his e-learning course Bruce Wilcox’s Go Dojo: Contact Fights. He uses the term "unstable" (< 5 liberties), "stable" (5 liberties) and "super-stable" (> liberties).

[Bill Spight]

1. The five liberties aspect: this is part of something I first came across in Korea and called it "five alive" - in local situations your groups needs (or can make) at least five liberties otherwise you shouldn't fight with it as is. Victor Guang Chow also told me a range of similar shape concepts was taught in Chinese classes (at least in Guilin), and I think Clive Hunt may have collected (and published?) examples when he and Victor were both living in South Africa. I have seen nothing like this in Japanese, Korean of Chinese books but someone may be able to provide a bit from more oral sources (e.g. the Guilin Qiyuan or Myongji University?).

Bruce Wilcox uses the same theory in his e-learning course Bruce Wilcox’s Go Dojo: Contact Fights. He uses the term "unstable" (< 5 liberties), "stable" (5 liberties) and "super-stable" (> liberties).

Bruce Wilcox is the source of the five alive concept, but not the name. In working on his go playing computer program he derived heuristics that humans can use. This is one of them. The sector lines concept is another. He taught at go camps in the summer, and some of his kyu level pupils there advanced four stones in two weeks. Wilcox published his ideas in a series in the AGA Journal in the mid-1970s.

[Knotwilg]

I would say that "awareness of liberties" is the most underestimated aspect of beginner/amateur go playing and perhaps teaching. At the very minimum it protects against stupid moves like self-atari or failing ladders. It leads to an understanding of local shape. As mentioned, it gives a heuristic for fighting versus potential sacrifice. And so on ...

Richard Hunter is another influential author who has given a lot of attention to the subject of liberties in fights. He has raised my awareness - at least for some time

[Bill Spight]

2. Your examples are binary: A or B (jump or crawl). But often real life is ternary. A knight's move slide C is often the best move - it combines imlied extra liberties with extra distance. And kosumi is very occasionally a tesuji in such cases.

Well worth repeating.

[Uberdude]

2. Your examples are binary: A or B (jump or crawl). But often real life is ternary. A knight's move slide C is often the best move - it combines imlied extra liberties with extra distance. And kosumi is very occasionally a tesuji in such cases.

Well worth repeating.

Until AlphaGo came along and suggested the below variation in the common approach a 4-4, pincer and take corner joseki, humans were blind to that 3rd option here, instead opting for the a-d exchange to gain liberties and allow the jump. But it damages the approach stone, and sometime AG (and other bots since) say it is better to directly slide because then white has some more powerful future aims of bringing out the approach stone and annoying black's group there (those aims are lessened if black plays the bot-and-now-standard human crawl under at d instead of connect at e).

$$W
$$ | . . . , . . . . . , .
$$ | . . . . . . . . . . .
$$ | . . . . . . . . . . .
$$ | . . . . . . . . . . .
$$ | . 7 . . . . . . . . .
$$ | . . . 6 . . . . . . .
$$ | . . 5 X . . . . . , .
$$ | . . 3 4 e 1 . 2 . . .
$$ | . . c a b d . . . . .
$$ | . . . . . . . . . . .
$$ +----------------------

Click Here To Show Diagram Code

[go]$$W
$$ | . . . , . . . . . , .
$$ | . . . . . . . . . . .
$$ | . . . . . . . . . . .
$$ | . . . . . . . . . . .
$$ | . 7 . . . . . . . . .
$$ | . . . 6 . . . . . . .
$$ | . . 5 X . . . . . , .
$$ | . . 3 4 e 1 . 2 . . .
$$ | . . c a b d . . . . .
$$ | . . . . . . . . . . .
$$ +----------------------[/go]

[Bill Spight]

Jaeja - I think there is a limit to what you can do with AI analysis - maybe it is related to your ranking.

Once you get into the amateur dan ranks AI becomes more useful - I had a game of mine analysed some 50 years ago, when I was 1d - the feedback from Haruyama 9p was that I was not aggressive enough - there were several times when I was failed to play kikashi. Once you know that, you can use AI to spot such things. What I don't think you can do (yet) is to use AI to build such generic advice.

To refine that a bit, with AI we have entities whose behavior we would like to understand, but they provide no explanations. Humans are very good at coming up with explanations. OC, not all of these pan out. Thor does not cause thunder. Now we know what does, but it took millennia to figure it out. We did so by means of science. We can use a scientific approach to analyzing the play of AI bots. While knowing current concepts, such as kikashi, can help, strength at go is not necessary.

I was one of the first to notice that AlphaGo played pincers less often than humans. In the Master games, given the first chance to play a pincer, human pros did so around 20% of the time, while Master did so around 10% of the time. A 10 kyu could have shown that. I also noted that when Master did pincer, it almost always had backup strength on one side or the other. That way, if the pincered stone jumped out, it could only counterattack in one direction. This observation may be on the SDK level. These observations produce a heuristic: Wait to pincer until you have backup. OC, it is only a heuristic, there are exceptions. But we can study those in a scientific manner, as well.

Recently I have come up with another observation in the Elf commentaries of GoGoD games. It is usually right to occupy the last open corner. I found it while looking for examples of the last play of the opening. In vain. Elf does not seem to recommend those plays. Again, it did not take much go knowledge or expertise to make those observations or to come up with that heuristic. Persistence was the main virtue.

Edit: BTW, many people have noticed that today's bots play kikashi more often than humans. Why, we don't know.

[John Tilley]

I was impressed to see that thirdfogie had independently developed as personal guidance one of Sonoda's proverbs:

He has a tactic for breaking into my potential area by attaching to the side of one of
my stones. When I answer with a hane, he crosscuts and then forces a way
in with a series of threats to capture. I have slowly and painfully developed
a new personal proverb to cover these cases: To preserve an area, don't hane
but back off to thwart the cross-cut. All I need to do now is to apply it sensibly.

I first became aware of this when John F recommended Sonoda's book (in Japanese) back in October 2016 苑田流格言のすべて:

Sonoda-ryu Kakugen no Subete (All about Sonoda's proverbs). The sub-title is about absorbing pro-level knowledge, and is conveyed through the heuristics (rather than proverbs) of Sonoda Yuichi. I found this an astonishing eye-opener......

He also has interesting and novel things about when to play orthogonally and when to play diagonally.

This guidance is:

自分の石数の多いときは、石をタテヨコに使う (When I have more stones, play "tateyoko" = criss cross)
自分の石数が相手より少ないときには、石をナナメに使う (When I have fewer stones, play "naname" = diagonally)

This is essentially what thirdfogie wrote - if your opponent is playing attach moves inside your territory, then play solid moves along the lines - don't play diagonally as a cross-cut helps the player trying to live or settle the position.

I learnt this heuristic some 3 years ago and have found it most useful both during actual play and when reviewing games.

Sonoda introduced a small number of heuristics in his book, illustrated them with examples and then commented on seven professional games. Its most interesting that thirdfogie developed one of them independently.

An earlier book of Sonoda's - "Go Strategy" - is well reviewed at Sensei's Library:

https://senseis.xmp.net/?GoStrategy

John Tilley

[Bill Spight]

I was impressed to see that thirdfogie had independently developed as personal guidance one of Sonoda's proverbs:

He has a tactic for breaking into my potential area by attaching to the side of one of my stones. When I answer with a hane, he crosscuts and then forces a way in with a series of threats to capture. I have slowly and painfully developed a new personal proverb to cover these cases: To preserve an area, don't hane but back off to thwart the cross-cut. All I need to do now is to apply it sensibly.

I first became aware of this when John F recommended Sonoda's book (in Japanese) back in October 2016 苑田流格言のすべて:

Sonoda-ryu Kakugen no Subete (All about Sonoda's proverbs). The sub-title is about absorbing pro-level knowledge, and is conveyed through the heuristics (rather than proverbs) of Sonoda Yuichi. I found this an astonishing eye-opener......

He also has interesting and novel things about when to play orthogonally and when to play diagonally.

This guidance is:

自分の石数の多いときは、石をタテヨコに使う (When I have more stones, play "tateyoko" = criss cross)
自分の石数が相手より少ないときには、石をナナメに使う (When I have fewer stones, play "naname" = diagonally)

This is essentially what thirdfogie wrote - if your opponent is playing attach moves inside your territory, then play solid moves along the lines - don't play diagonally as a cross-cut helps the player trying to live or settle the position.

Another word for criss cross (tateyoko), particularly in a game context, is rook-wise. As in, Go stones are rook-wise connected.