Life In 19x19

I was curious to see how an AlphaZero-like system discovers various Go patterns while training from self-play, from scratch.
So I did a little data-mining starting from the large SGF collection of self-play games from https://sjeng.org/zero/ (the all_sgf.xz file).
The version I did this analysis is about 3 weeks old now - the latest LZ version included in my analysis is af9ce63c from Feb 21st.
I skipped the first 1.5M games from that SGF collection to eliminate the initial random play, so the first version analyzed is 39fe0ce3 from Dec 28th, 2017.

I parsed the SGFs and extracted all the corner, 9x9 sized patterns containing exactly 10 stones (5 for each of white and black), and which appeared in at least 3 different LZ versions. Symmetries and color-swaps are taken into account, too.

I generated a report with top patterns from each version (versions are listed in chronological order). One can find the version IDs on the LZ project page here: http://zero.sjeng.org/

For each pattern, the plot next to it shows its frequency across the training versions. The x-axis numbers correspond to version index (from 0 to 45, since there are 46 versions between 39fe0ce3 and af9ce63c), while the y-axis numbers are the frequency of the pattern relative to other patterns in the same version, for instance 0.003 means this pattern appeared 3 times in each 1000 patterns in the games of a LZ version). Taller ticks mean that the pattern's relative frequency within a particular version is higher.

Each pattern is only shown in the earliest LZ version it occurred in - looking at the plot next to it should give an idea about popularity trend: if ticks for later versions are higher, the pattern became more popular in later versions, while if it is lower its popularity faded away.

I generated a report for the top one pattern in each version: http://361points.com/media/top1patterns/patterns.html
The page is very slow to load the patterns, it will show first some "....xxxooo..." text before showing image patterns.
Interestingly, we can see first the old-style 3x3 invasion against the opponent's 4x4 stone, with hane-tsugi in the end; LZ did this in older versions, but we see from the frequency plot how it abandoned it eventually.


Second pattern is the AlphaGo style of crawling one more time on 2nd line instead, to get sente; LZ discovered this later, but it seem to have abandoned it too; I bet it will show again in the most recent versions (after the switch to the 10-blocks, on March 5th), I will redo the analysis to incorporate more recent versions.


Third pattern is the double-hand and capture, which we see from the plot that it became super-popular in the 2nd half of the timeline. I expect that as LZ gets even stronger, it will abandon this in favor of the 2nd pattern, above.


For people interested in more data: I also generated a report for the top-10 patterns in each version: http://361points.com/media/top10patterns/patterns.html
This larger analysis may take several minutes to load - sorry about that, I hope I can figure out a better way.

Hi Sorin,

Thanks.

Which software did you use to generate the board image ?

EdLee wrote:Hi Sorin,

Thanks.

Which software did you use to generate the board image ?

Hi Ed,

I used jgoboard 3.0 from here:
https://github.com/jokkebk/jgoboard

Very interesting, thanks for your efforts in making this. We can see it discovering quite a lot of AlphaGo style moves, e.g. in the top 10 view near the end I spied the attach inside big high shimari after ignored knight's approach.

One suggestion: can you include a rank estimate or Elo rating along with network name? Doesn't need to be super accurate and of course depends on hardware, but would be nice to have an idea if the network half way through is nearer 8 kyu or 1 dan.

I expect the folks over at reddit/cbaduk would like this: would you like to post it there or can I? Is the image rendering server or client side, wouldn't want to overload your web server.

Uberdude wrote:Very interesting, thanks for your efforts in making this. We can see it discovering quite a lot of AlphaGo style moves, e.g. in the top 10 view near the end I spied the attach inside big high shimari after ignored knight's approach.

One suggestion: can you include a rank estimate or Elo rating along with network name? Doesn't need to be super accurate and of course depends on hardware, but would be nice to have an idea if the network half way through is nearer 8 kyu or 1 dan.

I expect the folks over at reddit/cbaduk would like this: would you like to post it there or can I? Is the image rendering server or client side, wouldn't want to overload your web server.

I was also very surprised to see that it follows pretty much all the peculiar patterns of AG. I wonder now if these are "fundamental truths" in Go (that humans just missed to discover), or just artifacts of the particular training algorithm used.

Good idea about attaching Elo to versions, I will do that.

No concern with reposting, rendering is client-side, thanks for your concern!

sorin wrote:I was also very surprised to see that it follows pretty much all the peculiar patterns of AG. I wonder now if these are "fundamental truths" in Go (that humans just missed to discover), or just artifacts of the particular training algorithm used.

Oh, I expect that there is path dependency, both in bots' training regimes and in human baduk history. For instance, if unfinished games early on are scored by proximity scoring, the bot might start off playing an influence oriented game. Scoring unfinished games by number of prisoners might produce more aggressive play, quien sabe?

For the 3-3 invasion patterns that you show in your first note, humans overlooked the crawl on the second line (second pattern) instead of the hane (first pattern). As a result of that, perhaps, we felt that the 3-3 invasion was somewhat dubious. Now, AlphaGo, in all varieties, likes it.

But what do we make of Black's two step hane, preventing White from crawling? That pattern seems to be superseding the second pattern. Also, it is a human joseki. Could it be that that pattern is so good for Black that in later development Leela Zero will reject the early 3-3 invasion? Nobody knows. But it looks like Leela Zero took a detour before returning to a human joseki, while humans never quite rejected the first joseki.

Anyway, a great study! Many thanks.

Edit: Corrected to refer to the right bot.

I'm nearly certain the second pattern disappeared because it was too good for the 3-3 side, and won't be coming back. At least, it's not likely to become more popular than the third pattern.

redreoicy wrote:I'm nearly certain the second pattern disappeared because it was too good for the 3-3 side, and won't be coming back. At least, it's not likely to become more popular than the third pattern.

Looking at AG games, it very rarely plays pattern 3 (for the 4x4 player). By that measure, I think pattern 3 may fade away as LZ gets stronger, and pattern 2 re-emerge... Time will tell (soon ).

sorin wrote:

redreoicy wrote:I'm nearly certain the second pattern disappeared because it was too good for the 3-3 side, and won't be coming back. At least, it's not likely to become more popular than the third pattern.

Looking at AG games, it very rarely plays pattern 3 (for the 4x4 player). By that measure, I think pattern 3 may fade away as LZ gets stronger, and pattern 2 re-emerge... Time will tell (soon ).

Ok, but I didn't find a single instance of pattern 2 in all the released games by alphagozero.

redreoicy wrote:

sorin wrote:

redreoicy wrote:I'm nearly certain the second pattern disappeared because it was too good for the 3-3 side, and won't be coming back. At least, it's not likely to become more popular than the third pattern.

Looking at AG games, it very rarely plays pattern 3 (for the 4x4 player). By that measure, I think pattern 3 may fade away as LZ gets stronger, and pattern 2 re-emerge... Time will tell (soon ).

Ok, but I didn't find a single instance of pattern 2 in all the released games by alphagozero.

I see, you are right!! I did a quick check here http://www.alphago-games.com/#agzero_vs_agmaster where one can see this pattern in many of the game pics, but then I realized that it is always AG Master playing that shape. Very interesting, I see - I wonder if it is possible that LZ has already discovered something that AG Master doesn't know!

I did some more analysis of LeelaZero patterns, using the latest available collection of training SGFs (up to version 1ccb7342 which is a little more recent and stronger than the version that defeated Hajin Lee 4p).

Also, this time I made the analysis more focused and hopefully more interesting, by picking relatively few patterns to analyze and compare.
Here is what I have so far:

• Overview: http://361points.com/media/gopatterns// ... terns.html
• How does LeelaZero use two moves in a row in an empty quadrant: http://361points.com/media/gopatterns// ... a_row.html
• How does LeelaZero approach a single opponent stone: http://361points.com/media/gopatterns// ... n_one.html
• Analysis of patterns from the recent, pro-level version 1ccb7342: http://361points.com/media/gopatterns// ... b7342.html

See below for some patterns not included in the above analysis, but nevertheless interesting, from analyzing up to version 1ccb7342.
Timeline graphs show relative frequency of the pattern in various training versions (older versions on the left, newer versions on the right). The red line highlights where the currently analyzed version is in the timeline.

Timeline for what I call "Go Seigen pattern" (possibly misnomer) - we see that this became more popular with LeelaZero relatively recently (but it is still a rare pattern overall, appears less than twice in ten thousand patterns on the average) AlphaGo-like two-space extension by black (maybe to prevent the peep by white, if black plays just one-space jump, like AlphaGo played against Lee Sedol in game 2) Timeline for another AlphaGo move, the double-hane at S6 (instead of the human move at Q4):

Wonderful stuff! Many thanks.

BTW, I think that the two space high approach originated with Honinbo Shuei. In his 21st Century Go set, Go Seigen likes it as if Black starts with a 3-4 point.

BTW, I think that the two space high approach originated with Honinbo Shuei. In his 21st Century Go set, Go Seigen likes it as if Black starts with a 3-4 point.

This has long been the opening that most fascinates me, for various reasons, some of which seem applicable to discussion of AI.



It appeared before Shuei, but in more of a middle-game context than pure fuseki, but what was noticeable about its adherents was not so much that Shuei was among them (he played it when very young and immature then abandoned it for a long while) but that it was taken up by Shuho and Shusai in a big way. It was also associated with Honinbo pupils. It seems therefore it must have been heavily discussed among the Honinbo family. I am not aware of any record of what they might have said, regrettably, but we can probably infer some things from their games, and in particular how they responded to it. For example, the common defence at A. Even today, whenever I see it, I marvel at this move being played by pros. It seems so counter-intuitive to me. It is down to Honinbo Shugen.

The analogous B (rather than the usual C) took a long time coming, and remains hyper-rare, but Takahashi Shigeyuki (Shusai's son-in-law) tried it in the 1930s, including in this amusing opening:



Another aspect of my fascination are the continuations at D and E. E had a try-out in the 1930s Shin Fuseki period and was promptly dropped (perhaps it was seen as helping White play towards the centre?) but D has had a reasonable run and has thrown up several unusual josekis. This looks very Shin Fuseki-ish but was introduced by Onoda Chiyotaro in the 1920s. However, as I have pointed out in "Old Fuseki vs New Fuseki" Shin Fuseki was part of an organic process starting in the Meiji period (and so this high approach was part of that). Shin Fuseki had an explosive hype moment in 1933 but was not really new (shin).

This opening also has some fascinating pincer josekis. The extra gap changes the dynamics of the pincer in several instructive ways.

The merits of the high approach are such that several major players have tried it not as an approach but as first move in the corner (or even as first move in the game). Sekiyama Riichi tried it first in the 1930s and was soon followed by the likes of Kitani and Go Seigen. It hasn't quite disappeared and I was amused to see Sekiyama Toshimichi (Riichi's grandson) play it again last year (2017) in a couple of games. The Sekiyama link is actually even stronger than it looks, as the first recorded instance of the high approach is by Sekiyama Sendayu in 1824.

At a more strategic level, the high approach has been discussed by Go Seigen, of course, but even more extensively by O Rissei - in both cases as part of the 21st Century Go movement.


PS: The following quote from the original post source needs correcting:

The official Japanese name is "nikken-takagakari" because it is a two-point jump (nikken), it is high ("taka(i)") (4th line instead of 3rd), and is an approach ("kakari" - turns into "gakari" just because it sounds better in the context).

There is nothing official about it, and there's rather more to lenition than just sounding better (see rendaku and Lyman's Law if you're interested), but the word nikken does not exist. It is niken, and it means two-space not two-space jump (three-space is sangen BTW, and one-space does indeed have -kk-: ikken). While we are on this topic, I'll mention that kakaru doesn't mean approach, although I think it's acceptable. But the usage in Japan is not quite agreed either. Some explain it as attack since e.g. teki ni kakaru means attack (fall on) the enemy. I think this is the commonest explanation, but others say it means getting a handhold (i.e. relating it to the word tegakari). But fuzzyists like me think it has both connotations.

PPS: By chance the first game I started transcribing after writing the above featured the two-space approach, with an interesting comment by Masubuchi Tatsuko (Black):



Masubuchi said she ignored White 4 rather than answering as usual at Q14 because that would mean falling in with White's plans (so maybe a feature of the high-approach is that it is more highly strategy-oriented than other approaches?) and played 5 as a kind of probe (yes, a probe this early - not that rare, actually). However, she seemed to be a little dissatisfied with the result as she said that White had a achieved an efficient position after 10.

John Fairbairn wrote: This has long been the opening that most fascinates me, for various reasons, some of which seem applicable to discussion of AI.



It appeared before Shuei, but in more of a middle-game context than pure fuseki, but what was noticeable about its adherents was not so much that Shuei was among them (he played it when very young and immature then abandoned it for a long while) but that it was taken up by Shuho and Shusai in a big way. It was also associated with Honinbo pupils. It seems therefore it must have been heavily discussed among the Honinbo family. I am not aware of any record of what they might have said, regrettably, but we can probably infer some things from their games, and in particular how they responded to it. For example, the common defence at A. Even today, whenever I see it, I marvel at this move being played by pros. It seems so counter-intuitive to me. It is down to Honinbo Shugen.

Thank you for your detailed post John, as usual lots of very interesting Go culture information!

I asked LeelaZero's opinion on Takahashi Shigeyuki's game after white's "niken takagakari" (thank you for the spelling correction also!) in the lower-left, and its top local choices are: first E3, second D3, third hasami at H3 - so the counter-intuitive looking kosumi (A in your quoted diagram) looks natural to AI for some reason!

Insightful analysis again Sorin, thanks!