pareto principle applied to go

[Kirby]

I've been reading about learning, and one method (listed partially below) is to:
1.) deconstruct a learning topic into small learnable units
2.) identify the ~20% of these units which can be learned to achieve ~80% benefit (i.e. the pareto principle)

I've been wondering to myself whether this type of learning could apply to go, and was curious to hear your thoughts. Aligned with the two questions above:
1.) do you think that go can be deconstructed into small learnable units? what do you think these units might be?
2.) do you think that pareto's principle applies? what "units" in go give you the biggest bang for your buck?

I have a couple of thoughts of my own, but was curious to see what L19-ers had to say

[EdLee]

Hi Kirby,

do you think that go can be deconstructed into small learnable units?

Yes, qualified: I thought all the millions of Go problems ( and sensei pages, books, videos, etc. ) demonstrate small learnable units, no ?
( The reverse is obviously different: the entirety of human Go knowledge since antiquity still falls very short of AG-Z. )

what "units" in go give you the biggest bang for your buck?

My sense is they vary according to our current level.
The units for a 10k are probably different from those for a top pro.

[Bill Spight]

I've been reading about learning, and one method (listed partially below) is to:
1.) deconstruct a learning topic into small learnable units
2.) identify the ~20% of these units which can be learned to achieve ~80% benefit (i.e. the pareto principle)

I've been wondering to myself whether this type of learning could apply to go, and was curious to hear your thoughts. Aligned with the two questions above:
1.) do you think that go can be deconstructed into small learnable units? what do you think these units might be?
2.) do you think that pareto's principle applies? what "units" in go give you the biggest bang for your buck?

I have a couple of thoughts of my own, but was curious to see what L19-ers had to say

Mainly to second Ed Lee's reply.

Go has already been deconstructed into learnable units. Not entirely, OC. Something like Pareto's principle applies. As a dan player, you have already learned 20%. Sorry.

What units give the biggest bang for the buck?

Those that have a name. Proverbs. (Real proverbs, not those made up by kyu players.)

I think that small in this context does not mean small scale, although small scale units in go are powerful, such as the concepts of snapback, ladder, and net. The trouble with small scale units in go is that they do not usually affect a lot of points. Small scale life and death problems are an exception.

The trouble with large scale units in go is that they are not very unitary. They are fuzzy, and have exceptions. Some units, such as thickness and kikashi, are not so easy to learn, but that is the counsel of perfection. They are not difficult to learn imperfectly. Gradus ad parnassum.

It is plain that much of the strength of AlphaGo and other strong AI programs comes from learning on a large scale. As a result, humans can point to small scale "errors" in their play. Some claim that those are not really errors, which is why I put errors in quotes. And surely that is true in many cases. We all know that it may be right to take a small scale loss to make a large scale gain. But surely in some cases they actually are errors, but small errors.

Go knowledge is quite substantial, but, IMO, it has not been very well systematized. The low hanging fruit (the "20%") has been, and is fairly readily available, even in English. The accumulation of go knowledge has been more folkloric than scientific, but there is a good bit of it, nonetheless. It is perhaps more error prone than scientific knowledge, but it is still worth knowing, and criticizing. Progress in go knowledge is dialectical, a clash of ideas.

As for doing one's own deconstruction, IMX, it is easiest to do so on a small scale. You can get large scale ideas, but who knows how effective they are. On the small scale you can prove things.

[John Fairbairn]

While agreeing with the last two posters, I'll add another consideration.

Obviously we can't measure the percentages of what we learn and what we gain, so that 20% and 80% are just approximations, which probably also vary over a great range. It seems likely that different "starter packs" which aim at 20% coverage could give startlingly different results. My guess is that the starter pack that evolved for the west was always, and probably still is, much inferior to the packs available for CJK players. The ingredients may be similar, or even the same, but the proportions and order of mixing vary uncontrollably in the west. They would need to be sorted out before we can reap the whole benefit.

Connected with that, I think, is what we might call inevitable "interference" by westerners. Unable to get Aleppo pepper to make their Turkish eggs, they have substituted black pepper, or even omitted pepper altogether. More adventurous souls substitute crème fraiche for yoghurt and wonder why the results are not quite right. This was inevitable in the past, when communications and resources were pitiful compared to what we have now. It's not necessary now, but I fear the habit of "interference" is still ingrained.

We need to lose that habit and just accept the CJK starter packs off the shelf. There's a famous Zen story (which I may have remembered inaccurately) where a monk made a show of his zennic asceticism by going to sit on top of a 100-foot pole to meditate. A great Zen master mocked him, pointing out that he would never make progress until he stepped off the pole.

As specific examples, I've mentioned here recently a few methods offered by Japanese pros, such as the one where you get a strong clue to your next move simply by counting the number of stones for each side in a selected half of the board. This is far from foolproof but my assessment is that it provides benefit in about 90% of games in the early phases, and clearly needs just a piddly few percentage points of learning (mainly to learn to select the right half of the board). That's not in any western start packs that I know of, but (in various different forms) it is in the CJK packs.

[Galation]

Hi, I am far from a good level as a Go player, still a big part of my professional skills are about data analysis and continuous improvement (planning and implementation). These are my 2 cents.

Pareto principle (widely promoted by Juran) is properly applied when there is a clear (i.e. quantified) sequence of problems/topics to be improved. It heavily rely on the statistical approach, meaning that you need a statistically significant sample.
Pareto rule theorizes that solving/improving the first 20% of the list will gain 80% of the result. Continuous improvement requires that you periodically prioritize the topics and again improve the worst 20%.

As Ed Lee said, the main topics are probably individually varying with Go strength and individual skill. My guess, applying this to Go learning, is that the problem is having enough data for understanding what is your Pareto istogram.
e.g. what are the main reason you lost (let's say) your last 100 games? Prioritize the bigger 20% and improve that first.

IMHO data entry is not easily defined nor statistically significant.
Otherwise, a Pro (or a Go teacher) could tell your bigger weaknesses and suggest how to improve that, but this is only partially resembling to a Pareto approach (again: not quantified only prioritized).

Answering your questions:
1. I think that it is entirely possible to deconstruct Go into learning units, but I do not think that this is a good way to study Go: Learning joseki loses two stones strength...

2. I do not think that there is an easy Pareto application valid for everyone and anyone, still there is a priority list (both individually varying and modifying during one's Go career).

Galation

[Pio2001]

1.) do you think that go can be deconstructed into small learnable units? what do you think these units might be?

Yes, they would be

Rules
Area counting, territory counting

Life and death
Two eyes
Vital points, Nakade
False eyes
Seki

Fighting
Ladder
Net
Double atari
Atari on the 2nd line
Snapback
Pin
Capturing races

Shapes
Extend, Kosumi, Jump, Knight move, Large knight move, Elephant's step etc
Attach, Hane, clamp, warikomi etc
Crosscut, Bamboo joint, turtle back, tiger mouth, dog's head, connection on the first line etc

Tesuji
Races
Cut / Connect
Shortage of liberties
Sacrifice
Magic in the corner

Opening
Corner first, edges next
Enclosures
Approaches
Extensions
Lines
Basic Jôseki
Classic openings

Opening strategy
Make a base / attack weak groups
Play in the bigger space
Stay away from strength
Make stones work together
Balance territory and influence

Endgame
Closing frontiers
Sente, gote
Value of moves
Neutral points, teire points

Positional Judgement
Strategy if late, strategy if in advance, strategy if balanced

Invasion
Sabaki

Reduction

Moyo

Ko fighting

Use of aji

Attack
Make profit while attacking
Use of thickness

Various
Avoid sente for the sake of sente
Efficiency, tewari
Sacrifice useless stones

2.) do you think that pareto's principle applies? what "units" in go give you the biggest bang for your buck?

I don't know...

[ez4u]

I think the question of "deconstructing" Go into small learnable units is a little off. You are Black and face the empty board. You have the choice of playing on one of the shaded 55 points below or some mirrored equivalent. Once you pick your first play, your opponent will face an expanded choice of their own (unless you play on tengen in which case they will have only 54 points to choose from). All human players have basically learned to ignore 80% of the available plays for Black's first play and an even higher percentage of White's. And so on. A more appropriate question seems like, "Can we chunk Go into a finite set of effective, learnable units?" Isn't that what all the heuristics, proverbs, etc. are about? Isn't doing so already a pareto-like process where we are rejecting any number of possible concepts in favor of others based on some sort of ranking of their effectiveness, even if we are not then ranking the final list from top to bottom?

$$c Choices for Black's first play.
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . ? |
$$ | . . . . . . . . . . . . . . . . . ? ? |
$$ | . . . . . . . . . . . . . . . . ? ? ? |
$$ | . . . , . . . . . , . . . . . ? ? ? ? |
$$ | . . . . . . . . . . . . . . ? ? ? ? ? |
$$ | . . . . . . . . . . . . . ? ? ? ? ? ? |
$$ | . . . . . . . . . . . . ? ? ? ? ? ? ? |
$$ | . . . . . . . . . . . ? ? ? ? ? ? ? ? |
$$ | . . . . . . . . . . ? ? ? ? ? ? ? ? ? |
$$ | . . . , . . . . . ? ? ? ? ? ? ? ? ? ? |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------

Click Here To Show Diagram Code

[go]$$c Choices for Black's first play.
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . ? |
$$ | . . . . . . . . . . . . . . . . . ? ? |
$$ | . . . . . . . . . . . . . . . . ? ? ? |
$$ | . . . , . . . . . , . . . . . ? ? ? ? |
$$ | . . . . . . . . . . . . . . ? ? ? ? ? |
$$ | . . . . . . . . . . . . . ? ? ? ? ? ? |
$$ | . . . . . . . . . . . . ? ? ? ? ? ? ? |
$$ | . . . . . . . . . . . ? ? ? ? ? ? ? ? |
$$ | . . . . . . . . . . ? ? ? ? ? ? ? ? ? |
$$ | . . . , . . . . . ? ? ? ? ? ? ? ? ? ? |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

[daal]

A more appropriate question seems like, "Can we chunk Go into a finite set of effective, learnable units?" Isn't that what all the heuristics, proverbs, etc. are about? Isn't doing so already a pareto-like process where we are rejecting any number of possible concepts in favor of others based on some sort of ranking of their effectiveness, even if we are not then ranking the final list from top to bottom?

We could probably just rephrase Kirby's question so: "what should I do to improve at go?" As demonstrated in Pio2001's post, even mid-level sdks know what the chunks are (though I would stick miai in somewhere), and although we in the West don't have the advantage of a CJK curriculum, I don't think that it is the finite units themselves that separate the weak from the strong. Aside from the abilities to read out sequences and remember patterns, I would say that strength correlates to the correctness of one's ranking of priorities in a given situation, and I don't think that this can easily be broken into learnable chunks.

[Bill Spight]

It is a fairly good guess, I think, that pros have learned around 50,000 to 100,000 chunks. My own guess is that SDKs have learned around 1,000 - 2,000.

[John Fairbairn]

Bill, knowing that in your case a guess will be rather more informed than just guess, how did you reach those figures?

And how do you think pros create or find chunks?

[Bill Spight]

Bill, knowing that in your case a guess will be rather more informed than just guess, how did you reach those figures?

The 50,000 - 100,000 figure comes from 20th century research on chess, and is the same order of magnitude as the size of adult vocabulary, across different languages. Without knowing a large number of chunks, how do chess pros, go pros, and checkers pros see the right play almost instantly on most occasions? (They also see other candidate moves, which they check out, given time.)

My guess for SDKs is speculative, but I think that it is not too difficult to learn 1,000 chunks or so, without really trying.

And how do you think pros create or find chunks?

Mostly they pick them up. They do study books, but not those written for amateurs. What pro has not worked through the classic problem books? They also play around with stones on the board. There is a lot we don't know, because nobody has done the research. Sakata said that when he was six years old, his teacher, Masubuchi, would come to the house and lay out a whole board position, and then ask him where he would play. He would show her and she would pat him on the head. How did he know where to play? {shrug}

[John Fairbairn]

The 50,000 - 100,000 figure comes from 20th century research on chess, and is the same order of magnitude as the size of adult vocabulary, across different languages.

Thanks, Bill. I expect I've seen such figures for chess as I've read a lot of books on chess training theory, but as they are numbers I've totally forgotten them

The implied link with language rings true to me. I've always felt strongly that learning chess and go can be likened to learning a language and so we should be studying go in terms of vocabulary and grammar rather than theory (i.e. you don't study linguistics to learn e.g. Japanese)). You also know as a long-suffering reader of my logorrhoea that I believe one of the biggest handicaps western players have imposed on themselves is to try to view too much in go in static terms (vocabulary lists) and to ignore the dynamic (grammar - or syntax, how the units work together). The katachi/suji (or haengma) dichotomy is one exemplary aspect but there are quite a few others such as choshi and the domino theory in fuseki - in fact proverbs in general have a lot to say about go grammar.

I have always been amazed (as I think you are) at the disdain shown by some people in the west for go proverbs, simply on the grounds they are sometimes wrong ("i before e except after c" is also wrong sometimes but I imagine most native English speakers are grateful to such little proverbs dinned into us at an early age).

Sakata said that when he was six years old, his teacher, Masubuchi, would come to the house and lay out a whole board position, and then ask him where he would play. He would show her and she would pat him on the head. How did he know where to play? {shrug}

New to me, but FWIW I wouldn't interpret this the way you seem to. I don't think the pat on the head was to do with the soundness of the move, but was an oblique way of instilling confidence in the youngster to choose his own move. He was obviously making plausibly sound moves already otherwise he wouldn't have been in Masubuchi's school.

[Kirby]

I like the comparison to language, which is partially what motivated this thread. When learning Japanese as an adult, it’s useful to study the joyo kanji, as you get more “bang for your buck” by learning those that are commonly used. Kanji can also be further deconstructed into radicals, which expediates the learning process- much more efficient than memorizing random strokes.

Even in English, learning the most commonly used 100 words gets you a long way.

I suppose that learning commonly appearing life and death problems, joseki, and practical shapes might be analogous, and might go a long way.

After that, maybe it’s time to hit the grammar books(?).

Perhaps the Pareto principle won’t solve everything. After all, even if you learn joyo kanji in Japanese, there’s a lot more to learn to truly become an expert in the language.

[EdLee]

Re:

joyo kanji

jōyo 常用.
Commonly used kanji.

[Knotwilg]

Pareto's rule is known to apply only 73% of the cases.

That aside, surely there are things to learn in each stage of your go career, that will make the biggest difference on your winning percentage.

As a relative beginner, the first thing to do is not resign, because that's the easiest way to lose a game. The second thing to do is not to run out of time, because that's the 2nd easiest way. The third thing to do is to keep focus at the end of the game and be aware of the liberties of your groups as liberties go down, because the 3rd easiest way to lose is by ignoring that one of your vital chains is in atari.

This is quite unpopular advice. Most beginners want to hear they should study joseki or fuseki and of course they should but it won't make any difference on their winning percentage. This leads to frustration: all that study and I'm still x kyu!

It is also unpopular advice because not resigning is considered to be impolite when far behind. My opinion is that there is plenty of time to develop the proper sportsmanship so develop gamesmanship first. So, the proverbial 80% of your victories as a beginner may come from

- fighting spirit
- time management
- counting liberties

Later in one's go career, I don't know.