Yilun Yang: The Fundamental Principles of Go

John Fairbairn · Post by **John Fairbairn** » Mon Aug 27, 2018 3:08 am

Until the advent of AlphaGo the 3-3- invasion was heresy.

How would the notorious early 3-3 invasion rank? Can it become a Class 2 move and, if so, what are the circumstances?

If we define "early" as >= move 10, early 3-3 was not really heresy among pros before AG but there was a very marked difference between White and Black.

Searching on 100,000 games in the GoGoD database, I used an empty 9x9 quadrant with the non-invader playing next. (Using a 10x10 quadrant made very little difference: e.g. 700 total games as opposed to 711 with a black 3-3 up to 2018-08-25).

With White being the 3-3 invader, there are 104 games before AG but only 10 with Black as the invader. Almost all these were komi games.

For the historical record, the first pro to try it was Segoe Kensaku in the Oteai in 1927, but it was not part of the New Fuseki. After 1928 there was an almost clear gap till 1974. But when it was tried it was generally the topmost pros who tried it (e.g. Cho Hun-hyeon, Yi Ch'ang-ho, Chen Zude, often multiple times). As a personal guess, I think they may have seen theoretical merit in the move, but found it (like tengen) a bit too hard in practice. In other words, it was eschewed not as heresy in defying accepted principles, but just for being too hard.

Overall, up to 2018, the pattern changes to a much smaller difference between White and Black 3-3: 711 for B 3-3 and 795 for W 3-3. Most of these are pros experimenting after AG. The total of AI games, out of 1506, is 189, split almost equally between B and W.

The AIs may have "seen" something that the early pros missed about early 3-3, but it may just be that, while all these games were with komi, the earliest ones had the smaller komi of 5.5 and that influenced the human pros. Looking at all the games together for each colour invasion, with A Black 3-3 the winning rates were Black 46% and White 54%, but for White 3-3 they changed to 41% and White 59%. This discrepancy seems to match the intuition of the early pro experimenters. The difficulty for the early humans may have been is that they had a blind spot in thinking they had to play the hanetsugi - that is what AI has really taught us, not the early 3-3.

My tentative conclusion about later developments is that the early 3-3 now so popular among human pros is part of a wave of experimentation which in no practical way undermines Yang's advice in this book, but for the experimenters both top human pros and AIs have detected something that favours an early White invasion (there is still a question mark over early Black invasion), but, like tengen, to take advantage of that needs extra skill that probably only AIs have. That means the real search for El Dorado is not on the fuseki shores of the game but deep in the jungles of the middle game, and that most of us human players are still better off holidaying in Hawaii or Torremolinos than up the Amazon.

Gomoto · Post by **Gomoto** » Mon Aug 27, 2018 4:13 am

I think Yilun Yang's ranking system is a valid teaching approach. Proverb like simplifications can help to develop your fuseki intuition, especially for people new to the game.

But the ranking in the book is in my opinion not up to date anymore.
(For example the passive play in the middle of a side has to be reevaluated.)

If you use such simplifications to develop your intuition, it is in our times essential, that you review your games with AI or with a stronger player and refine your moves.

Gomoto · Post by **Gomoto** » Mon Aug 27, 2018 4:22 am

The early 3-3 point invasion is in most positions in the same category as an approach to the 4-4 point.

I think it is very important for stronger players to have both options. Experimenting with the 3-3 invasion also encourages you to develop your shinogi strategy.

It is probably a "hard" move, but I am always suspicious if somebody suggests not to play "too difficult" moves. I am not afraid of go

Tami · Post by **Tami** » Mon Aug 27, 2018 4:25 am

Gomoto wrote:I think Yilun Yang's ranking system is a valid teaching approach. Proverb like simplifications can help to develop your fuseki intuition, especially for people new to the game.

But the ranking in the book is in my opinion not up to date anymore.
(For example the passive play in the middle of a side has to be reevaluated.)

If you use such simplifications to develop your intuition, it is in our times essential, that you review your games with AI or with a stronger player and refine your moves.

I pretty much agree...and I think Yang does too. He does after all write "Examining how the stones relate to each other is one of the most useful studies in go. Understanding these relationships, rather than merely memorising patterns, is the way to improve your ability and advance in the game" and continues to give the examples of following the ranking system, but without thinking about the stones' relationships, as cautionary tales. This topic is what Chapters Two and Three are all about: learning to think about the bigger picture, learning to make the stones work as a team. I suspect, at a more advanced level, some of the ideas that Robert Jasiek and others have come up with could prove to be useful additions to the toolbox. But, getting back on track for this book review, I recommend this one for players of about my level. It has given me just what I needed to start moving up from where I was, and I am sure other players hovering just outside the dans would find it a similarly refreshing change of perspective compared with A&D, SCOG and other such classics.

Bill Spight · Post by **Bill Spight** » Mon Aug 27, 2018 5:13 am

Tami wrote:Are we even going to be able to draw any general, working conclusions at all?

Absolutely. It's one of the things that humans do well.

A recent example, outside of go, comes to mind. We are entering the era of Big Data and Algorithms (i.e., machine learning), of which go bots are a part. One proprietary (and therefore secret) algorithm, based upon reams of data, indeed performed better than human experts had. But some academic researchers studied the algorithm's outputs and came up with a formula that did just as well. The formula only had three parameters. The algorithm was a definite advance, but then humans figured out how to simplify it effectively.

I am confident that the same thing will happen with go AI. Especially by young people who have little to unlearn first.

Or will the future involve mastering an ever more dauntingly large array of specific situations about which concrete knowledge is sine qua non?

Joseki books will definitely be rewritten.

They are showing us new joseki variations that would have taken humans a century or more to come up with and accept as joseki.

Bill Spight · Post by **Bill Spight** » Mon Aug 27, 2018 5:41 am

John Fairbairn wrote:The AIs may have "seen" something that the early pros missed about early 3-3, but it may just be that, while all these games were with komi, the earliest ones had the smaller komi of 5.5 and that influenced the human pros. Looking at all the games together for each colour invasion, with A Black 3-3 the winning rates were Black 46% and White 54%, but for White 3-3 they changed to 41% and White 59%. This discrepancy seems to match the intuition of the early pro experimenters. The difficulty for the early humans may have been is that they had a blind spot in thinking they had to play the hanetsugi - that is what AI has really taught us, not the early 3-3.

The realization that the hane-tsugi is not good is what takes the early 3-3 invasion out of the realm of the experimental and into the mainstream.

like tengen, to take advantage of that needs extra skill that probably only AIs have. That means the real search for El Dorado is not on the fuseki shores of the game but deep in the jungles of the middle game,

Without the hane-tsugi, it seems to me that the skill required to handle the relatively weak wall is the main question. The 3-3 invasion trades territory for influence. I would say thickness, but, as you have pointed out, whether the wall can be considered thick is a real question. Until we get familiar with these patterns, it seems to me that the main burden is on the invadee.

Bill Spight · Post by **Bill Spight** » Mon Aug 27, 2018 6:12 am

Tami wrote:I am quite keen to modify Yang's ranking system based on AI-derived information. That is to say, can we amateurs also use AI to construct new general principles while we wait for the pros to do it better?

Here is Yang's ranking system, paraphrased in a nutshell:

Class 1
Empty corners

Class 2
Approaches/Enclosures to assymetric corners
Mid-point of facing positions (star point/enclosure)
Starting a joseki

Class 3
Mid-point of side with fertile corner
Move that makes one side stronger and other side weaker
Approach/Enclose 4-4 or 3-3

Class 4
Completing 4-4 enclosure
Developing side with non-fertile corners
Other extensions

When you say mid-point, do you mean a play on the side near the 10-4 point? If so, I think that we have to demote those. (We also may need a fifth class.) Go Seigen had already started to demote the wariuchi back in the 1990s, and the sanrensei has generally been demoted.

Approaching or enclosing the 4-4 has already been promoted, but maybe not to the level of approaching the 3-4, as Uberdude points out. We have to add the early 3-3 invasion at the same level.

I'm not sure what you mean by starting a joseki, but if you mean playing a pincer, I observed that AlphaGo played fewer pincers than humans, and others have noticed that, as well.

1) How would you rank a tenuki when one side plays an approach to a 4-4 point? (My Lizzie prefers to low-approach Black in the lower right when Black attempts a Kobayashi-style Fuseki)

Back in the 90s, as part of a contest by John Fairbairn, I proposed the following proverb: Tenuki is always an option. Today's bots tenuki more than humans, and I think that there is an important lesson or two there. It means that we will have to play more flexibly, that is, more lightly, and will need to develop our understanding of sabaki, aji, and furikawari.

3) When you have low-approached a star point, how urgent or non-urgent is it to play out a joseki if the opponent answers with a knight's move?

We need to let go of the idea of finishing a joseki. As a general principle, the more stones played in an area (unless a mistake has been made), the less the value of a subsequent play. You can see this on the whole board: as the game progresses, plays generally become smaller. The same is true in local situations. That's why tenuki is always an option.

Bill Spight · Post by **Bill Spight** » Mon Aug 27, 2018 6:55 am

An interesting position shown by Tami:

Click Here To Show Diagram Code: [go]$$Bc Lizzie gives the highest rating (47.3%) for Black 3 $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . 1 . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . 2 . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . 3 . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

AlphaGo Teach also likes

. What top bot today doesn't?

This may be the new orthodoxy. But, as I have said, we shouldn't strain after gnats. There are other plays that AlphaGo likes almost as much, and cannot be dismissed.

Click Here To Show Diagram Code: [go]$$Bc Approach the bottom corner $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . 1 . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . 2 . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . 3 . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

There is little to choose between this play and the 3-3 invasion.

Click Here To Show Diagram Code: [go]$$Bc Large knight's enclosure $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . 1 . . . . . . 3 . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . 2 . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

This might be dubbed an inaccuracy, but I don't think we can call it a mistake. It came as a surprise to me. Not on my radar.

Click Here To Show Diagram Code: [go]$$Bc One space jump $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . 1 . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . 3 . . . . . . . . . . . . . | $$ | . . 2 . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

I think of this jump as a Go Seigen/AlphaGo play. It has become rather popular, hasn't it?

Perhaps we can classify all these plays as Class 2?

Next, high plays on the side, which AlphaGo rates a bit lower.

Click Here To Show Diagram Code: [go]$$Bc Class 3 plays? $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . 1 . . . . . . . . . . . . . | $$ | . . . O . . . . 3 3 . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . 2 . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . 3 . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . O . . . . . 3 . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

Probably premature to play one of these.

Click Here To Show Diagram Code: [go]$$Bc Class 4 plays? $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . 3 . . . . . . . . . . . . . . . | $$ | . . . . . 1 . . . 3 . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . 2 . . . . . . . . . . . . . . . . | $$ | . . . 3 . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

According to AlphaGo, I think we can call these mistakes at this point, despite being joseki. I have always wondered about the slide. Why the others are demoted I'm not sure.

Vesa · Post by **Vesa** » Mon Aug 27, 2018 9:14 am

One of the rare go books translated into Finnish. Wait, perhaps the only one.

https://www.kirjavinkit.fi/arvostelut/g ... perusteet/

Cheers,
Vesa

Bill Spight · Post by **Bill Spight** » Mon Aug 27, 2018 10:51 am

While playing around with AlphaGo Teach to explore how to punish (if that's necessary)

in this line, I came across an interesting position at

. AlphaGo plays three different moves. The winrates differ, OC. They were calculated independently by running 10,000,000 simulations from each position. What I find interesting is that their spread is 3.2%!

Now, that does not mean that a human play with a winrate that is less than from AlphaGo's by less than 3% is not a mistake, but it indicates that there is more leeway than a lot of people suppose. Certainly we should not worry about a winrate difference of less than 1%. No point straining after gnats.

The main lesson here, I think, is that the pincers in this and similar positions are deprecated.

Also, IMO, the case for the 3-3 invasion is strengthened, even though AlphaGo chose the keima and one space jump, as well. Nothing says that AlphaGo can't make mistakes, too.

John Fairbairn · Post by **John Fairbairn** » Mon Aug 27, 2018 10:59 am

Although I and, I think, most others are assuming the early 3-3 invasion by bots is somehow connected with, or justified by, their whole-board play much later on.

I'm beginning to wonder if that's really the case, though. After all, if a bot starts at 4-4 or 4-3 we don't assume it's seeing something around move 100 that makes these the best moves (maybe we should, but that's a huge ask...)

Now if we don't do that for 4-4 and 4-3, why should we do it for 3-3?

AlphaGo Zero played the 3-3 on move 2. Even if it did that every game, I think we'd find it hard to accept it knew there was something deep in the game that made it the best move - it would almost imply go was solved. But in fact there has only been one example of that play. So it and other bots assume in this case that other moves are better?

Yi-Tianrang played the triangled 3-3 against Deep Zen on move 3. There is now extra information on the board. But, still, other bots haven't copied this move.

In this case, the position from the book sample, when it was last White's turn, Leela Zero did indeed choose the triangled move. But it actually considered (only) three moves and the other two were 3-3 invasions on the right side. These even had a higher win rate but much inferior number of visits/rollouts. As I understand it, that means LZ is saying, "I don't like these higher scoring moves because there's too much uncertainty about them." If it's saying anything like that, that seems to imply LZ is not really seeing deep in the game. If it were a human, we would assume it is choosing these candidate moves purely on the basis of general principles or heuristics (as we would do) - in effect, choosing "local" moves. Of course it may not be doing that in reality, but if we apply reverse engineering to the process, we could easily start with that assumption. The strategic element may not be something specific deep in the middle game but simply an awareness by the bot that (say) the fourth line and third lines are not really equal (as we are taught) but the third line is significantly better.

This is all very reminiscent of the (human) debate over high approach over low approach. The popularity of the high approach has waxed and waned over the years, influenced no doubt by komi changes but possibly also by fashion. It was discussed in strategic terms but in reality it all came down to local plays. As I have indicated above, the 3-3 invasion has already been through a similar ebb and flow, and we can probably expect bots to behave in like fashion as refutations emerge. We see the same sine curve of fashion with chess openings.

gowan · Post by **gowan** » Mon Aug 27, 2018 11:25 am

This diagram from Bill's last post above:

Click Here To Show Diagram Code: [go]$$B One space jump $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . 1 . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . 3 . . . . . . . . . . . . . | $$ | . . 2 . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

reminded me of something I saw in pro games thirty or so years ago:

Click Here To Show Diagram Code: [go]$$B 19x19 diagram $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . O . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . O . . . . . | $$ | . . . . . . . . . . . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . O . . . . . , . . . . . X . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

.

I think it was attributed to Kajiwara and, depending on Black's response, might be followed by another one-space jump.

John Fairbairn · Post by **John Fairbairn** » Mon Aug 27, 2018 12:02 pm

reminded me of something I saw in pro games thirty or so years ago:

I have no record of that precise position but the position with the five stones in the upper right quadrant has appeared in 120 games (once in 1956, and then from 1973 onwards) and what I found interesting is that (like the 3-3 invasion experiments above) it was tried by many of the very best players.

I suspect the position you recall (great memory if so!) is the one you show plus a Black stone on the upper star point, because that appeared in the Japan China Supergo in 1987 when Nie Weiping beat Takemiya with it. Go World 50 has a commentary on it (and see also Go World 60), and I think it may also have appeared in one of Yuan Zhou's books on the styles of top pros. There was a bit of psychology at play there, I suspect, because the White jump in that position had first been introduced by Takemiya in 1976. Takemiya only ever played it once more but a clutch of players tried it against him.

Bill Spight · Post by **Bill Spight** » Mon Aug 27, 2018 1:27 pm

John Fairbairn wrote:

In this case, the position from the book sample, when it was last White's turn, Leela Zero did indeed choose the triangled move. But it actually considered (only) three moves and the other two were 3-3 invasions on the right side. These even had a higher win rate but much inferior number of visits/rollouts. As I understand it, that means LZ is saying, "I don't like these higher scoring moves because there's too much uncertainty about them." If it's saying anything like that, that seems to imply LZ is not really seeing deep in the game. If it were a human, we would assume it is choosing these candidate moves purely on the basis of general principles or heuristics (as we would do) - in effect, choosing "local" moves.

dfan may have a better and clearer explanation. But, IIUC, Leela Zero uses the Monte Carlo Tree Search strategy, even though it evaluates leaf nodes using a neural network instead of using Monte Carlo playouts. The winrate evaluations guide the tree search as it expands promising nodes. That means that nodes with high evaluations get more visits. It has been shown, I believe, that picking the play to the node with the most visits is a better strategy than picking the play to the node with the highest evaluation. That does not mean that that is the better play, if the two differ, just that it is good enough to play well. As you indicate, the program does not currently have enough information to be confident of a play with a high winrate but a low visit count.

As for seeing deeply into the game, that depends not just upon the calculation of variations of the current game, but upon the training of the evaluation algorithm (neural network). That training depends upon games that are played out, so we may say that Leela Zero's evaluation is based upon experience with a large number of similar positions. It is based upon breadth, not upon depth, per se. However, because of its breadth of experience, Leela Zero may be sensitive to a move that occurs 100 moves later than now. (Unlikely, but possible.) So it is basing its evaluations upon heuristics, but that does not mean that it is using local heuristics. Both its tree search and its training are based upon the whole board.

dfan · Post by **dfan** » Mon Aug 27, 2018 2:15 pm

In addition to Bill's post, I'll add a little precision about how Leela Zero et al are making their decisions.

They continue to look at variations, starting from the current position, until they hit some budget limitation (number of variations, or amount of time spent), then choose the move from the root position that has been visited the most.

Its tree of variations starts out with just the root position. Each variation traversal moves through the tree greedily by a heuristic to be described in the next paragraph, and ends when it reaches a node in the tree that has not been seen yet. The next traversal starts from the root again.

The heuristic is Q + U, where Q is a guess at the winrate from making this move, calculated by averaging all the winrates (generated from the value network) in the subtree starting with that move, and U is the output of the policy network (how often would it make this move based purely on intuition) multiplied by a factor that penalizes moves that already been visited a lot and rewards moves that haven't been looked at much. So, as you would expect, it is incentivized to look at moves that seem good but also incentivized to look at moves it hasn't considered yet.

So if a move has a high displayed winrate (Q) but doesn't have a lot of visits, it means that either 1) it just found the reason that it is a good move pretty recently, so the visit count is rising but isn't that high yet. or 2) it has a low U factor, which probably means that the policy (intuition) network doesn't like it much and would rather concentrate for the time being on moves that it thinks has a higher chance of panning out. These programs do not currently really have a sense of amount of uncertainty in an evaluation other than by measuring the number of variations starting from that position have been explored.

I would emphasize that this just happens to be how AlphaGo Zero works, and it's totally possible that other systems could work even better.

Life In 19x19

Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go

Re: Yilun Yang: The Fundamental Principles of Go