Life In 19x19

0.15 points? Is that significant?

Here are a couple of positions from recent pro games in the Elf commentaries that may be relevant to the question of the response to the two space high approach.

Click Here To Show Diagram Code

[go]$$B Sakai Hideyuki (W) - Yukawa Mitsuhisa
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . . . . . . 9 . . . . |
$$ | . . . , . . . . . , . . . O . , X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . a . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . O . . . |
$$ | . . . . . O . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

GoGoD 2018-06-06b

Elf's top choice is a, but it gives a 1½% higher winrate. KataGo is stronger than Elf, but it's hard to say that is a mistake.

Click Here To Show Diagram Code

[go]$$B Yamashiro Hiroshi (W) - Komatsu Hideki
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . 8 . . . . . . . . . . . X . . |
$$ | . . O , . X . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . a . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . O . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . , X . . |
$$ | . . . . . X . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

GoGoD 2018-03-05e

Again, Elf's top choice is a. This time it gives a lower winrate by 3%. I think this is borderline, but probably not a mistake.

AFAICT, there is little, if any, research into whether a play that a bot rates as inferior is actually a mistake. Especially in the opening.

Bill, I'm getting similar results from KataGo on the positions you posted.

In the first position, its top choice alternated between and a many times as I let the analysis run.

In the second position, it prefers a over by about 0.3 points, or 2.4% win rate. Actually, its second-favorite move after a is not , but the one-space low pincer.

Given that the two engines seem to agree, but there is some variance in joseki choice from position to position, I guess the whole board needs to be taken into account to choose the strictly best response to this approach.

Harleqin wrote:0.15 points? Is that significant?

I interpret this question in two ways. 1: Should we trust KataGo's judgment on a 0.15 point difference? 2: Is it worth studying a move that is only 0.15 points better?

Regarding (1), I personally choose to follow KataGo until I see a specific reason not to. I doubt this will lead me into worse habits than it leads me out of.

Regarding (2), I think there are multiple factors that affect which mistakes we should spend time fixing. One factor is the size of the mistake; another is how often the mistake is repeated; another is how easy it is to fix.

Generally, I think mistakes in the very early game can be repeated quite often, and are also very easy to fix, so many such mistakes are worth fixing even at 0.15 points. However, right now my opinion is that this particular move doesn't need much more attention yet, given Bill's further evidence this is a whole-board strategy question and not a simple clear-cut joseki preference.

hakuseki wrote:

Harleqin wrote:0.15 points? Is that significant?

I interpret this question in two ways. 1: Should we trust KataGo's judgment on a 0.15 point difference?

I rather intended: should we interpret a 0.15 point difference as a clear judgement? My hunch is: no, this is just line noise. Katago is telling us »this is fine too«.

Take a look at the very last moves of a game. There are often a lot of equivalent moves, but Katago assigns some arbitrary values to them, sometimes +0.1, sometimes -0.4 etc., and these also don't correspond to theoretical values from game theory. Shouldn't a computer be able to do an exact calculation at that point?

Harleqin wrote:Shouldn't a computer be able to do an exact calculation at that point?

Don't forget! The whole machine learning revolution for the last half-decade with neural nets means that it's the exact opposite now. Computers are bad at calculating things exactly and making sure to exhaustively account for every possibility. What they're good at is making a fuzzy approximate holistic judgment. Relative to humans they perform most best when choosing among a lot of finegrained options and smoothly-varying strategies and possibilities, rather calculating with a smaller number of sharp or deep tactics or reading out a precise outcome.

Anyways, 0.15 points is indeed comparable to or under the magnitude of the noise level in a lot of cases, and the noise level will be especially worse in many kinds of endgame, or in the middle of sharp fights, or if the game isn't close (because KataGo won't blatantly give up points like older zero bots, but still will care less about exactly optimizing the game) and better in the opening or in moments when there are a lot of whole-board choices.

But that doesn't mean 0.15 points is meaningless either. Let's suppose you evaluate all moves to roughly equal depth (you should compare neither scores nor noise based on moves that were searched highly dissimilar amounts). If you always pick a move that is estimated between 0.1 and 0.2 points worse than the original preferred choice whenever available, I'm pretty sure you would win noticeably less often. Even if you can't confidently judge whether the difference was worse or just noise on any single move, on average the difference should add up.

It's also worth remembering that even weak humans can correctly identify one move as better than another in positions based on standard human knowledge that a bot thinks are LESS than 0.15 points difference. So the idea 0.15 points difference moves are only understandable by 9d+ players is wrong. That the difference between right and wrong in opening problems for DDKs vol 1 can be so small is worthy of reflection. Sometimes it shows the human theory has cleverly extracted fine principles info understandable proverbs and ideas. Other times it shows the move we thought was wrong wasn't actually as bad as we thought.

Bill wrote:Again, Elf's top choice is a. This time it gives a lower winrate by 3%. I think this is borderline, but probably not a mistake.

AFAICT, there is little, if any, research into whether a play that a bot rates as inferior is actually a mistake. Especially in the opening.

What would you consider a valid research method for checking bot recommendations? We already know that bot ratings change as we allocate more visits/playouts to a position and that different bots produce different recommendations. Do you think that some form of human analysis would be more valid?

Seems like a first pass at this would be pretty straightforward, if a bit computationally expensive. Still, very doable if someone has a beefy GPU and is willing to invest the GPU time.

Pick an normal opening position with a move A and a move B that the bot rates as worse by, say, 3% winrate.

Run a script that plays about 2000 games from A and 2000 games from B, (using a fairly low visit or playout cap to make it feasible to run that many games). 2000 games on each is about how many you'd need to such that if there were a true 3% difference, that would be about 2 sigmas.
(Of course, we know bot winrates only estimate the winning chances under the *precise conditions of self-play training games*, rather the games you'd run in such a test, in addition to being limited by the neural net's learning capabilities. So one should be prepared for the high likelihood that the difference will not be 3% even under the positive scenario of the bot being "right" - indeed one should expect it to be different).

And ideally repeat for maybe a couple dozen positions.
Because playing a couple thousand games reduces the statistical noise for measuring the difference between moves for one particular position, but doesn't reduce the noise from due to the one-shot chance of ultimately being "right" or "wrong" for that position or not, if what you are interested in is the distribution of reliability in general, across opening positions. So you also want multiple positions.

And yes, comes with the caveats that a particular bot's self-play is not always measuring that are general to other players, particularly humans. You could also try mixing it up and testing with other bots besides the one making the initial judgment.

But still, it seems straightforward to do, and would probably be a worthwhile starting point. Anyone want to try?

Recent thoughts

I can divide my studies into three domains:

1. Problems whose solutions can be completely verified. Mostly endgame and life and death problems.
2. Patterns that can be memorized and applied in many games, i.e. joseki and some whole-board opening considerations.
3. The bulk of the midgame: can neither be solved completely, nor applied from memory.

For (1) I study tsumego; currently BadukPop is my preferred resource.

For (2) I used to rely on books and databases; now I have this study journal.

And (3) has mostly resisted systematization. I recognize proverbs and checklists as an attempt to make midgame knowledge explicit. But mostly I think I need to train my instincts to distinguish good and bad moves.

So lately I've been using AI Sensei to create hundreds of midgame problems from my games.

AI Sensei does an automatic game analysis, but I think it's still no substitute for interactive analysis with KaTrain in deciding which moves I should have played. To me the main benefit of AI Sensei is that it lets me create the problems faster than I can through other interfaces, and lets me study my own problem collection with spaced repetition.

I think in a couple months I'll have an idea of whether has improved my midgame substantially.

Anyway, on to game 23.

I played black in this game and lost about 26 points by move 80.

Game 23, position 1

Click Here To Show Diagram Code

[go]$$Bc No mistakes yet
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 4 . . . . . , . . . . 5 , 1 . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 2 . . . . . , . . . . 9 3 7 . . |
$$ | . . . . . . . . . . . . . a . 8 6 . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

I think aims to develop the right side, while a is more flexible to develop either the right side or the lower side. Since black already has a shimari on the upper right, I chose .

Game 23, position 2

Click Here To Show Diagram Code

[go]$$Wc Biggest area for black?
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . 3 . . . . . . . . . . . . . |
$$ | . . . O . . b . . , . c . . X , X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . d . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . 2 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . e . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . X X X . . |
$$ | . . a . . f . . . . . . . . . O O . . |
$$ | . . . . . . . . . . . . . 1 . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

I chose e, but that actually seems like the worst of the choices shown, although the margins are small. After review I like c to enlarge black's moyo. This extension is novel to me.


Game 23, position 3

Click Here To Show Diagram Code

[go]$$Bc Answering the pincer
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . . . . . . . . . . . |
$$ | . . . O . . a . . , . b . . X , X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 2 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 1 . c . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . X X X . . |
$$ | . . d . . . . . . . . . . . . O O . . |
$$ | . . . . . . . . . . . . . O . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

After white's pincer at , I jumped into the corner at d. However, in this situation the correct local move is to jump out at c. A move on the top side would be a bit better still.

Actually, several moves in this sequence were evaluated a bit differently than I expected, and I wonder if that is related to a potential ladder from the upper right.

Game 23, position 4

Click Here To Show Diagram Code

[go]$$Bc A ladder forms
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . . . b 6 5 9 . . . . |
$$ | . . . O . . . . . , . . 7 4 X , X . . |
$$ | . . . . . . . . . . . . c 8 . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . O . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . 2 . . . . . . . . . . . . . . . . . |
$$ | . 1 O O . . . . . , . . . . X X X . . |
$$ | . 3 X X a . . . . . . . . . . O O . . |
$$ | . . . . . . . . . . . . . O . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

In this sequence, is considered slightly worse than a, which I think could be related to the ladder.
However, is too slow, and is the first big (~4 points) mistake in this game. Black should instead atari at b; extending at c would not be bad either.

ez4u wrote:

Bill wrote:Again, Elf's top choice is a. This time it gives a lower winrate by 3%. I think this is borderline, but probably not a mistake.

AFAICT, there is little, if any, research into whether a play that a bot rates as inferior is actually a mistake. Especially in the opening.

What would you consider a valid research method for checking bot recommendations? We already know that bot ratings change as we allocate more visits/playouts to a position and that different bots produce different recommendations. Do you think that some form of human analysis would be more valid?

I am going to mix in my answer with comments and questions about lightvector's post below.

I. Science is basically about learning, public learning. With today's powerful computers and machine learning programs and massive databases, modern AI has had some spectacular successes, such as AlphaGo, and some questionable results, such as facial recognition and assessing teacher effectiveness. For the most part, these programs are proprietary, which means that the learning is not public. A good review of the field is Weapons of Math Destruction, by Cathy O'Neil, AKA Mathbabe. There is no question that computers are revolutionizing science, but progress is not smooth.

Let me address briefly a problem I know a bit about. To standardize sentencing by US Federal judges and make it more objective, judges make use of a program to predict reoffense. The more likely a convicted defendant is to reoffend, the longer the recommended sentence. Now, as an old-time scientist you might think that the accuracy of the programs would be tested against actual rates of reoffense and also what effect the length of sentences might have. But no such research has been done. Why? Because it would take years to do that research. So the programs are trained on proxies for reoffense, using certain variables. All of this is proprietary, so we don't really know what the proxies are or what variables are used. (This is based on what I read some years ago. Maybe things have changed for the better. But maybe not.) It turns out, no surprise, that the predicted probabilities of reoffense are higher for African Americans than for other racial groups. This does not mean that race itself is one of the variables of the algorithm; other variables that correlate with race may be. However, we know that illegal drug use is not correlated with race, but convictions for illegal drug use is. So the algorithm seems to be learning the injustices of the current system.

II. There is no one theory of learning, but it has been studied for well over a century. One of the most effective theories of learning is reinforcement. The learner, whether human, animal, or machine, is treated as a black box. We don't know how it generates its behavior. What we do with humans and animals is to reward good behavior, which reinforces whatever went on the black box that produced that behavior. For go, presumably winning a game rewards good behavior, but it reinforces every move by the winner, good or bad, crucial or not. Besides, to provide enough reinforcement to create a good play could take a very long time. One way of training the learner is to reward good moves. That gives immediate feedback, and reinforces good behavior.

But what if the trainer does not know whether a move is good or bad? Now we are getting closer to the situation with computer learning. One approach is to play over top level games, reinforcing choosing the same play as the top player. That approach was tried, with limited success. One problem being that there were just not enough pro games for training, and another being that the program could not learn to play better than humans.

Then Teasauro created a program that learned to beat humans at backgammon, using a new technique, temporal difference learning. Instead of predicting the best play, which was unknown, the program predicted the chance of winning the game. (I'm hazy here, because backgammon has a cube to change the payoff for winning, but this will apply to go.) That may be fine for the last move, but what about earlier moves? Tesauro's brilliant idea was to predict the prediction after the next play. The closer the prediction was to the next prediction, the more the method used to make that prediction was reinforced. This chain of predictions, after learning, converges to the prediction of the final payoff, which in its simplest form is a win or loss. The next prediction is a proxy for the final payoff.

B. F. Skinner, a pioneer of reinforcement learning in humans and animals, based it upon natural selection. It is not individuals or species that are being selected, but behaviors. One necessary part of natural selection is variation. Animals and humans typically exhibit a variety of behaviors which can be selectively reinforced. In computers, variety is introduced by randomness, one way or another.

Variety also gives us an error function. Error functions are useful for science, because we may be able to measure the success of a theory by the errors of its predictions.

What bugs me about current go AI, and, I gather from some recent reading, bugs others about other current AI, is that the algorithms do not come with error functions, so we cannot use the size of errors to assess how well the AI is doing.

I'll continue this tomorrow.

lightvector wrote:Seems like a first pass at this would be pretty straightforward, if a bit computationally expensive. Still, very doable if someone has a beefy GPU and is willing to invest the GPU time.

Hmm. This is just the output of the value network on the current position, minus the output of the value network on the previous, right?

When this forum first started discussing AI output, Bill often mentioned "margin of error". The commonest reaction to that, apart from ignoring it, was tantamount, to my inexpert eyes, to a bit of hand-waving - "oh, that, yeah, well, whatever." Bill's reaction to that reaction, it seemed to me, was to plod on by re-mentioning it patiently. In the end he seemed to tire of making the point (a common reaction on forums), but I was left with a strong (though possibly irrational) feeling that he was saying something profound and almost everybody else was missing the point. The fact that he has returned to the topic today reinforces my feeling.

In my case I can only really claim to understand the three words: margin, of, error.

That's not as humble as it sounds. Take 'of'. Most people consider it as a function word in a phrase such as 'A of B', meaning A's B: or a B belonging to A or somehow qualified by B. Most definitely not B's A. But the Book of Job is not the same as Job's Book. And where does 'of' fit in with phrases such as 'of a morning' or 'the book I am thinking of'? Even more curious is the case where 'A of B' = 'B of A': 'things of 'different kinds' means the same as 'different kinds of things'.

But I understand all that. When it comes to understanding the triumverbate 'margin of error' in the same way, though, I could only mumble a claim to understand that on a good day when the wind is in the right direction.

Bill has already been kind enough to write error in the correct British way, rather then the American errrr. Since he is so kindly disposed, I would like to ask him therefore if he would consider writing a brief "Margin of Error for Dummies" when he resumes his thoughts on this topic. I have an inkling it will open more eyes than just mine.

Ideally, we would quantify an AI's error margin for its suggested next move as the difference between its score and the perfect play score. We can only do so if we have mathematically proved the latter. Otherwise, definitions of error margins become difficult. Distributions, even if measured, of sampled, unproven scores (or worse: probabilities of moves) are not error margins that would indicate the correct score.

What we can measure is winning percentage of a particular AI against given opponents. Then, I guess, somehow an error margin can be defined probabilistically but what would it tell us?!

John Fairbairn wrote:Bill has already been kind enough to write error in the correct British way, rather then the American errrr. Since he is so kindly disposed, I would like to ask him therefore if he would consider writing a brief "Margin of Error for Dummies" when he resumes his thoughts on this topic. I have an inkling it will open more eyes than just mine.

In the US we normally run into the term, margin of error, with regard to polls and elections. I never know what that means, exactly, but I always take such polls with a large grain of salt, anyway. Still, if a poll say that 66% of Americans like thus and such, with a margin of error of 4%, and I am pretty sure what thus and such means, I figure that most Americans like it. I do not believe that between 62% and 70% of Americans like it, however. Most public opinion pollsters don't do a very good job, IMHO. The margin of error of an estimate is usually indicated by the ± sign. E.g., 66% ± 4%.

Go has some very good examples of the use of margin of error. For instance, consider the simple gote position with a Black/White game tree, {2|0}. I.e., Black to play gets a local score of 2 points, while White to play gets a local score of 0. We may estimate the value of that position as 1 ± 1. The margin of error is 1 point.

There is another way that we use margin of error. For that position, what is the margin of error of its average value? It is 0, because the average of 2 and 0 is 1, with no error.

Modern bots do not think that way, and they wouldn't even if they reported margins of error. They (or their programmers) think statistically. Statistically you do not generally have such strict margins of error, and usually you use such terms as standard error. Unless you are a pollster.

Suppose that we have a winrate estimate for Black in a given position with White to play of 87%. What does that mean? Well, it is said that it means that Black has an estimated 87% probability of winning the game. That's ambiguous. First, there are two main schools of probability, frequentist and Bayesian. The Bayesian school is the oldest and newest. The frequentist school replaced it in the late 19th to early 20th century, but Bayesian probability made a comeback in the late 20th century. Both are in use today. The frequentist school says that a single event has no fractional probability; it either occurs or it doesn't. The probability is 0 or 1. By that thinking the margin of error of the 87% estimate is 87%. That's no help.

But suppose that we were to play out the game several times from that position with White to play, where the AI plays both sides with the same randomized, non-deterministic strategy. Then sometimes Black will win and sometimes White will win. These results will give us a second and different, statistical estimate of Black's winrate under those conditions of play. Furthermore, we can calculate the standard error of that winrate estimate and from that derive a margin of error within which we expect that the actual winrate lies. Typically we find the margin of error by multiplying the standard error by 2. We expect that the actual winrate will lie within that margin of error approximately 95% of the time. Obviously, this is not how the program makes its original winrate estimate. If the difference between the program's original winrate estimate and the statistical estimate is greater than the latter's margin of error, we get suspicious of the original estimate.

But note that we have only calculated the margin of error of the statistical estimate, not the margin of error of the original winrate estimate, which is what we want. What I want, anyway. We have found 1 error estimate, that's all.

Without getting further into the research methodology, there are two meanings of the margin of error, absolute and statistical. The territory estimate for the gote position has an absolute margin of error. The statistical margin of error is found by producing a randomized sample, calculating the standard error of its mean, and multiplying it by some number, depending upon what percentage of the time we want the actual winrate to lie within the margin of error. Obviously, there is a good bit of judgement involved with the statistical margin of error.