Life In 19x19

daniel_the_smith wrote:You're saying, if I read correctly, "Given that I experienced a loss, Bayes says we should expect my mistakes must have been bigger". In isolation, yes. But you're not done, you also have to run some other hypotheses through, like the one that "my mistakes must have been more numerous", and the one that, "my opponent's mistakes were fewer and/or less severe". You can't use Bayes unless your evidence distinguishes between those hypotheses, i.e., it has to actually be evidence. Without knowing the player's mistake frequency and size distributions, I don't think the fact that there was a loss favors any of those explanations.

Here's what I had in mind. Let L be the event of a game loss. Let M be the event of one particular type of significant mistake for the player in question (an example of a particular type of mistake would be "missing a snapback"; another would be "needlessly allowing a group to be enclosed").

Now what I want you to allow me is that P(L|M) > P(L|not M), i.e. the probability you lose if you make that type of mistake in your game is larger than the probability you lose if you don't make the mistake. (No Bayes Theorem yet.) If you don't allow me this, so be it, but keep reading for where I would have used Bayes Theorem if you had.

If so, then some algebraic manipulation using Bayes Theorem (i.e. P(A|B)P(B) = P(B|A)P(A)) allows me to conclude P(M|L) > P(M|not L), i.e. the probability you make that type of mistake in a lost game is larger than the probability you make that mistake in a won game. I'll leave out the algebraic manipulation unless this last bit is really what you're challenging.

If this counts as meaningless, sorry!

prokofiev wrote:Here's what I had in mind. Let L be the event of a game loss. Let M be the event of one particular type of significant mistake for the player in question (an example of a particular type of mistake would be "missing a snapback"; another would be "needlessly allowing a group to be enclosed").

Now what I want you to allow me is that P(L|M) > P(L|not M), i.e. the probability you lose if you make that type of mistake in your game is larger than the probability you lose if you don't make the mistake.

I'll totally grant that. But M is actually a class of hypotheses. So M₁ is playing a bad forcing move, M₂ is losing sente, etc., up to M_N is running out of time.

prokofiev wrote:If so, then some algebraic manipulation using Bayes Theorem (i.e. P(A|B)P(B) = P(B|A)P(A)) allows me to conclude P(M|L) > P(M|not L), i.e. the probability you make that type of mistake in a lost game is larger than the probability you make that mistake in a won game. I'll leave out the algebraic manipulation unless this last bit is really what you're challenging.

This is true but only half the story. The thing is, that logic doesn't distinguish between various classes of mistake. Yes, it means M₁ is more likely, but it also means M₂ is the same amount more likely. It also raises P(OW) (where OW = Opponent played well) by the same amount. Just knowing that you lost raises the probability of all potential causes of your loss, including ones that have nothing to do with you. To make the particular conclusion you're trying to make, you need evidence that would look different depending on what exactly caused your loss.

To put it differently, you need evidence that changes the relative frequency of the possible explanations.

Suppose M₅ is missing a snapback and M₆ is self atari. If you're normal, you probably miss more snapbacks than self-ataris, so the prior for M₅ is larger than the prior of M₆. If M₅ was 10% more likely than M₆ to begin with, it will still be 10% more likely after observing a loss and applying Bayes. Even though the probability of both M₅ and M₆ will have gone up, their relative frequency didn't change.

To get to your conclusion, you'd have to have evidence to the effect that larger mistakes result in losses more often than more numerous small mistakes. Then you can plug actual numbers into Bayes and watch it change the relative frequencies of the hypotheses.

I'm hungry so hopefully that makes sense

Daniel, why are you so concerned about biggest mistakes? I realize someone may have said something about "biggest mistakes" but what we're really worried about is the mistakes that have the highest "weighted measure"* of bigness and frequency. And isn't the Bayesian argument just transparent there?

If I have a 120 point mistake that happens one in one hundred games, that sucks, and I'd like it fixed, but I'll still care more about a 2 point mistake that happens in half my games.

* That's a weasel phrase. I don't mean anything technical.

hyperpape wrote:Daniel, why are you so concerned about biggest mistakes? I realize someone may have said something about "biggest mistakes" but what we're really worried about is the mistakes that have the highest "weighted measure"* of bigness and frequency. And isn't the Bayesian argument just transparent there?

Depending on how you define "weighted measure" (which I think is a useful concept, and maybe one we should attempt to define), I think that becomes a tautology, no need for Bayes at all.

For example, if the WM of a particular mistake is the percentage of lost games it appears in, then this is a tautology.

hyperpape wrote:If I have a 120 point mistake that happens one in one hundred games, that sucks, and I'd like it fixed, but I'll still care more about a 2 point mistake that happens in half my games.

Yeah, totally.

I don't see any particular reason to continue the discussion (and promised myself I wouldn't! ). Ed and I each had our say, I doubt Round 3 would be any more profitable than Round 2. But for the developing argument about Bayes' Theorem, it's worth distinguishing between my highly focused point, which I think is influencing Daniel, and a broader point, which prokofiev and hyperpape were elaborating with Bayes' Theorem.

Broad point: if you randomly select one of your won games and one of your lost games, ex ante it's more likely that the lost game will have an example of any particular error you make. On inspection there may be more of a particular class of error, large or small, in the won game, but but ex ante you wouldn't suspect this, would you? Relative proportions don't matter.

Focused point: Perfect play gains nothing, so saying that a move loses X points already assumes that the opponent will respond correctly. Errors that are part of larger sequences where the opponent screws up or doesn't notice the mistake don't affect the point total, and so don't affect who wins. To use our formal terminology, something like {bad ko threat & opponent ignores} is a mistake class that loses points and is more likely to show up in a lost game, while {bad ko threat & opponent answers} actually gains points and so is no more likely to show up in a lost game, even though the bad ko threat was an equally large mistake in either case. Similar examples: {capture in ladder instead of net & opponent plays ladder breaker}, {mess up l&d & opponent kills}, {make weak group & opponent attacks}. All of these mistakes are equally bad regardless of what the opponent does, but in some of them telling the player what the correct play is is much easier than getting him to appreciate how bad things could have gotten if his opponent had responded correctly.

I was calling these "big mistakes" in a colloquial way, but if we want feed them into a probability function we would have to specify that it is the opponent's lack of mistakes in responding to them that is making my original, point-losing move both (i) "bigger," and so more likely to be found in a lost game, and (ii) "glaring," and so easier to learn from.

Back to the broad point: here's a heuristic argument that may show that large errors are relatively more likely in lost games. I haven't actually formalized it, so it may be garbage. Let's say I play an opponent, always taking black, hundreds of times, and we randomly vary the komi between zero, 7, and 14 in each game. Using 14 komi is like making a 14 point mistake on the first move, 7 komi is like me making a 7 point mistake. If we're even with no komi, perhaps he wins 70% of the games with 7 komi and 90% of the games with 14 komi. So while I make this particular 7 pt. mistake with pr=.33 and the 14 pt. mistake with pr=.33, the pr(14 komi game | i lost)= .43, while pr(7 komi game | i lost) = .33. I don't think this relies on the fact that pr(14 komi |7 komi)=0. So the relative frequency of large mistakes should be higher in lost games.

But I'm not trying to say that I think errors of large magnitude are necessarily more important, and this doesn't affect any argument about which games to review.. My point was always narrowly about how palpable the consequences of the error are, which depends on independent events later in the game. In fact, I think that one thing prokofiev has shown is that a lost game is much more likely to contain multiple cases of the same error, which would make it much easier to correct common small errors in lost game. (Again, this goes back to making things manifest: when you can point out that someone made the same endgame mistake four times, he'll understand why the game suddenly turned against him, and won't do it again.)

daniel_the_smith wrote:This is true but only half the story. The thing is, that logic doesn't distinguish between various classes of mistake. Yes, it means M₁ is more likely, but it also means M₂ is the same amount more likely. It also raises P(OW) (where OW = Opponent played well) by the same amount. Just knowing that you lost raises the probability of all potential causes of your loss, including ones that have nothing to do with you. To make the particular conclusion you're trying to make, you need evidence that would look different depending on what exactly caused your loss.

To put it differently, you need evidence that changes the relative frequency of the possible explanations.

Suppose M₅ is missing a snapback and M₆ is self atari. If you're normal, you probably miss more snapbacks than self-ataris, so the prior for M₅ is larger than the prior of M₆. If M₅ was 10% more likely than M₆ to begin with, it will still be 10% more likely after observing a loss and applying Bayes. Even though the probability of both M₅ and M₆ will have gone up, their relative frequency didn't change.

To get to your conclusion, you'd have to have evidence to the effect that larger mistakes result in losses more often than more numerous small mistakes. Then you can plug actual numbers into Bayes and watch it change the relative frequencies of the hypotheses.

I'm hungry so hopefully that makes sense

I'm not sure what my conclusion was supposed to be. If you want something not specific to a particular type of mistake, how about this:

Call a type of mistake M for which you'll allow P(L|M) > P(L|not M) a "relevant" type of mistake. Then the conclusion I had for each type of mistake individually, i.e. P(M|L) > P(M| not L) for "relevant" M, implies that the expected number of "relevant" *types* of mistakes in a lost game is higher than the expected number of "relevant types of mistakes in a won game. (No mention of frequency of mistake within a given type.)

The expected number of "relevant" types of mistakes present seems a reasonable measure of what sort of game is good to review.

An aside: There are presumably types of mistakes that are not "relevant" in the above sense. I imagine that certain endgame errors are much more likely in won games than in lost games, at least for some individuals.