The Probability of a Monkey Defeating Yi Chang-ho

[topazg]

What you all are saying amounts to a claim that 20k players are *worse than random*. I can't understand this. The random player is going to have to play like 10¹⁰⁰ games just to get one as good as an average 20k game...

Numbers scientifically pulled out of a donkey.

Not at all. What we are saying is that a random player can, and will, play the perfect game 1 in x times. The application of human knowledge applies filters to the way we think (hence why we even have "bad habits"), which limits the floor and ceiling of performance by applying set criteria over what moves are acceptable and what are not. The level of understanding affects how these levels move.

I'm arguing that a 20k player will have a range of equivalent performance, maybe between 12k and 28k, and a random bot will have a range of performance from 12p (whatever theoretically perfect play would be) to probably 150k, with an enormous skew towards the lower end. So a random bot will almost never achieve 50k performance, but has the potential to play perfectly, but 20k will play better than the random bot in almost every single game, but never reach such dizzy heights.

[daal]

Seriously? You guys have too many pens in your shirt pockets.

[HermanHiddema]

As I already said: I understand the argument, I just don't agree.

Well, sure, but the point of a discussion is to state a point of view and then defend it

We're getting somewhere now, tho, since you've indicated that you disagree with the statement that 20k's don't have the right moves in their list of candidates. I'm racking my brains to think how this could be possible.

How about this - Herman's 20k is someone who has a sound grasp of the correct strategy, and is able to evaluate the right place to play and the correct direction to play in, but is 20k due to a particularly poor tactical ability. In this game against topazg, he is somehow able to fluke his way thru the tactical situations, and his superior stategic ability proves to be a match for topazg?

The opinion that 20k's don't have the right moves in their inventory is backed up by nothing, it is just a gut feeling you have. Perhaps you feel that it is obviously true. I don't. I think it is obviously false. The only way to defend either point of view here would be with some data. Since there is none, and it is impossible to come by, the point is moot.

Really, at the probabilities we are talking about, it also becomes probable for the stones to randomly hop out of the bowl and into the right position, purely because quantum fluctuations caused all the molecules to move in the same direction at the same time, or some such thing.

[topazg]

The opinion that 20k's don't have the right moves in their inventory is backed up by nothing, it is just a gut feeling you have. Perhaps you feel that it is obviously true. I don't. I think it is obviously false. The only way to defend either point of view here would be with some data. Since there is none, and it is impossible to come by, the point is moot.

There is none for Go, but as I alluded to, there is sparse data for chess that imply that, for example, an ELO 1600 rated player cannot beat an ELO 2800 rated player (at least in serious games from which the limited data are drawn). The difference between a 9p and 20k is hugely greater, and on top of that, the room for game-critical losing errors in a Go game is significantly lower.

I'm not sure if the burden of proof lies with either side here more than the other (although the fact you think it is "obviously" false implies some requirement of a burden of proof I think!), but there's still only any data provided for one side of the argument. Like I say, I'm interested in data you have supporting your side more than a simple refutation of the argument.

[HermanHiddema]

The opinion that 20k's don't have the right moves in their inventory is backed up by nothing, it is just a gut feeling you have. Perhaps you feel that it is obviously true. I don't. I think it is obviously false. The only way to defend either point of view here would be with some data. Since there is none, and it is impossible to come by, the point is moot.

There is none for Go, but as I alluded to, there is sparse data for chess that imply that, for example, an ELO 1600 rated player cannot beat an ELO 2800 rated player (at least in serious games from which the limited data are drawn). The difference between a 9p and 20k is hugely greater, and on top of that, the room for game-critical losing errors in a Go game is significantly lower.

The sample size on that is nowhere near enough.

If every particle in the visible universe was a random go monkey (or a 20k), and all of them could simultaneously play against a player the strength of Lee Sedol, and each could finish a billion games every nano-second, and they had been at it since the big bang, the sample size would still be laughably insufficient.

[quantumf]

The opinion that 20k's don't have the right moves in their inventory is backed up by nothing, it is just a gut feeling you have. Perhaps you feel that it is obviously true. I don't. I think it is obviously false. The only way to defend either point of view here would be with some data. Since there is none, and it is impossible to come by, the point is moot.

Actually I do have a lot of evidence. I've taught a lot of players from beginner level, and played teaching games with 20k's, and reviewed games after playing 20k's. So I'm aware of the kinds of gaps they have in their thinking. We're not talking about a lot people, perhaps a few dozen. However, I've certainly never come across a 20k who could consistently demonstrate some capability of finding 1d level moves. No doubt your experience is similar. Do you reject this data?

[quantumf]

My experience with Go is that your rank is close to the weakest part of your game. If you're 5d at fuseki and direction, 5d at understanding joseki and hamete, 4d at common tactical tesuji, but 1k at reading complicated issues around a fight, you'll be 1d, or 2d at best - I think this is true even if you change around the skills at will. As a result, a 1d player and a 20k player will have such a wide disparity in skills and knowledge that it would be extraodinarily rare case that 20k was even up to the 1d standard in any aspect of the game. You talk about "correct stategy", but tactical knowledge and understanding determine the correct strategy - it's particularly clear that knowing what aji and what sequences are available in a given position play a huge part in deciding what the correct strategy should be at my level, let alone the level of a Go title holder.

I entirely agree with all this. My example player was merely to and understand what kind of 20k could beat a 1d, even theoretically.

Note also that a game is also long enough that 1d could see soon enough where a 20k was strong and weak, and focus on the weak aspects (if his mortgage was riding on it)

[quantumf]

If every particle in the visible universe was a random go monkey (or a 20k), and all of them could simultaneously play against a player the strength of Lee Sedol, and each could finish a billion games every nano-second, and they had been at it since the big bang, the sample size would still be laughably insufficient.

Yeah, I know. Which is why I proposed the thought experiment as a way of bypassing the overwhelming effect of the miniscule probabilities involved.

[topazg]

The sample size on that is nowhere near enough.

If every particle in the visible universe was a random go monkey (or a 20k), and all of them could simultaneously play against a player the strength of Lee Sedol, and each could finish a billion games every nano-second, and they had been at it since the big bang, the sample size would still be laughably insufficient.

Then you're misinterpreting the point of the data. It's not to collect enough games between a 20k and a 9p to disprove the 10^x games required for a win theory, it's the fact that rating variation appears to have a reasonably close to linear relationship with winning probabilities, over a few tens of thousands of games of data. The data all point towards a trend line that hits 0 and 100%, even though there's no real data collected between players far enough apart to be on this line.

It's pointless claiming that "my theory is dubious until I've collected enough billions upon billions of games of data to prove beyond doubt that it is impossible for a 20k to beat a 9p", that's a very blinkered way of assessing or debunking the veracity of theories. Sure, it would be ideal if those games could be collected, but they can't be for obvious reasons, and there are other ways to support or refute the theory (such as trend modelling on existing data of win probabilities across disparate ranks).

Regarding the comparison of ideas, data with a non-compelling sample size is still better than no data at all. All your arguments are aimed towards poking holes in my theory (which is fine, and important), but you aren't allowing anyone to view the supporting evidence of your theory with a critical eye, by the very nature of not providing any supporting evidence. Even a data set that seems to indicate a true s-curve between rating difference and performance would be compelling, but you haven't shown one.

[HermanHiddema]

The opinion that 20k's don't have the right moves in their inventory is backed up by nothing, it is just a gut feeling you have. Perhaps you feel that it is obviously true. I don't. I think it is obviously false. The only way to defend either point of view here would be with some data. Since there is none, and it is impossible to come by, the point is moot.

Actually I do have a lot of evidence. I've taught a lot of players from beginner level, and played teaching games with 20k's, and reviewed games after playing 20k's. So I'm aware of the kinds of gaps they have in their thinking. We're not talking about a lot people, perhaps a few dozen. However, I've certainly never come across a 20k who could consistently demonstrate some capability of finding 1d level moves. No doubt your experience is similar. Do you reject this data?

Have you ever come across a random monkey who could consistently demonstrate some capability of finding 1d level moves?

I know how weak 20 kyu players are. But random players are staggeringly much weaker.

[quantumf]

Have you ever come across a random monkey who could consistently demonstrate some capability of finding 1d level moves?

Sure - the Monte Carlo bots

I know how weak 20 kyu players are. But random players are staggeringly much weaker.

Only in a statistical sense. They are not constrained by faulty logic and flawed reasoning and hence could play a perfect game, or at least perfect enough to beat topazg.

[HermanHiddema]

Then you're misinterpreting the point of the data. It's not to collect enough games between a 20k and a 9p to disprove the 10^x games required for a win theory, it's the fact that rating variation appears to have a reasonably close to linear relationship with winning probabilities, over a few tens of thousands of games of data. The data all point towards a trend line that hits 0 and 100%, even though there's no real data collected between players far enough apart to be on this line.

It's pointless claiming that "my theory is dubious until I've collected enough billions upon billions of games of data to prove beyond doubt that it is impossible for a 20k to beat a 9p", that's a very blinkered way of assessing or debunking the veracity of theories. Sure, it would be ideal if those games could be collected, but they can't be for obvious reasons, and there are other ways to support or refute the theory (such as trend modelling on existing data of win probabilities across disparate ranks).

Regarding the comparison of ideas, data with a non-compelling sample size is still better than no data at all. All your arguments are aimed towards poking holes in my theory (which is fine, and important), but you aren't allowing anyone to view the supporting evidence of your theory with a critical eye, by the very nature of not providing any supporting evidence. Even a data set that seems to indicate a true s-curve between rating difference and performance would be compelling, but you haven't shown one.

I'm sorry, but I don't understand the argument. I think there is a clear relationship between rating difference and performance. The larger the difference, the worse the performance of the weaker player. And since the random player is weaker than the 20k, therefore his performance is worse than the 20k, when playing the same opponent. In your first paragraph, you seem to be arguing the same thing, proposing to extrapolate from the linear relation between rating and performance. But then later you claim that data that shows a relationship between rating and performance would be compelling, but I haven't shown it. So now I'm a bit lost here. It seems to me that you are arguing that the data that you yourself provided supports my position. But that can hardly be what you mean, can it?

[topazg]

I'm sorry, but I don't understand the argument. I think there is a clear relationship between rating difference and performance. The larger the difference, the worse the performance of the weaker player. And since the random player is weaker than the 20k, therefore his performance is worse than the 20k, when playing the same opponent. In your first paragraph, you seem to be arguing the same thing, proposing to extrapolate from the linear relation between rating and performance. But then later you claim that data that shows a relationship between rating and performance would be compelling, but I haven't shown it. So now I'm a bit lost here. It seems to me that you are arguing that the data that you yourself provided supports my position. But that can hardly be what you mean, can it?

I'm arguing that the support data collected argues towards a relationship between rating and performance where, with a large enough rating gap, the chance of the weaker playing is 0%. There is limited by existant data supporting this by demonstrating that, for chess at least, the relationship is closer to linear than logistic.

You are arguing that the relationship between rating and performance exists, but the lower and upper bounds of winning chance never reach 100% or 0% - a true logistic function. I'm asking for some data that supports the view that a true logistic function is more reliable model than a function that is linear with logistic elements.

[HermanHiddema]

I'm sorry, but I don't understand the argument. I think there is a clear relationship between rating difference and performance. The larger the difference, the worse the performance of the weaker player. And since the random player is weaker than the 20k, therefore his performance is worse than the 20k, when playing the same opponent. In your first paragraph, you seem to be arguing the same thing, proposing to extrapolate from the linear relation between rating and performance. But then later you claim that data that shows a relationship between rating and performance would be compelling, but I haven't shown it. So now I'm a bit lost here. It seems to me that you are arguing that the data that you yourself provided supports my position. But that can hardly be what you mean, can it?

I'm arguing that the support data collected argues towards a relationship between rating and performance where, with a large enough rating gap, the chance of the weaker playing is 0%. There is limited by existant data supporting this by demonstrating that, for chess at least, the relationship is closer to linear than logistic.

You are arguing that the relationship between rating and performance exists, but the lower and upper bounds of winning chance never reach 100% or 0% - a true logistic function. I'm asking for some data that supports the view that a true logistic function is more reliable model than a function that is linear with logistic elements.

Oh, I see.

Well, the basis of the Elo rating system and similar systems is logistical. There is no rating difference for which the formula returns 0 or 1. The data fits that curve reasonably well, AFAIK. The fact that a certain result, which according to the formula should have a very small but non-zero chance, has not in fact happened, does not in any way constitute proof that it cannot happen.

[HermanHiddema]

Anyway, I really don't know how I got dragged into this argument.

It is my opinion that neither side has any chance of proving that they are right. All I have is an opinion. I also think that that is all you have, an opinion. I'm not claiming that the burden of proof is on either side. I'm not claiming that there is any support for my position at all. In fact, I think there is no meaningful support for my position. And that there is also none for yours. I do not think that thought experiments have value in a discussion like this. I do not think there is any meaningful way to extrapolate from the data we have.