www.goratings.org is online with ratings of top professional

[hyperpape]

Do we judge that Japanese pro are weaker than Chinese/Korean based on few games? I say 'few games' from algorithm's point of view because if it has enough game, with enough result, it must be shown in the ranking, and Iyama Yuta wouldn't be rank so high, would he?
Since everyone claim that Japanese pro rarely play a game in international match, then how could we be so sure that they're being overestimated? Based on those 'rarely' played games? that maybe enough to convince our sense that Japanese pro is weaker, but unless more game is played right now it's not enough to convince the ranking's algorithm.

#My reply is not an answer to Hyperpape's question (which is why I don't quote him) He ask a different question

There is absolutely enough data to conclude that the strongest 20 Japanese professionals are weaker than the strongest 20 Korean/Chinese professionals. The only questions are "how precisely can we estimate that difference?" and where Iyama stands.

Also, with the exception of what we say about Iyama, common opinion and goratings are in sync: there are fewer than 10 Japanese players in its top 100.

[hyperpape]

Kirby, I don't think your analogy is very good. In the case of your school, you have no standard of comparison whatsoever. In the case of go, we have some, it's just limited, in ways that we can make rather precise. The upshot is that we don't know, but we can state our assumptions, we can explain the model, we can evaluate success. We just can't definitively answer the question we want to answer.

[pookpooi]

Or the simple answer is, maybe Iyama Yuta is really that strong and deserve that place I hope he finally hold the seven japan major simultaneously.

[Kirby]

Kirby, I don't think your analogy is very good. In the case of your school, you have no standard of comparison whatsoever. In the case of go, we have some, it's just limited, in ways that we can make rather precise. The upshot is that we don't know, but we can state our assumptions, we can explain the model, we can evaluate success. We just can't definitively answer the question we want to answer.

My example was extreme for the purposes of illustration.

I added an underline to your quote. The more limited the comparison with the larger data set, the more limited your confidence can be in your analysis. You cannot confidently assess the relative strengths between the sample and the population when you don't have many games to do so with.

So I stand by my original conclusion:

You take a handful of Iyama Yuta's games from a smaller population, try to extrapolate it onto a larger population... well, you're just shooting in the dark.
...
The only way to know for sure is to get real data from the larger population (i.e. get everyone to play games with one another worldwide, since that's what this list represents).

By the way, your other reply to 'pookpooi':

There is absolutely enough data to conclude that the strongest 20 Japanese professionals are weaker than the strongest 20 Korean/Chinese professionals.

I think we need to quantify our certainty with such statements. There will never be "absolutely" enough data to make this conclusion - we are only as certain as we are that the data points to a correlation.

There's also the point that player A can regularly beat player B, who regularly beats player C, who regularly beats player A.
---

So basically, my take is that we cannot have an accurate placement for Iyama Yuta if he is not actively playing many games against those we are comparing him against. I think my analogy is consistent with that viewpoint.

[pookpooi]

I'm trying to understand hyperpape, you(hyperpape) may think that it has absolutely enough data (to conclude that JP is weak) because you look at the rank and almost all Japanese pro are ranked at bottom, less than 10 Japanese pro are ranked at the top 100. But if it really has absolutely enough data why you(hyperpape) make Iyama Yuta an exception (for overestimating) he may be already at the right place, don't you think so?

[Kirby]

Case in point: consider simply Iyama Yuta vs. Lee Sedol. Iyama Yuta is ranked higher than Lee Sedol, yet of the 6 games they played one another, Lee Sedol won 4 of them. It could very well be that Iyama Yuta is stronger than Lee Sedol - or the other way around.

But for me to think that one guy is stronger than the other, they'd need to play more games. With the current data, I don't see a reason to say that Iyama is stronger than Lee Sedol.

[wineandgolover]

For some reason, I can't access page 1 of this thread on any browser or device.

This URL: "http://www.lifein19x19.com/forum/viewtopic.php?f=13&t=12048" gives me this message:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator and inform them of the time the error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.

Apache Server at http://www.lifein19x19.com Port 80

I hope this post sufficiently complies with those instructions.

This has happened to at least one other thread before. Most odd.

[Kirby]

For some reason, I can't access page 1 of this thread on any browser or device.

This URL: "http://www.lifein19x19.com/forum/viewtopic.php?f=13&t=12048" gives me this message:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator and inform them of the time the error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.

Apache Server at http://www.lifein19x19.com Port 80

I hope this post sufficiently complies with those instructions.

This has happened to at least one other thread before. Most odd.

I'll take a look tonight. Thanks.

[hyperpape]

Case in point: consider simply Iyama Yuta vs. Lee Sedol. Iyama Yuta is ranked higher than Lee Sedol, yet of the 6 games they played one another, Lee Sedol won 4 of them. It could very well be that Iyama Yuta is stronger than Lee Sedol - or the other way around.

But for me to think that one guy is stronger than the other, they'd need to play more games. With the current data, I don't see a reason to say that Iyama is stronger than Lee Sedol.

Your other post needs a more detailed reply, but this post doesn't make much sense to me. Ke Jie, the strongest player in the world, has a 0-1 record against Fu Chong. We can still compare the two and say that Ke is stronger, because we have so much information from the games that the two players have played against third parties. The head to head games are almost irrelevant.

Having six games doesn't change much. It's more games, but it still doesn't outweigh the dozens-hundreds of games each player has played against a broad set of opposition. Those games are what we judge their strength against.

I also agree that a player can outperform their strength against a particular opponent, or have an opponent who they lose against, even though they seem like they're stronger (Rui Naiwei and Lee Changho had that relationship at one point, I believe). Assigning a single value of strength to a player is an abstraction over those detailed head-head comparisons, and usually it's a more productive one, but I acknowledge it's an abstraction.

[hyperpape]

I'm trying to understand hyperpape, you(hyperpape) may think that it has absolutely enough data (to conclude that JP is weak) because you look at the rank and almost all Japanese pro are ranked at bottom, less than 10 Japanese pro are ranked at the top 100. But if it really has absolutely enough data why you(hyperpape) make Iyama Yuta an exception (for overestimating) he may be already at the right place, don't you think so?

Good question. There's no such thing as "enough data", just enough data to make particular conclusions.

1. There are a bunch of Japanese players who have roughly the same strength below Iyama, but there's only one Iyama. So we can look at the collective results of the weaker players, and see that while each one of them only has a few games, and the results might not be statistically significant, when you look at the big picture, there is a clear pattern. But you can't do that with Iyama. There is no one to compare him to. He's so much better than his domestic opponents right now, and he doesn't play enough against international opponents to make a clear comparison there. So we have to look at the model, and say "how could this be right/wrong? Or do we just have to say we have no idea?"
2. Iyama's results are better than the other Japanese players. So when I eyeball the results, I say Iyama's results don't look great against the top 20, but they also don't look terrible--it's hard to pin him down. He could be 3rd, or he could be 30th. Hard to say. When I look at the other Japanese players, I say "their results really look more like they could be 50th, or they could be 150th. Hard to say." (Don't put much stock in the particular numbers I'm choosing. I'm talking about eyeballing them).

[Kirby]

hyperpape, all I am arguing is that we need more data to make more confident assumptions. Lee Sedol and Iyama Yuna haven't played one another much, so we don't know much about what their relative placement should be.

In the same way, Iyama Yuta hasn't played a lot of people on the list, so it's hard to know what his actual placement should be.

The set of people that have played each other more often - those guys - we know more about their relative placement.

End of story.

[Kirby]

Also, I should clarify something.

I am not saying that it is impossible to project an assumption about Iyama's position internationally. Goratings.org has done that, and in my opinion, the placement is probably pretty good. But I do believe that more data among players in the population makes the placement assumption more accurate. If Iyama Yuta keeps playing those nearby him in the list, we can have more confidence in where he should be. So naturally, players that have played one another more on this list - we can be more confident of our assumptions about those guys, because we have more data.

It would be nice to quantify our certainty in some way of the ratings. So for players that have played most of the people on the list many times, we'd be able to see more confidence in our assumption about their placement. Whereas if you have a player that has only played 2 games, we could express our confidence in that guy's placement as being weak (e.g. maybe he got lucky).

[hyperpape]

Also, I should clarify something.

I am not saying that it is impossible to project an assumption about Iyama's position internationally. Goratings.org has done that, and in my opinion, the placement is probably pretty good. But I do believe that more data among players in the population makes the placement assumption more accurate. If Iyama Yuta keeps playing those nearby him in the list, we can have more confidence in where he should be. So naturally, players that have played one another more on this list - we can be more confident of our assumptions about those guys, because we have more data.

It would be nice to quantify our certainty in some way of the ratings. So for players that have played most of the people on the list many times, we'd be able to see more confidence in our assumption about their placement. Whereas if you have a player that has only played 2 games, we could express our confidence in that guy's placement as being weak (e.g. maybe he got lucky).

I agree with this post, though I would reframe it a bit:

There's already a concept of confidence in the model and I believe Remi tweaked the display to show the lower bound of the confidence interval: viewtopic.php?p=190653#p190653.

With Iyama, I suspect that the model has normal-high confidence in his rating, since he's played many games. We're all standing around speculating as to whether the model reliably/accurately reflects reality.

[Kirby]

With Iyama, I suspect that the model has normal-high confidence in his rating, since he's played many games.

I guess the only part where my opinion differs is that I think that high confidence should come from the rating if he has played many games against many different people from the population that he is being ranked in.

To give an extreme example, if he only played one guy on the list a million times, I'd personally have little confidence in whatever ranking we give him, even if he won (or lost) all of the time against that guy.

[pookpooi]

With Iyama, I suspect that the model has normal-high confidence in his rating, since he's played many games.

I guess the only part where my opinion differs is that I think that high confidence should come from the rating if he has played many games against many different people from the population that he is being ranked in.

Last month I count the nation of pro in goratings
Japan 333
China 251
South Korea 193
Taiwan 28

So maybe this is the problem, since Iyama Yuta play with the biggest group of the population (Japan), it'll continue to give him a good record
But the model will still give a good prediction, since it's more likely that he'll continue to play a lot of domestic games. If you lower the Elo of Iyama Yuta, it might reflect a better prediction for game against Korean/Chinese pro (which he rarely play) and worse prediction for domestic game (which he'll play a lot)