Could you explain why Iyama is currently ranked #4, and excluding AlphaGo, #3 human player in the world?

Iyama's ranking seems unrealistically high. Seeing that Japanese players rarely compete internationally, how are Japanese pros' games weighed?

I found it amusing that Thomas Debarre, the 6d amateur from France, is ranked about Rin Kaiho 9p from Japan/Taiwan, who although getting rather old could surely easily beat Thomas.

There's no concept of nationality in algorithm, if Japan pro has like 60% of game database and international game has 40% the international game is treated as a minority here.

Edit: For algorithm, Japan game means group A that usually love to play among each other, and international game means group B that usually play among each other, and if group A is larger algorithm doesn't see why it has to give more weight to group B.

Right. Algorithm doesn't bias based on nationality, but depends on the games we give it. So rating will inherently reflect the data the algorithm is given.

_________________
be immersed

Here's a simple idealized story. There's enough international play for the algorithm to judge that the best Chinese and Korean go players are much better than Yamashita, Kono, Takao, Yuki, Yu, Kyo, and other Japanese players. The algorithm also judges that Iyama is much better than those players. Specifically, it estimates that he'll beat them something like 75% of the time. It estimates that Park Junghwan will beat them slightly more, Lee Sedol slightly less. Based on those estimates, it says Park is stronger than Iyama, Lee is a little weaker.

Are these estimates accurate? Absent a lot of head to head games between Iyama and players at his level, it is hard to be confident. I can think of three ways they could fail to be accurate:

1. Those Japanese players that Iyama is being compared to are themselves overrated. That would be visible in the data: the model would predict that they win more games against Korean/Chinese players than they do.
2. The assumptions about how strength is distributed are bad--maybe if you look at the top five players in the world, the Chinese/Koreans beat the 70th-80th rated players more than Iyama does, but for some reason, the math works out that he has a high rating[0]. This would *also* be visible in the data.
3. Iyama's performance against Japanese players does not predict his performance against Korean/Chinese players. Maybe he's gotten into the heads of the other Japanese players, but can't translate that into wins against international competition. In that case, playing against international competition would harm his rating (there's a sense in which the model used predicts that if two players play a series of games, they should not expect their ratings to change). This possibility can't be judged based on the game records we have.

In Remi's original paper on KGS, he evaluated his models predictions with regard to the KGS users. Eventually someone should do the same for professionals.

[0] I don't even know if the math for this works out. I suspect it could, but who knows?

_________________
Occupy Babel!

If Goratings had sufficient Data...

Is it the elo rating system, etc. it assumes that if a player is rated A points higher and has X excess winning percentage, and if rated B points higher should have, for demonstrative sake, 2X excess win percentage. It could be overestimating the difference in skill level between Iyama and the rest of the Japanese pros by believing "that for him to maintain such a high winning percentage against all the other Japanese professionals, he must be X points above them!". The Japanese pro's are generally underrated, Iyama is overatted as there is little international calibration?

_________________
On Go proverbs:
"A fine Gotation is a diamond in the hand of a dan of wit and a pebble in the hand of a kyu" —Joseph Raux misquoted.

Do we judge that Japanese pro are weaker than Chinese/Korean based on few games? I say 'few games' from algorithm's point of view because if it has enough game, with enough result, it must be shown in the ranking, and Iyama Yuta wouldn't be rank so high, would he?
Since everyone claim that Japanese pro rarely play a game in international match, then how could we be so sure that they're being overestimated? Based on those 'rarely' played games? that maybe enough to convince our sense that Japanese pro is weaker, but unless more game is played right now it's not enough to convince the ranking's algorithm.

#My reply is not an answer to Hyperpape's question (which is why I don't quote him) He ask a different question

The algorithm does not look at Iyama Yuta's game records to assess his strength; Chinese and Korean pros do.

It is difficult to extrapolate assumptions about the general population from a small sample when you are looking for an absolute ranking for the general population. For example, I was the valedictorian of my small high school of 82 people, and also had the highest standardized test scores. Naively, you could say that I was in the top 1~2%, academically. With no other data about the distribution of academic ability among students from other schools, I could try to say I was in the top 1~2% of students in the world. But that is bull****. Once I entered an environment having a greater number of students, competition grew, and I realized that there were many other skilled students. Among the students at my university, if I had to guess, I might guess that I were in the top 30% academically, but there's no way I could claim that I were in the top 1~2%. Competition was much more severe. And this is just talking about percentages. Even if my school were representative of the general population, being in the #1 position at my school in no way suggests that I will be #1 in a larger population.

So however you want to frame the mathematics, it's always going to be dependent on the dataset. You take a handful of Iyama Yuta's games from a smaller population, try to extrapolate it onto a larger population... well, you're just shooting in the dark. Maybe you're right, and maybe you're wrong.

The only way to know for sure is to get real data from the larger population (i.e. get everyone to play games with one another worldwide, since that's what this list represents).

_________________
be immersed

Again, whatever Chinese/Korean pro might say about Iyama Yuta, from algorithm's point of view we don't know for sure unless more data are inserted. They may be right though. But the problem is, they're able to give a better prediction of match's result by using different set of mind to evaluate each pro's strength by looking at the game record while Remi use only game result with single algorithm (and still give the best result among other rating method). This is why it'll be fun if we evaluate pro by have AlphaGo look at the game and give winrate in each move (given that AlphaGo is much stronger than it is right now and never lose to human again)

There is absolutely enough data to conclude that the strongest 20 Japanese professionals are weaker than the strongest 20 Korean/Chinese professionals. The only questions are "how precisely can we estimate that difference?" and where Iyama stands.

Also, with the exception of what we say about Iyama, common opinion and goratings are in sync: there are fewer than 10 Japanese players in its top 100.

_________________
Occupy Babel!

Kirby, I don't think your analogy is very good. In the case of your school, you have no standard of comparison whatsoever. In the case of go, we have some, it's just limited, in ways that we can make rather precise. The upshot is that we don't know, but we can state our assumptions, we can explain the model, we can evaluate success. We just can't definitively answer the question we want to answer.

_________________
Occupy Babel!

Or the simple answer is, maybe Iyama Yuta is really that strong and deserve that place I hope he finally hold the seven japan major simultaneously.

My example was extreme for the purposes of illustration.

I added an underline to your quote. The more limited the comparison with the larger data set, the more limited your confidence can be in your analysis. You cannot confidently assess the relative strengths between the sample and the population when you don't have many games to do so with.

So I stand by my original conclusion:


By the way, your other reply to 'pookpooi':


I think we need to quantify our certainty with such statements. There will never be "absolutely" enough data to make this conclusion - we are only as certain as we are that the data points to a correlation.

There's also the point that player A can regularly beat player B, who regularly beats player C, who regularly beats player A.
---

So basically, my take is that we cannot have an accurate placement for Iyama Yuta if he is not actively playing many games against those we are comparing him against. I think my analogy is consistent with that viewpoint.

_________________
be immersed

I'm trying to understand hyperpape, you(hyperpape) may think that it has absolutely enough data (to conclude that JP is weak) because you look at the rank and almost all Japanese pro are ranked at bottom, less than 10 Japanese pro are ranked at the top 100. But if it really has absolutely enough data why you(hyperpape) make Iyama Yuta an exception (for overestimating) he may be already at the right place, don't you think so?

Case in point: consider simply Iyama Yuta vs. Lee Sedol. Iyama Yuta is ranked higher than Lee Sedol, yet of the 6 games they played one another, Lee Sedol won 4 of them. It could very well be that Iyama Yuta is stronger than Lee Sedol - or the other way around.

But for me to think that one guy is stronger than the other, they'd need to play more games. With the current data, I don't see a reason to say that Iyama is stronger than Lee Sedol.

_________________
be immersed

For some reason, I can't access page 1 of this thread on any browser or device.

This URL: "http://www.lifein19x19.com/forum/viewtopic.php?f=13&t=12048" gives me this message:



I hope this post sufficiently complies with those instructions.

This has happened to at least one other thread before. Most odd.

_________________
- Brady
Want to see videos of low-dan mistakes and what to learn from them? Brady's Blunders

I'll take a look tonight. Thanks.

_________________
be immersed

Your other post needs a more detailed reply, but this post doesn't make much sense to me. Ke Jie, the strongest player in the world, has a 0-1 record against Fu Chong. We can still compare the two and say that Ke is stronger, because we have so much information from the games that the two players have played against third parties. The head to head games are almost irrelevant.

Having six games doesn't change much. It's more games, but it still doesn't outweigh the dozens-hundreds of games each player has played against a broad set of opposition. Those games are what we judge their strength against.

I also agree that a player can outperform their strength against a particular opponent, or have an opponent who they lose against, even though they seem like they're stronger (Rui Naiwei and Lee Changho had that relationship at one point, I believe). Assigning a single value of strength to a player is an abstraction over those detailed head-head comparisons, and usually it's a more productive one, but I acknowledge it's an abstraction.

_________________
Occupy Babel!

Good question. There's no such thing as "enough data", just enough data to make particular conclusions.

1. There are a bunch of Japanese players who have roughly the same strength below Iyama, but there's only one Iyama. So we can look at the collective results of the weaker players, and see that while each one of them only has a few games, and the results might not be statistically significant, when you look at the big picture, there is a clear pattern. But you can't do that with Iyama. There is no one to compare him to. He's so much better than his domestic opponents right now, and he doesn't play enough against international opponents to make a clear comparison there. So we have to look at the model, and say "how could this be right/wrong? Or do we just have to say we have no idea?"
2. Iyama's results are better than the other Japanese players. So when I eyeball the results, I say Iyama's results don't look great against the top 20, but they also don't look terrible--it's hard to pin him down. He could be 3rd, or he could be 30th. Hard to say. When I look at the other Japanese players, I say "their results really look more like they could be 50th, or they could be 150th. Hard to say." (Don't put much stock in the particular numbers I'm choosing. I'm talking about eyeballing them).

_________________
Occupy Babel!