Beyond feeling too enervated to do anything else - shellshock after taking two little grandsons to see Yogi Bear - I'm not quite sure why I'm posting this as the subject bores me, but I think there's a lot of interest among others.

The Hanguk Kiwon has for a little while now published quarterly world rankings. The latest batch takes us to end 2010. Yi Se-tol took top place from Kong Jie in the middle of last year and has held on to it. The big surprises were Pak Cheong-hwan zooming up to third place from 8th, and Yi Ch'ang-ho continuing his remorseless slide down to 13th. The Koreans dominate the top ten, with six out of ten places, but also hogging the higher placings. The whole top 20 is not dissimilar, and after that it becomes mostly China, China, China. The top Japan-based player is Cho U at 34, and the top Japanese is Iyama Yuta at 42.

The methodology has been published, but I haven't read it all, and probably wouldn't follow it if I had. But the gist of it seems to be as follows.

There are three stages. Stage 1 is to compute an initial ranking based on domestic results. This uses a method called "maximum probabilities". I have no idea what that is, but the Koreans appear to have been influenced by Ales Cieply's EGF ratings system. Ales described his method at an ICOB in Korea, so that's no surprise. However, the Koreans found Ales's results too dominated by Japanese results. No surprise there, either - Ales used the GoGoD database as his core, and at the time that was heavily biased towards the Nihon Ki-in. It's rather different now, but Ales has been too busy to resume his ProGor ratings.

Even leaving aside biases in the collection of raw data, there is a problem with overemphasis of domestic results. Stage 2 is therefore a corrective based on international results. A coefficient is produced which is attenuated over time. If I understand it correctly, the latest domestic results, of which there are many, are used, but in the case of international games, of which there are few, data are drawn from several previous years. The coefficient is applied in full for recent results, but is reduced to 0.5 of its value for results a year ago, to 0.25 for two years ago, and so on. This coefficient seems to have a rapidly noticeable effect. Good Chinese results internationally gave them good placings in the first half of 2010, but a good Asian Games by the Koreans turned these round in a major way, and explain Pak Cheong-hwan's rapid rise. To my very inexpert eye, this oscillation seems highly suspect.

In Stage 3 some correction is made for the cases where players regularly play opponents who live close by. This apparently results in what is called regional scatter. Since reference is made to a problem in Europe where players of one country tend to play each other much more often than they play foreigners, I assume this may be another concept borrowed from Ales Cieply, but I'm totally unclear how it is applied in the Far East. Maybe there's some coefficient for games between, say, Shanghai players, or games between Kansai Ki-in players.

The top ten on that basis is as follows. I give also the win-loss figures, as these show that Koreans play significantly more games than the Chinese, who in turn play more than the Japanese (Cho U's record was 31-12 and Iyama's was 33-19). All Korean unless marked by C = Chinese.

1. Yi Se-tol 76-18
2. Kong Jie (C) 46-22
3. Pak Cheong-hwan 65-23
4. Ch'oe Ch'eol-han 78-27
5. Kang Tong-yun 68-33
6. Heo Yeong-ho 65-21
7. Gu Li (C) 53-38
8. Xie He (C) 50-21
9= Weon Seong-chin 64-26
9= Li Zhe (C) 40-18

Although I have no rational basis, I do wonder whether the large number of Korean games is a skewing factor. Also, my gut instinct tells me that a table that purports to show the best 60 players in the world but has no room for Cho Chikun, Rin Kaiho or Cho Hun-hyeon yet finds room for a minor low-dan like Sun Tengyu in 41st place with a very modest 19-20 record (that is, one spot above Iyama Yuta who had a 33-19 record) is having a little trouble with its coefficients. It also seems a little precious to try to iron out regional clusters while ignoring the vast differences between tournament formats (of course both factors work in Korea's favour, which may or may not be significant).

Thanks John, very interesting.

My gut feeling is like yours, one of relative skepticism at some of the players and ordering. Where are Mok Jinseok and Zhou Ruiyang out of interest?

You can't fit an elephant into a room.

John, there are some other world rankings that can help you survive grandparenthood. The go community cannot afford the enervation of its leading poster.

see rottentomatoes.com, or specifically http://www.rottentomatoes.com/m/yogi-bear/

For kids movies, anything 80 or over is bearable, 95 or over enjoyable. Anything less than 20 is torture that even the most doting grandfather should not endure.

The ranking system is also briefly discussed on Go Game Guru.
http://gogameguru.com/top-20-pro-go-2010/

_________________
Google Site
Bay Area High School Go

A statistical analysis is always subject to what is actually being measured. It is very difficult to "analyze an analysis" without knowing more precisely how the measurement is calculated.

I'd be interested in seeing the referenced published methodology. Otherwise, any interpretations I'd make of the data are purely speculation.

_________________
be immersed

As with all rating systems, it is very hard to measure whether the system is objective. To judge whether a system is objective, we would have to confirm that it puts stronger players higher on the list. How do we know who the stronger players are? Well, we would need an objective rating system for that. A sort of catch-22 of rating systems.

Of course there are methods to get around this catch-22. One good way to judge a rating system is to judge its ability to predict future results. This, of course, is very common to science in general. Scientist confirms their ideas by making predictions based on them, then measuring whether those predictions are accurate. As more and more experiments confirm the predictions, the theory will gain more and more acceptance as being correct. The best theories are those that have stood the test of time, through countless experiments. The same is true for rating systems. Only in the long term can we say whether a rating system is truly accurate.

In the short term, most people judge a rating system based on their preconceived notions of what it should say. The criticism leveled at Ales Cieply's ratings of professionals at the ICOB is an example. Surely the world rankings should not be dominated by Japanese players? In that case, of course, we have a case of biased input, because the data used (GoGoD) was at that time biased towards more data from Japan.

In this case, the developer of the rating system seems to have tweaked the parameters until the results aligned with his preconceived notions of what they should be. That is a dangerous thing, and I am therefore skeptical as to its accuracy. But only time will tell.

There's an interesting article by Malcolm Gladwell in the February 14 & 21 issue of The New Yorker magazine that has some bearing on the topic of this thread. Gladwell's article discusses the ranking systems used in several published media for ranking such things as cars, colleges and universities, countries by number of suicides per hundred thousand people, etc. Gladwell reminds us of the well-known fact that reducing a multi-dimensional phenomenon to a single number is fraught with biases and inaccuracies almost by nature of the enterprise. His concluding sentence is particularly relevant: "Who comes out on top, in any ranking system, is really about who is doing the ranking." I don't think it is going too far to say that there is no such thing as a truly objective ranking system.

I disagree. It is probably logistically impossible to create one for any large scale phenomenon that is multi-dimensional, but for simple phenomena or on small scales it is not impossible or even always difficult to create a truly objective rating system.

I mostly agree with your post, but I do not know how to come to a conclusion on the last bit about tweaking parameters, unless I know exactly how the analysis was performed (eg. How do I know if he tweaked the parameters to his liking or not, without adding my own bias?).

So I guess I will ask directly what I tried to hint at before: Do we have a link or name of the publication of the referenced methodology?




I don't think it's that far of a stretch to claim this, either. I would say that you can put together an analysis, but in order to interpret it well, it's necessary to look at what data is actually being presented.

_________________
be immersed

I think you're eliding the important distinction. For multidimensional phenomena, it is conceptually (not logistically) impossible to create a ranking. For simple phenomena (i.e., ones that we want to rank along a single transitive dimension) the ranking is conceptually trivial, but may be logistically difficult when we can't directly observe the dimension in question (and need infer values on the hidden dimension from the multiple dimensions we can observe).

For example, consider the question, "Who is the world's best Go player?" This seems like a simple question, but only because we are able to impose a semantic equivalence that doesn't exist in the brute data. It's obvious to you and me that "playing Go" includes playing with 10 minutes main time, and playing with 8 hours; playing in Tokyo, and playing in Seoul; playing for fun, and playing for a half-million dollars; and so on. If you've never sliced up the concept of "playing Go" like this before, your intuition is probably that they must be linked, since they're all examples of "playing Go", but in fact these are just relatively autonomous dimensions of a single concept, and there's no actual reason to believe in advance that a ranking of players along one dimension will fit especially closely along another dimension.

I had a bit of trouble following what you are saying, so let me know if my response indicates any kind of misunderstanding.

Yes, but this is because you're discussing a large scale and hence inherently general question. Consider a multi-dimensional but still local case of a single match between two players in which these conditions remain (roughly) the same. In such cases caeteris paribus is much more applicable and hence an objective ranking is possible. In this case the 'dimensions' might consist of "Playing on Tuesday" and "Playing on Wednesday". Now, supposing one of the players has a cold on Wednesday it will be apparent that these two dimensions do not map.

This was my point. Single-dimensional phenomena such as, say, human weight can be objectively measured on any scale in which one has the logistical capability of measuring the entire pool. Multi-dimensional phenomena require much more accounting for, and so can generally only be objectively ranked in local situations when potential factors can are apparent and accountable for.

I'm not claiming that one-hundred percent accuracy is possible, but accuracy and objectivity are not the same thing.

No, it's objectivity that's the problem, because there is no neutral way to decide how much to weight each dimension. Until you've deciding on a weighting of the dimensions, there's no unitary underlying phenomenon to measure, accurately or inaccurately; and there's nothing that makes one weighting of the dimensions more "objective" than another.

Then please explain how it is that people discover new dimensions to a phenomenon if the underlying phenomenon doesn't pre-exist the determination of its dimensions. It seems to me you're suggesting that someone conducting studies on "falling bodies" who discovers wind resistance is in actuality concocting a wholly new phenomenon rather than discovering a new dimension to the same phenomenon. What makes them think that it is relevant to falling bodies? Isn't it the fact that it clearly affects them? Can't this effect on falling bodies then be quantified and weighted according to that quantification?

Fortunately, you're asking about a literally one-dimensional problem. When we're investigating the rate at which bodies fall, we are only interested in their distance from the ground. That's one dimension. Lots of factors might go into figuring out that distance at a given t: the force of gravity, air resistance, air pressure, jet propulsion, etc. But it's one dimension, and once we've figured out the coordinate where a given set of falling bodies will be at t (and as you say, it may be difficult to do this accurately), it's trivial to rank the bodies according to their position.

But what if we were interested in distances from the ground, and the rates of change of those distances, and the rate of change of that rate of change? Now we have three dimensions of information about the falling bodies, and it's no longer possible to rank them -- unless they happen to fall in a line. This isn't troubling to us, because there's no one concept that x, dx/dt, and d^2/dt^2 all seem to be examples of. If we could get an ordinal ranking of a bunch of falling bodies along those three dimensions (in the case where they lined up perfectly), it's not clear what on earth that ranking would mean. But all of the games that pros play are clearly examples of "playing go", so it seems intuitive that all the different dimensions of the concept should line up nicely.

@jts In theory, its possible that there is an underlying factor that has high explanatory value in predicting success across the various playing conditions.  At some level this has to be true for Go.  While some professionals perform comparatively better at blitz or slow games, there's no such thing as a player who can win blitz tournaments, but wouldn't meet professional standards at slow play.

_________________
Occupy Babel!

Absolutely. The relationships between the dimensions are not wholly random. But unless the correlations are extremely tight, they're useless for establishing objective rankings - you still need to decide how much to weight each dimension. Even if all pros are about evenly matched in both blitz games and slow games, ranking systems that weighted blitz games twice as heavily, that weighted slow games twice as heavily, and that weighted them both equally could each give us completely different top ten lists. Since there are multiple different ways to weight the dimensions, there are multiple different possible rank orderings. This is true whether we're talking about best Go player, or best university, or best country, or whatever the flavor of the week is among people who love rankings.

Then we are using the terms differently. In any case, I think Go is closer to falling bodies, which is why it seems more intuitively obvious as a phenomenon than the example you give later involving ranking objects according to a variable, the rate of change of that variable and the rate of rate of change together. The ranking is determined by a single measure: game results. Inasmuch as there are not any significant differences in scoring, this ranking can operate. Other conditions may vary, such as game length, location etc... just like air pressure may vary. Ideally these should be accounted for or, even more ideally, made equal.

Right, I think you're close, but you're being misled into thinking that "game results" is a homogeneous category by the fact that we apply "game" to a lot of different things. Question: which falls faster, a feather, or a tiny rocket (much smaller profile than the feather) which is being propelled upwards with a force that is equal to a very small fraction of the (downward) force of gravity?

Answer: