KGS Ranking adjustment?

[jlt]

I don't know about the US, but I don't think that in France there is a big problem about players stuck at some given rank below their real level. According to

http://ffg.jeudego.org/echelle/echelle_algo.php

(1) A 6k winning a game against a 6k gains about 24 points.

(2) If during a tournament a player gains more that 60 points, then his rating is automatically adjusted prior to the tournament.

So it's not uncommon for a player around that level to gain 2-3 kyus after winning 5/5 games in a tournament.

[gennan]

Let me put it in another way.

Let a 6k play a 10 game match on even against a 7k. The AGA system expects the 6k to win 8-2 (80%). Anything less and the 6k will lose rating points.

But from historical data, we should expect the match to end in 6-4 (60%) and nobody should gain or lose rating points in that case.

If we look for a player that the 6k can beat 8-2 (80% winrate) in a 10 game match on even, the historical data suggests we should match him against an 11k (5 ranks weaker corresponds to 80% winrate around these ranks).
The match will probably end 8-2 and nobody should gain or lose rating points when that is the case.

But the AGA system would expect a near 100% score, so if the 6k loses 1 or 2 games out of 10 against the 11k (as might be expected historically), he would lose rating points.

[gennan]

I don't know about the US, but I don't think that in France there is a big problem about players stuck at some given rank below their real level. According to

http://ffg.jeudego.org/echelle/echelle_algo.php

(1) A 6k winning a game against a 6k gains about 24 points.

(2) If during a tournament a player gains more that 60 points, then his rating is automatically adjusted prior to the tournament.

So it's not uncommon for a player around that level to gain 2-3 kyus after winning 5/5 games in a tournament.

Right, so the FFG has some extra mechanisms to update the ratings besides game results. The EGF also has such mechanisms. And I suppose the AGA has them too. (Even chess rating systems with weaker players have such mechanisms).

These corrective measures reduce the problem of unrealistic winrates by the system.

[gennan]

I don't know about the US, but I don't think that in France there is a big problem about players stuck at some given rank below their real level.

The FFG system is closer to the EGF system than the AGA system, so I would expect it to be less of an issue. But still there is this French tournament organiser that seemed to experience issues along the lines of deflation of strong kyu ranks: viewtopic.php?p=233885#p233885

[jlt]

That tournament organizer had some impressions, but they are probably not backed up by statistics, so it's hard to say if his impressions are correct or not. In this French tournament

http://ffg.jeudego.org/resultats/affich ... ?idt=14291

games were played at handicap minus one. The winning ratio W:B was:

Handicap 1: 3:1
Handicap 2: 6:2
Handicap 3: 2:2
Handicap 4: 2:5
Handicap 5: 3:0
Handicap 6: 1:1
Handicap 7: 1:0
Handicap 8: 3:3
(I am not taking into account H9 games which were sometimes played against players who were much more than 9 ranks apart).

Overall, White won a bit more than Black, as expected (since handicap minus one was used).

[dfan]

I have similar opinions about the AGA system, largely based on running the WHR rating algorithm on American Yunguseng Dojang games (there are over 5000 serious even games by now) and seeing how the resulting ratings compare with AGA ones.

There are a few problems that I believe I observe about the rating system. One is that the emphasis on handicap games means that it's hard for the system to be incredibly surprised at any particular result. In a chess tournament, you might be (the equivalent of) 10k and beat a couple of 5ks, and rocket up. If those are 5-stone handicap games, the fact that you won a couple of coin-flips isn't so impressive. Add to this a large amount of inertia in the rating system (and apparently the fact that it imposes unreasonable demands on winrate) and you get a relatively stagnant rating pool.

There is thankfully a pressure relief valve in the form of a rule that if you enter a tournament at least three ranks up from your established rating, and win at least one game, your rating resets entirely. The bad news is that this seems to be by far the most efficient way to get even a modest increase in skill recognized. If you are rated 4k and think you are 3k, I believe that the best way to achieve it is not by regular play but by entering tournaments at 1k until you win one game. That doesn't feel like a healthy system.

[gowan]

Some of the discussion above agrees with the idea that rating has no inherent meaning. Too many people fixate on raising their rating and we see concern with how to increase rating fast. If I am a 4k player and want to raise my rating to 3k and I do it by repeatedly entering tournaments at self-rated 1k until I win a game against a 1k, am I going to quit playing after my rating increase? Because if my "real" rating is 4k I'll probably lose the artificial 3k rating quickly after resuming tournament play. There is no guarantee that rating will increase over time. There are some time tested ways to improve your play. For example, you can take lessons and work on using what you are learning in tournament games. Or you can just play and analyze your games to see what mistakes you are making. The thing is you have to think about the moves you are making, so, probably, slower games are important, where you can think about your moves; that's how new ideas can be integrated into your play.

Actual ratings are not very meaningful, their purpose is to make it possible to play opponents at the same level, but they do not determine any particular playing strength. The name "3k" describes players of different strengths in the AGA and in the EGF. Even 1p strength is different in USA, Japan, and Korea.

[jlt]

The rating is not absolute, but can be useful to measure progress. The whole discussion shows that 1-stone improvements are hard to detect on KGS because KGS ratings are not always based on reliable anchors, and the idea of readjusting KGS ratings according to known AGA ratings is not good either because some AGA players are stuck a few stones below their "real" level (e.g. they are AGA n-kyu but of the same level as the median AGA (n-3)-kyu).

[gennan]

If you see rating only as a means to measure short term progress, then a simple Elo system would be sufficient. But problems arise for long-term players if you assign hard go ranks to ratings and the rating system slowly deflates or inflates.

My country had about 10 6d EGF players 10 years ago. Now they were all forced to demote themselves to 5d, because the rating system says so. Did they all become 1 stone weaker? Perhaps they are a bit weaker now (there seems to be a trend that players get a bit weaker as they age). But some of those players aren't that old. My feeling is that at least part of the reason is deflation. You could ask who cares if 6d in 2007 are forced to be 5d in 2017. But I think it matters. If the EGF rating system deflates over time, long-term players who stopped improving will start to avoid playing in tournaments, because they know it will hurt their rating (=rank!) and thus their percieved status in the go community. I fear deflation has a negative effect on the go community.

[gennan]

That tournament organizer had some impressions, but they are probably not backed up by statistics, so it's hard to say if his impressions are correct or not. In this French tournament

http://ffg.jeudego.org/resultats/affich ... ?idt=14291

games were played at handicap minus one. The winning ratio W:B was:

Handicap 1: 3:1
Handicap 2: 6:2
Handicap 3: 2:2
Handicap 4: 2:5
Handicap 5: 3:0
Handicap 6: 1:1
Handicap 7: 1:0
Handicap 8: 3:3
(I am not taking into account H9 games which were sometimes played against players who were much more than 9 ranks apart).

Overall, White won a bit more than Black, as expected (since handicap minus one was used).

The number of games in your example is much too low for statistical analysis.

You really need large samples of statistical data to distinguish between 65% and 75% winrate with any reliability,.
The EGD data contains about 100.000 handicap games (tournament games in Europe tend to be even games in McMahon tournament system). And if handicaps are used in tournaments, it's usually a reduced handicap, which favours white, obscuring any issues of handicaps being too large. Even "normal" handicaps favor white, as black's 1st handicap stone is only half a move advantage. But it's possible to correct the expected statistics for skewed handicaps.

I did collect those statistics in 2017 and as far as I can tell, over the whole of Europe in a period of 20 years, handicaps seem to be pretty consistent with rank differences. But when I zoom in to smaller samples (like only one country for a period of 5 years), the sample becomes too small for reliable statistics. The "signal" is pretty small and it becomes blurred by the "noise".
For example, the observed winrate between 6k and 7k in even games is about 57%. When the 6k player gives the 7k player 1 handicap, the winrate may be about 53% and with 2 handicap it may be about 45%. You need pretty large samples to detect such small winrate differences reliably.

[Harleqin]

Just because your system spews out numbers to four significant digits doesn't mean that these numbers have the meaning you ascribe to them.

Specifically, it doesn't even mean that these numbers are internally consistent, not even in the long run (which we don't have, due to lack of data volume). It also doesn't consistently mean that if the number goes up/down, the strength goes up/down.

[gennan]

I have similar opinions about the AGA system, largely based on running the WHR rating algorithm on American Yunguseng Dojang games (there are over 5000 serious even games by now) and seeing how the resulting ratings compare with AGA ones.

There are a few problems that I believe I observe about the rating system. One is that the emphasis on handicap games means that it's hard for the system to be incredibly surprised at any particular result. In a chess tournament, you might be (the equivalent of) 10k and beat a couple of 5ks, and rocket up. If those are 5-stone handicap games, the fact that you won a couple of coin-flips isn't so impressive. Add to this a large amount of inertia in the rating system (and apparently the fact that it imposes unreasonable demands on winrate) and you get a relatively stagnant rating pool.

There is thankfully a pressure relief valve in the form of a rule that if you enter a tournament at least three ranks up from your established rating, and win at least one game, your rating resets entirely. The bad news is that this seems to be by far the most efficient way to get even a modest increase in skill recognized. If you are rated 4k and think you are 3k, I believe that the best way to achieve it is not by regular play but by entering tournaments at 1k until you win one game. That doesn't feel like a healthy system.

I didn't know the AGA rating system works like that, but it doesn't suprise me that such measures are needed to overcome what I would call flaws of the system. The EGF system has different measures, but it's basically a similar fix for a similar problem.
But these arbitrary measures are really needed to keep a reasonable alignment between the rating system and reality. Abolishing measures like these would break the rating systems.

I suppose some go players are aware of this, but improving the rating systems is hard. And by that I don't mean the technical part. Go players are humans and humans tend to prefer a status quo.

[gennan]

It goes against the grain, I suppose, but I have long believed that ranking in terms of handicap stones, despite the fact that skill in giving or taking handicap stones is variable, it better than ELO ratings or score differences in differentiating skill levels. The main reason for my belief is that, over a large range of skill levels, handicap stones have a roughly linear relationship. E. g., if player A is three ranks stronger than player B, that means that a handicap of 3 stones with White giving komi normally gives each player around a 50-50 chance of winning the game. If player B is four ranks stronger than player C, then if A gives C a handicap of 7 stones, White giving komi, that will also normally give each player around a 50-50 chance of winning. Elo ratings, OTOH, will tell us that A will crush C in an even game.

I think go ranks have no meaning without handicap. Handicap defines ranks. If go had no handicap system and all games would be even games, we would just use an Elo rating system.

But the go rating systems all try to align winrates to handicaps. Tournament games in Europe tend to be even games, so you don't really need ranks for that, but it's nicer to have opponents of your own level in even games. Go players tend to have ranks in their clubs bases on handicap, so you want to use those club ranks in tournaments to put players in evenly skilled McMahon groups. It's also reasonable to update ratings based on tournament game results.

This puts a responsibility on go rating systems to convert well between ratings and ranks and to keep this conversion consistent in the long run. I do think the go rating systems are reasonably effective in this respect, but there is clearly room for improvement.

[Bill Spight]

I think go ranks have no meaning without handicap. Handicap defines ranks. If go had no handicap system and all games would be even games, we would just use an Elo rating system.

But the go rating systems all try to align winrates to handicaps.
{snip}

This puts a responsibility on go rating systems to convert well between ratings and ranks and to keep this conversion consistent in the long run. I do think the go rating systems are reasonably effective in this respect, but there is clearly room for improvement.

Back in the 1970s I devised a rating system for the New Mexico Go Association. The AGA had already devised an Elo system, but I took the handicap basis seriously and did not do so. OC, there was no theory at the time about how to base a rating system on handicaps (Is there one now? ), so I drove by the seat of my pants. I divided ranks into two, so that a strong shodan would take White against a weak shodan, receiving ½ pt. komi, etc. Instead of complicating the calculation of ratings, I had each game count the same number of points, and increased the point range of each successive rank by 5%, going upwards. Since most of our players were in their 20s I added inflation points to the winners' scores. After three years, based upon the results of visiting players and of our players playing elsewhere, I promoted everybody by ½ rank and added more inflation points. Two years later I had no reason to change the system.

OC, such a system faces the problem of players who make very rapid progress at the SDK level and above. Other systems do, as well, but the very short ranges of ranks in the DDK range do not present much of an obstacle. Human intervention may be required. I might have done so for Janice Kim, but after her summer as a go student in Korea, she was cautioned not to play with amateurs. So that problem solved itself.

[jann]

This puts a responsibility on go rating systems to convert well between ratings and ranks and to keep this conversion consistent in the long run.

so that a strong shodan would take White against a weak shodan, receiving ½ pt. komi

There is another problem with "handicap" in this case (faux H1). Suppose a player plays 100 games like this and loses all by 3-4 pts (no komi). This would correspond to winning all by 3-4 pts (with komi). I doubt there is a completely correct handling of such results.

OC a typical system calls this 0-100 and drops his ratings. But in another universe he decides to reject "handi" games and play even (same opponents, same move sequences). There these (the same performance) are called 100-0 and huge rating gain.