KGS ranking system is nonsense

[ez4u]

Don't the EGF and AGA rating systems basically work similarly to KGS? Looking at the EGF Rating page at SL, it seems even more severe in the expected winning percentage against people rated half a stone (50 points) weaker. Where are all the complaints about those systems?

[uPWarrior]

Given that nobody has addressed what to me is a key issue here, I'll leave this note: ranks, as people think of them in go, establish a strength difference between people of different ranks through handicap games. 2 stone difference* means 50% win probability when the opponent has 2 stones on the board, 3 stones mean 50% when they have 3 stones on the board and so on. We never talk about win probability against people of the previous rank in even games. It quickly follows that no ranking system is good if players only play against people of the same rank, as it quickly stops being connected to stone differences.

I don't have a practical remark from this note other than that if even games were forbidden the ranking would be a lot more accurate.

* ignore all the half stone mambo jambo due to no komi not meaning symmetry.

[Uberdude]

Don't the EGF and AGA rating systems basically work similarly to KGS? ... Where are all the complaints about those systems?

The British Go Journal and gotalk mailing list.

But the EGF rating system is quite different to KGS in that it is essentially ELO so your rank is purely a function of yor last rank and the results of your last set of games from a tournament, rather than constantly recalculating based on past opponents' current ranks as in KGS. EGF ranks have zero inertia; if you go on a win streak your rank goes up quite fast.

[lemmata]

The conflict between these two statements is my personal source of struggle with the KGS system. I looked myself up on kgs.gosquares.net and I have over a 65% win rate over the last two months (20+ games played per month), but my rank hasn't increased. This is troubling for me - I consistently feel that I'm playing at a level above my rank indicates, and the statistics reflect this, but the KGS ranking system does not... In fact, it took multiple months of win rates greater than 70% for me to achieve my last rank promotion (which was then mercilessly taken away from me during the recent anchor shift in late December).

KGS rankings are continuous, are they not? With a 65% winning percentage, you should been 7.7 dan or something like that according to the rank graph, which is better than 7 dan (if you were 7 dan at the beginning of those two months). You may have felt underrated because KGS rounds down all displayed ranks, but you were probably properly rated according to your raw rank.

What does it mean to be playing at a level above your KGS rank when 8 dan level means winning more than 70% of your games at 7 dan?

That said, I do prefer something like the Tygem system, as it is simple and allows a player to experience discrete variations in rank more frequently (adding variety/spice to life). I would agree with the general sentiment in this thread that the KGS ranking system is not as fun as it could/should be, but it is by no means bs. It's just different.

[ez4u]

The conflict between these two statements is my personal source of struggle with the KGS system. I looked myself up on kgs.gosquares.net and I have over a 65% win rate over the last two months (20+ games played per month), but my rank hasn't increased. This is troubling for me - I consistently feel that I'm playing at a level above my rank indicates, and the statistics reflect this, but the KGS ranking system does not... In fact, it took multiple months of win rates greater than 70% for me to achieve my last rank promotion (which was then mercilessly taken away from me during the recent anchor shift in late December).

KGS rankings are continuous, are they not? With a 65% winning percentage, you should been 7.7 dan or something like that according to the rank graph, which is better than 7 dan (if you were 7 dan at the beginning of those two months). You may have felt underrated because KGS rounds down all displayed ranks, but you were probably properly rated according to your raw rank.

What does it mean to be playing at a level above your KGS rank when 8 dan level means winning more than 70% of your games at 7 dan?

That said, I do prefer something like the Tygem system, as it is simple and allows a player to experience discrete variations in rank more frequently (adding variety/spice to life). I would agree with the general sentiment in this thread that the KGS ranking system is not as fun as it could/should be, but it is by no means bs. It's just different.

Actually KGS ranks are not continuous, only the ratings are. In the end isn't the KGS rating system working just as advertised? However, the KGS ranking system and the KGS pairing system (based on the ranking system rather than the rating system) are the culprits, aren't they? According to their rating, as someone approaches promotion (e.g. their rating is 1.99, just short of jumping to 2d) they are systematically over-handicapped (receiving too much) against the average stronger player and under-handicapped (giving too little) against the average weaker player. That is why they need to win at very high rates in order to jump to the next level (and when they fail will join in the chorus here on how unfair KGS is). Once they make it over the hump to a 2.01 rating and are ranked as 2d they will switch to being systematically under-handicapped (receiving too little) against stronger players and over-handicapped (giving too much) against weaker players. As a result, they can survive at 2d despite losing half their games but they are unlikely to feel that they deserve it. Both situations are depressing.

This use of ranks rather than ratings for pairings was a conscious choice that wms has commented on previously. However, it seems to me that it is the main cause of the perceived 'stickiness' of ranks. The use of 'neutral' komi (i.e. with 50% chance of winning) would seem to offer relief from the problem. DGS does this with its 'proper' setting. WING used to do it by default when I played on it years ago. Such an approach does benefit in terms of transparency from an extra step to inform the players of the prospective komi/handicap in a game under negotiation. That is what WING used to do. Will it help? Well, at least it might change the repeated threads here from being complaints about sticky ranks to being complaints about Mickey Mouse komi so that we could enjoy a little variety.

[EdLee]

According to their rating, as someone approaches promotion (e.g. their rating is 1.99, just short of jumping to 2d)
they are systematically over-handicapped (receiving too much) against the average stronger player
and under-handicapped (giving too little) against the average weaker player.

Hi Dave, does this also happen on IGS ? Is this part of the reason
that they introduced the finer divisions within each rank (e.g. 1k and 1k+), to try to reduce both the over-handicaps and under-handicaps ?

[hyperpape]

I have a pet belief that we should play with 1-2 stone reduced handicaps on go servers. I do not know whether it would actually help people move up more quickly in ranks (my intuition is that it ought to, but I have never sat down to figure out the exact ratings math), however it would certainly give you a range of opponents to play against, addressing shapenaji's concern.

[HermanHiddema]

I've seen people stuck with 60% win rate on KGS analytics never moving at all. And if Mef is correct, that's by design.

Isn't the same true for many rating systems?

In the EGF rating system, if you have a rating of 1649 (strong 5k) and you play 60% against people with an average rating of 1600 (average 5k), your rating will not shift one bit. At 49 rating difference (almost half a stone), you are expected to win that much. If you were 2449 (strong 4d) and got 60% against 2400 (average 4d), your rating would actually drop, because the EGF system expect stronger players to play more consistently and expects a 65% win rate at that level.

At tygem you need 67%, 70% or 75% depending on your rank, if you consistently remain between 55% and 65%, you will never promote either.

Of course you can quibble about what percentage represents a rank difference, but there will always be some percentage higher than 50% at which you will not be promoted. And most servers do seem to agree that 60% is too low to represent a full rank difference.

[HermanHiddema]

I have a pet belief that we should play with 1-2 stone reduced handicaps on go servers. I do not know whether it would actually help people move up more quickly in ranks (my intuition is that it ought to, but I have never sat down to figure out the exact ratings math), however it would certainly give you a range of opponents to play against, addressing shapenaji's concern.

It IGS, they underhandicap you and compensate with komi. They work with half ranks and at 1 rank difference you play even but black receives a reverse komi of 5.5, they only start placing handicap stones from 1.5 ranks difference (2 stones, 0.5 komi at that point).

[Boidhre]

I've seen people stuck with 60% win rate on KGS analytics never moving at all. And if Mef is correct, that's by design.

Isn't the same true for many rating systems?

In the EGF rating system, if you have a rating of 1649 (strong 5k) and you play 60% against people with an average rating of 1600 (average 5k), your rating will not shift one bit. At 49 rating difference (almost half a stone), you are expected to win that much. If you were 2449 (strong 4d) and got 60% against 2400 (average 4d), your rating would actually drop, because the EGF system expect stronger players to play more consistently and expects a 65% win rate at that level.

At tygem you need 67%, 70% or 75% depending on your rank, if you consistently remain between 55% and 65%, you will never promote either.

Of course you can quibble about what percentage represents a rank difference, but there will always be some percentage higher than 50% at which you will not be promoted. And most servers do seem to agree that 60% is too low to represent a full rank difference.

Isn't the issue in the EGF system at least that if you don't know the list of past opponents that someone's win rate is fairly meaningless? There's an expected win rate for a certain GoR difference but that's calculated on a game by game basis and has no relationship to someone's EGF rank really.

[HermanHiddema]

Isn't the issue in the EGF system at least that if you don't know the list of past opponents that someone's win rate is fairly meaningless? There's an expected win rate for a certain GoR difference but that's calculated on a game by game basis and has no relationship to someone's EGF rank really.

I'm not sure what you mean by this?

Win rates depend on how strong or weak your opponents are. If you have a win rate of 20% but played only players more than 3 stones stronger, then that is a good result. If you have a win rate of 80%, but played only players more than 3 stones weaker, then that is a bad result.

Of course, with most tournaments being McMahon, most people probably play a lot of opponents around their own level.

At the EGD you can find win rates for players, but only on their specific player pages, not in rating lists, because it wouldn't be meaningful there.

EGF ratings are calculated on a per event basis. If I visit an event and meet 5 opponents and win 3 games, it does not actually matter which of the games I win. As long as I play the same 5 opponents, any combination of 3 wins and 2 losses results in the same rating change.

[mitsun]

I've seen people stuck with 60% win rate on KGS analytics never moving at all. And if Mef is correct, that's by design.

Isn't the same true for many rating systems? ....

IGS is the outlier in this respect. In the IGS rating system, you get points for every win, and you get promoted when your points reach a threshold level, so any win rate over 50% will eventually get you promoted. Of course right after such a promotion, you may have a much lower win rate and get demoted fairly quickly. This rating system is not as good as ELO type systems for evaluating player strength or predicting win/loss probabilities, but it has other properties some people like.

[Boidhre]

Isn't the issue in the EGF system at least that if you don't know the list of past opponents that someone's win rate is fairly meaningless? There's an expected win rate for a certain GoR difference but that's calculated on a game by game basis and has no relationship to someone's EGF rank really.

I'm not sure what you mean by this?

Win rates depend on how strong or weak your opponents are. If you have a win rate of 20% but played only players more than 3 stones stronger, then that is a good result. If you have a win rate of 80%, but played only players more than 3 stones weaker, then that is a bad result.

Of course, with most tournaments being McMahon, most people probably play a lot of opponents around their own level.

At the EGD you can find win rates for players, but only on their specific player pages, not in rating lists, because it wouldn't be meaningful there.

EGF ratings are calculated on a per event basis. If I visit an event and meet 5 opponents and win 3 games, it does not actually matter which of the games I win. As long as I play the same 5 opponents, any combination of 3 wins and 2 losses results in the same rating change.

I mean raw win rates aren't very interesting unless you know who people are playing and thus the expected win rate for them. Tournaments, eh, in the larger go countries sure, but the smaller ones won't be giving you opponents of similar level in most of your games, there'll normally be some handicap (-1 or whatever) involved.

[HermanHiddema]

I mean raw win rates aren't very interesting unless you know who people are playing and thus the expected win rate for them.

Oh yeah, sure, but the same is true for any system. It is quite possible to maintain a 90% win rate on KGS and never be promoted, you just have to play lots of weaker players without handicap.

[lemmata]

The conflict between these two statements is my personal source of struggle with the KGS system. I looked myself up on kgs.gosquares.net and I have over a 65% win rate over the last two months (20+ games played per month), but my rank hasn't increased. This is troubling for me - I consistently feel that I'm playing at a level above my rank indicates, and the statistics reflect this, but the KGS ranking system does not... In fact, it took multiple months of win rates greater than 70% for me to achieve my last rank promotion (which was then mercilessly taken away from me during the recent anchor shift in late December).

KGS rankings are continuous, are they not? With a 65% winning percentage, you should been 7.7 dan or something like that according to the rank graph, which is better than 7 dan (if you were 7 dan at the beginning of those two months). You may have felt underrated because KGS rounds down all displayed ranks, but you were probably properly rated according to your raw rank.

What does it mean to be playing at a level above your KGS rank when 8 dan level means winning more than 70% of your games at 7 dan?

That said, I do prefer something like the Tygem system, as it is simple and allows a player to experience discrete variations in rank more frequently (adding variety/spice to life). I would agree with the general sentiment in this thread that the KGS ranking system is not as fun as it could/should be, but it is by no means bs. It's just different.

Actually KGS ranks are not continuous, only the ratings are.

You stated what I actually meant in more precise terms.

In the end isn't the KGS rating system working just as advertised?

Agreed.

However, the KGS ranking system and the KGS pairing system (based on the ranking system rather than the rating system) are the culprits, aren't they? According to their rating, as someone approaches promotion (e.g. their rating is 1.99, just short of jumping to 2d) they are systematically over-handicapped (receiving too much) against the average stronger player and under-handicapped (giving too little) against the average weaker player. That is why they need to win at very high rates in order to jump to the next level (and when they fail will join in the chorus here on how unfair KGS is). Once they make it over the hump to a 2.01 rating and are ranked as 2d they will switch to being systematically under-handicapped (receiving too little) against stronger players and over-handicapped (giving too much) against weaker players. As a result, they can survive at 2d despite losing half their games but they are unlikely to feel that they deserve it. Both situations are depressing.

Indeed, this is a good description of effects that could make the activity less fun for the players. It does not make the system bogus though.

Here's one more argument from me in favor of more rank volatility: Discourage this weird tendency toward rank protection. If players feel like they could erase the effects of some games played under less than ideal conditions fairly quickly, then they would be willing to play more often under more circumstances. The most important feature of a go server (to me personally) is how quickly I can find a decent game under common time settings. The reason I play on Tygem despite its horrific abomination that is the Tygem client is because I know that I can find a game fairly quickly there. Tygem does have a larger user base, but the number of users connected to a specific Tygem server (China 2/Korea 1) is only around 2-3 times larger than the number connected to KGS. Perhaps more rank volatility encourages people to push the automatch button more often. As little as I care about rank, I do care a lot about the number of people who are willing to push the automatch button on a server. Alas, I doubt that KGS will ever change...

Aside: OGS ladders are perfect in this regard. You can find a standard time-setting game on that server at any time of your convenience.