KGS ranking system is nonsense

[hashimoto]

One clarification here -- it is incorrect to say that that the KGS rating system has stability as a goal. The rating system has accuracy as a goal. The purpose of the rating system is to do the best job it can of predicting the outcome of a game that has yet to be played between any two given players rated by the system.

The confusion here is because the KGS ranking model assumes that an accurate rank is a stable one. KGS model tries to converge to a stable rank, because the model interprets rank to work this way. This is debatable.

[daal]

A player should have a reasonable random walk around their rank.

I'd like to see something like this too*. One model that I like is brain workshop's n-back game. Here, after crossing a threshold to a new level a single loss (or in the case of n-back, failure to achieve a certain percentage of correct answers) will not get you demoted. Instead, you stay at the higher level until you have accumulated three losses. This sort of system allows you to get a taste of higher level play without fear of immediate demotion, and although your rank may not be quite as accurate for some of the games, you still get a chance to collect some valuable experience.

*Of course, you could just join the ASR where you play even games against both stronger and weaker players.

[shapenaji]

I don't see how this is a bad thing. Who would want a rank that auto changes if you have a few bad games in a row? If i have a bad swing and lose 4 games in a row i don't want to lose 1-2 ranks, so why should the opposite happen?

Well, the typical method in club environments is to adjust handicap after 3 victories or 3 losses (Heck, I play 1-game kadoban with some of my closest rivals, we change the handicap after every game).

This creates reasonable challenges to reward improvement, but it also gives you consequences for losing.

Believe me, if you win 20 games in a row, no matter how "stagnant" your rank is, you'll be getting closer and closer to a higher rank anyway.

The rank system does fine in making the large adjustments associated with someone who is so far off their rank that they can win 20 games in a row... it's much worse when the rank system is 1 stone off, and we're talking about a 60% win rate rather than a 50%.

People just want to think they're stronger than they really are. Your rank is just a number that people get way too tied up over. Its not like not like anything will really change if you're "stronger". There is no secret KGS menu for a KGS 7d that we know of yet.

People thinking that they're stronger than they really are is how people get stronger. If you never believed that you were capable of more, you'd never try.

Also, Things absolutely change when you get stronger, even a difference in a rank or 2 can be a major difference in the primary focus of the games. Getting stronger is addictive for just that reason, you never know what's past the next rank horizon.

[shapenaji]

A player should have a reasonable random walk around their rank.

I'd like to see something like this too*. One model that I like is brain workshop's n-back game. Here, after crossing a threshold to a new level a single loss (or in the case of n-back, failure to achieve a certain percentage of correct answers) will not get you demoted. Instead, you stay at the higher level until you have accumulated three losses. This sort of system allows you to get a taste of higher level play without fear of immediate demotion, and although your rank may not be quite as accurate for some of the games, you still get a chance to collect some valuable experience.

*Of course, you could just join the ASR where you play even games against both stronger and weaker players.

Sounds similar to the Tygem system, of which I'm definitely a fan.

[Mef]

...when the rank system is 1 stone off, and we're talking about a 60% win rate rather than a 50%.

For KGS, one stone off would be closer to 80%. I think this is another one of those things that often throws people off on their expectations of the rating system.

[FortyRock]

Keep in mind the people you're playing also see their ranks shifting. So it's not just a matter of basing progress on a comparison of two single numbers.

I like to compare it to trust. Difficult to gain, easy to lose.

[often]

For KGS, one stone off would be closer to 80%. I think this is another one of those things that often throws people off on their expectations of the rating system.

yes this

in my opinion there really isn't that much of a difference between a 1 stone rank anyway.

Well, the typical method in club environments is to adjust handicap after 3 victories or 3 losses (Heck, I play 1-game kadoban with some of my closest rivals, we change the handicap after every game).

This creates reasonable challenges to reward improvement, but it also gives you consequences for losing.

People thinking that they're stronger than they really are is how people get stronger. If you never believed that you were capable of more, you'd never try.

Also, Things absolutely change when you get stronger, even a difference in a rank or 2 can be a major difference in the primary focus of the games. Getting stronger is addictive for just that reason, you never know what's past the next rank horizon.

what you're arguing here is preference, which is fine
like i said earlier, thats why there are plenty of other go servers with different methodologies. if they are what you like, so be it. i'm ok with how KGS does things.

that's why i don't think the tagline of this thread is valid (calling it bs)

i personally, can't say that if i beat 3 (1d) ranks in KGS that i would say "well make me 2d".

[shapenaji]

...when the rank system is 1 stone off, and we're talking about a 60% win rate rather than a 50%.

For KGS, one stone off would be closer to 80%. I think this is another one of those things that often throws people off on their expectations of the rating system.

That doesn't make any sense, how do you have more range between ranks than other systems and also have those ranks similarly populated from 9d down to 30k?

[shapenaji]

yes this

in my opinion there really isn't that much of a difference between a 1 stone rank anyway.

There is a considerable difference in a 1 stone difference, in one case I have komi, in the other I don't.
Also, you entirely missed his point, this would suggest a large difference between KGS ranks, not a small one.

what you're arguing here is preference, which is fine
like i said earlier, thats why there are plenty of other go servers with different methodologies. if they are what you like, so be it. i'm ok with how KGS does things.

I'm not just arguing for my preference (although, you should really try it, it can do wonders for your advancement), your rank is not as stable as you think it is, and it's inaccurate to fix it so tightly.

But also, you're not the only user of KGS, my preferences are valid concerns.

[mitsun]

For KGS, one stone off would be closer to 80%. I think this is another one of those things that often throws people off on their expectations of the rating system.

Yes, this is a big difference between KGS and other rating systems. If you have a sustained win rate of 60% in even games (at the dan level), the KGS rating system will never promote you. In some other rating systems (IGS in particular), a sustained win rate over 50% will eventually get you promoted.

[often]

Please make the proposal and argument for a different ranking system then.

From your posts this is what i've read into it:

In my experience, this is not really true. People respond to positive feedback. If a player starts winning games, but the challenge doesn't increase, it's easy for a player to lapse into bad habits.

Likewise, a player who experiences variation in style and challenge of games has more to learn from and improves faster.

This seems to be an issue for a player by player basis. Couldn't the argument be made that if a player sees his rank fluctuate wildly he gets frustrated and never knows how strong he is? Also, this is supposing that playing on KGS you wouldn't interact with a variety of styles at your given rank.

This creates reasonable challenges to reward improvement, but it also gives you consequences for losing.

This is (possibly incorrectly interpreted) like saying that winning isn't reward enough.

I still don't see why being a 6k is a bigger reward than being a 7k. You're saying that there will be bigger challenges and different play styles, which i would argue against. A person who might be 6k will still receive certain challenges playing people at a 7k rank if they were "incorrectly" given a 7k rank.

Also, as to your point of "more challenges" there would be no merit to most players playing people 1 level below their rank if we took your thought process. If you're a 5d and can win 80% of the time against a 4d, the problem might be that you're higher than a 5d instead.

Perhaps I missed Mef's point, but let's talk about this quote also:

There is a considerable difference in a 1 stone difference, in one case I have komi, in the other I don't.

This is depending on the level. I feel that if you pit two 5 kyus against each other the komi would rarely come into a deciding factor. I think for the most part komi is something only the stronger go players really have to worry and factor into, for the rest of us it is something to take into account at the counting process at the end of the game. If you can show me a kyu game where they actively figured the komi into their play style i'd be impressed.


All in all, yes it comes down to preference and yes the dissenting opinions should have their say. I feel that most people are arguing over something really inconsequential and that most of these people who are arguing over something like this don't even play enough for it to matter anyway.
If you're saying its an issue of challenge I think you're missing the point as there will always be a challenge in Go. The person who argues the ranking system and says "i'm so bored of winning all the time and my rank never goes up" probably forgot all the other times he loses to people of that rank.

In fact, i urge most people to check out the KGS analytics site: http://kgs.gosquares.net/ Most of the time you're winning and losing 50% of your games anyway. If that figure were to ever be "incorrect" then of course the ranking system wouldn't be doing its job. But since it is typically 50% for almost everybody then it -is- doing its job.

[Longstride]

Yes, this is a big difference between KGS and other rating systems. If you have a sustained win rate of 60% in even games (at the dan level), the KGS rating system will never promote you. In some other rating systems (IGS in particular), a sustained win rate over 50% will eventually get you promoted.

In fact, i urge most people to check out the KGS analytics site: http://kgs.gosquares.net/ Most of the time you're winning and losing 50% of your games anyway. If that figure were to ever be "incorrect" then of course the ranking system wouldn't be doing its job. But since it is typically 50% for almost everybody then it -is- doing its job.

The conflict between these two statements is my personal source of struggle with the KGS system. I looked myself up on kgs.gosquares.net and I have over a 65% win rate over the last two months (20+ games played per month), but my rank hasn't increased. This is troubling for me - I consistently feel that I'm playing at a level above my rank indicates, and the statistics reflect this, but the KGS ranking system does not... In fact, it took multiple months of win rates greater than 70% for me to achieve my last rank promotion (which was then mercilessly taken away from me during the recent anchor shift in late December).

I know KGS's algorithm involves a lot more than simple win/loss ratios, but... From a psychological standpoint, it's disheartening to me that winning two games for every game I lose only results in stagnation of my rank, not growth. Even though I know there are other factors, this doesn't sit particularly well with me. On the other hand, though, it does make every promotion that much more sweeter since it was so much more difficult to attain.

[shapenaji]

Please make the proposal and argument for a different ranking system then.

Already did, I pointed out that I was a fan of Tygem's approach. I'm also a fan of IGS and WBaduk's ranking systems. The single reason I stay with KGS is that it is a community hub for North American go players.

This seems to be an issue for a player by player basis. Couldn't the argument be made that if a player sees his rank fluctuate wildly he gets frustrated and never knows how strong he is? Also, this is supposing that playing on KGS you wouldn't interact with a variety of styles at your given rank.

I think that's fairly unlikely, I don't think anyone would have an issue saying "I range between 4d and 6d"

This creates reasonable challenges to reward improvement, but it also gives you consequences for losing.

This is (possibly incorrectly interpreted) like saying that winning isn't reward enough.

Winning is definitely not enough of a reward. I play go to develop a better understanding for the game, a win is just a sign that I'm walking that path correctly. But if I'm stuck playing the same kind of games over and over again, it's possible that I can muddle by with the same understanding that I have. When you vary the difficulty of the game, you are adapting to a players understanding, and giving them room to learn new techniques.

KGS is extremely slow at noticing differences in understanding and offering new challenges, and it's detrimental to a large portion of their player base (At least the ones that don't make a new account every 4-6 months).

I still don't see why being a 6k is a bigger reward than being a 7k. You're saying that there will be bigger challenges and different play styles, which i would argue against. A person who might be 6k will still receive certain challenges playing people at a 7k rank if they were "incorrectly" given a 7k rank.

It's not about being a 6k, or a 1d, or a 5d. It's about playing the hardest game you can handle.

Also, as to your point of "more challenges" there would be no merit to most players playing people 1 level below their rank if we took your thought process. If you're a 5d and can win 80% of the time against a 4d, the problem might be that you're higher than a 5d instead.

I'm afraid I don't see what you're getting at, most ranking systems have ranks separated by ~66% win rate. Yes, my preferred system uses a lower percentage win rate for a shorter period of time before handicapping becomes involved... and?

Perhaps I missed Mef's point, but let's talk about this quote also:

There is a considerable difference in a 1 stone difference, in one case I have komi, in the other I don't.

This is depending on the level. I feel that if you pit two 5 kyus against each other the komi would rarely come into a deciding factor. I think for the most part komi is something only the stronger go players really have to worry and factor into, for the rest of us it is something to take into account at the counting process at the end of the game. If you can show me a kyu game where they actively figured the komi into their play style i'd be impressed.

Even if I didn't notice komi, it doesn't mean it doesn't have a large impact on the game. If I suddenly take an even match and then give one side an advantage, I will skew the results.

They don't even need to know how to count for komi to play a role.

All in all, yes it comes down to preference and yes the dissenting opinions should have their say. I feel that most people are arguing over something really inconsequential and that most of these people who are arguing over something like this don't even play enough for it to matter anyway.
If you're saying its an issue of challenge I think you're missing the point as there will always be a challenge in Go. The person who argues the ranking system and says "i'm so bored of winning all the time and my rank never goes up" probably forgot all the other times he loses to people of that rank.

It's not really that inconsequential, KGS has an absurd system, under which "The more you play, the less you move".

It is one of the only servers which I have ever heard publicly advocate for not playing on their site for a bit if your rank is changing. That's silly. Especially since, during periods of heavy game play, your rank is likely to change the most.

In fact, i urge most people to check out the KGS analytics site: http://kgs.gosquares.net/ Most of the time you're winning and losing 50% of your games anyway. If that figure were to ever be "incorrect" then of course the ranking system wouldn't be doing its job. But since it is typically 50% for almost everybody then it -is- doing its job.

[/quote]

I've seen people stuck with 60% win rate on KGS analytics never moving at all. And if Mef is correct, that's by design.

[mitsun]

That doesn't make any sense, how do you have more range between ranks than other systems and also have those ranks similarly populated from 9d down to 30k?

If by "range" you mean winning percentage per rank difference, the answer is simple: you compress the kyu levels and stretch the dan levels. At the dan level, a one-stone rank difference corresponds to a 79% win rate, while at the kyu level, a one-stone rank difference corresponds to a 70% win rate. (See "KGS Rating Math" at Sensei's Library.) That actually makes some sense if you think about it -- the stronger you get, the more likely it is that an extra handicap stone will affect the game result.

[Mef]

...when the rank system is 1 stone off, and we're talking about a 60% win rate rather than a 50%.

For KGS, one stone off would be closer to 80%. I think this is another one of those things that often throws people off on their expectations of the rating system.

That doesn't make any sense, how do you have more range between ranks than other systems and also have those ranks similarly populated from 9d down to 30k?

I have no idea what sort of predispositions you have toward other rating systems. I simply know what KGS assumes for a 1 stone difference, and how well it would appear to assess that difference.

If you are trying to claim that other rating systems do a poor job of spacing their ratings, I will not try to argue with you.