Oddities in KGS ranking system

[tj86430]

At the end of May I was barely 6k at KGS. I have since then played 5 ranked games, of which I have won 4. Now I'm 3k.

It sounds like too rapid improvement of rank to me.

Data: http://valkonen.kapsi.fi/keksi.php?user=tj86430

[judicata]

Your rank is affected by the improvement in your opponents' rank (even after you played them).

[tj86430]

Your rank is affected by the improvement in your opponents' rank (even after you played them).

All the opponents I won during this period are still the same rank as when I played against them. Only the one I lost to has gained two kyu since we played.

It must be the opponents I have played a long time ago.

[Mef]

The server had a very low confidence in your rank (the 5k?) going into July (There's an exponential decay on game weight)...you go 3-1, then win again in August...basically now the server treats you as if you've gone 4-1.......And then it looks at the fact that your loss was basically handicapped as a 3k. So the majority of your rank is based on the fact you have won 4 games as a 4k (with no losses), and suffered 1 loss as a 3k.

[ez4u]

Why is any rating movement based on too few games "odd"?

[Kaya.gs]

Consider i will soon make a whole new thread about rating systems, finding one for Kaya.gs i do want to mention an odditiy about kgs rating system.


Besides accounts being heavy and such, there is an impressive psychological aspect of the system that does not feel to affect point-based systems like in Wbaduk or Tygem.

Back then when playing with danigabi[5d] account i have played certain 2ds giving them 3 handicap stones. I would win & lose, and i think i won a tad more than lost (say 60%). The impressive happens later. Right after losing a game, i would log back in with Rakuen[7d], and play the very same player with 6H. Suddently, i would win almost 80%.

How is it possible that increasing many stones , my chances to win go up. My current account, DexMorgan, has been brought up to 7d with a similar effect.

I think this is a specific anomaly of this history-based rating system, where the psychology of the palyers deeply affect the end results and hence its accuracy.

This is not a single or a few more games, it is proven by many strong players that giving enormous amounts of handicap give a really good edge. (While playing the default handicap is disadvantageous)

[daniel_the_smith]

Consider i will soon make a whole new thread about rating systems, finding one for Kaya.gs i do want to mention an odditiy about kgs rating system.


Besides accounts being heavy and such, there is an impressive psychological aspect of the system that does not feel to affect point-based systems like in Wbaduk or Tygem.

Back then when playing with danigabi[5d] account i have played certain 2ds giving them 3 handicap stones. I would win & lose, and i think i won a tad more than lost (say 60%). The impressive happens later. Right after losing a game, i would log back in with Rakuen[7d], and play the very same player with 6H. Suddently, i would win almost 80%.

How is it possible that increasing many stones , my chances to win go up. My current account, DexMorgan, has been brought up to 7d with a similar effect.

I think this is a specific anomaly of this history-based rating system, where the psychology of the palyers deeply affect the end results and hence its accuracy.

This is not a single or a few more games, it is proven by many strong players that giving enormous amounts of handicap give a really good edge. (While playing the default handicap is disadvantageous)

I think that is entirely explicable with psychology and how bad 2d's play in general. You try harder with more stones, the 2d tries less hard and also has the effect of "ah, I just won, now I can be lazy". The 2d could be bad at using handi, you could be good at making a whole board fight where additional stones don't help that much. Etc. I personally seem to be 4-5 stones weaker in casual games...

All rating systems make the incorrect assumption that strength can be expressed as a single value-- cycles where A beats B beats C beats A obviously happen all the time. You've just discovered such a cycle with 2 players and differing handicap stones. What I'm saying is that you'd have to invent a multidimensional rating system to find one that could make sense of the data you just reported-- I'm not aware of any that handle more than one dimension of strength.

TL;DR: similar inconsistencies can be found in ALL rating systems.

Oh, and-- I believe in the KGS system, white is expected to win 60% of the time in handicap games. This is because a 2 stone handi is really 1.5 stones (black should have 6.5/7.5 reverse komi to make it genuinely a 2 stone game).

[karaklis]

The ranking system of KGS assumes that the improvment development of go players is the same. Other systems such as that of IGS assume that it does not change if you don't play (there).

Both assumptions are not correct, but how do you want to measure the real improvement in order to obtain matches with a balanced win percentage? It's actually impossible. A compromise would be to keep the rank/rating on not playing, but to make it more volatile, so that you can quickly make it to your real strength after a longer playing break. I don't have detailed knowledge of the ranking/rating system of OGS, but it seems that this idea has been implemented there. Maybe it's not a bad idea to have a look at the system there.

[hyperpape]

The ranking system of KGS assumes that the improvment development of go players is the same.

What do you mean by this?

[daniel_the_smith]

WHR (Whole History Ratings) does correctly what KGS is attempting to do; KGS will change your rating now if the guy you beat a month ago gets stronger. WHR will change your rating of a month ago if it decides the guy you beat a month ago was actually at that time a stone stronger that the rating system thought he was.

KGS computes a rating for each player (the chart is created by appending the rating it calculates once per day).

WHR computes an entire rating history for each player each time it is run. IOW, every day, WHR will compute a (slightly) different chart of your rating over time.

WHR is the most theoretically advanced rating system I'm aware of, I really wish it were used somewhere...

Hm. And my last post made me want to invent a multi-dimensional rating system...

[snorri]

How is it possible that increasing many stones , my chances to win go up.

That's a pretty impressive example. Maybe the best rating system is a blind one where you don't know how strong your opponents are until later

There are some questions as to whether the traditional handicap system / star-point placement really compensates for strength differences accurately. I once heard a pro say that he wanted to define amateur 1d as a certain amount of reverse komi taking black against a pro rather than as a multi-stone handicap because it is "easy for the stronger player to make the handicap stones useless and it doesn't depend so much on the number of stones, but points are points and their value can't be erased."

[uPWarrior]

The problem is that adding a new stone has less and less impact as the number of stones grows.

E.g.:
7d vs 2d at 5 handi and -6.5 komi, the expected win rate is 50%.
However, 7d vs 2d at 6 handi and -6.5 komi, the expected win rate is 79% for black.

Does anyone believe that a single new handicap stone could produce this difference in winning percentages? Even if black started winning 75% of his games with this new stone, his rating would go down.

I think that any sufficiently robust system should have this into account, either by decreasing the compensation value of each handicap stone as stones get added or by decreasing the impact of high handicap games in rating itself.

[daniel_the_smith]

The problem is that adding a new stone has less and less impact as the number of stones grows.

E.g.:
7d vs 2d at 5 handi and -6.5 komi, the expected win rate is 50%.
However, 7d vs 2d at 6 handi and -6.5 komi, the expected win rate is 79% for black.

Does anyone believe that a single new handicap stone could produce this difference in winning percentages?

Sure, I could believe it, but I'd also like to know where your figures came from

BTW, AFAIK, EGF, AGA and KGS all do, in fact, take things like that into account.

[xed_over]

The ranking system of KGS assumes that the improvment development of go players is the same.

What do you mean by this?

he means that even if you don't play for a while (on KGS), the next time you logon, your rank will have increased, because the opponents you played in the past have increased.

KGS tries to make the assumption that even if you're not playing on KGS, that you're still playing somewhere and improving. And it does that by comparing you with your past opponents.

That's how I became SDK -- I quit playing.

[hyperpape]

Is that what it assumes? Daniel's explanation fit what I understood better--it's not that KGS believes that you're improving, it's that as it gets more information on your opponents' skill (because they play more games), it changes its estimate of your strength ("that guy you beat was actually 2 dan, not 3 kyu, so that's way more impressive).

In fact, that seems to be based on the assumption that your strength as well as the strength of your previous opponents are relatively stable. Otherwise, it's meaningless that your opponent from three months ago was 2 dan. Another factor is that older games are weighted less heavily than more recent games.