30k? to 2d? by not playing for a few months?

[blade90]

Imagine that over the last 180 days, I have played either 20 or 1000 games on KGS. In the first case, the system has no idea how much time I'm spending on go. In the later, it knows that I'm spending lots of time.

This sounds like making the simple comlicated, why should any rating system care how much time I spend playing on Go? I could play one game each month but still study a lot, thus spending a lot of time on go.
It doesn't really give a reason for increasing strenght. But to be fair: good results like a winning streak does also not really prove of increasing strenght.

After a winning sreak, always comes a losing streak (sooner or later). But if I lose 10 games straight, then I want to get demoted! The current rating system on KGS gives you the option to simply "lock" your rank by playing many games. People who want to keep track of their growth through KGS will get stuck at some point. Shapenaji called it "the barrier".

I encouterd a few 2k's that played worse then most 5k's on KGS, I had handicap of 2-4 stones but still they felt weaker. These people play 10+ games per day, even if they get stronger or weaker their rank will nit change at all.

Sometimes I'm amazed to see someone being stuck on 6k for a year while I just past it in a few months without studying at all, the reason is simple: I play not as much as them! Their rank is simply "frozen".

I believe (however I am not certain) that WMS has stated that there is a halflife of 45 games.

I think I read somewhere that the last 6 months are used to calculate the rating, because if you do not play for 6 months you will lose your rank.

[wms]

I know that 179 day old games already count less--what I'm proposing is that for two players, one of whom is playing more on the server, their older games count even less.

I believe (however I am not certain) that WMS has stated that there is a halflife of 45 games.

45 days, not 45 games, but otherwise yes, that is the case.

[wms]

But all this has me wondering if it's possible to create a system that is somehow intermediate between counting all games from the past 180 days and one that only counts the most recent n games?

Imagine that over the last 180 days, I have played either 20 or 1000 games on KGS. In the first case, the system has no idea how much time I'm spending on go. In the later, it knows that I'm spending lots of time. So it should think the probability that my strength is going up is higher, without looking at the results of the games. Could it therefore count the more recent games more heavily in the later case? Has this ever been tested?

One difficulty is that game weights must be symmetric in the KGS rank system. That is, a win that pushes you up in rank must push your opponent down equally strongly. When I experimented with asymmetric weights I got weird little groups of people who would spiral crazily up or down. So if the game fades fast for you, it must also fade fast for your opponent; if you play a lot and they don't, then what do we do? If we choose an intermediate half life, then you would get really broken stuff, where some of your games would fade away faster than others for reasons that have nothing to do with what they say about your strength.

I was happy to see Remi's results. Even though the games were even, and thus within ±1 stone, this is by the KGS rank system. If the rank system had been badly broken, then more accurate systems would have come up with different ranks for players and predict winners more accurately. Elo and glicko are very well respected rank systems, and KGS outperforming them tells me that my rank system is as accurate as it needs to be.

This also indicates to me that people who complain about getting stuck are seeing patterns that are not statistically valid. In basketball there is a strong belief amoung a lot of people that a player gets "hot hands" where they are shooting baskets exceptionally accurately. A statistician examining this phenomenon and found it to be invalid - players who hit a series of baskets were no more likely to hit the next than normal, it's just that human brains always look for patterns in sequences of events, and sometimes end up finding them where they don't actually exist. Sometimes KGS players do genuinely improve faster than the rank system can accommodate, but more often when people win or lose a lot and are disappointed by how little their rank changes, they are seeing patterns that are statistically outweighed by other data.

On the other hand, maybe accuracy isn't so important in a rank system. People like to see their rank go up and down. Maybe building an intentionally inaccurate rank system that responds too quickly to streaks would lead people to enjoy their games more even though it made their rank less meaningful. Not something I have time to experiment with now, but it is an idea.

[shapenaji]

Bear in mind that there is a lot of wishful thinking going on. If you have a group of people who all mistakenly believe that they're stronger than they are and create new accounts, a substantial fraction of them will end up with a higher rank, at least for a good long while.

In my progress to 4d KGS, I've had ~4 accounts. Each got stuck somewhere, I started winning, and decided that progression by winning 67%+ of my games was going to take forever. I started a new account, and after ~10-15 games, I was back to 50%.

Yes, there are people who mistakenly believe they are stronger, but the fact is that there is a lot of inertia, and go is just not nearly as fun if you are not getting a challenge.

There's a lot of wishful thinking on Tygem/WBaduk too. Folks set their rank too high. But, just as it is easier to gain ranks, it is also easier to lose ranks. So people eventually gravitate to the right rank anyhow. There's just a bit more noise.

But all this has me wondering if it's possible to create a system that is somehow intermediate between counting all games from the past 180 days and one that only counts the most recent n games?

Imagine that over the last 180 days, I have played either 20 or 1000 games on KGS. In the first case, the system has no idea how much time I'm spending on go. In the later, it knows that I'm spending lots of time. So it should think the probability that my strength is going up is higher, without looking at the results of the games. Could it therefore count the more recent games more heavily in the later case? Has this ever been tested?

I don't know about this, this doesn't factor in the considerable amount of time a person may be spending studying or playing over the board, and would seem to unfairly punish those forms of training.

I will say that, as it stands now, if you play a lot of games, those will weigh you down for a long time. There's a 45 day half-life (Thanks speedy for that stat btw), BUT the distribution of games is not uniform.

It would seem to me that every day should be treated as a single data point, NOT every game.

Games are just measurements of how a person is playing at a particular time, if they play 20 games in a day or 5 games in a day, those are both measuring the same parameter. So, you should store a performance rating and a confidence interval for each day. (Those might even be interesting stats to publish for the stat-hounds out there)

Those performance ratings should be combined (and older ones given less weight) to give you a players total rating.

That would make more sense to me.

[shapenaji]

And following that idea up,

Take the performance ratings, if the performance ratings spike (whether up or down) magnify their impact.

Basically, if a person is playing at or near their weighted average, consider that noise. If there is a spike, the signal-to-noise ratio should go way up. That should tell you that something has happened. You should be able to weight based on that parameter.

[wms]

Take the performance ratings, if the performance ratings spike (whether up or down) magnify their impact.

Basically, if a person is playing at or near their weighted average, consider that noise. If there is a spike, the signal-to-noise ratio should go way up. That should tell you that something has happened. You should be able to weight based on that parameter

I'm not sure what this even means. "Performance ratings"? Of the rank system or the players? And signal-to-noise is a communications term, I'm not sure what it means with respect to ratings or game plays.

[shapenaji]

Performance ratings of the players, A LA what they do for chess GM's during tournaments. They calculate a performance rating for that player based on their results against the other players during that tournament.

if a person rated 6d has the following results for a given time period (you could calculate the performance ratings on a per-day basis or a per-week basis, whatever)

7 wins-6 losses Performance Rating: 6.8d
6 wins-8 losses Performance Rating: 6.1d
5 wins-4 losses Performance Rating: 6.5d
7 wins-4 losses Performance Rating: 7.3d
8 wins-3 losses Performance Rating: 7.5d



Now, the signal-to-noise stuff comes out of looking at the pattern of the performance ratings

suppose I have 10 weeks of performance ratings that are well fit by a normal: mean = 6.5 sd = 0.4

Now suddenly the next data point changes the standard deviation dramatically. That spike in the standard deviation is a signal that something has changed.

This is different from, say, a sociological experiment. In this case, recent unusual results are more important than the average.

[wms]

I think you are searching to find an algorithm that will fit what you see as a pattern. As I said in my earlier message, I trust statistical calculations more than humans.

KGS can underrate quickly improving players. I have always stated this. Nick, I would not be surprised if you were one of the few based on what I know about your playing strength and how quickly you got there. But very few players improve quickly enough for that to bite them, and even fewer truly become weaker that quickly. If 99% of the players are well suited by the current algorithm, then I would not call it more accurate to make a change to better fit the few if it causes false changes in the rest; and calling a few wins or losses a trend and adjusting because of it will definitely cause false rank changes in most players.

[blade90]

@wms: Now that is something I understand!
But one question remain: is it true that it takes longer to rank up if you play more games, because I had the feeling that you deny that fact?
I know it is true that you rank up fast if you don't play many games.

[Uberdude]

Let's say you have played 400 games at 4d and won half of them (randomly spaced out). You then win 20 in a row. Shapenaji, do you want this to make you rank up to 5d? And if you lost 20 in a row go down to 3d? I think that would be wrong as all those old games let us know this spike was not significant and you are still probably a 4d who just had a nice winning streak. If, on the other hand, you had 20 games at 4d and had won half of them, and then won 20 in a row, your rank should go up as they are more significant. According to my understanding of the KGS rating system, this is how it works (I'm sure wms can correct me if I'm wrong).

I would agree though that KGS ranks can have rather a lot of inertia if you play loads of games (and there's drift), but I went from 30k all the way to 3d in 2 years on my single Uberdude account without feeling the rating system holding me back. I was typically playing a few games a day and had a high win rate (up to 85%, man I miss those days when improving was so easy ). In fact I did make new accounts for short periods, but that wasn't because of rating inertia but because my bad connection made me a chronic escaper.

[speedchase]

Uberdude, I think what shapenaji is saying, is that in order for a 4d to go on a 20 game win streak they have to be playing like a 5d, and if they go on a 20 game lose streak, then they are playing like a 3d. I personally agree. your rank should reflect how you are playing, not the accumulation of your games over the past months. If you think about it, it doesn't really make sense that how you were playing 45 days ago is half as important to your rank as how you are playing today. in order to provide accurate handicaps, ranks need to reflect the current playing strength of the player, not a strength that they can maintain over a long period of time.

[hyperpape]

Twenty game winning streaks are quite a thing. But how does the rating system handle the guy who wins twenty then loses ten?

[speedchase]

he is promoted and then demoted.

[hyperpape]

According to which standard? The accelerated one or the normal one?

[shapenaji]

I think you are searching to find an algorithm that will fit what you see as a pattern. As I said in my earlier message, I trust statistical calculations more than humans.

KGS can underrate quickly improving players. I have always stated this. Nick, I would not be surprised if you were one of the few based on what I know about your playing strength and how quickly you got there. But very few players improve quickly enough for that to bite them, and even fewer truly become weaker that quickly. If 99% of the players are well suited by the current algorithm, then I would not call it more accurate to make a change to better fit the few if it causes false changes in the rest; and calling a few wins or losses a trend and adjusting because of it will definitely cause false rank changes in most players.

So I do see a pattern in advancement, most folks who are advancing tend to hop forward. They'll be at 16k then suddenly jump to 12k, then 6k, then 1d, then 3d

It's true that I jumped through those hoops rather quickly, but I see the same pattern in others, they just sometimes spend more time at certain barriers.

My hypothesis has always been that advancement comes with a paradigm shift in how you see the game, and when someone finds that, they suddenly jump.

As you say, the KGS system doesn't always deal with rapid advancement well. But the way I see it, if a person really is at a barrier, a system with less inertia will effect them very little. They might have a good day and go up a rank, or a bad day and go down one, but they'll hover around the same spot.

The risk, in this case, is that those players might end up playing 1 stone off their "proper" handicap. I don't actually see this as a problem. Folks will fight harder to win when there's more of a risk of falling down, and more of a gain in winning.

Also, being one stone off the proper handicap is unlikely to be a blowout, at any rank.

This would make ranks considerably more fluid, along the lines of what they are at clubs. Imo a person's strength variance is usually on the order of a stone, I don't see a huge problem with their actual rank variance representing that.