Life In 19x19

Mef wrote:
As an aside -- If we actually performed this thought experiment, Bob's rank would never be stable. You would expect his rank to jump +-2 stones everyday simply from random noise.

Why would you expect this? Bob could very well have a stable rank after having gone to go congress.

Kirby wrote:

Mef wrote: Tygem does necessarily not do this. If you maintain a win rate of 33% on Tygem, you do not demote.

If you are able to win against a 5d for 33% of your games after 10,000 games, your strength must be close to 5d. It would be highly unlikely for a kyu player to be able to achieve this.

In any case, this is unrelated to the point I am getting at - Tygem predicts changes in strength more easily than KGS does when users have played many games. A direct example of this can be seen in the "thought experiment" I provided.

Since it would seem you did not read it, I shall rewrite the thought experiment:



And regardless of thought experiments, as the actual study using 2.3 million KGS games as data showed: KGS predicts game results with higher accuracy than less sophisticated systems.

Kirby wrote:

Mef wrote:
As an aside -- If we actually performed this thought experiment, Bob's rank would never be stable. You would expect his rank to jump +-2 stones everyday simply from random noise.

Why would you expect this? Bob could very well have a stable rank after having gone to go congress.

Playing 1000 games/day that are properly rated is equivalent to flipping 1000 coins in a row. Random variations will allow for streaks like winning 12/15 quite readily when you are playing that many games. You will even expect an 8 game winning streak once every 500 games.

Mef wrote:

Since it would seem you did not read it, I shall rewrite the thought experiment:

...

Yep, I read it again. And it still relys upon Tygem's flaw in allowing the initial setting of ranks. If initial ranks are accurate to begin with, 33% win rate against a 5d is close to 5d.

Mef wrote:
And regardless of thought experiments, as the actual study using 2.3 million KGS games as data showed: KGS predicts game results with higher accuracy than less sophisticated systems.

Yes, someone wrote a paper on this, and he liked KGS's system. This is within the bounds of his definition of accuracy.

And I'll say it again, in case you are not reading what I've written: In the example I provided, KGS does worse job of determining difference in Bob's strength.

Kirby wrote:Yep, I read it again. And it still relys upon Tygem's flaw in allowing the initial setting of ranks. If initial ranks are accurate to begin with, 33% win rate against a 5d is close to 5d.

No, the problem is that the rating system does not converge. If your rating system requires you to have perfect knowledge of your own rating prior to joining then why do you have a rating system in the first place?

Mef wrote:...

Playing 1000 games/day that are properly rated is equivalent to flipping 1000 coins in a row.

Nope, not in this case. Here, Bob has > 50% chance of winning against even opponent since he has increased in strength.

Kirby wrote:

Mef wrote:...

Playing 1000 games/day that are properly rated is equivalent to flipping 1000 coins in a row.

Nope, not in this case. Here, Bob has > 50% chance of winning against even opponent since he has increased in strength.

I am talking about the case prior to the go congress. If Bob is playing 1000 games/ day, he will never hold a rank for more than a few hours simply from random variation. Saying that Bob "converged" to 3d prior to the congress is meaningless because he would be oscillating around it.

Mef wrote:No, the problem is that the rating system does not converge.

Given the numbers of users on Tygem, for people that use automatch, ranks should come close to converging. The case you mentioned is for people that deliberately provided a bad initial rank.

Mef wrote: If your rating system requires you to have perfect knowledge of your own rating prior to joining then why do you have a rating system in the first place?

Agreed, I don't care for this aspect of Tygem's system.

Mef wrote:I am talking about the case prior to the go congress. If Bob is playing 1000 games/ day, he will never hold a rank for more than a few hours simply from random variation. Saying that Bob "converged" to 3d prior to the congress is meaningless because he would be oscillating around it.

Sure, I agree with this. Though, it's still easier for him to get a game matching to his strength than it is for Sally.

Kirby wrote:

Mef wrote:I am talking about the case prior to the go congress. If Bob is playing 1000 games/ day, he will never hold a rank for more than a few hours simply from random variation. Saying that Bob "converged" to 3d prior to the congress is meaningless because he would be oscillating around it.

Sure, I agree with this. Though, it's still easier for him to get a game matching to his strength than it is for Sally.

Sally's games would be handicapped off by a stone for 10 days. Whereas Bob will have a significant number of his games with the wrong handicap on any given day.

Mef wrote:
Sally's games would be handicapped off by a stone for 10 days. Whereas Bob will have a significant number of his games with the wrong handicap on any given day.

Bob has oscillation around his "true" strength on any given day, but it is faster for him to move up in rank when he improves like this. It takes Sally 10 days before she can start playing 2d evenly. Because it is helpful to play against stronger players, the oscillation is preferable for improvement, because you will get more exposure to higher level play. By the time 10 days passes and Sally is playing against 2d, Bob has already seen moves of 4d Tygem and maybe 5d Tygem for a long time now.

Because of this, I suspect Bob has better chance to improve faster, because he gets exposure to higher level of play more quickly.

To summarize so far, I made mistake in calling Tygem's methodology one that "converges" to a rank. As Mef points out, it oscillates around a rank, with some variation when you play a lot, since there are a static number of wins and losses required to be promoted or demoted.

But I believe that the point remains that Tygem's system is more volitile and when you do improve in rank, Tygem is able to detect it sooner. Further, if you have some bad games on KGS vs. if you have bad games on Tygem, it is easier to "fix" your rank on Tygem, because of this volitility.

Some people prefer the slow moving, but more stable KGS system. I understand this, and it can work for predicting a rank and matching people up. But this type of slow moving system makes me afraid to play games, because I know that if I play crappy today, I will have to play many more games to fix it.

This is why my preference is for volatility over stability - it responds faster to game results, and I like that in a ranking system.

You can identify scenarios where KGS ranking is more accurate than Tygem, but the converse can also be true for periods of a player's growth when they are changing in rank. In these times, a volitile rank is better than a slow moving one.

Kirby wrote:

Mef wrote:
Sally's games would be handicapped off by a stone for 10 days. Whereas Bob will have a significant number of his games with the wrong handicap on any given day.

Bob has oscillation around his "true" strength on any given day, but it is faster for him to move up in rank when he improves like this. It takes Sally 10 days before she can start playing 2d evenly. Because it is helpful to play against stronger players, the oscillation is preferable for improvement, because you will get more exposure to higher level play. By the time 10 days passes and Sally is playing against 2d, Bob has already seen moves of 4d Tygem and maybe 5d Tygem for a long time now.

Because of this, I suspect Bob has better chance to improve faster, because he gets exposure to higher level of play more quickly.

Except that we are talking about rating accuracy. Bob will be playing a significant portion of his games with an inaccurate rank even under the best conditions. Sally will have a relatively small fraction played at an inaccurate rank when all is said and done.

Mef wrote:Except that we are talking about rating accuracy. Bob will be playing a significant portion of his games with an inaccurate rank even under the best conditions. Sally will have a relatively small fraction played at an inaccurate rank when all is said and done.

Bob's portion of games played at inaccurate rank are not significant, considering the number of games he is playing. Despite the oscillation, most games will be played around this center of oscillation.

P.S. If KGS is so concerned about accuracy, we could delve back into its escaper policy, which we know is an accurate solution!

Anyway, I think when we talk about "accuracy", we can have different interpretations.

KGS's definition of "accuracy" is good in that it takes into account game history, keeps ranks relatively stable and steady, and provides a more precise definition of what a rank is.

Within this stable, steady system, however, there can be people that start to improve and change - or conversely, forget things and get worse. Tygem's interpretation of rank accuracy allows for this to be picked up faster at the cost of having more variance in ranks to begin with. Tygem's ranks are more volatile and are less sensitive to past history.

For someone interested in improving and playing stronger players, I think that this can be a good thing. For someone interested in a system that has slower-moving, yet stable, ranks, I think that KGS's method can be preferred.

Most importantly for me: I am not afraid to play many games on Tygem, because I am not afraid of being "stuck" in the slow-moving ranking system that KGS has. With KGS, if I play a lot of games and get set as X rank, then it will take a lot of wins to change that, and when I'm studying and trying to improve at the game, this is disheartening. God forbid, I become an anchor... But that's beside the point.