Life In 19x19

Kirby wrote:P.S. If KGS is so concerned about accuracy, we could delve back into its escaper policy, which we know is an accurate solution!

When I'm at a computer I may work out a model for Bob's misranked games (which on an infinite timeline trend toward infinity!), however in the meantime...The KGS escaper system is designed around protecting ranks (which has been demonstrated ad nauseum in this forum previously)...For a person who plays a significant number of games they can at best raise a 50% win rate to 55%. This is much less than half a stone of inaccuracy...

Mef wrote:When I'm at a computer I may work out a model for Bob's misranked games

I'm sure you can. Since Tygem's system has more volatility, you can obviously find more games that are not *precisely* at Bob's strength. But the point is as I've stated earlier, this volatility, while a negative, can also be a strength.

Mef wrote:The KGS escaper system is designed around protecting ranks...

I don't intend to start up the discussion on this here, which is why I mentioned this in a "P.S." note, but I believe you indicated previously that escaping is a valid solution to not finishing a game you don't want to finish. This is not an accurate measure of a game result.

Anyway, I think, as usual and as we are here, we'd continue to reiterate the same points, neither party changing his minds. As a KGS admin, you will, of course, maintain the arguments for the current KGS implementation. Conversely, I will, of course, maintain the arguments I have stated in the past.

Kirby wrote:

Mef wrote:When I'm at a computer I may work out a model for Bob's misranked games

I'm sure you can. Since Tygem's system has more volatility, you can obviously find more games that are not *precisely* at Bob's strength.

But the point is as I've stated earlier, this volatility, while a negative, can also be a strength.

Not when you are discussing accuracy.

Mef wrote:
Not when you are discussing accuracy.

Volitility, even randomness, is more accurate than stable systems for some scenarios. One of these scenarios can occur at times when one increases in strength - the KGS system is slow to pick this up. Here's an example: You increase in strength from 5k to 1d. KGS system will maintain the 5k status for awhile. A volatile system might think 6k a bit, then 4k a bit, then 3k maybe, then 5k, then 2k, maybe, then 1d maybe, then maybe 2d, then 1k, etc.

But the point is, the oscillation is moving upward, closer to the *accurate* strength. So volatility *can* be more accurate than a stable system.

You are confusing "stability" with "accuracy".

Kirby wrote:

Mef wrote:
Not when you are discussing accuracy.

Volitility, even randomness, is more accurate than stable systems for some scenarios. One of these scenarios is when one increases in strength.

On the whole as a go server it significantly reduces accuracy.

Mef wrote:

Kirby wrote:

Mef wrote:
Not when you are discussing accuracy.

Volitility, even randomness, is more accurate than stable systems for some scenarios. One of these scenarios is when one increases in strength.

On the whole as a go server it significantly reduces accuracy.

On the whole, yes. A stable system like KGS is more likely to be "accurate" than a volatile system. That's because the assumption is that most people are not changing in strength.

For such situations, though, a volatile system can be more "accurate".

I am interested in such situations, because improvement is something I am interested in.

Kirby wrote:

Mef wrote:On the whole as a go server it significantly reduces accuracy.

On the whole, yes. A stable system like KGS is more likely to be "accurate" than a volatile system. That's because the assumption is that most people are not changing in strength.

For such situations, though, a volatile system can be more "accurate".

I am interested in such situations, because improvement is something I am interested in.

If everyone else on the server has an inaccurate rank, how is it meaningful for you to have an "accurate" one?

Mef wrote:
If everyone else on the server has an inaccurate rank, how is it meaningful for you to have an "accurate" one?

Because you can get the opportunity to play stronger players. Have you ever tried playing on Tygem? You should give it a shot. You might be offset by the volatility of the ranks at first, but you will have the opportunity to play stronger players, and I would bet that you'd get stronger in the longrun.

And it's not like the ranks on Tygem, as a whole, are random and inaccurate, either. They are just more volatile than KGS ranks, so now and then you see someone with different strength.

As you approach dan levels, ranks become more stable like KGS's, because even with volatility, it's a lot harder to beat those stronger players, even by chance!

Kirby wrote:

Mef wrote:
If everyone else on the server has an inaccurate rank, how is it meaningful for you to have an "accurate" one?

Because you can get the opportunity to play stronger players.

This has nothing to do with rating accuracy. For the record, I have played on tygem (and virtually every single other go server that is brought to my attention, even the extremely short lived eurogoserver), I have nothing against the server personally, however to try and claim it's rating system is designed for any sort of accuracy is a farce. As I said up thread, I wouldn't be surprised if it were intentionally set this way to introduce uncertainty into the system (much like the rating systems of baduk clubs where they gamble).

Mef wrote:...however to try and claim it's rating system is designed for any sort of accuracy is a farce.

If you sincerely believe this, then I can't believe that you've played that much on the server, particularly if you've gotten to the dan ranks. Like I said, while not as stable as KGS, ranks do get more stable as you level up in rank. Are you sure you weren't playing on wbaduk?

Kirby wrote:

Mef wrote:...however to try and claim it's rating system is designed for any sort of accuracy is a farce.

If you sincerely believe this, then I can't believe that you've played that much on the server, particularly if you've gotten to the dan ranks. Like I said, while not as stable as KGS, ranks do get more stable as you level up in rank. Are you sure you weren't playing on wbaduk?

As we've already covered...Even for an ideal case, if a person who plays a large number of games their rank is going to be +-2 stones simply because of how the system is designed (and this is ignoring the fact that their opponents also will be varying +-2 stones!). That is not a system aiming for accuracy.

I have played on wbaduk as well (well, admittedly it was more on cyberoro...but some on wbaduk).

Mef wrote:
As we've already covered...Even for an ideal case, if a person who plays a large number of games their rank is going to be +-2 stones simply because of how the system is designed (and this is ignoring the fact that their opponents also will be varying +-2 stones!). That is not a system aiming for accuracy.

I am talking about your personal experience. Did you feel that the ranking system was a "farce" when you were playing? I play on Tygem often, and I find it hard to believe, because I feel that the ranks do become more stable as you increase in rank.

Of course, this is just my feeling, but you can look at actual numbers. I'm logged into one of the Tygem servers right now, for example. I have the list of users on the right, and I scrolled to a random spot - ended up on some 4d players.

Here's a snapshot:


The 3rd column is the win count and the 4th column is the lose count. Maybe these numbers aren't as even as they are in all cases on KGS, but we have numbers like: 4702 wins and 4358 losses; 4803 wins and 4622 losses; 8045 wins and 8036 losses.

Sure, there are exceptions, but these numbers are pretty close - people are winning similar to the number of times that they are losing. What also sticks out to me are the pure number of games these people play. How many people on KGS do you know that have played over 8000 games on their account?

Whether you enjoy the variance in the system or not, we have numbers here that show two things:
* People play a lot of games
* They win in numbers proportionate to the times they lose

Playing on Tygem myself, I find myself in a similar situation:
* I am matched against opponents that seem similar to my strength
* I am encouraged to play a lot of games.

So we've discussed a lot about how KGS has design to be more stable, but I have difficulty believing that you believe the system is a farce, if you've sincerely played a lot on the server.

Kirby wrote:I am talking about your personal experience. Did you feel that the ranking system was a "farce" when you were playing? I play on Tygem often, and I find it hard to believe, because I feel that the ranks do become more stable as you increase in rank.

Of course, this is just my feeling, but you can look at actual numbers. I'm logged into one of the Tygem servers right now, for example. I have the list of users on the right, and I scrolled to a random spot - ended up on some 4d players.

Here's a snapshot:


The 3rd column is the win count and the 4th column is the lose count. Maybe these numbers aren't as even as they are in all cases on KGS, but we have numbers like: 4702 wins and 4358 losses; 4803 wins and 4622 losses; 8045 wins and 8036 losses.

Sure, there are exceptions, but these numbers are pretty close - people are winning similar to the number of times that they are losing. What also sticks out to me are the pure number of games these people play. How many people on KGS do you know that have played over 8000 games on their account?

Whether you enjoy the variance in the system or not, we have numbers here that show two things:
* People play a lot of games
* They win in numbers proportionate to the times they lose

Playing on Tygem myself, I find myself in a similar situation:
* I am matched against opponents that seem similar to my strength
* I am encouraged to play a lot of games.

So we've discussed a lot about how KGS has design to be more stable, but I have difficulty believing that you believe the system is a farce, if you've sincerely played a lot on the server.

In my personal experience there was a very wide disparity in skill from opponent to opponent. To me that suggests that there is much more than 1 stone of strength variation inside a given "rank". I had used my KGS rating at the time when I signed up, which I later learned meant that I was probably a bit underrated (at 1d), but the players I faced off against ranged from being a good match that came down to endgame to being someone who was clearly several stones weaker than me. The best comparison I personally have is the low dan sections of the US open (where you have the guy from a Japanese tour group who entered at his Japanese rating, the kid who plays 1 tournament/year and has to play as a 2d even though he has a 7d KGS account, and 4 people who lie somewhere in between). For the AGA the problem is clearly sample size, tygem does not have that problem.

Winning a similar numbers of times to what you are losing does not mean that games are evenly matched. You could pair people 4000 times with 30ks and 4000 times with 9ds and they would all end up with 50% win rates. This would not be an accurate rating system. The true measure of accuracy in a rating system is being able to predict in advance who will win a game. Tygem does not do that, and makes no attempt at doing that.....but trying to pretend it does is a bit silly.

Mef wrote:Winning a similar numbers of times to what you are losing does not mean that games are evenly matched. You could pair people 4000 times with 30ks and 4000 times with 9ds and they would all end up with 50% win rates. This would not be an accurate rating system. The true measure of accuracy in a rating system is being able to predict in advance who will win a game.

Okay, with no particular bias, here's a listing of the last several games of the first player referenced in the list I provided earlier:


As you can see, all of his games are against 4d. He likely uses automatch, just like most other people on Tygem, and hence plays people of the same rank. Around half of this *4d* player's ranks against other *4d* players are wins, and about half are losses. Setting his rank to 4d would, therefore, be an accurate indicator to predict in advance who would win the game, would it not?

Feel free to look up other players on Tygem. You'll likely find similar results, especially at higher levels.

Your argument is based on how you believe an "accurate" system should be. You like KGS's system - it's stable. I get it.

But I am showing you real data. Take a random player on Tygem, and there's a good chance they'll have around 50% win rate if they've played a lot of games. And take a look at that player's history. They've likely played players of the same rank, since pretty much everyone uses automatch. I have personal experience that the system works, and I'm showing you real data. If you don't want to believe it, that's fine. But you are missing out on a great resource for playing games, given the number of players of various ranks, and the wide availability of games.

Kirby wrote:But I am showing you real data. Take a random player on Tygem, and there's a good chance they'll have around 50% win rate if they've played a lot of games.

This is precisely the problem...many of them SHOULDN'T HAVE A 50% WIN RATIO.

A 50% win ratio implies that there is no difference between a weak 4d and a strong 4d...there are only average 4ds. It implies that you have no predictive capacity, because the result of every game is, in fact, random chance. As that study I pointed to showed, all of those other rating systems do significantly better than random chance, because they have predictive capacity. KGS just happens to do better than most of them (admittedly WHR did even better than KGS, but critics of KGS will hate WHR even more...so there's that).