Is your rating accurate ( too low? too high? )

[topazg]

Hmm another way can be playing 100 games in a row while drinking whisky-vodka mixed with tequila-cognac. Then you will probably be really underranked

Mainly due to masses of losses on time from being in the restroom I would have thought

I don't really understand this thread. My rating is based on my performance, so at any given snapshot it will be accurate based on how I've played. Same with everyone else's. It's pointless thinking "I could have won that game if I hadn't been so tired", or "I'm really stronger than this but I don't try that hard online" because for all you know this could be the same (or even more so) with your opponents.

All a rating does is reflect your performance, not your strength. Over time and enough games played at a similar seriousness to your opponents, the rating will reflect your comparative strength to theirs pretty accurately, but it's only ever a general proxy.

My rating is what it is. I am as strong (or weak if you'd rather!) as I am. Beyond that, it's just a number anyway.

[hyperpape]

All a rating does is reflect your performance, not your strength. Over time and enough games played at a similar seriousness to your opponents, the rating will reflect your comparative strength to theirs pretty accurately, but it's only ever a general proxy.

"All the data in scientific papers is just reports of measurements, not of the underlying reality. We never find anything out about the world, just about the dials on our instruments."

One must tread carefully here.

[judicata]

That's exactly what I was trying to say. "Underranked" and "currently improving" are different things. Otherwise almost everybody would be underranked, which would make the ranking system void.

This sort of depends on the accuracy and efficiency of the rating system, and what you mean by "underranked." For example, on KGS, if I have improved rapidly. If I start a new account, it would immediately be ranked higher than my current account--probably within hours. Whereas, because I've played so many games on my other account, it takes a long time to increase. So, is the new account overranked? Is the old one underranked?

Important note: this is not a criticism of the KGS rating system. I personally like the system.

[topazg]

"All the data in scientific papers is just reports of measurements, not of the underlying reality. We never find anything out about the world, just about the dials on our instruments."

One must tread carefully here.

Sure, but that's quite a paraphrase / change of flavour of what I wrote. All data in scientific papers are indeed merely observations that we hope reflect the aspects of reality being studied. That doesn't necessarily mean they do, and it always leaves open the possibility of false data due to the dials on our instruments. It's unavoidable that these things exist, and the better awareness we have of them, the more likely we are to know what's signal and what's noise, and how to investigate further.

However, on the actual subject in question, I don't actually have a rating as such. I have a GoR that is out of date, a rank on KGS, a rank on IGS, and they are all different depending on a range of factors. They are arbitrary with no particularly stable reference point (ok, I know KGS has semi-stable anchors, but I mean inter-rank reference points), so putting any great faith in any of them seems a bit unreasonable. As a result, it's hard to feel my rank could be accurate enough to comment on it being 0.5 or 1 stone too high/low - my initial reaction would always be "compared to what?"

[ethanb]

"All the data in scientific papers is just reports of measurements, not of the underlying reality. We never find anything out about the world, just about the dials on our instruments."

One must tread carefully here.

Sure, but that's quite a paraphrase / change of flavour of what I wrote. All data in scientific papers are indeed merely observations that we hope reflect the aspects of reality being studied. That doesn't necessarily mean they do, and it always leaves open the possibility of false data due to the dials on our instruments. It's unavoidable that these things exist, and the better awareness we have of them, the more likely we are to know what's signal and what's noise, and how to investigate further.

However, on the actual subject in question, I don't actually have a rating as such. I have a GoR that is out of date, a rank on KGS, a rank on IGS, and they are all different depending on a range of factors. They are arbitrary with no particularly stable reference point (ok, I know KGS has semi-stable anchors, but I mean inter-rank reference points), so putting any great faith in any of them seems a bit unreasonable. As a result, it's hard to feel my rank could be accurate enough to comment on it being 0.5 or 1 stone too high/low - my initial reaction would always be "compared to what?"

Internet games are so variable - it's too easy to get one any time, whether you're really up for it or not. I feel like my rating is most properly represented by my AGA rating, which is untainted by blitz games, pizza deliveries mid-game, etc. Yeah, performance on given tournament days may be off, but as an average of times when I'm playing most competitively I think it works.

So topazg, in my opinion your BGA rating would be more accurate than any of the ones you listed above. Of course, that's different for quickly improving players - tournaments may be too far apart to properly track the progress of somebody from 24k to 14k (which could happen within 3-6 months if somebody's both talented and really studious.) That's why most tournament directors will allow self-promotion (among double-digit kyus at least) if the players feel they're up to it and can convince the TD.

[topazg]

So topazg, in my opinion your BGA rating would be more accurate than any of the ones you listed above. Of course, that's different for quickly improving players - tournaments may be too far apart to properly track the progress of somebody from 24k to 14k (which could happen within 3-6 months if somebody's both talented and really studious.) That's why most tournament directors will allow self-promotion (among double-digit kyus at least) if the players feel they're up to it and can convince the TD.

I would agree, if I played enough games. For example, on KGS, I have played over 60 games over the last three months, in real life tournaments I've played 20 games in the last two years. Not only that, but the volatility of the GoR system at the strong SDK end means that to gain a stone in strength with the sort of winning probability expected against those a stone and a half stronger I would have to play a good 30 or so games - at my rate, that's 2013 on the guestimate that it's currently about a stone too low.

Eventually, something like GoR should be very accurate - the likelihood is that all games are serious, and normally against pretty good anchors, which are both really good things. In reality, with the difficulty in getting to tournaments or playing regular games, the lag time on GoR is HUGE. Unlike the AGA, we can't have ordinary games ranked yet.

[daniel_the_smith]

What kind of games don't count towards GoR that the AGA would rate?

[topazg]

What kind of games don't count towards GoR that the AGA would rate?

I thought the AGA allowed people to do their own tournaments, or even games played and recorded in clubs, and submit those into the rating system? If I'm wrong, it's almost definitely somewhere else - the point wasn't really supposed to be "It's easier in the US" though, more that I wish we _could_ do that in the UK.

There are some good proposals to do so, but none of the mechanisms to actually make it happen are in place yet.

[hyperpape]

The AGA allows rated games to be played at a club. I don't recall ever seeing anyone do it--the closest are self-paired rating "tournaments" that some clubs hold every month or so. The requirement is that the game be public, played with reasonable time controls or none, and that the players agree in advance.

A small point: the idea of a ratings accuracy can be based entirely on the standards of the rating system. So it's perfectly conceivable to say "Topazg is rated 1.5, but there was only a 1% chance that he would've been rated < 2 given his actual strength"

On the more abstract side, do you think scientists study bacteria? Or do they study measurements of bacteria? Or observations of measurements of bacteria...?

Edit: removed "%" typo.

[daniel_the_smith]

Ah, yeah, it's allowed, but nobody does it.

[topazg]

Ok, I thought I'd run a simulation, to see whether my suspicions are at all valid that there can be genuine lag in real life (in this case, BGA/EGF) systems. The idea is to see how quickly a 1951 player (me ) can hit 2051+ if playing consistently with a performance of 1.5 stones stronger, at 2100:

Simulation rules

One tournament per quarter of a year, starting with 4th quarter 2010
All EGF Class B tournaments, 3 rounds.
(... This reflects a reasonable equivalent of my tournament attendance ability)

Rating at beginning of first tournament = 1951
Actual ability (used to calculate winning chance) = 2100
First round opponent = current rating +/- 250 GoR points (Randomised)
Second round = same code for first round, but +50 if 1st round was a win, -50 if loss (simulates McMahon drawing system)
Third round = same code as previous rounds, but +100 if on 2/2, -100 if on 0/2, +0 if on 1/2

Results:

Code: Select all

Year            Run 1     Run 2     Run 3     Run 4     Run 5     Average
----------------------------------------------------------------------------
Jan 2011        1965      1943      1964      1974      1964      1962 (+11)
Jan 2012        2029      1939      1999      1957      2011      1987 (+25)
Jan 2013        2041      2022      2013      2005      2055      2027 (+40)
Jan 2014        2087      2059      2065      2049      2068      2066 (+39)

So, even pretending I have an ability of 1.5 stones above my official rating, I gain 1 stone by the beginning of 2013 in only one run (by 4 points), and even by 2014 I haven't reached it in one of the runs. In contrast, on KGS I can just go play 20 rated games a month for a couple of months and it will fairly accurately reflect my rating based on performance. Player consistency and seriousness of the match is likely to be higher, and more consistent, in real life rated games, but there just aren't enough of them to put much faith in their up-to-date-ness.

[ethanb]

The idea is to see how quickly a 1951 player (me )

Wow, I wouldn't have thought you were THAT old!
I'd have guessed you weren't born a day before 1961, myself.

Seriously though, maybe the BGA system doesn't age old games fast enough? The more recent games should count more than older ones, I'd think.

EDIT: oh, just noticed 3 games per tournament - usually the tournaments around here are four or five games; that may make a (small) difference too.

[Laman]

One tournament per quarter of a year, starting with 4th quarter 2010
All EGF Class B tournaments, 3 rounds.

well, with this number of games you really can't expect any fast changes in any rating system... and, i wonder, do you have so many 3-round B-class tournaments in Britain? in Czechia we have some B-class, but almost all tournaments are 5-round. and last, i think most active players (= players you can play at tournaments with) play more than 12 B games a year, so their rating should be much more accurate

Seriously though, maybe the BGA system doesn't age old games fast enough? The more recent games should count more than older ones, I'd think.

BGA (if it is the same as GoR) doesn't age old games at all, does it? it just makes calculations with the new ones

[topazg]

well, with this number of games you really can't expect any fast changes in any rating system... and, i wonder, do you have so many 3-round B-class tournaments in Britain? in Czechia we have some B-class, but almost all tournaments are 5-round. and last, i think most active players (= players you can play at tournaments with) play more than 12 B games a year, so their rating should be much more accurate

I agree, and I don't, hence my argument over which ratings I would view as most accurate

Most tournaments are B in the UK, and most one days are 3 rounds (I can't attend multiple day because of young family commitments). 5 rounds don't exist for one-day tournies here except for a single class C kyu players only tournament.

[Exologist]

Just because you are winning over half your games does not mean you are under-ranked. It just means you are steadily improving (assumin the handicap is correct).

Well yeah, that's what I mean. Steadily improving probably sums it up nicely. But if you can manage to win over half your games playing 2k, then you probably shouldn't be 3k. I'm getting some good victories in, so it's only a matter of time before it corrects.