France has a committee to forcibly apply rating resets. My earlier point about players who win more than 100 points in a tournament - well that is an idea I think would be useful to forcibly address the issue of countries who do not use rating resets. Gaining more than 100 points should be an indication that your original rating was a lie.

_________________
North Lecale

I can only say that this is the way the EGD worked for 20 years. I suppose even before the EGD, this was just the way things were done. I'm not involved in the policies used by the EGD and tournament organizers. I'm merely studying the EGD system and studying areas where it might be improved, because it happens to interest me. I'm not an EGF official of any kind.

The issue of reliability of newcomers' ranks was not one of the things I intended to study, but now that you and others mention this being a problem, I can think of a way to deal with it more gracefully: the system could use a grace period for a newcomer (say 10-15 games). During this period, the system would use a reduced K factor for the newcomer's opponents and an increased K factor of the newcomer herself. Then gradually, the system would settle the K factors towards their normal values as the newcomer's rating settles.
Some Elo rating systems have this kind of feature (like the FIDE rating system), but the EGD doesn't. I could add this feature to my revised system.

I think the method of having initial ratings determined by playing several tournament games is not free from flaws. For example, what do you do with an unrated visitor from Korea who wants to play in a tournament but won't likely play in any other tournaments? Some online servers have people have ratings with a ? until several games have been played. The problem with that in face-to-face tournaments is that opponents get no effect from winning (or losing) games. I think that in practice the best method of determining initial ratings is to have organizers or officials judge what the initial rating should be, taking into account self rating, online ratings and ranks from other organizations. Of course there will be some improper ratings but any effects will be smoothed out over time.

The EGD uses quite a large K factor for lower ratings. A newcomer who's rating is about 20k to the best of her knowlegde, could win 3 or 4 games in her first 5 game tournament (playing opponents ranging from 20k to 18k) and the EGD rating system could well award her more than 100 points for this result. This is not an exceptional situation IMO and I would definitely not call her a liar. She did nothing wrong. If you want to put blame somewhere, perhaps you should blame the large K factor the EGD uses at lower ratings.


But if I build a grace period into my revised system (see my previous response to Schachus), it would also involve a large K factor for the newcomer herself while her rating settles. That could mean that she gains many points during the grace period, more than 100 points even: perhaps she estimated her rank at 19k while in hindsight, 17k was closer to the truth, but she just didn't know any better. I think that a perfectly calibrated grace period system should award her 200 points before her grace period ends to conserve her future opponents' ratings. Would you consider such a system morally wrong?

That is indeed a good argument against a grace period: suppose an upcoming 5d plays a "real" korean 7d newcomer and manages to win a hard fought battle. Indeed, it would be disappointing for the 5d to not gain any points for this achievement, because his opponent is still in his grace period.

If one actually want to solve the issue, a system like Rémi Coulom's WRH system is probably a better solution than an Elo-like system, because WRH would be robust against misranked newcomers while still granting the 5d mentioned above his deserved points for winning against the real 7d newcomer.

If no grace period is used in an Elo-like system, some players will "undeservedly" gain or lose points against misranked newcomers. But perhaps undeservedly gaining or losing points is just a fact of life as a go player: "undeserved" wins and losses happen quite often in regular games anyway. As long as misranked opponents are not too common, the overall effect should even out in a similar way as undeservedly winning or losing against regular opponents.

In these discussions, doesn't Herman usually come around and mention that for kyu players, self-estimated ratings are more accurate than the ones the system dispenses?

_________________
Occupy Babel!

@hyperpape: If you insist

Is a link to a previous post sufficient? viewtopic.php?p=74100#p74100

Hi,
I don't know how the european system works, but in France, there are several adjustments made to the rating system.
For new players, an iterative system is performed to correct the rank. If the variation after the first tournament is more than -50 or +50, the player's registration grade is replaced with it's final grade, and the calculations are performed again. The final grade is fed in place of the initial grade as long as the variation is more than 50.
It allows to start the ranking of a new player according to his/her performance during the tournament, rather than guessing.
The system works well except if the first ranked game is a single game in the club. If the result is not meaningful, then the same adjustment can't occur during the second tournament of the player.

However, a second type of iterative adjustment is possible : if, during any tournament, the grade increases more than 60 points, then the iterative system is performed. This only works for positive variations. There are no iteration for negative variations, except during the very first tournament of new players.

These correction are also useful for their opponents, as the correction occurs before all the calculations.
For example, if a young player has gone from 5 kyu to 1 dan during the holidays, and they get a +200 correction, their opponents are considered to have lost against a 3 kyu player instead of a 5 kyu.

Also, the french federation uses a weighting parameter for handicap games that is biased towards the higher grades : the variation of the players for a handicap games are multiplied by (1 - H/10), except if while looses. In this case, the variation of white's grade is again multiplied by (1 - H/10).
It means that handicap games are weighted at 90 % of their values for a 1 stone handicap, 10 % for a 9 stone game, and that strong players have a special protection when they play handicap games against weaker players. For a 9 stone game, their variation is only 10 % of the calculation if they win, but 1 % if they loose (their opponent still have 10 % in this case).

It does seem quite complicated to invent a rating system to robustly accomodate quickly improving players that play few tournaments. I suspect it's just about impossible to get it right. You may add many rules and variables, but basically the system just does not have enough information from tournament game results alone.

For kyu players especially, just believing the declared rank seems like a better strategy (like Herman says). A system that liberally resets kyu ratings tends to be closer to the truth than a system that adheres much more weight to the previous rating than the currently declared rank. If a 9k club player became 6k in the period after his most recent tournament 6 months earlier (based on his personal experience in handicap club games played in between), why not just have him declare himself 6k in the next tournament and have the system adhere much belief to that information (like the tournament organizers)?

Do you think that kyu players tend to balatantly underrate or overrate themselves when declaring a rank in real life tournaments? I don't believe this is true. I think most kyu players are quite judicious when declaring a rank in a real life tournament. I think that a rating system that claims to map well to go ranks, should follow players declared ranks rather than the other way around, especially for lower ranked players that play few tournaments.

In the (higher) dan region the system should be insensitive to changes in declared ranks. It is unlikely that a 4d becomes 7d within a few months. Also, (higher) dan players play more tournament games, so the system tends to have enough information about them. In that case, adhering more weight to the previous rating than to the declared rank seems like a reasonable policy to me. So for a 4d to become 7d in the rating system, he'll just have to earn those points in tournaments. The system won't automatically apply a reset when he changes his declared rank from 4d to 7d.
If he happened to study in China for a year and really became 7d without playing tournaments in Europe, the system administrator could reset him by special request, but I think this happens very rarely (has it ever happened?).

I'm now experimenting with adhering more weight to declared ranks, particularly lower ranks. Not like the current system with a hard reset to the new rank when skipping a rank (which is a bit too crude IMO), but a bit more sophisticated.

Is it possible to do a comparison of goratings.org and goratings.eu?

Perhaps to somehow place goratings.org into the eu version and see how euro players compare to aisan pros?

goratings.org is made by Rémi Coulom and goratings.eu is made by me (Dave de Vos). There is no connection other than a similarity in name. The similarity in name comes from the fact that both are about go ratings, but I think the similarity stops there.

goratings.eu focusses on finding a rating system that maps to European amateur go ranks well (like the EGD), in the past, present and future. In such a system, 100 rating points difference corresponds to one go rank difference for all go ranks (at least this is the intention). The winning odds between go ranks varies considerably, depending on the rank of players (this is different from normal Elo ratings). Overall rating drift is undesirable, because it would mean that the mapping to European amateur go ranks would be lost over time. I think this kind of system mostly serves European amateur tournament players.

I think goratings.org is intended as a world ranking list to order the world's top players by their current competitive success rate. There are European top players in that list (in the lower regions). In my understanding these ratings are like standard Elo ratings: it means 100 points rating difference corresponds to about 2:1 winning odds for all ratings. That means that ratings in that list have no direct relation to go ranks (amateur or professional). I suppose the overall rating values may also drift over time, because that does not matter for the order in the list. Only rating differences have a meaning for the order of the list.

So comparing ratings from goratings.org to ratings from goratings.eu is like comparing apples and oranges IMO.

I did make a crude estimate for conversion at the bottom of this page, by matching extrapolated winning odds from goratings.eu and guestimating an offset by comparing some players that occur in both lists, but I don't think you should attach much meaning to it. They work differently, because they serve different purposes. They use different scales and the offset between them may drift significantly over time.

Why did the player in your example not declare himself 1k after the holidays? Was he unaware that he became so much stronger during the holidays? I find that a bit hard to believe.

Perhaps his national go association did not allow him to declare himself 1k? I know that some countries won't allow it, but I think such a policy is counterproductive, because it inevitably causes problems like these. And then you need a complicated system to fix it and arrive at the same result as when that player were just allowed to declare himself 1k.

I suppose such policies are intended to prevent rank inflation, but is that really a big problem? I can understand a desire to have some regulation for (higher) dan ranks, but for (lower) kyu ranks, the cure seems worse than the disease. I think it leads to kyu rank deflation. I read the topic that Herman linked to a few posts back and some posters in that thread claim that kyu rank deflation was clearly an issue in France in 2011.



I think that handicap differences define go ranks. So handicap games are an important benchmark for the spacing between ranks. So I would rather not reduce the weight of large handicap games as long as they are fairly additive (so that in practise, 8.5 ranks differences matches 9 stones handicap equally well as 2.5 ranks difference matches 3 stones handicap). I feel that the smaller K factor of the stronger player is already sufficient "protection" for the stronger player's rating (assigning more inertia to his rating). (I don't know if the French system uses a variable K factor).

I haven't come to it yet, but I intend to study handicap game statistics from the EGD game history. There are almost 100,000 handicap games in it (more than 10% of the total), so there is plenty of data. For example, I hope to find out to which degree handicap games can serve as long range benchmarks for rank spacings, because even though even game winning odds are a reasonable local measure of the rating curve, they are a bit too volatile with large differences to accurately determine the global shape of the rating curve.

I played with the rating reset policy.

I find that the tweaking the rating reset policy works well to regulate overall rating drift (inflation / deflation).
I chose a rating reset policy that is a bit more liberal in the kyu range (basically adhering more belief to self-promotions) and more conservative in the high dan range (compared to the EGD rating reset policy), optimized to the point that the apparent rating drift over the years look about 0.
In fact it seems that tweaking the reset policy works so well to regulate drift, that I was able to throw away the epsilon parameter that was intended to regulate rating drift.

The effect of this can be seen in the rating distribution charts. For example, the EGD rating distribution shows skewed distributions an deflation in the mid-dan range. But the revised rating distribution look much more regular. For me, this result proves that it is good idea to believe declared self promotions, particularly in the kyu range.

There are underranked and overranked players in the EGD and in the revised system. But I think it's only to be expected that the distributions are more or less regular gaussian distributions over the entire rating and rank ranges. That is just how regular statistics behave.

The problem is that many players see the rank as a title or a reward.

The FFG allows to ask for a re-evaluation for players who have improved a lot without playing official games. But the process is regulated :
The reevaluation must have the backup of an administrator of the club, and be sent with sgf of some games, that are then analysed by a comitee, who will decide if the player deserve a reevaluation or not.

But in my club, the top players are proud of their rank, and do not like to see people getting high rank without winning their games in actual tournaments. I remember when they purposely underevaluated the reevaluation form of a player because they said that otherwise, the player would be ranked above another one and that would be "unfair". For them, a rank is a trophy and should not be awarded freely.

This behaviour has its root in asiatic tradition, where the dans were honorary distinctions that were awarded only to the best champions.

In my opinion, this attitude is consistent as long as we are dealing with the players that are usually above the McMahon Bar in tournaments. For players under the McMahon bar, it is necessary to have the most accurate ranking possible for the McMahon system to work properly and pair together players of equal strength. In this case, reevaluations are useful.

I assume the majority of the French go community prefers to keep these regulations (otherwise, why are they still used by the FFG?), so it is of little consequence how I feel about these policies. I may be able to show if these policies affect French rank distributions, though:

When I inspect the overall rating distribution, it seems that the distribution of ranks is about gaussian.

In the revised rank distributions (of 2009 for example), I see there is not much skew (asymmetry) and each rank distribution happens to have a consistent width (FWHM) of about 160 rating points over the whole rating range and they are quite evenly spaced. Because of the consistency and regularity of these distributions, I feel this may be close to the natural, unbiased variation of go ranks.

The EGD rank distibutions are similar, but (again looking at 2009) around 1k the rank distributions tend to be wider (about 250 rating points), more skewed and about 50 points deflated. To me this hints at systemic biases, which I'm trying to eliminate in my revised system (while keeping it as simple as possible).

The revised rank distributions look quite unbiased to me, so from the statistics of all European players I cannot tell if the FFG policies leads to kyu rank deflation. At least it's not visible in the total European data.

I think it would be interesting to present rank distributions per country to see if there is any correlation between rank bias and national rank regulation policies.

I added rank distributions per country, and I don't really notice an big difference between countries with large go populations. For example, comparing France 2011 with Germany 2011 or Russia 2011, I don't notice a clear difference between strict rank regulation and no rank regulation. If anything, kyu rank regulation seems to be less harmful than I expected. Perhaps it is even beneficial to evenly distributed ranks.

I notice that countries with a smaller go population have a greater variance in rank distribution. But perhaps this is not surprising, because smaller numbers usually means greater statistical deviations.
But comparing medium sized go population countries like Czechia 2011 with Netherlands 2011, the Czech kyu ranks look more evenly distributed.

Very interesting,
Thank you for your work !



The truth is that nearly nobody understand how the rating system works, except the people in charge.

Players are aware of the existence of special adjustments, because they can see it in their rating, but the details are mostly unknown to them.

Thank you

I tweaked the rank distributions: I removed the normalization and I added extra buttons to jump to the first and last year.

I also added markers to show the amount of rank shift in the data.
For example, comparing 2012 EGD rating distribution with 2012 Revised rating distribution, the rank shift of the EGD data in the lower dan area become much clearer.

I find some curious differences in EGD rank shift per country. For example when I compare 2012 UK EGD with 2012 UK revised, the EGD distribution has a strong bias in the mid-dan region. But when I compare 2012 CZ EGD with 2012 CZ revised, the EGD distribution does not have much bias.

What could explain this difference? The Czech distribution shows a relatively low number of games in the mid-dan region, while the UK distribution shows a relatively high number of games in that region. So perhaps the 2012 UK mid-dan players were more affected by the EGD biases, simply because they were more active in 2012 than their Czech counterparts.

So I looked up a year where the Czech mid-dan players were active: 1998. Comparing 1998 CZ EGD with 1998 CZ revised, the EGD distribution does show a bigger bias in the mid-dan region.

I cannot say that this conjecture holds in all cases, but there is theoretical support for it, because of the the parameter values that the EGD uses, in particular a (see 1/a). Because of this choice of a, the expected odds of the EGD are especially unfavourable for players around 1d, causing them to lose rating points when they are in fact not underperforming. So if these players want to conserve their EGD rating, the best strategy would be not to play in tournaments. I tried to make my revised system neutral in this aspect.

I added rating lists, for example Spain: ES revised rating list