Hi all,

I'm Dave de Vos, a Dutch go player.

I wanted to investigate the EGD rating system. I attempted to make a revised system that fixes some issues that I notice in the EGD rating system.

The EGD rating system was originated by Aleš Cieply and it is explained here. The EGD manager Aldo Podavini kindly provided the game history from the EGD for me to play with. He also suggested to reverse engineer the EGD rating system and reproduce the EGD rating history and go from there to tweak it. That is what I did: http://goratings.eu. On the About page I explain a bit more. I used part of the introduction from that page here.

My main concern is the a function. It is used to compute an expected game result, so it should predict winrates reasonably well. But the expected winrates from the a function used by the system don't match all that well with observed winrates. (See 1/a predicted EGD vs 1/a observed EGD). The expected odds are about twice the observed odds, so the expectations of the EGD are clearly too high. Only around rating 100 and rating 2700 its predictions come closer to the observations.

This means that all players lose more than the system expects against a lower rated player and win more than the system expects against a higher rated player. Over time, this will contract the rating range: 1 grade difference will correspond to less than 100 points rating difference. The most frequent opponent has a rating of about 1700, and the frequency tapers off below and above. Because of this, I expect that the rating range will contract towards 1700. But this trend may be obscured by other deflation or inflation effects.

For some time, I've had this feeling that there is gradual deflation in the mid-dan region of a few rating points per year. I suspect that to some degree this deflation may be attributed to the the above cause. And even if it isn't, I see no reason to use a model that doesn't match with observations. So I implemented a revised rating system that uses an a function that matches the observed winrates better.

On the Player Rating History page you can compare the rating histories computed with this revised rating system. It is still anonymous, so you'll have to look up the player ID (PIN) on the EGD search page.

This site is still under construction. As it is now, it's just a quick and dirty contraption to share my thoughts and results. I'm still tweaking the system, so the charts may also evolve over time.

I welcome your questions, remarks, suggestions and other feedback.

I hope your work goes well, and that any positive changes you identify can be implemented.

I wondered for some time if it was possible to make a second iteration of the ratings algorithm to correct really out of date ranks. If exit_rating - entry_rating exceeds 100 (or perhaps 150) for 1 or more players in a tournament, then compensate this probability defying event by 1 re-iteration of the algorithm.

_________________
North Lecale

Nice project davos (and I remember many fun games together back in old OGS days). I too have the feeling at mid-dan that loses to weaker players happen more than the system expects: in many smaller UK tournaments I go to (not London open or Brit championship) I'm the highest rated by a stone or two, so I'm a 4d (2395) playing 1-2ds. For these sort of typical games I need to win something like seven games for every one I lose to maintain the same rating. I am a bit of a byo-yomi blunderer but I think even for a typical player that ratio is too high for such a strength difference. From a rating standpoint these tournaments end up rather "nothing to gain and everything to lose", but I don't care so much about that anymore. Also I think the EGD doesn't award enough rating points for beating much stronger players: if I beat someone my rating I get 8 points, if I beat some 2700+ super-strong like Ilya, Fan Hui, Hwang Inseong etc I just get 16 points. I feel that deserves a lot more points than 2 wins against a 4d, and is also likely indicative of an improving player: a normal 4d won't beat an 8d but an improving 4d who will be 5d or 6d soon might so boost their rating so as not to hurt/deflate the normal 4ds they are crushing along the way. Also the 2800 doesn't have to lose the same number of points (and indeed only loses ~10 in EGD) so such an event is a good opportunity to inject extra points into the system to reflect the growing total strength of the player population.

I don't think expected results should match actual results, because some of the player population is improving, and therefore they are underrated. In the simplest case consider a rating system with only two players. One is rated 10kyu and one 1dan. Suppose the 1dan player's true rating remains 1dan, and the 10kyu player's true rating rapidly improves to 1dan. They play even games, but the 10kyu will be winning more than expected.

Maybe the 'a' function does need changing, but I don't think you can use the method you described. To find the correct 'a' function by looking at actual results you need to only look at games between players that are already accurately rated. Even in that case there will always be some uncertainty in the ratings, and that uncertainty will cause expected vs actual to be slightly different. I think I read a paper on chess Elo studies about this but I can't find it now.

Maybe another way to look at the problem you see, given what I think problem is (improving players), if the rating system responded to improving players faster it would reduce the problem.

I thought that the idea was not that they should match, but that the fit should be slightly better than it currently is.

It would be quite nice to see some kind of histogram for the rating distribution at (say) 2006 and 2016 for the 2 models

_________________
North Lecale

I'm not sure I understand what you man by out of date ranks. I just reprocess the full game history of all players in every test run.


I haven't looked into details like that. I'm looking at the full EGD game history (12,000 tournaments, almost 900,000 games). I assume rating defying tournaments like that are rare, so I think they won't affect the statistics very much.

Thanks and yes, I remember them too


In a standard Elo rating system, the maximum points gained or lost in a game is determined by the K factor. In chess, it is usually around 24. Some systems use 16 for higher ratings and 32 for lower ratings.
In the EGD system, it is called the con factor, which ranges from 10 to about 100, with a value of 24 at 1d.

In my revised system I use similar values in the dan region, but it does not grow as big for lower ratings to reduce wild rating oscillations at lower ratings. You can compare http://goratings.eu/Probabilities/Points_EGD with http://goratings.eu/Probabilities/Points_Revised to see the difference. Those charts also include the epsilon term. The revised system used a bigger value for epsilon. It seemed neccessary to reduce deflation over the years.

I think it is not so small, looking at http://www.europeangodatabase.eu/EGD/cr ... dgob=false

_________________
North Lecale

Improving players are a source of deflation, lowering the ratings of the other players.
The EGD system has 2 mechanisms to handle this issue:

1: A go players enter a tournament with a declared rank. If a player improves quickly, he may skip a rank in the next tournament and the EGD will then reset that players rating to the new rank, so he won't have to earn those points (removing points from the system in the process). This is called a rating reset in the EGD system.

2: The EGD has an epsilon parameter which is intended to handle the issue of improving players. Every player gets some free points for every game to even out the points lost on average to improving players. It is implemented in such a way that lower rated players get more than higher rated players. Determining a good value for epsilon is tricky. You need to collect statistics to estimate the average points lost to improving players. My feeling is that the EGD uses a value that is too small, so I used a larger value. I chose it so that on average, it keeps a good match between declared ratings and computed ratings in the kyu range. My value tapers off in the mid-dan region to avoid inflating dan ratings to values greater than declared.

On average, it seems that declared ranks are a reasonable indication of peoples rating. I assume many kyu players who don't play many tournaments determine their declared ranks from casual handicap games against stronger players in their club. The EGD also contains handicap games, but I haven't come to analyze their statistics yet.

A good idea. I will do that.

I'm not sure how you can judge the relative size of epsilon from that list.

Sorry, I was not very clear there. If you sort the last rating change in this page you will find several instances of positive rating change over 100 points in size. I suppose that these come from 30kyu entered as 20kyu, or some guy who hasn't played in a rated event for N months but has improved several stones in strength in the meantime.

So my idea there is essentially to resubmit the tournament result with the starting rank for such players adjusted to the value with which they exited on the first application on the algorithm. I hope that's a bit clearer. It's what the old FFG system used to do.

_________________
North Lecale

Ok. I think your observation applies to the K factor rather than the epsilon parameter. The EGD uses a rather large K factor at low ratings that allows large oscillations like this (one could call it a quirk of the EGD rating system), but for higher ratings it is stable enough, I think.

I did use a smaller K factor and a larger epsilon to have smaller oscillations while still allowing the system to follow quickly improving players as good as the EGD. For example, this is the history of Mateusz Surma: http://goratings.eu/Home/History?PIN=12837968. But for quickly improving players, I think the EGD rating reset policy is quite effective (and for compensating the potential deflation from quickly improving players, it is more important than than the epsilon parameter). I used the same rating reset policy in my revised system.

But this policy is rather crude. The epsilon parameter tries to compensate for slowly improving players (improving less than 2 ranks between tournaments they participate in), of which there are many more than quickly improving players. The deflation effect of that is quite subtle (for the average of all active european tournament players, I estimate it at 2 rating points per year more then the EGD estimate).

A normal Elo system does not do iterations to find some equilibrium rating values and neither did I in my revised system. That kind of system sounds more like the WRH rating system from Rémi Coulom: https://www.goratings.org/en/. Rémi's ratings reflect the relative skill / succes of players to make a ranking list for the world's top players, but those ratings don't map to go ranks. For amateurs, a reliable mapping between ratings and go ranks is the most important feature of the EGD rating system IMO. I want to keep that feature and improve it. A normal Elo-like rating system or WRH-like system is not anchored. They are designed to maintain a ranking list where only the order and relative distance matters. They have no mechanism to compensate for subtle long term overall deflation or inflation. A mapping to go ranks that stays reliable over a 20 year time span is a different matter.

BTW, I'm still playing with the system, so some of these parameters will change.

This behaviour is normal in an Elo-like rating system. Elo (and EGD) award rating changes by probabilities and those don't go over 1 (times the player's K factor when converting to rating points change). For the behaviour that you'd like, the system should award rating point change by odds, not by probabilities. It would be more like a betting system than an Elo system. Perhaps your preference is not unexpected, because Brits seem to rather like betting .

But I think it's an interesting idea. I'll probably give it a try.

I added histograms for every year (since 1996) for both the EGD and the revised system: http://goratings.eu/Histograms.
I also fixed a bug and made some changes to the parameters.

I improved the histograms. They are now normalized rating distributions.
For example EGD rating distribution 2013 vs Revised rating distribution 2013.

When looking at the EGD distributions, it seems that over the years, many players graded around 1k (between 6k and 5d) have grown weaker than their rank.
In later years, this trend seems to be reversed a bit, but this could be due to players finally complying to the EGD rating system and demoting themselves.
But I suspect this is an artifact of the parameter values of the EGD system.

When choosing different parameter values (based on observations), the picture changes.
In the revised system, over the years, many players around 13k (between 18k and 8k) have grown stronger than their rank.
But I think it's understandable that they are conservative in promoting themselves, because that would mean 'disobeying' the EGD rating system.

Hi,
I'm in charge of the registration of new players in the club of Lyon (France).

The only relationship between the ranks and the real strength that is enforced by the system is that the bottom rank is 30 kyu (I won't talk about grades because we don't use the european system in France).
Besides that, it is we, people in charge of giving their first rank to new players, that may shift the entire scale up or down in comparison to an objective playing strength. If we underevaluate the level of the players, the whole scale goes down, if we overestimate their abilities, the whole scale goes up.

I personally use to register beginners at 20 kyu, but some people rather register them at 30 kyu. We have no directions in this matter.

For players who are not beginners, the most widely used estimation is the kgs rank, but with a correction. When I started, 3 years ago, I was told that there was about 3 kyu of difference between the KGS scale and the EGF scale.
But it seems that year after year, the difference between the scales is getting bigger in the 10 kyu region. 10 years ago, 8 kyu KGS was around 11 kyu EGF. Now it's rather around 13 kyu EGF.

A problem is that now that we know this, instead of registering new players 3 ranks below their KGS rank, like we used to, we tend to register them 5 kyu below...
Which in turn widens the gap between the two scales, since we are thus deflating the EGF scale. This is an endless loop. In 10 years, maybe we will have to register people 7 kyu below their KGS rank, which will in turn get the two scales 9 ranks apart...

OGS made a survey of their members ranks around all the ranking systems in the world.
It turned out that the KGS scale was higher than all other scales in the world in the kyu region, and that the Tygem scale was skewed relatively to all other scales (the "size" of their ranks is different).

If I understand correctly, your issue is not about deflation of EGD kyu ratings. Your issue is about deflation of KGS kyu ratings relative to real life European kyu ranks.

With any of these internet rating systems, it's not easy to verify that one is more true than the other. Only with lots of data one could verify that rating distances match handicap distances. I think the EGD is better in this aspect than KGS (but even the EGD can be improved on IMO, that's what I'm trying to do here), but ofcourse you cannot use the EGD when players started on the internet and improved there before they started playing in real life (in a club or tournament).

When players start playing in real life and have an unknown real life rank, I think the best way to establish their real life rank, is to have them play a dozen or so real life handicap games with players having a "known" real life rank to find the equilibrium rank of the new player. That is the way it was done before internet go servers and rating systems existed and I think it's still the preferred way to do it. Isn't this method the thing that defines real life go ranks?

I don't know how the KGS rating system works and I don't know if it's possible to download a game results table somewhere to collect statistics and derive its characteristics for conversion to real life European ranks. AFAIK, only anecdotal data exists to guestimate a conversion and as you say, it changes over the years. So I don't see a good way to fix this issue. I'll have a look at the OGS survey data (but if it is not recent, it may be obsolete soon as all these systems drift away from this one data point in time). Also, internet games tend to be rather quick and I don't know for sure what effect this has on handicaps, but I suspect that short time limits increase handicaps between players. So perhaps internet handicaps inherently have a poor connection to real life handicaps, which means that internet ranks map poorly to real life ranks, which tend to be based on longer time limits.

A different (but connected) problem from handicap distances is that ideally the ratings of players should not go up or down much over the years if their actual playing strength stays the same. In real life, it is easy enough: That player just keeps the same rank. But in computed rating systems, it's not so easy to prevent slow rating drifts over the years. How to establish if a player's strength stays the same? I can only assume that higher ranked players are more stable than lower ranked players, so overall fitting to minimize drift of dan ratings compared to declared ranks seems like a reasonable strategy. So that is what I'm doing.

BTW, I consider "grade" and "rank" synonyms.

Is that so? 3 years ago, I(in Germany) played my first tournament. I was 7k on KGS, so I registered as 8k and played 3:1. I believe it is fair to say that 8k was the right rank to register at given the local opposition, registering at 10k or even 12k would have been sandbagging.

The problem is more, that maybe in france or somewhere else, players of the same strength would call themselves 10k or 12k. And the rating system would encourage them to do so, because the rating system accepts self-estimated ranks way too much, the system that initial rating comes from self estimation instead of beeing calculated solely from performance in the first one or two tournaments, means that in different local societies ranks are skewed or shifted against one another, than also the ratings are shifted to fit the ranks, because new players bring ratings that fit the local rank scale. This way, the rating scale would need a lot more game of players from the different local regions against one another to uniformize, than would be needed, if ratings would work intrinsicly and not correct themselves to fit the rank people claim to have.

e.g, tell me why it was usefull to initialize this player at 2700 GoR:
http://www.europeangodatabase.eu/EGD/Pl ... y=18437485
Yes, he claimed to be 7d, but he played a lot of games at his first event, that clearly show, his strength is nowhere near 2700 GoR. A good rating system should calculate some kind of performance from his games and initialize him on that (would be around 2400 GoR maybe). There is no sense a initializing him on way too high rating and thereby gifting his opponent rating points (if your 5d and beat a 7d, that a lot points, in truth he was maybe a 4d EGD, so it wasnt that remarkable the 5d beat him).

As long as your revised history gives him clearly more current rating than the official rating, I dont think your revision improved the major issues

There are big differences in attitude/frequency of rating resets around Europe. I believe the EGD system allows (and perhaps even encourages them) for external changes of at least 2 ranks in strength. In Britain we do rating resets, but in other countries they are much less common and even frowned upon (e.g. Czechia, France). I seem to recall they aren't even allowed at dan grades in France (which also has its own internal rating system). I remember at the Cambridge University club we had a Czech student whose official rating was something like 13 kyu but he was 5 kyu in strength. He was adamant he shouldn't reset to 5 kyu: the rating system is not meant to reflect one's strength, it is points you have to earn and resets are cheating that.

Also as Scachus says if a new player's declared rating is obviously wrong you should change it when you submit the results after the tournament. We do try to do this in Britain but it can be hard if you don't realise after the first tournament (e.g. Stephen Hu AGA 5d entered his first British tournament as 3d and won all 4 games (but only 2d- opposition), and then 5-2 at London against stronger opposition so clear 4d was more appropriate but we didn't: I think because resets <2 ranks aren't allowed (though you can fiddle by swapping between English and Chinese names!) but maybe incompetence.