It is currently Thu Mar 28, 2024 9:14 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 115 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next
Author Message
Offline
 Post subject: Re: Revised European go ratings
Post #81 Posted: Mon Nov 06, 2017 12:43 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
In my kids club I use ranks down to 40k and Kalli Balduin (go teacher with currently 600 pupils in the Berlin area) even uses ranks down to 50k for his kids clubs. In my experience ranks below 20k are perfectly valid ranks when they are supported by handicap differences against stronger players.

So the revised system does not use a lower bound (for example, see http://goratings.eu/Home/History?PIN=16949867). With a liberal reset policy, not having a lower bound is perfectly fine as long as Go associations allow beginners to declare their rank freely in tournaments. When this is the case, the rating system doesn't need a special confidence factor for beginners (like a very large K factor).


Last edited by gennan on Mon Nov 06, 2017 1:39 pm, edited 8 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #82 Posted: Mon Nov 06, 2017 12:54 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Pio2001 wrote:
Do handicap games count as much as games without handicap ? (in France, they are weighted with a coefficient equal to 1-H/10, and if White looses, her variation is again multiplied by 1-H/10).


The EGD uses a weight of 1 for handicaps up to 9 stones (for both players). For higher handicaps it uses a weight of 0 (for both players).

The revised system uses a weight of 1 for all handicaps. The reason is that I think handicap games help to maintain long range calibration. Reducing their weights would throw away valuable information.

I did include a mechanism in the revised system to reverse engineer ranks below 20k from apparently overhandicapped games against '20k' players. But still, most of the information about ranks below 20k is just lost, I'm afraid.

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #83 Posted: Mon Nov 06, 2017 1:31 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
BTW, using the revised system's beta-function, I added a mapping to the Elo ratings used in DeepMind's papers (see today's Addendum on http://goratings.eu/Home/About)
According to this mapping, the different AlphaGo versions would have the following estimated ranks in the revised rating system:

Quote:
AlphaGo Fan 7p
AlphaGo Lee 12p
AlphaGo Master 18p
AlphaGo Zero 20p

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #84 Posted: Mon Nov 06, 2017 1:34 pm 
Lives in gote

Posts: 418
Liked others: 9
Was liked: 83
Rank: kgs 5 kyu
KGS: Pio2001
So, is your revised system complete ? Can we start to use it ? :geek:

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #85 Posted: Mon Nov 06, 2017 2:29 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Pio2001 wrote:
So, is your revised system complete ? Can we start to use it ? :geek:


I think the EGD is basically doing a decent job, so it never was my intention that goratings.eu would replace europeangodatabase.eu (I have no desire to become a system administrator of anything). I only made the revised system to test the validity of EGD ratings and to find potential improvements. I do think the revised ratings are closer to "the truth".

The revised ratings are getting stale, because it hasn't been updated with tournament data since 2017-08-19. I suppose that keeping it updated would not significantly change my conclusions, but it would be nicer to keep the revised ratings current. So I am considering to add a mechanism to update it automatically by scraping newer tournament data from the EGD.

Then again, I talked to several people about my little project (not just on this forum) and I get the impression that very few people care about improving the rating system, so perhaps I should just not bother and abandon it.

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #86 Posted: Mon Nov 06, 2017 3:34 pm 
Lives in gote

Posts: 418
Liked others: 9
Was liked: 83
Rank: kgs 5 kyu
KGS: Pio2001
That would be a shame.

Maybe what we need is to translate it into directly applicable guidelines :

New rules (auto-reevaluation, bottom rank etc)
A summary of the mechanism (to replace the page that describes the current system)
The list of changes from the previous system (the new values of "a", "con" and epsilon, etc)

And maybe the various national lists could then be joined into a unique european list. For example maintaining both the ratings of french players and european players means to have the work done twice.

I just discovered that I was currently 12 kyu in France, but with a 13 kyu rating and an 11 kyu ranking in Europe.


This post by Pio2001 was liked by: gennan
Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #87 Posted: Tue Nov 07, 2017 12:56 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Pio2001 wrote:
That would be a shame.
Maybe what we need is to translate it into directly applicable guidelines :

New rules (auto-reevaluation, bottom rank etc)
A summary of the mechanism (to replace the page that describes the current system)
The list of changes from the previous system (the new values of "a", "con" and epsilon, etc)

I kind of did that, but I guess the details are distributed over several pages and some details are only explained in this forum. Some editing could make it clearer, though. I could also publish the source code of the core of the system to remove any ambiguity about the details (if you can read C# source code). The core of the rating system is really quite small (I think less than 100 lines). I could even publish source code to reproduce the EGD ratings. The core of the EGD rating system may be a bit bigger, because it has more special rules, but not much.

Pio2001 wrote:
And maybe the various national lists could then be joined into a unique european list. For example maintaining both the ratings of french players and european players means to have the work done twice.

I just discovered that I was currently 12 kyu in France, but with a 13 kyu rating and an 11 kyu ranking in Europe.

I never saw the French rating list before, but I found it just now: http://ffg.jeudego.org/echelle/echelle_niveau.php
These are the French ratings according to the EGD: http://www.europeangodatabase.eu/EGD/createalleuro3.php?country=FR&dgob=false
These are the revised French ratings from 2017-08-19: http://goratings.eu/RatingList?country=FR
I suppose these different systems should agree with each other fairly well, if these lists if all are well calibrated and if all systems are fed with the same tournament data.
For your rank, the systems seem to be in agreement within a variation of about one rank, which is not bad. I suppose 13k is your EGF rank and 11k is your revised rank?

So what you suggest already exists, I think. I don't know which national go associations maintain their own rating system and why. Before I started my project I knew Belgium had their own rating system. Only on this forum I found that France also maintains their own rating system (which is almost the same as the EGD since 2013).
I think most countries don't have their own rating system, but some countries regulate ranks. For example, the Netherlands don't have a separate system, but we do have a classification committee (currently lead by HermanHiddema) that processes tournament results to regulate dan ranks. I know some countries regulate kyu ranks as well and other countries don't regulate ranks at all.

I don't know why the FFG maintains their own rating list. If they submit al results to the EGD, they could just use the EGD rating list. But I suppose the FFG wants to apply their own rules, which are somewhat different from the EGD system (like a lower bound of 30k, which modification has been requested to the EGF by the FFG, but it has not been granted yet).

Perhaps it's difficult to make one european rating system that complies to the different wishes of each national go association. Even if it were possible, I guess some national go associations will still want to use their own system because they just prefer to keep some sovereignty (even though it takes effort to maintain their own system).

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #88 Posted: Tue Nov 07, 2017 2:28 am 
Gosei

Posts: 1494
Liked others: 111
Was liked: 315
I think that the rating database deserves some improvements, so I hope your work isn't discarded. Isn't there a committee that is supposed to be investigating such projects?

The faux argument is often made that allowing ranks below 20 kyu will introduce inaccuracies into the system. You just have to look at the rating distribution to scoff at that - players are competing at 30 kyu in tournaments, and they are recorded instead as having competed at 20 kyu. There is no genuine problem to extending the system.

_________________
North Lecale

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #89 Posted: Wed Nov 08, 2017 1:36 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Javaness2 wrote:
I think that the rating database deserves some improvements, so I hope your work isn't discarded. Isn't there a committee that is supposed to be investigating such projects?

The faux argument is often made that allowing ranks below 20 kyu will introduce inaccuracies into the system. You just have to look at the rating distribution to scoff at that - players are competing at 30 kyu in tournaments, and they are recorded instead as having competed at 20 kyu. There is no genuine problem to extending the system.

From my emails with Aldo Podavini (the EGD manager since 2009) and reading the reports of the Annual General Meeting (AGM) of the EGF (see Minutes of Annual General Meetings at https://www.eurogofed.org/egf/) and a summary at https://www.eurogofed.org/egf/commissions.htm stating The AGM of 26th July 2012 agreed to reduce the number of EGF commissions [...] The Ratings Commission was suspended for review., it seems such a committee does not exist anymore. And from my emails with Aldo, I get the impression that the mathematical details of the rating calculations are not really his concern.

It seems this is the situation since about 2011: As the EGD manager, Aldo Podavini validates tournament result files, he reports annual participation trends to the EGF board and AGM meetings and he implements changes decided by the EGF board. But since its inception by Aleš Cieply in 1996, the calculations haven't changed much AFAICT. I can only find the increase of epsilon (from 0.014 to 0.016) in 2009 and some changes in the rules for tournament classification (which determines the weight factor of each tournament).

So perhaps I'd have to lobby the EGF board directly to have them take my suggestions in consideration? I kind of doubt that general executives would concern themselves with the mathematical details of rating calculations.

Perhaps I should just go ahead and scrape updates from the EGD and have goratings.eu coexist with the EGD.

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #90 Posted: Wed Nov 08, 2017 2:33 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Interestingly, I just found this document by Aleš Cieply from 2001: http://gobase.org/studying/articles/myong-ji/reports/Ales-Cieply.ps.gz. The format may be somewhat inconvenient, so I converted it to pdf and uploaded it here: http://goratings.eu/content/ales-cieply.pdf. (I think it's from 2001, because the modification data of the ps file inside is 2001-05-21)

Apparently, he did a similar statistical analysis of winning probabilities (with data from 1996 to 2001) as I did and he found the same discrepancy: see figure 2 on page 7 and compare it with http://goratings.eu/Probabilities (the dashed line in his figure is the black line in my figure, the solid line in his figure is the black line in my figure). He notes the discrepancy in the lower half of page 8, but he offers a different explanation in the upper half of page 9 (he thinks it's caused by the McMahon pairing system that most tournaments use), but he does offer a fitting function 1/a = λ => 7 / (3300 - rating), which is similar to what I used in an earlier version of the revised system. The EGD uses 1/a = λ => 20 / (4100 - rating).

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #91 Posted: Thu Nov 09, 2017 12:37 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Here you can compare the different functions: http://fooplot.com

Red is the EGD winrate prediction, black is my fit to observed winrates, blue is Ales' fit to observed winrates.


This post by gennan was liked by: ez4u
Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #92 Posted: Thu Nov 09, 2017 5:36 am 
Lives in gote

Posts: 418
Liked others: 9
Was liked: 83
Rank: kgs 5 kyu
KGS: Pio2001
I have linked to this discussion in the french forum http://go-on.forumactif.com/t3078-echel ... e-revision

Sylvain answered that the european ratings are linked to the national ratings in the following way (my translation) :

- Jean is 12 kyu in both the european and french lists.
- Jean registers in a tournament "A" at 12 kyu and wins 5 games out of 5.
- Results of the tournament A are taken into account in the french list. With the special adjustment iteration used in the french system, Jean is now 9 kyu in the french list.
- The results are also taken into account in the european list. The european rating of Jean is now 11 kyu.
- Jean registers as 9 kyu in the tournament B, and wins 2 games out of 5.
- Results of tournament B are taken into account in the french list. Jean looses some points, but remains 9 kyu.
- Results of tournament B are taken into account in the european list. His registration rank being 2 levels above his best european rank, his european level is resetted to 9 kyu before the calcuation of the variation of the tournament B (reset policy). He looses some points, but remains 9 kyu in the european list.


What Sylvain is saying is that in this example, it is the special rules of the french list that have an effect on the european list.
Here, the reset policy is not the ability of Jean to choose his registration rank, it is the automatic adjustments of the french ratings that are copied into the european list.

So the question is to what extend does the european list owe to its own rating calculation, and to what extend is it a mere compilation of the various national lists.


This post by Pio2001 was liked by: gennan
Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #93 Posted: Thu Nov 09, 2017 12:22 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Pio2001 wrote:
I have linked to this discussion in the french forum http://go-on.forumactif.com/t3078-echel ... e-revision

Sylvain answered that the european ratings are linked to the national ratings in the following way (my translation) :

- Jean is 12 kyu in both the european and french lists.
- Jean registers in a tournament "A" at 12 kyu and wins 5 games out of 5.
- Results of the tournament A are taken into account in the french list. With the special adjustment iteration used in the french system, Jean is now 9 kyu in the french list.
- The results are also taken into account in the european list. The european rating of Jean is now 11 kyu.
- Jean registers as 9 kyu in the tournament B, and wins 2 games out of 5.
- Results of tournament B are taken into account in the french list. Jean looses some points, but remains 9 kyu.
- Results of tournament B are taken into account in the european list. His registration rank being 2 levels above his best european rank, his european level is resetted to 9 kyu before the calcuation of the variation of the tournament B (reset policy). He looses some points, but remains 9 kyu in the european list.


What Sylvain is saying is that in this example, it is the special rules of the french list that have an effect on the european list.
Here, the reset policy is not the ability of Jean to choose his registration rank, it is the automatic adjustments of the french ratings that are copied into the european list.

So the question is to what extend does the european list owe to its own rating calculation, and to what extend is it a mere compilation of the various national lists.

As long as the declared rank does not trigger a reset, the European list (and the revised list) ows only to its own rating calculations.
When the declared ranks triggers a reset, the European list (and the revised list) ows to that declared rank. I don't think the system needs to know the reason for the promotion. In some countries it's a self-promotion and in other countries it's a promotion by the National Go Association.

In your example, Jean is 12k in tournament A and he is 9k in tournament B (in all the systems). The only difference between the French rating list and the European rating list is the Jean's rank between tournament A and B. That doesn't mean much for the rating system, but if it's important for Jean's National Go Association, they could send an email to the EGD manager to adjust Jean's rank to 9k before tournament B (I assume that would be the normal procedure).

If this happens all the time and if it's really important to adjust ranks between tournaments, the National Go Association could ask the EGD manager to make a special web page where the National Go Association can manually reset their players ranks without the EGD manager's help.


Last edited by gennan on Thu Nov 09, 2017 12:46 pm, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #94 Posted: Thu Nov 09, 2017 12:36 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Pio2001 wrote:
I have linked to this discussion in the french forum http://go-on.forumactif.com/t3078-echel ... e-revision

I just read the discussion (using Google Translate). I thank you for calling attention to my investigation and explaining my findings so clearly!


Last edited by gennan on Thu Nov 09, 2017 1:54 pm, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #95 Posted: Thu Nov 09, 2017 12:58 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Pio2001 wrote:
What Sylvain is saying is that in this example, it is the special rules of the french list that have an effect on the european list.
Here, the reset policy is not the ability of Jean to choose his registration rank, it is the automatic adjustments of the french ratings that are copied into the european list.

So the question is to what extend does the european list owe to its own rating calculation, and to what extend is it a mere compilation of the various national lists.

From your description of the French system it seems that one of its purposes is to be a platform where the FFG officially publishes player promotions.

I don't think that's a purpose of the European list. It just follows players' declared ranks and their tournament games results.

Also, I don't think various national lists exist (I know only of France and Belgium). As far as I know, most countries just use the European list.

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #96 Posted: Thu Nov 09, 2017 2:53 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Pio2001 wrote:
I have linked to this discussion in the french forum http://go-on.forumactif.com/t3078-echel ... e-revision

I have one correction to your explanation on that forum:

You state that I found that two times 3 stones handicap equals 6 stones handicap, but that is not really what I found. I found that two times 3 stones handicap equals two times 250 rating points difference = 500 points, which corresponds to 5.5 stones handicap (6 stones handicap with komi for white), not 6 stones handicap.

The EGD and the revised system use this formula: Rating difference = (handicap - 0.5) * 100, because theoretically, the first handicap stone is only half a move advantage: it's the same as an even game without komi. My finding is that this formula is consistent with the observed statistics of handicap games (although I cannot say this with high accuracy / confidence, because the data is a bit too sparse for high handicaps).

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #97 Posted: Thu Nov 09, 2017 3:16 pm 
Lives in gote

Posts: 418
Liked others: 9
Was liked: 83
Rank: kgs 5 kyu
KGS: Pio2001
gennan wrote:
Also, I don't think various national lists exist (I know only of France and Belgium). As far as I know, most countries just use the European list.


Ah, that's good.
My concern was that we can't tell a "declared rank" from a "national rating overwriting a European rating".
If all resets would have come from national calculations (and not from players declaration), and if all countries suddenly stop using their rating system and replace it with yours, then all resets would have disappeared from the system.

In France, it has not been allowed to register in a tournament at a different rank than the one published by the French federation (if it exists) since 2010.
It is possible to ask for a reevaluation if the rank of a player becomes too different from his/her real strength, but this is a heavy process : the reevaluation can only by asked by the administrator of a club, and must be supported by sgf files proving the real level of the player.
In 2017, a total of 10 reevaluations have been made. There were 8 in 2016, and 22 in 2015. All other "resets" are just the French system calculating a different rating than the European one.

gennan wrote:
From your description of the French system it seems that one of its purposes is to be a platform where the FFG officially publishes player promotions.


The French rating system was created in 1981 and have been the official rating list used in France since then. It has undergone several transformations. The use of an ELO-Style core calculation, borrowed from the EGF, came in 2010.

I have been in charge of the pairing and the results of local tournaments for two years. I discovered the existence of the European rating list in my pairing software, And I learned that someone was calculating my rating in the European list just now, reading this thread.
Since I never communicate our tournament results to anyone else than the French federation, I was assuming that someone was just copying the French ratings into the European list !

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #98 Posted: Thu Nov 09, 2017 3:59 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Pio2001 wrote:
Since I never communicate our tournament results to anyone else than the French federation, I was assuming that someone was just copying the French ratings into the European list !


I assume that the FFG communicates your tournament results (and all other French tournament results) to the EGD.

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #99 Posted: Fri Nov 10, 2017 1:39 am 
Beginner

Posts: 10
Liked others: 0
Was liked: 13
Rank: EGF 2 kyu
gennan wrote:
Also, I don't think various national lists exist (I know only of France and Belgium). As far as I know, most countries just use the European list.

Ukraine also has its own ranking system. As far as I know, there are three major differences from EGF ratings:
  1. Dropping for more than 100 points is allowed.
  2. After-tournament ratings are calculated sequentially, i.e. ratings "change" during the tournament. However, this doesn't affect seedings.
  3. There is a term "abnormal tournament result". If player performed in tournament far better than his(her) rating, then he(she) gets a new one (the result of calculation) and the whole tournament is recalculated. This protects other players from losing too many points. Also, the "abnormality threshold" is different for every initial rating.

Top
 Profile  
 
Offline
 Post subject: Re: Revised European go ratings
Post #100 Posted: Fri Nov 10, 2017 2:49 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
gennan wrote:
Also, I don't think various national lists exist (I know only of France and Belgium). As far as I know, most countries just use the European list.

In Britain we use the European rating system, but we used to apply a linear transform to convert an EGF rating to a British dan/kyu rank. We would look at the average rating of a player declared as a 1 dan in Europe (it was a bit less than the theoretical 2100) and also the average rating gap between declared ranks (not sure exactly over what range, it was a bit less than the theoretical 100) and then apply extrapolate these, e.g. 1 dan is 2070 and 95 points per rank so 3 dan is 2070 + 2*95 = 2260 rather than 2300. However, this adjustment was abandoned earlier this year.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 115 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group