It is currently Thu Mar 28, 2024 2:40 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 75 posts ]  Go to page Previous  1, 2, 3, 4  Next
Author Message
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #41 Posted: Tue Apr 06, 2021 1:11 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Adin wrote:
I've noticed an important change, points gain/loss seem to be much lower than before. For example some years ago I had a nice tournament where i gained 109 points rising from 2k to 1k. But in the new system I only gained 59 points.

It was mentioned before in this thread that on average a player has roughly 10 rated games/year, which means roughly 2 tournaments. Now that it's become more difficult to gain/lose points and with players having 2 tournaments/year on average, doesn't it mean that the gap between real strength and EGF rating risks getting bigger? Essentially having a lot of underrated players (due to slowed down rating increase not being able to match improvement) and also having more overrated players.


1) We felt that ratings of lower rated players were overly volatile in the original system, so their con/K factor was reduced.

2) The system is not clairvoyant. It has no way of knowing if a player has improved 3 stones between 2 tournaments 1 year apart. When players improve so quickly that their calculated rating doesn't keep up (because they play few tournaments relative to their improvement rate), they should just declare their new rank in the next tournament instead of sandbagging with their obsolete rank. When their new rank is 2+ ranks higher than their previous highest declared rank, this triggers an automatic rating reset in the system, so their rating is just updated to their new rank before updating it with the tournament game results.
This rating reset mechanism already existed before the rating system update. However, some countries have quite conservative policies in this regard and don't allow it, even for low rated players, effectively forcing those players to sandbag until the rating system catches up.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #42 Posted: Tue Apr 06, 2021 1:29 am 
Dies in gote

Posts: 28
Liked others: 0
Was liked: 2
Rank: 1 kyu
gennan wrote:
some countries have quite conservative policies in this regard and don't allow it, even for low rated players, effectively forcing those players to sandbag until the rating system catches up.

This is a very good reason to not make it even harder for those players by lowering the rating gains.

In real life players are much more likely to overrate themselves. They will happily think they are stronger than they truly are. The new system will reinforce the idea that "I'm actually much stronger and I can't go up fast enough by playing so I need to declare a stronger rank". Then as the rating losses are also significantly reduced, the overrated player will take a longer time to reach its real rating.

Essentially instead of rewarding or punishing players based on objective results in games the new system promotes setting your own rating based on subjective opinions.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #43 Posted: Tue Apr 06, 2021 1:41 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
Quote:
In real life players are much more likely to overrate themselves. They will happily think they are stronger than they truly are. The new system will reinforce the idea that "I'm actually much stronger and I can't go up fast enough by playing so I need to declare a stronger rank".


Not sure about that. Even though my online ranks are still improving, and even though my EGD rank is a bit underrated, the improvement rate is not spectacular so I keep thinking that I am stuck, especially after losing against a weaker player. So I wouldn't declare a stronger rank at the next tournament even if it is allowed.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #44 Posted: Tue Apr 06, 2021 1:49 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Adin wrote:
gennan wrote:
some countries have quite conservative policies in this regard and don't allow it, even for low rated players, effectively forcing those players to sandbag until the rating system catches up.

This is a very good reason to not make it even harder for those players by lowering the rating gains.

In real life players are much more likely to overrate themselves. They will happily think they are stronger than they truly are. The new system will reinforce the idea that "I'm actually much stronger and I can't go up fast enough by playing so I need to declare a stronger rank". Then as the rating losses are also significantly reduced, the overrated player will take a longer time to reach its real rating.

Essentially instead of rewarding or punishing players based on objective results in games the new system promotes setting your own rating based on subjective opinions.

There are both optimistic and pessimistic players when it comes to self-promotions (this was also the case before the rating system existed). The rating system has a rating bonus to compensate for any imbalance between those types of players.
Your gut feeling may be that players tend to be too optimistic on average, but the overall statistics of the rating system indicate that it needs a positive rating bonus instead of a negative rating bonus to minimize long term overall deflation/inflation of the system. This supports the view that on average, European players tend to be (slightly more) pessimistic rather than optimistic in regard to self-promotions.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #45 Posted: Tue Apr 06, 2021 2:06 am 
Dies in gote

Posts: 28
Liked others: 0
Was liked: 2
Rank: 1 kyu
Trust me, as a KGS admin for almost 20 years I've seen how much some players are willing to do just to increase in rank. From all kinds of dishonest stuff to things like specifically choosing their opponents based on profiles that they think give then better chance of winning. I'm not saying that most players are like that but certainly most players want to have a better rank. I've also seen a lot of offline/EGF rank special promotions that made no sense approved by my national federation (Romanian one). I even made an analysis shared with Romanian players community that showed how the majority of special EGF rank promotions have proven to be overrated over the following years. It's human nature to want to appear better to others.

Ideally ranks should be based just on game results. That's not always possible in offline ranks, but in my opinion lowering the impact that games have on rating is a big step in the wrong direction.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #46 Posted: Tue Apr 06, 2021 5:40 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
So when comparing airbagging and sandbagging, airbagging is the bigger issue on KGS?
That's funny, because on OGS the general perception seems to be that sandbagging is the bigger issue (at least in their forums I don't see nearly as many complaints about airbagging as I see about sandbagging).

Ignoring malicious airbagging/sandbagging, both overly optimistic and overly pessimistic players (both in good faith) have always existed. And I don't think this is a big problem, as long as those populations balance each other out. If a player underpromotes or overpromotes themselves, their rating will settle to a more realistic value soon enough (but perhaps opinions will differ on how soon is soon enough).

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #47 Posted: Tue Apr 06, 2021 6:38 am 
Dies in gote

Posts: 28
Liked others: 0
Was liked: 2
Rank: 1 kyu
Yes, for sure more overrated players than sandbaggers. Specially since AI became widely available cheating with it became a huge issue. Not just on KGS of course, people are largely underestimating how serious this is in all online games. KGS is dealing with this and it will be kept under control. EGF should also take it very seriously.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #48 Posted: Tue Apr 06, 2021 8:34 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
I don't think it's really fair to conflate concerns of KGS ratings with concerns of EGF ratings. Each of those rating systems has their own specific concerns (at least outside of online EGF rated events in pandemic times).

1: I feel that airbagging and cheating are two diferent things. Cheating is more directed at gratification from beating high dan players, while not caring to play by yourself.
I think airbagging is inflating your online rating by exploiting some loopholes of online play (picking opponents, perhaps exploitable bots, but not neccessarily cheating with an AI). I suppose the gratification is that your inflated rating allows you to be matched against stronger players than usual, but you will still play by yourself (mostly?).

2: I think neither cheating, nor airbagging is very common in IRL tournament games. So I feel that these are not great concerns for the EGF rating system.
Cheating is not so easy IRL. Yes, it may happen every once in a while, but that won't compromise the rating system much overall.
Airbagging is also not easy IRL. For one, you don't get to pick your opponents. Yes, you may get away with a malicious rating reset once in a while. But that only gets you so far and you cannot repeat that trick over and over. Participating in IRL tournaments is not anonymous, so it's easy for a tournament organiser to check your EGF rating history when you register and they should notice when strange things are going on in there. For example, if you double promote in 5 consecutive tournaments while scoring 0 wins, it should raise some eyebrows.

But in any case, responsibilties to check registrations and prevent cheating belong to tournament organisers. It is their responsibilty to feed the rating system with reliable data. It's not the responsibility of the rating system itself and these issues are outside the scope of this particular rating system commission.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #49 Posted: Tue Apr 06, 2021 11:36 am 
Dies in gote

Posts: 28
Liked others: 0
Was liked: 2
Rank: 1 kyu
This is my new EGD profile: https://www.europeangodatabase.eu/EGD/P ... y=14537380

The fact that tournaments make you go up or down much less than before makes it that for look like for 8 years I've been fairly stable 1k and next 6 years pretty much stable 1d. That is simply not true. My strength did go up and down mostly due to circumstances in my life when I was more or less able to focus on Go. It certainly wasn't as smooth as this graph looks.

The old graph is not only more accurate but also much more exciting: https://www.europeangodatabase.eu/EGD21 ... y=14537380 It has a moment when I'm nearly dropping to 4k and that is when I had some health problems that affected my play. It has another moment when I'm going up more than 100 points in a 5 games tournament which is certainly not easy when you're 2k and it gave me a morale boost.

You might think that "excitement" does not matter as long as the ranks are on average statistically accurate. But most people don't travel for years to offline tournaments to say: "yeah, I've started as 1k and 5 years later I'm still 1k, never went up or down in rank, yay!" That is just not motivating as far as rating is concerned. It's like gambling with pennies, win or lose you do not care as much.


This post by Adin was liked by: gennan
Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #50 Posted: Tue Apr 06, 2021 1:14 pm 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
You gained 100 points because you won 5/5 games in a tournament, and most of your opponents were stronger players. Wouldn't that already be enough to boost your morale?

Conversely, if you lose all your games in a tournament, wouldn't that be already depressing enough? Why would you insist on losing >100 points in addition? (This happened to me once...)

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #51 Posted: Tue Apr 06, 2021 1:30 pm 
Gosei

Posts: 1494
Liked others: 111
Was liked: 315
A big thank you for updating the rating system

_________________
North Lecale


This post by Javaness2 was liked by: gennan
Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #52 Posted: Tue Apr 06, 2021 1:55 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Adin wrote:
This is my new EGD profile: https://www.europeangodatabase.eu/EGD/P ... y=14537380

The fact that tournaments make you go up or down much less than before makes it that for look like for 8 years I've been fairly stable 1k and next 6 years pretty much stable 1d. That is simply not true. My strength did go up and down mostly due to circumstances in my life when I was more or less able to focus on Go. It certainly wasn't as smooth as this graph looks.

The old graph is not only more accurate but also much more exciting: https://www.europeangodatabase.eu/EGD21 ... y=14537380 It has a moment when I'm nearly dropping to 4k and that is when I had some health problems that affected my play. It has another moment when I'm going up more than 100 points in a 5 games tournament which is certainly not easy when you're 2k and it gave me a morale boost.

You might think that "excitement" does not matter as long as the ranks are on average statistically accurate. But most people don't travel for years to offline tournaments to say: "yeah, I've started as 1k and 5 years later I'm still 1k, never went up or down in rank, yay!" That is just not motivating as far as rating is concerned. It's like gambling with pennies, win or lose you do not care as much.

So you prefer volatile ratings, because that reflects ups and downs in your life better than more stable ratings. I can understand that.

But I think there is also something to say for more stable ratings. At least when your rating reaches a new high level, you can be more confident that you have really reached that level with some consistency, instead of it being just some random noise, a fluke or a "lucky" draw of opponents in a single tournament.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #53 Posted: Tue Apr 06, 2021 4:08 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Adin wrote:
gennan wrote:
some countries have quite conservative policies in this regard and don't allow it, even for low rated players, effectively forcing those players to sandbag until the rating system catches up.

This is a very good reason to not make it even harder for those players by lowering the rating gains.

In real life players are much more likely to overrate themselves. They will happily think they are stronger than they truly are. The new system will reinforce the idea that "I'm actually much stronger and I can't go up fast enough by playing so I need to declare a stronger rank". Then as the rating losses are also significantly reduced, the overrated player will take a longer time to reach its real rating.

Essentially instead of rewarding or punishing players based on objective results in games the new system promotes setting your own rating based on subjective opinions.

If you check the new rating histories of young European top players, such as Ilja Shikshin or Artem Kachanovskyi, and compare with the old rating histories, you can see for yourself that the new version is quite capable of keeping up even with the progress of prodigies.
It's just generating less noise than the old version, while tracking their progress.

So I don't think it's really justified to assume that the new lower volatility means that it cannot keep up with rapid progress.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #54 Posted: Wed Apr 07, 2021 12:09 am 
Dies in gote

Posts: 28
Liked others: 0
Was liked: 2
Rank: 1 kyu
I don't understand why you see these two as good examples of rank progression with the new system. Ilja started as 4k and went to 262 tournaments. Artem had a special promotion to 6k very early in his career and went to almost 200 tournaments. It's obvious that when you start around 5k and go to hundreds of tournaments then it's possible to rise to high dan level.

The real issue is with the average player who goes to 2-4 offline tournaments a year (even after corona is over). If he is a new player now starting at 30k it might take years for his EGF rank to catch up to his real strength. You are saying that he can just request a special promotion as Artem did, but there are a couple of issues with that. The first is that as you said some countries do not allow you special promotion and just force you to sandbag. At least before the higher volatility allowed a shorter sandbag period. The second is that quite a lot of players would gladly take advantage of special promotions becoming more common to overrate themselves. Don't think only in terms of countries where the average age of players is 30-40 and they are mostly well educated adults. Countries like Romania have *a lot* of kids and teens that see their rank as a way to stand up among peers and some of them are ready to use any opportunity to raise it regardless of real strength.

Even taking a player like myself who started as 2k as I was playing a lot online before going to offline competitions and who did not have very large rank changes. If I had just floated from 1k to 1d for 14 years as in my new graph I would get no motivation from my rating. Going down can hurt but that hurt is common in all sports and it's necessary to motivate you to improve. Also going up is obviously a motivation factor. You might say that's just a personal preference and other people prefer stability to feel comfortable. That may be true but the real issue is that the new rank graph does not reflect reality. In these 14 years I was playing a lot online and certainly did not have a smooth ride there just floating around the same rank. Even though the sample size was hugely larger online I was going up and down quite a lot as my play certainly was changing and it still does today as I use a lot of AI for review lately. What you are doing is just cushioning the real raises and falls and creating a much flatter line that in statistics will look good (after all someone going from 3k to 2d can be well approximated for statistical purposes as a stable 1k) but hides the real evolution of the player.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #55 Posted: Wed Apr 07, 2021 1:02 am 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
To allow a quicker rating increase, the volatility for low ratings is much larger than the volatility for high ratings (this was also the case in the old system). This volatility is determined by the con function (in chess Elo rating systems this is called K and it also typically varies from large values for low ratings to low values for high ratings).

I tried to look up a young Romanian DDK player that didn't use rating resets (special promotions) to compare the old with the new rating history. This may be an example that meets your criteria: https://www.europeangodatabase.eu/EGD/Player_Card.php?&key=18437639, but I still fail to see that the lower volatility is a problem. The volatility of their new rating history is a bit smaller, but it is still large enough to keep up.

As for higher volatility giving higher motivation. I think that's very much a matter of personal taste. I understand that you prefer a high volatility, but I don't feel that motivation of players should be a design concern for rating systems. We cannot make a rating system where everybody gets to personalize it to their own preferences or their own perception of reality. The rating system merely exists to track the level of all European tournament go players, to allow tournament organisers to make good tournament pairings where players get to play opponents of a similar level.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #56 Posted: Wed Apr 07, 2021 1:30 am 
Gosei

Posts: 1494
Liked others: 111
Was liked: 315
"The real issue is with the average player who goes to 2-4 offline tournaments a year (even after corona is over). If he is a new player now starting at 30k it might take years for his EGF rank to catch up to his real strength"

Sorry, but why wouldn't he just enter his next tournament at a rank which matched his actual strength?
We know that we can sometimes see kids improve 5 ranks around the level of 30k to 20k practically overnight, something just clicks in their understanding of the game. The stronger players at the local club can surely manage that kind of change.

_________________
North Lecale


This post by Javaness2 was liked by 2 people: gennan, silk
Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #57 Posted: Wed Apr 07, 2021 2:33 am 
Dies in gote

Posts: 28
Liked others: 0
Was liked: 2
Rank: 1 kyu
gennan wrote:
I tried to look up a young Romanian DDK player that didn't use rating resets (special promotions) to compare the old with the new rating history. This may be an example that meets your criteria: https://www.europeangodatabase.eu/EGD/P ... y=18437639, but I still fail to see that the lower volatility is a problem. The volatility of their new rating history is a bit smaller, but it is still large enough to keep up.

That's a player who took part in 29 tournaments over 3 years and a bit. Certainly 9 tournaments per year is way above average.
Javaness2 wrote:
Sorry, but why wouldn't he just enter his next tournament at a rank which matched his actual strength?

Because, as gennan said, some countries do not allow you to do it. And it gets even worse now because you are starting at 30k, not 20k.

And for countries that do allow you to set a higher rank, the lower volatility is likely to become an excuse for players overrating themselves. They will say: "look I'm winning but it's too hard to reach my real level by playing because the points gain per win are too small". I already heard that with the old system.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #58 Posted: Wed Apr 07, 2021 3:03 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
The French go Federation allows rating resets, but it has to be approved by a committee:

https://ffg.jeudego.org/informations/of ... uation.php

In the 11k-30k range, only resets by 4 levels or more are allowed. If an application is rejected, then the player has to wait 3 months before applying again.

Maybe other countries should adopt a similar rule.

Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #59 Posted: Wed Apr 07, 2021 3:35 am 
Gosei

Posts: 1494
Liked others: 111
Was liked: 315
I really think that the national federations have to reform if they think rating resets are something special in the outer kyu range. Rating volatility can never fully compensate for completely out of date ratings. The system is not designed to do that.

I know that the Romanian Go Federation has already had some fun discussing who is in control, which rules are right and which are wrong, etc. Here is another opportunity to create some fruitful discussions to bring joy to the community. :)

Seriously though, the EGF does need to standardize the practices of its federations regarding their use of the rating system. For instance
    Best practice on submission of rating results - some member nations have never submitted tournament results
    Produce clear guidelines on the limitations of the rating system - i.e. why rating resets are necessary
    Agreement on modality of rating resets. Example: a 10k has a record of 10k+ 9k+ 7k+ 6k+ 5k+ in an event. The supreme beings decide he will be promoted to 7k. Is it valid to change his entry rank to 7k before submitting the tournament file for rating, or can it only be reset for their next tournament.

_________________
North Lecale


This post by Javaness2 was liked by 2 people: gennan, Harleqin
Top
 Profile  
 
Offline
 Post subject: Re: EGF Rating System Commission Report 2020
Post #60 Posted: Wed Apr 07, 2021 5:05 am 
Dies in gote

Posts: 26
Location: Karlsruhe/Germany
Liked others: 1
Was liked: 6
jlt wrote:
The French go Federation …

In the 11k-30k range, only resets by 4 levels or more are allowed.


two points.

It is sad, that it's different from the European rule (that allows 2 stones).

But having Sandbaggers by design means, that people who do not like sandbagging, will not join the tournament.

When I promote Tournaments in France I always get complaints about the French rating and most players do not like sandbaggers.

As a tournament director I want players having good games and I accept ratingreset, if comprehensible (an online rating per se is not comprehensible).

Greetings from Germany

Wilhelm


This post by karlsgo was liked by: Harleqin
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 75 posts ]  Go to page Previous  1, 2, 3, 4  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group