KGS Ranking adjustment?

[Bill Spight]

For a random thought experiment I could even imagine a system that does not throw away most of this data, ie. a server that ignores komi, records each game as board results B+3, B+8, W+1 etc, and manages player ratings using the whole result from each game (instead of truncating to 1-1 bits, which is dubious for H1).

Now I think I know why you think that there is something special about H1. You have been misled by those who say that the proper handicap for a one amateur rank difference is for the lower ranked player to take Black with ½ pt. komi. In fact, the proper handicap is for White to give komi. That is so for all handicaps, not just H1. Given incorrect knowledge of proper handicaps, there is something special about H1 with Black giving a 6½ or 7½ pt. komi.

Should there be a rating system based upon gradations of komi? Such a system is certainly possible. It is unclear how much that buys you, however, given the irreducible uncertainty of ratings, especially amateur ratings.

[John Fairbairn]

Because the traditional handicap for a one amateur rank difference was for the lower ranked player to take Black without komi, 50 years ago most amateurs did not know that that was the wrong handicap.

That's not true in any respect.

(a) In traditional play there were no separate amateur ranks. They used the same grade systems as pros.

(b) They had non-komi handicaps using colour alternation in series of 2,3 or 4 games. That's why they had grades. The system is well known here from Japan, but it was common in China. Indeed they must have invented it. Grades are mentioned as early as the Wei dynasty (3rd century), and as for the mention of series, we have this explained from the hand of Ming Dynasty author Xu Zhongye (Ming was 1368~1644), e.g. "A grade 1 [=9d] gives him [a grade 2 = 8d] first move in two out of three games."

(c) After WWII separate amateurs emerged as a result of the 1949 Japanese rules and their first tournaments used 4.5 komi.

(d) Both amateurs and professionals in Japan knew early on that a komi handicap was needed to make a single game properly equal. The earliest known komi game is 1751 involving what we may call amateurs (i.e. unofficial players), though go may have been their professionals gamblers. And in 1759 we have an example of White giving komi. Shusaku and Shuwa also famously discussed komi and the earliest komi game (1832) involving pros includes Shuwa.

(e) Western amateurs knew about komi at the very least in the 1960s (first international amateur tournaments).

(f) What no-one really knew was simply how much komi should be. Setting it at around 5 points was probably out of deference to the two Shus, but smaller komis were tried, especially in the 30s, though sometimes in conjunction with the series-type handicap.

No AI bots were harmed in the writing of this post.

[jann]

Now I think I know why you think that there is something special about H1. You have been misled by those who say that the proper handicap for a one amateur rank difference is for the lower ranked player to take Black with ½ pt. komi.

Not at all, I'm perfectly aware of the half stone (komi) error of traditional handicaps. And as you wrote, this equally affects all handicaps (and is easy for rating math to account for).

What's special about H1 games is that both komi and no-komi is very common for half rank difference, open for the players to choose, and in fact I have seen several cases where one player were even unaware if the game he plays has komi or not (set automatically). The players simply play the same game on the board, but the recorded 1-bit result (which is all a typical rating system sees) is determined by this (external) setting for the B+1 - B+6 range.

[Bill Spight]

Now I think I know why you think that there is something special about H1. You have been misled by those who say that the proper handicap for a one amateur rank difference is for the lower ranked player to take Black with ½ pt. komi.

Not at all, I'm perfectly aware of the half stone (komi) error of traditional handicaps. And as you wrote, this equally affects all handicaps (and is easy for rating math to account for).

What's special about H1 games is that both komi and no-komi is very common for half rank difference, open for the players to choose,

Well then there is your problem, right? That's why you say that the choice of komi is arbitrary.

[Bill Spight]

Because the traditional handicap for a one amateur rank difference was for the lower ranked player to take Black without komi, 50 years ago most amateurs did not know that that was the wrong handicap.

That's not true in any respect.

(a) In traditional play there were no separate amateur ranks. They used the same grade systems as pros.

Then we are talking about different traditions. I'm talking about the one adopted later by amateurs.

IMX, amateurs in both Japan and the US typically used Black first as the correct handicap for a one stone difference in strength. (Edit: At no time in my experience did any amateur, whether dan or kyu, whether in Tokyo, Kyoto, Los Angeles, New York, San Francisco, or Santa Fe, whether at the Nihon Kiin or Kiin associated clubs, or informally, at home, tell me that taking Black with no komi was not the correct handicap for a one stone amateur rank difference. No pro told me that, either.) Finally, in the late 1970s I introduced White giving komi in New Mexico. Even as recently as the late 1990s on rec.games.go I had to argue that simply letting Black play first was the wrong handicap for a one amateur rank difference. Don't tell me there was no such amateur tradition.

[John Fairbairn]

Then we are talking about different traditions. I'm talking about the one adopted later by amateurs

Well, I find it spooky to talk about something that happened in my lifetime (and yours) as a tradition, especially in a field that is already very ancient. There is something similar perhaps going in the antiques field. We have tv programmes here in which experts bemoan the trend to class objects 50 (or even 25) years old as antiques, instead of using the 100-year criterion that has long been used in their field. By that modern trend both you and I would be classed as antiques. I don't feel like one, and I very strongly suspect you don't either (maybe for simplification we'll leave getting out of bed in the morning out of it )

But that was just scene setting. My main focus was

50 years ago most amateurs did not know that that was the wrong handicap

That would have been around the time I started go. Early on then, I came across the German system of counting grades with simple numbers, 1 down to whatever, splitting each Japanese grade in two and making no distinction between dan and kyu. I think 1-dan was 18/19 from memory.

Japanese visitors found this (and other novelties) very peculiar and would write articles in Kido and so on, describing western practices in the style of "dogs there can walk on their hindlegs." The thought police hadn't reached there yet. But even so they were a bit misguided, because they didn't take enough account of the fact that we had relatively few opponents, and so had to play lots of handicap games. They also overlooked the fact that we had few ways of getting a reliable rating, and in particular few ways of changing a rating quickly. The German system, sporadically adopted elsewhere, was an attempt to provide a workable ratings framework for Europe.

It didn't really work - for several reasons, I suppose. It was a creature of mathematicians, and not all go players were mathematicians. Most other players seemed in thrall to the Japanese system and just mimicked it, without properly realising it was not fit for purpose here perhaps, or, much more likely, if they did realise, they didn't care - "we are just weak amateurs, really" (and we really were then!) and "it's a storm in a teacup." I certainly was in the latter don't-care camp.

Yet I believe most of us - even someone as bored by decimal points as me - still registered what the mathematicians were saying: that Black as a one-stone handicap is skewgee, and, in addition, that the rest of the Japanese handicap scale wasn't anywhere near linear. There was even a translated book - wasn't there? - in which Ishida Yoshio played other pros at all the various handicaps, and showed the lack of linearity. We knew we were surrounded by imperfections.

So, my contention is that "50 years ago most amateurs did not know that that was the wrong handicap" should have read "50 years ago most amateurs did not CARE that that was the wrong handicap."

I would also go further and say that is even more the case nowadays, since we can now find opponents easily and don't need to rely on a handicap system to get a rating.

That might not seem to marry with the representation of the go world currently given by L19, but I still do believe most go players in the real world are still not mathematicians and computer scientists.

[Bill Spight]

By that modern trend both you and I would be classed as antiques.

Watch it, Sonny!

I don't feel like one, and I very strongly suspect you don't either (maybe for simplification we'll leave getting out of bed in the morning out of it )

If you fall asleep in your chair you don't have to go to bed.

But that was just scene setting. My main focus was

50 years ago most amateurs did not know that that was the wrong handicap

That would have been around the time I started go. Early on then, I came across the German system of counting grades with simple numbers, 1 down to whatever, splitting each Japanese grade in two and making no distinction between dan and kyu. I think 1-dan was 18/19 from memory.

Japanese visitors found this (and other novelties) very peculiar and would write articles in Kido and so on, describing western practices in the style of "dogs there can walk on their hindlegs."

There is evidence from extant handicap game records that, like the German system except for the naming of ranks, the Japanese system in Dosaku's time used half-stone rank differences.

So, my contention is that "50 years ago most amateurs did not know that that was the wrong handicap" should have read "50 years ago most amateurs did not CARE that that was the wrong handicap."

Good point.

That might not seem to marry with the representation of the go world currently given by L19, but I still do believe most go players in the real world are still not mathematicians and computer scientists.

Right. I think that it was unfortunate that so many Western go players in the mid-20th century were mathematicians. It sort of put people off until the hippies came along and thought that go was some kind of mystical right brain game. (At least, in the US. ) Still, in the US I have found that go players are rather cultured by comparison with the rest of the population. I could drop a Shakespeare quote at the bridge table and people would go huh? While over supper go players would discuss the interesting properties of the verb, cleave.

Edit: I doubt if the amateur "tradition" started as late as after WWII. I read in a book by Segoe his opinion that an amateur could feel satisfaction as a shodan because that meant that he would take only four stones from a pro. I read that at a time when pros were giving amateur shodans seven stones. Taking amateur rank inflation into account, I think that pushes the time when amateur ranks separated from pro ranks and adopted one handicap stone per rank as the standard into the late 19th century.

[Javaness2]

The robot players cannot handle handicap games 'correctly'. It's a bad set of data to pick.
It would be nice if KGS could simply block handicap games with bots.

I am curious. Which part of the results do you think indicates this lack of ability?

I read what their developer's wrote.

[gennan]

What does happen is that the 5k's rating goes up and the 7k's rating goes down.

Did you mean the opposite? Given that your next sentence was "So the rating system's high winrate expectations contracts the ratings toward the middle."

Yes, I did mean the opposite .

But the system is pretty much anchored at the top, so over time, the deflation above 5k will push everybody below downwards as well.

I don't understand this sentence. What do you mean by "anchored at the top"? Why does everyone get deflated?

I'm sorry, I wasn't very clear. Anchoring, contraction and downward pressure are all different aspects of the issue.

I'll try to clarify what I mean with an example:

7d EGF is pretty much fixed in practise (the expected winrates in the EGF system are basically correct for high dans and high dan ratings have a large "mass" in the system. The rating system only updates their rating by a tiny amount per game). So once he is at 7d, his rating is not affected much by some quirks that the EGF system may have.

But from 5k to mid dan, you would have this "rating contraction effect" from exaggerated winrate expectations. The low dan ratings slowly go down (being pulled towards 5k). Suppose a 2d's rating dropped below 2100 at some point (2d EGF is supposed to be 2200 and 2100 is supposed to be 1d). His rating didn't drop because he got weaker, but just because the rating system is slowly contracting over many years. Now the 2d may eventually feel forced by his low rating to demote himself to 1d (many of his peers already preceded him).

Then there is this 5k player in his club whose skill didn't change and his rating didn't change either (even though he also plays tournaments, a lot actually, but those are mostly even games against other 5k players). His rating is not affected by the general contraction to 5k.

Since the skill gap between him and the (former) 2d remained the same in reality, the (former) 2d can still give the 5k a handicap of 6-7 stones when they play. And they play each other regularly.
So the 2d's demotion can have a domino effect in his club. The 5k knows he still needs 6-7 stones handicap and he may start to doubt his own rank. After all, the dan rank of his opponent has more weight than his own kyu rank. He may even feel some pressure to demote himself. This is what I meant with "pushing everybody below downward".

You may think this is all speculation, but I really believe such effects are really happening (very slowly, I think this process has been going on for at least a decade).

You might ask what happens if the (former) 2d plays a match against a 7d with 6-7 stones handicap (the correct handicap for a 1d against a 7d). Won't that make it clear that the 2d's demotion wasn't really called for? Perhaps, but there aren't many 7d around in Europe, so this doesn't happen nearly as much as club games between 2d and 5k. So the demographics work against such "self-correcting" mechanisms.

[gennan]

Early on then, I came across the German system of counting grades with simple numbers, 1 down to whatever, splitting each Japanese grade in two and making no distinction between dan and kyu. I think 1-dan was 18/19 from memory.

Japanese visitors found this (and other novelties) very peculiar and would write articles in Kido and so on, describing western practices in the style of "dogs there can walk on their hindlegs." The thought police hadn't reached there yet. But even so they were a bit misguided, because they didn't take enough account of the fact that we had relatively few opponents, and so had to play lots of handicap games. They also overlooked the fact that we had few ways of getting a reliable rating, and in particular few ways of changing a rating quickly. The German system, sporadically adopted elsewhere, was an attempt to provide a workable ratings framework for Europe.

It didn't really work - for several reasons, I suppose. It was a creature of mathematicians, and not all go players were mathematicians. Most other players seemed in thrall to the Japanese system and just mimicked it, without properly realising it was not fit for purpose here perhaps, or, much more likely, if they did realise, they didn't care - "we are just weak amateurs, really" (and we really were then!) and "it's a storm in a teacup." I certainly was in the latter don't-care camp.

In the Netherlands there are still some places where that German system is used. We call those numbers "classes". I know they still use it for the ongoing blitz competition in Tilburg and I use it to calculate handicap tables for my own club. The last Dutch dan diploma that I recieved in 2005 also stated the class equivalent of my promotion (4d, class 13). It was actually once common to have half rank dan promotions in the Netherlands, but I think it fell in disuse in the last decade.

[gennan]

And while I'm complaining about some rating drift at the higher end of the EGF rating system, that's nothing compared to the mess that can be found in the kyu range (in general, not EGF in particular):
.

(image from https://www.emptytriangle.com/archive/26)

[dfan]

Yikes, do we we really have to make fun of the pronunciation and grammar of people for whom English is not their native language?

[gennan]

Yikes, do we we really have to make fun of the pronunciation and grammar of people for whom English is not their native language?

That was not my intention. I just posted it for the irony of the situation, not to make fun of the Japanese boy or to offend anyone.

[gennan]

Back to what started this conversation --

Yes, KGS rankings seem to be pretty far off, and have not been closely tied to appropriate anchors in a very long time. I am trying to gradually tweak the system, without making major shockwaves. Our goal would be for KGS ranks to align more closely with AGA ranks. There are so many servers, and such wide rank variations among them, that picking a standard is very hard. The merit of AGA ranks is that we have extensive records of in person play for hundreds of individuals.

If you find your rank, or someone else's has changed in a way that is totally irrational, let me know here. I can't promise to fix anything, but my actions may cause unintended fluctuations, and knowing about them can help me to make better decisions in the future.

_________________
Paul Barchilon,
AGF Vice President
KGS Manager

If the AGA can put resources behind this, it strikes me that this would be a great topic for a Kaggle competition. Can you publish a big file with the (anonymised/de-identified?) results of all games played in 2019 plus some metadata? Metadata might include handicap, komi, time settings, AGA ratings of the players where known, exact date and time when the game was played, which country/region people were logged in from if known.

I suspect the correlation between AGA and KGS rank would be weak, as some people play better online than over the board, some people the other way round, and different people take online games more or less seriously.

I also wonder whether you'd find cliques in the KGS players. For instance, group A plays mostly fast games, group B plays mostly slow games, they hardly ever play against each other, and 2k in group A is a different strength from 2k in group B. Or it could split up by time of day, country, or something else.

The goals of the Kaggle competition could be to design a better ranking system from scratch, or to suggest how to improve the current system without radical change, or to explain the various factors making ranks (appear to be) unstable or inconsistent.

Does anybody know if such historical data from KGS and/or AGA is accessible for rating system nerds like me?

[jlt]

Then there is this 5k player in his club whose skill didn't change and his rating didn't change either (even though he also plays tournaments, a lot actually, but those are mostly even games against other 5k players). His rating is not affected by the general contraction to 5k.

Since the skill gap between him and the (former) 2d remained the same in reality, the (former) 2d can still give the 5k a handicap of 6-7 stones when they play. And they play each other regularly.
So the 2d's demotion can have a domino effect in his club. The 5k knows he still needs 6-7 stones handicap and he may start to doubt his own rank. After all, the dan rank of his opponent has more weight than his own kyu rank. He may even feel some pressure to demote himself. This is what I meant with "pushing everybody below downward".

Did you actually observe people who play in a lot of tournaments and demote themselves? Personally I've never seen that.

Some people may register at a rank below their real strength, especially those who haven't participated in a tournament for a long time, but other people register at an unrealistically high rank, like in the cartoon you posted, so the two effects might cancel each other.