Well, komi is not arbitrary.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

In the NM system, the average difference between strong shodan and weak shodan is defined by a winrate of 50% with ½ pt. komi. We could even get more fine-grained and distinguish between levels according to exact komi. The problem with that, as I see it, is that skill at go is a vector; it is only average skill that is reducible to a number. Trying for too much precision is self-defeating.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Sure, but as I wrote above, this strongly depends on the assumption that result range -1 to -6 is not over- or under-represented. This assumption can be wrong (especially if the sample count is low), and these samples carry less info than others (can be seen as both wins or losses).

And in any case this means the rating will carry more random variance, since normally only the players' performance varies, while for "h1"/even the rating will vary and depend on an external factor outside of the player's board performance.

Such as whether the player takes Black or White at chess?

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

What kind of mechanism would cause a sharp probability drop between 6 points win on the board and 7 point win on the board? Without any special mechanism, the probability would drop smoothly. Is this some specially rigged version of go, like a game on a 2x3 board with area scoring and mirror plays forbidden?

We could imagine a player A, rated 1.75 dan, and a player B, rated 1.45 dan, so that on average, if A takes white, then A's score is (say) -3 points + komi on average, but we don't know anything about the standard deviation. If A's playing style is such that the standard deviation is very small, then A will lose too many games with komi 0 compared to what A's and B's Elo rating would predict. On the other hand, A also sometimes takes black with komi 0 against player C, rated 2.05 dan, and A wins too many games against C, so that the two effects hopefully cancel out.

And if they don't, then A will be demoted to 1.49 dan at some point, and will win against B again, so it's not really a big deal.

Sure, in the long run (and as I wrote if the ratings are adjusted frequently there is no real problem).

But suppose that you only allowed to observe 10 games of a new player, and need to make the best guess of his strength. With 10 even games I'm confident that I can make a decent guess regardless of the outcomes (even if he happens to lose or win all).

But with 10 of such half-stone "H1" games, if most of them happen to be in the -1 - -6 range (not too hard to imagine), I cannot even guess if he is stronger or weaker than his opponents.

I estimate an average human 1d player would lose about 150 points over a whole game (compared to perfect play). You are hypothesizing what would happen when the probability distribution of his total error has a very sharp peak, so that the standard deviation is as small as 1 point. My question is: Why?
I'm pretty sure that such players don't exist in real life (unless you build an AI that has exactly this behaviour), so I fail to see the point of this scenario. IMO it has little to do with real world statistics.

@gennan: I agree with you, I was trying to interpret jann's messages.

Oh, I'm sorry. I missed that it was you instead of jann.

Well yes, 10 games is not a lot of data if you only look at the win/loss ratio. And when the winrate is close to 100% or 0%, the information content of the data is even lower.

You could extract more data by analyzing the quality of play in those 10 games. Then you have much more data (something like 1200 moves to analyze). I suppose an AI could do that for you. For example, you could extract an error probability distribution of his moves using KataGo and compare it with error probability distributions of typical players of known ranks.

My point was the relative value: 10 even games are a decent amount of data even for 0% or 100%, but 10 "H1" games can be much less informative.

In the EGF system, komi can be 1 point for an even game. That is entirely valid.

_________________
North Lecale

And your point is?

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Rereading jann's posts, I might make a guess about jann's point:

Handicaps provide fairly large increments of advantage/disadvantage. He worries that with small sample sizes and/or players with very small standard deviations, statistical anomalies could hide in those large increments.

My suggestion to handle this issue: If you worry about this, you could use komi in addition to handicap stones. Then you can make the handicap increments as fine-grained as half-point increments.

Use this as match conditions: If one player loses a game, you change the komi by 0.5 points in his advantage. If the komi goes below 0 or goes up to 14 points, you add or remove a handicap stone. If it's a jigo, you keep the same handicap.

Over the course of a match, the handicap + komi should gravitate to some specific value which would give a pretty good indication of the rank gap between these players.

If it's still unclear what I pointed at I doubt I could make it clearer. The above example of what could you tell after 10 komi / 10 no komi games looks like my best attempt summarizing it.

I see no problem with actual handi (H2+) games because in practice those are almost always played without komi. I also don't see a problem if there are a huge amount of games with frequently changing handicap / updated rating.

But without extra stones - and with the everyday server practice of offering a free choice (ranked differently OC, but that only helps in the long run) between the auto / nominal handicap or an even game with normal komi - there is a peculiar trait of those "h1" games: about 1 out of 3 games have no clear winner or loser (the recorded winner depends on the players' mentioned choice, which is arbitrary).

Here is what I meant. Any percentage other than 0% or 100% indicates variability in the correctness of answers to questions or of wins and losses.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Well, it seems that you somehow think that komi is arbitrary.

Aside from that, you seem to be arguing something similar to the following:

There are two chess players who are close in strength. In ten games in which one player always took Black and the other player always took White. All ten games were draws. Who can tell which player is stronger? There is something wrong with having one player take Black and the other take White.

(The first player has the advantage in both chess and go. Go without komi is what you call H1. There is no komi in chess.)

I do not mean to parody you, but chess without odds seems to me to be analogous to what you are calling H1 without komi.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

For half stone rank diffs it is the players' arbitrary choice, both 0.5 and whole komi is very common. Lack of this choice is also why the chess analogy seem to fail.


For 10 draws there is only one interpretation: B performed slightly above W. OC this is nowhere decisive (just 10 games afterall) but more than nothing. B being stronger is a bit more likely than the opposite.

This is not the same as 10 no-komi games all B+3. In that case there are two interpretations: W performed better (winning all with komi, instead of the expected 6 or 5) or worse (losing all without komi, instead of the expected 5 or 4). Harder to say who performed better (W OC, but only if you allowed to look at the scores, not just win%).

In terms of calculating ratings, there is no problem here. The only problem is using the ambiguous word "better". For this scenario, black performed above expectations, therefore black's rating needs to be adjusted upwards. Whether or not white is a stronger player than black is impossible to say from this data. It's possible that black is several stones stronger than white but chose to play slack moves (or safe moves) when ahead.

----

But there's a more fundamental issue here. We're getting hung up over a very implausible scenario. These two hypothetical players who can play a ten-game series and get exactly the same score (plus or minus one point) every time:

• First, Larry and Moe here are also playing games against other people, and those games also feed into the rating calculation, right? (Or if they're not playing anyone else, then does their rating calculation actually matter at all?)
• But even more important, this scenario is a one-in-a-thousand freak event, not business as usual. Humans just aren't that consistent. Even strong AI isn't that consistent yet. Look at the LZ promotion matches or the CGOS game results. Look at the many "golden era" challenge matches played without komi. Below pro level, if you can play me without komi ten times in a row and win every single time, then you're strong enough that you should be giving me at least three stones. (Or else this was a freakish run of games, and our next ten-game series will look very different). And if you're winning by exactly two or three points every time, you're choosing to play in a non-greedy style and you're very good at counting.

Of all the ways in which a rating system could go wrong, this is one of the last things we need to worry about.