I did a study of KataGo's opinions about the "fairest" komi for each square board size using the final 40 block network, for every size that the 40-block network has had direct training experience on. These opinions should reflect pretty well KataGo's experiences during self-play about its empirical winning frequency on these board sizes for different values of komi on those board sizes, and although KataGo is only one bot, might be indicative of what "fair" komi would generally be among bots trained to a comparable level on these board sizes.

Firstly, from separate earlier studies, we know (probable) perfect komi for smaller sizes:

Next my interpretation of the raw data. Most "fair" komi, in KataGo's opinion, assuming quality of play at KataGo's self-play-training level, plus some notes about that komi.

Keep in mind the fact that in Chinese rules, absent seki, odd komi are the ones that matter to cross on odd boards, while even komi are the ones that matter to cross on even boards. So, on odd sized boards, 5.5, 6, 6.5 are all the same for example, while 6.5, 7, 7.5 are all different. On even boards, it's the other way around. So there's this weird oscillatory behavior for Chinese rules on board size parity.

Lastly, the raw data. These are tables of the observed winrates for KataGo from the opening position for different board sizes under different rules. This is the full range of board sizes that KataGo has had actual training on - size 7 to size 19. These were done with the final g170 40 block net with 100ks of playouts on the smaller boards, diminishing to 10ks of playouts on the large boards since those took longer, and using somewhat wider-than-normal search settings.

On the large boards the results appear quite stable even if searching longer, since the search is mostly just smoothly averaging over the large variety of permutations of the opening what the neural net says based on its self-play experience. On the smaller boards, tactical complications start happening much faster, and the values never stop fluctuating as the tactics are explored ever deeper, I often just chose an arbitrary point to cut it off, different runs might get different results.

Also of course, technically these are "KataGo Japanese" and "KataGo Chinese" rules, which might differ from the actual Chinese and Japanese rules in sufficiently pathological positions.

Japanese rules winrates:


Chinese rules winrates:


And the same two tables, interleaved and paired if you want to more easily compare Japanese and Chinese together for just a single komi:

There are of course other things to observe in the raw data too.

Anyways, this was fun. Hope someone finds this interesting.

Fascinating that fair komi hardly changes between board size 9 and 19, and that we've known this all along.

I find this very interesting!

I made charts for odd board sizes 9-19.

So for odd board sizes 9-19, the fairest komi is 6+ for Japanese rules and 7- for Chinese rules.

With Japanese rules, 6 komi looks like perfect komi.
When draws are unacceptable, 6.5 komi looks like the fairest alternative (but then white has an advantage on smaller boards).

With Chinese rules, 7 komi looks like perfect komi.
When draws are unacceptable, 6.5 komi looks like the fairest alternative (but then black has an advantage, more so on smaller boards). With 7.5 komi, white has a fairly large advantage.

Evidence is mounting that 7.5 komi, as commonly used with Chinese rules (and AGA rules), is too much.
Does anyone know if there are plans in China or the US to lower the komi to 6.5?

Has any AI research been done on area scoring with integer komi and a Button to break ties?

With area scoring rules, you have 361 pts to share, so if there's no seki, the score difference on the board is always an odd integer (If black score on the board is B, and white score on the board is B, then W+B=361)

The only case where there's a difference between 5.5 and 6.5 komi is when there's an odd number of irreductible dame.

Yes, the lower graph also shows that 5.5 and 6.5 komi have basically the same winrate with Chinese rules.

But black has 55% winrate with 5.5 or 6.5 komi, while white has 60% winrate with 7.5 komi. So 7.5 komi seems to be more unfair than 5.5 or 6.5 komi.

Well, why not a komi of 7 points? Even pros would probably have ties less than 5% of the time. Chess gets by with a fairly high rate of draws.

I still like Button Go, though.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

@lightvector

How much experience did katago have at the different komi levels/rule sets on different boards? I assume that the katago should be weaker at assessing the alternatives simply due to the difference in experience levels. It would be extremely interesting to see your thoughts in this regard.

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21

KataGo supports area scoring + button Go as an evaluation method. I have not studied what its evaluations are or how they differ from those here (I suspect they would largely match the Japanese rules evaluations, but I haven't checked). I welcome anyone to do their own study.



Generally KataGo played easily 10x (order of magnitudewise) as many 19x19 games as any other board size. But also smaller boards are easier to learn - whereas if you only a have a few GPUs it takes weeks or months to reach pro level on 19x19 for example, I think it could be done on 9x9 in days. Also an enormous amount of knowledge generalizes between the board sizes, especially on the larger sizes. You can see this in the smoothness of the winrate estimates for the larger board sizes. If it were systematically the case that smaller board size play were abnormally "lower-quality" by a lot than the 19x19 play, you'd expect that empirically in self-play a given degree of komi-unfairness would not result in as pronounced an advantage for one side or the other, and then the neural net's output would reflect those statistics (fitting smooth-ish simple curves to statistics is very easy for neural nets), so you might expect a discontinuity in the sharpness of the winrates as you go from 19x19 to any smaller size. But to the contrary it looks quite regularly increasing in sharpness, as you'd expect if quality of play was fairly regular and the dominant effect was simply the higher variance of the large board due to the board being bigger / games lasting longer, given comparably high-quality play.

This makes intuitive sense to me. I'd think that 95+% of the "knowledge" between the board sizes is shared for board sizes 13x13 and larger. You learn the subtle ways that playing on 15x15 should adjust your overall judgment of territory vs influence and other fuzzy aspects of evaluation, and then modulo those shades of difference in judgment, almost everything you know about fighting and good shapes and what moves are effective in obtaining different kinds of trades carries over.

You can take the winrates as a fairly direct reflection of the self-play training statistics. For example on 9x9 with Japanese rules, KataGo must have experienced roughly fair results with 6 komi, and adjusting it merely to 5.5 komi gives black 70%/30%, more than a 2-1 advantage, whereas adjusting it to 6.5 gives white 74%/26%, almost a 3-1 advantage. While more extreme komis than this will have increasingly limited data, there should be quite many games at all of 5.5, 6, and 6.5, so presumably these are roughly the statistics from the games. Also self-play conditions have added noise and use only 200-1000 playouts - so basically you can say that slightly drunk ultra-blitz KataGo is roughly of a level on 9x9 where, at least against itself, a half-point advantage somewhere between doubles and triples your winning likelihood over the opponent. With the tens of thousands of playouts you'd get in match conditions, it should be yet much sharper.

One thing I find interesting:

For 3,4,5,6,7 you see a pattern where the odd sizes have a larger advantage for black than the even sizes. Something to do with tengen being a uniquely effective move on small boards, whereas when the center is a 2x2 square, white can also split the board and you get a more balanced result? It sort of makes intuitive sense. But, size 8 breaks the pattern. It has a *larger* advantage for black than either 7 or 9.

Size 9 is the last size in which tengen is a good move - and in fact, contrary to what you hear circulated in some forums, it's not even necessarily the best move - size 9 is where non-tengen moves finally "catch up" and become just as good in quality, 4-4 and 4-5 openings are probably also draws under fair komi. After size 9, non central moves overtake tengen. Past size 9, it also becomes increasingly clear that the bots are no longer "nearly solving" the game, and the game becomes smoother and not dominated by one single highly swingy tactical fight. Both players have enough choices of how to play and maintain average move efficiency that aren't dictated by a high density of one-way-street tactics, that I'd conjecture that perfect komi remains near 6 or 7 perpetually after, near the practical fair komi. As opposed to being much more jittery like the small boards and it merely being beyond anyone's reach to find the needle-in-a-haystack line that makes you multiple points better than the current komis that appear to be fair.

So anyways, that would seems to leave board size 8 as an oddity, where the single-big-tactical-fight of the game results in black having a bigger advantage despite an even size, rather than a smaller advantage, and one of the last board sizes for a chance for this to happen before one might conjecture smooth averaging behavior to start to dominate.

Great stuff lightvector. I didn't realise KataGo now prefers white on Japanese 6.5, it used to prefer black. The 20b network I just tried in the BadukAI app still does. Any idea when this changed?

Uberdude - Not sure. But all the nets are here, feel free to check.

https://d3dndmfyhecmj0.cloudfront.net/g ... index.html