Temperature graphs according to KataGo

[Bill Spight]

In the limit as KataGo grows very very strong, the tendency for the above overestimation might be expected to shrink to 0, but let me think if there's a better way. It's a little tricky, since ideally one would like to still tell the komi to the neural net to allow it to explicitly adapt to knowing whether it's ahead or behind. Hmmm...

Well, statistical komi is the median, which may be available during training on millions of positions. I suppose that's more work than just finding the game winner, however.

[xela]

Thanks for all the suggestions! Bill's and dfan's ideas are easy to implement, but I've got a busy few days ahead of me, so I'm unlikely to post the updated graphs before next week. Of course I've already published the source code, so someone else might leap in and make those changes before I get to it...

[xela]

OK, I now have graphs for the Spight Adjusted Temperature Method! The raw_temperature line is the same numbers from the first version, and the adjusted_temperature uses Bill's formula. Game_temperature is just the minimum of all the preceding adjusted temperatures.

Example 1: most recent GoGod game, same as post number 1 in this thread.

2019-10-07a_temperatures.jpg (95.92 KiB) Viewed 12294 times

2019-10-07a_temperatures_adjusted.csv

(8.52 KiB) Downloaded 580 times

[xela]

Example 2: AlphaGo-Lee Sedol game 1, post number 2 above

2016-03-09a_temperatures.jpg (91.88 KiB) Viewed 12293 times

2016-03-09a_temperatures_adjusted.csv

(6.55 KiB) Downloaded 641 times

This three attachments per post limit is getting a bit annoying...

[xela]

Example 3: you can probably guess which game...

2016-03-10a_temperatures.jpg (84.87 KiB) Viewed 12293 times

2016-03-10a_temperatures_adjusted.csv

(7.31 KiB) Downloaded 597 times

[xela]

Example 4

1954-07-05a_temperatures.jpg (94.41 KiB) Viewed 3652 times

1954-07-05a_temperatures_adjusted.csv

(5.57 KiB) Downloaded 379 times

[xela]

Example 5

2008nationals6-hanysz-ormerod_temperatures.jpg (77.51 KiB) Viewed 3652 times

2008nationals6-hanysz-ormerod_temperatures_adjusted.csv

(4.66 KiB) Downloaded 361 times

That's all for tonight. I'll look at the environmental go game on the weekend, as it may need a bit of special treatment, not sure if all the passes will require my Python code to be modified, and I don't want to get into that at this time of night. (Only the first game record is available, not the second?)

[Bill Spight]

OK, I now have graphs for the Spight Adjusted Temperature Method! The raw_temperature line is the same numbers from the first version, and the adjusted_temperature uses Bill's formula. Game_temperature is just the minimum of all the preceding adjusted temperatures.

Example 1: most recent GoGod game, same as post number 1 in this thread.

2019-10-07a_temperatures.jpg

2019-10-07a_temperatures_adjusted.csv

Very interesting, thanks.

It appears clear that the endgame in this game starts around move 173. At least, the era of extended high temperature fights is over. And it appears that the middle game has started by move 50, maybe earlier. If the nature of the middle game is extended high temperatures, that is.

Unclear what is happening around move 90 and other places with temperature drops. But it may well be that getting the last play before the drop is worth fighting for.

[xela]

Interesting stuff xela. The environmental go game at http://britgo.org/results/env/index.html would be a good candidate for analysis to see how well the players' thoughts on what the temperature was via when they took the points cards corresponds with KataGo.

Good suggestion, thanks!

SGF at http://britgo.org/files/games/firstenv.sgf -- tried to embed here but EidoGo doesn't like the format, so you'll need to download and view it with different software.

firstenv.jpg (120.59 KiB) Viewed 3586 times

firstenv_temperatures.csv

(25.99 KiB) Downloaded 369 times

firstenv_gtp_log.txt

(1.73 MiB) Downloaded 361 times

[xela]

OK, I've kind of fixed the SGF, it had non-standard "result" and "handicap" tags. Now it works in CGoban and q5Go, but still not in EidoGo.

firstenv_edited.sgf

(5.33 KiB) Downloaded 392 times

While I was doing this, I realised that I forgot to set the komi in KataGo, so the above analysis assumes komi=7.5. I'll re-run with komi=9.5 to see if it looks any different. Back in an hour or so.

[xela]

Oh, wait, KataGo thinks white is miles ahead for most of the game, because black is taking more cards. I need to adjust the komi every time there's a pass. Watch this space for updates.

[xela]

OK, try this one instead! (You can see the komi adjustments in the gtp log, if you want to wade through many many lines of text.)

Actually not a huge difference, but before the endgame there's slightly more gap between KataGo's idea of temperature and the top card. We've already noted above that KataGo seems to be overestimating in the opening. The endgame looks pretty good.

firstenv_take2.jpg (119.95 KiB) Viewed 3539 times

firstenv_temperatures_take2.csv

(16.65 KiB) Downloaded 368 times

firstenv_take2_gtp_log.txt

(1.7 MiB) Downloaded 351 times