It occurred to me that if KataGo can estimate the score, then it can also estimate the temperature. Just evaluate black's best move, then white's best move from the same position, then take the difference. Repeat for every move, and make a graph of the numbers. Here's my attempt at doing that in Python.

For my first test, I just picked the most recent game from GoGoD. If you want to make a game out of it, read through the SGF first, think about what a temperature graph might look like, and then compare your impressions with KataGo's.

SGF game record:


Temperature graph:


My impressions:

If I spent as much time actually playing go as tinkering with software, I might start getting good at this game...

Example 2: the first game of a match that we've been discussing recently.

SGF game record:

Temperature graph:

Same match, game 2

SGF game record:


Temperature graph:

I thought I'd look for an older, gentler game to see whether the last big point of the fuseki stands out more clearly.

SGF game record:

My guess:

Temperature graph:

Last one for today, let's see how much messier an amateur game looks.

SGF game record:


Temperature graph:

Interesting stuff xela. The environmental go game at http://britgo.org/results/env/index.html would be a good candidate for analysis to see how well the players' thoughts on what the temperature was via when they took the points cards corresponds with KataGo.

It's interesting that KataGo judges even the first couple of moves as being worth ~20 points. My understanding was that the value of the first couple of moves was approximately 2x komi, or 13-15 points.

_________________
We don't know who we are; we don't know where we are.
Each of us woke up one moment and here we were in the darkness.
We're nameless things with no memory; no knowledge of what went before,
No understanding of what is now, no knowledge of what will be.

Very interesting.

For one thing, your temperature graphs fit my intuition many years ago about the general shape of a go game. Namely that the average size of plays did not drop much until the endgame.

Another thing is that the error bands for the temperature estimates should be quite large. It is true that, in an ideal environment, with perfect play, each player will gain the same by playing first, approximately half as much as the largest gote gains, or as much as the largest reverse sente gains. The difference is approximately the temperature. But in a non-ideal environment, with perfect play the maximum error is equal to the temperature.

I am not sure about KataGo's score estimates, but at first glance at your charts, they seem high. That may be because they estimate the mean score instead of the median. The median result should be about 7 pts., which seems to correct komi, but the estimate appears to be about 10 pts. That's rather high, but large scores for the first player could be jacking up the mean, if that is what is being estimated. And large scores could be the result of the second player making risky plays. I don't know what is going on there, but that may not matter, if the estimates remain proportionally high throughout the game.

A word about sente. Let's assume an ideal environment with a certain temperature, T. Let's also assume a simple sente with a large gote follower. Let the temperature of the follower be 3T. Before the sente is played, we expect the first player to gain T/2. The sente move is played, gaining 3T. Now we expect the first player to gain 3T - T/2 = 2½T. That is, because of he temperature drop after the play in the follower, the gain is more than half the temperature of the follower. In this case the temperature of the sente will be estimated as 5T instead of its actual temperature of 3T. So the temperature of a sente will be systematically overestimated. A better estimate for the temperature of the sente follower is (D+T)/2, where T is the temperature just before the sente and D is the difference between the Black estimate and the White estimate for the follower. Your graphs often show sente spikes over 40. Such large plays are rare. Surely the sente temperatures are being overestimated.

Another thing is Professor Berlekamp's idea of a game temperature distinct from the board temperature. The game temperature never rises. OC, sente plays and extended high temperature fights can raise the board temperature, but eventually the temperature will come back down, perhaps with a resignation. OC, the temperature of sente plays will not show up in the game temperature. I tried eyeballing the game temperatures of your graphs and you could see possible phase changes. Anyway, a game temperature graph eliminates the sente noise. OC, you can add the game temperature to your graphs simply by drawing it as a floor to the rest of the graph. (It will be too low if the players are weak, but at least for pro games, probably not by much.)

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

About the last big play of the opening or the last big yose

It is true that the last big play of the opening and the last big yose are supposed to be worth getting. That may or may not be so. The value of the last big play at any level of the game depends upon how much the temperature drops after that play. If there is no temperature drop, who cares?

The temperature drop for the last play of the game is typically 1 pt., when it is worth getting. And, OC, it only matters if the game is close. How much is a significant temperature drop?

Well, suppose that the temperature for the first move is 14 pts., and 240 moves later the temperature is 0. The average temperature drop per move is then approximately 0.06 pt. The fact that it is so small is what makes the go board an nearly ideal environment, and why the largest play as traditionally calculated is almost always best. It also means that a temperature drop of 1 pt. at any time in the game is significant. Not that getting the last play before the drop makes the difference between winning and losing, but it might, and it is worth trying to get that last play.

Recently I have discovered a different last play, occupying the last open corner. OC, this happens as a matter of course in a parallel fuseki, but in the Elf commentaries it is usually the right thing to do. There are sometimes plays in the opening that are better, but that is true of any last play situation. How much does occupying the last open corner gain? It depends on the situation, OC, but around 2% seems usual. I am not writing a paper on it yet. In terms of eventual scores, that gain is surely less than 1 pt. on average. I doubt if it would be noticeable in a typical temperature graph. I doubt if the last big play of the opening would be noticeable, either. But that does not mean that it is not worth some effort to get it. Quien sabe?

One thing that I did notice by eyeballing the game temperature in your graphs is that the endgame phase, where temperatures are dropping more rapidly than before, did show up in some graphs. Getting the last play before that phase change could be significant.

Edit: Eyeballing the game temperature in your graphs, it is not unusual to see large drops in the game temperature. Getting the last play before those drops could be important.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

A note on Berlekamps' Sentestrat strategy

Professor Berlekamp was a kyu player, and I was skeptical of his sentestrat strategy. OC, he was an excellent mathematician, and his point was impeccable. The basic idea is to answer sente plays immediately instead of making an inbetween play that raises the temperature even further. The danger of not answering sente is that raising the temperature further raises your possible loss. Answering sente immediately minimizes your maximum loss. As a 5 dan, I felt that sentestrat was like following your opponent around, which is in general a bad idea. OC, one danger of following your opponent around is answering a gote by mistake. I also had confidence that I would be able to judge when to answer a sente and when to make an inbetween play. I was also aware that pros often get into extended fights at high temperatures.

Looking at your graphs you can see extended periods of play where the temperature drifts up and stays high for quite a while. Did the players foresee who was going to get the last play before the temperature came back down? I kind of doubt it. It struck me that there is a certain wisdom in sentestrat. It relies upon accurately judging whether a play is sente or not, and it is a conservative strategy. Minimizing your maximum loss does not show fighting spirit. But it does have an appeal if you want to preserve a lead.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

On the other hand, you are more likely to be a world class go analyst than a world class go player. So keep doing us and yourself a favor

Well, perhaps one is less implausible than the other. But more of a meta-analyst really. I'll just make a new tool and see what Bill and others can do with it :-)

Thanks for your kind words.

These are really interesting, thanks! One thing that would make them easier to compare by eye is to put them all on the same vertical scale, say 0-100, or maybe put horizontal lines at multiples of 20 or something.

Of course all the graphs start at the same value so one can always use that to anchor them.

Yeah, I suspect too the estimates are a little high. Given that the score mean estimates the expected score that will be achieved in self-play between the two sides, this probably does mean the neural net is predicting a mild tendency for the side that is behind to do something "risky" and end up on average losing by slightly more than the amount of points that would equalize the game.

SAI will probably do better in this area once it is fully-trained, although I suspect that its use of a sigmoid for modeling winrate vs komi may make it difficult for SAI to understand details like "5.5 komi chinese = 6.5 komi chinese" since a sigmoid may force those two to be different.

In the limit as KataGo grows very very strong, the tendency for the above overestimation might be expected to shrink to 0, but let me think if there's a better way. It's a little tricky, since ideally one would like to still tell the komi to the neural net to allow it to explicitly adapt to knowing whether it's ahead or behind. Hmmm...

More on better estimates

I pointed out the problem with using the straightforward difference, which I shall call D0, to estimate the follower of a sente, and suggested an improvement based upon the temperature estimate of the previous play. But as I was going to sleep, that a better estimate comes from looking forward, not back.

Assuming best play:

Let D0 be the difference for the current position.
Let D1 be the difference for the next position.
Let T be the temperature estimate for the current position.



If the current position is sente or ambiguous, then D0 is the best estimate. If the current position is gote, then it is the average of D0 and D1. To see why, consider the last play for Black. If it is a sente, {2*s | 0 || -r}, s > r, then the temperature is r, and D0 = r. If it is a gote, {2*s | 0} then the temperature is s, but D0 = 2*s. Since it is the last play, D1 = 0, and (D0 + D1)/2 = s, which is the correct temperature.

(OC, we could do more reading and calculation and come up with better estimates, but that's more work, and ballpark estimates are better than what we have now. )

So I would suggest this way to make the initial temperature estimates, and to keep track of the game temperature estimate, which is the lowest temperature estimate to date, and graph it, as well. Higher temperatures are in theory temporary, and the drop below the game temperature can be significant information.

Edit: What if Black's last play is a gote with a follower? {2*s | 0 || -r}, s < r. The temperature is (r + s)/2. D1 = 2*s, and I suppose that D0 = r. In this case the formula I gave will overestimate the temperature. When 2*s ≥ r, then T = r, and the overestimate is (r - s)/2. When 2*s < r, then T = (r + 2*s)/2, and the overestimate is s/2. Earlier in the game I suppose that D0 will be closer to the actual temperature and this problem will tend to disappear. That's the theory, anyway. And since the temperature is typically low near the end of play, this systematic overestimate will be small, and we may not care.

I have thought about working backwards, but that assumes correct play to the end, which we cannot do.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Well, statistical komi is the median, which may be available during training on millions of positions. I suppose that's more work than just finding the game winner, however.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Thanks for all the suggestions! Bill's and dfan's ideas are easy to implement, but I've got a busy few days ahead of me, so I'm unlikely to post the updated graphs before next week. Of course I've already published the source code, so someone else might leap in and make those changes before I get to it...

OK, I now have graphs for the Spight Adjusted Temperature Method! The raw_temperature line is the same numbers from the first version, and the adjusted_temperature uses Bill's formula. Game_temperature is just the minimum of all the preceding adjusted temperatures.

Example 1: most recent GoGod game, same as post number 1 in this thread.

Example 2: AlphaGo-Lee Sedol game 1, post number 2 above

This three attachments per post limit is getting a bit annoying...

Example 3: you can probably guess which game...