It is currently Sun Dec 08, 2019 10:22 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 10 posts ] 
Author Message
Online
 Post subject: Score mean versus probability
Post #1 Posted: Sun Nov 17, 2019 7:14 am 
Gosei

Posts: 1346
Location: Earth
Liked others: 510
Was liked: 218
I think the points difference shown by Katago is a better learning tool for me than the win probability. I can evaluate and compare moves in every phase of the game without having to adjust to the vastly differences in probability swing of a 2 point mistake in the opening and in the endgame for example.

Nowadays I turn off probability and only look at the points during analyzing with KataGo. Only when I compare interesting variations with ELF and LZ I switch probabilities on again.

Thank you very much lightvector, I enjoy analyzing go even more with KataGo. It is such a nice program.

("even more" and "such a" are reminiscent to two of my great human teachers by the way. Perhaps some of you can guess them correctly ;-))


This post by Gomoto was liked by 3 people: Bill Spight, gennan, lightvector
Top
 Profile  
 
Offline
 Post subject: Re: Score mean versus probability
Post #2 Posted: Sun Nov 17, 2019 2:58 pm 
Lives with ko

Posts: 187
Location: Adelaide, South Australia
Liked others: 61
Was liked: 90
Rank: Australian 2 dan
GD Posts: 200
I agree with most of this. Especially the thanks to lightvector and all the other open source programmers who share so freely :-)

Just a word of caution: it's possible to have a positive score together with a bad winrate . Example: if I can kill my opponent's big group, I win by 50 points; if I can't kill, I lose by 2 points. Probably I can't kill, but the small possibility makes my average score look good even though I'm losing the game.

If you look exclusively at scores, you might learn some risky behaviours. Perhaps best to have both numbers in front of you, look mostly at the scores, but keen an eye open for times when the winrate tells a different story?


This post by xela was liked by 3 people: Gomoto, lightvector, mhlepore
Top
 Profile  
 
Offline
 Post subject: Re: Score mean versus probability
Post #3 Posted: Sun Nov 17, 2019 3:14 pm 
Honinbo

Posts: 9174
Liked others: 2812
Was liked: 3099
xela wrote:
I agree with most of this. Especially the thanks to lightvector and all the other open source programmers who share so freely :-)

Just a word of caution: it's possible to have a positive score together with a bad winrate . Example: if I can kill my opponent's big group, I win by 50 points; if I can't kill, I lose by 2 points. Probably I can't kill, but the small possibility makes my average score look good even though I'm losing the game.


This is why I prefer the median score to the mean. Statistical komi is a median, for instance. :)

Quote:
If you look exclusively at scores, you might learn some risky behaviours. Perhaps best to have both numbers in front of you, look mostly at the scores, but keen an eye open for times when the winrate tells a different story?


If you look at scores you have to consider the temperature, as well. For instance, if you are 2 pts. behind but the temperature is 6, (Edit: and you have the move,) you have a good chance of winning, as a rule. Winrates, whatever their flaws, are predictive in themselves. You do need to know the number of visits or playouts as a confidence measure, but the bots can tell you that. I agree that combining estimated scores with estimated winrates is a good idea. :)

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

"Once in a very great while his eyes light up for a moment, and he says "Whee!" very quietly."
— Lion Miller


Last edited by Bill Spight on Sun Nov 17, 2019 5:01 pm, edited 1 time in total.

This post by Bill Spight was liked by 2 people: Gomoto, mhlepore
Top
 Profile  
 
Offline
 Post subject: Re: Score mean versus probability
Post #4 Posted: Sun Nov 17, 2019 4:11 pm 
Lives in gote

Posts: 331
Liked others: 67
Was liked: 293
Rank: maybe 2d
I second what xela said. Do take some care when you find yourself in a situation that involves a critical large dragon, or a difficult semeai, or a massive and yet unclear ko, or other situation that may make the game very very swingy in a way that KataGo has some uncertainty about.

But otherwise, glad to hear it. :)


This post by lightvector was liked by 2 people: gennan, Gomoto
Top
 Profile  
 
Offline
 Post subject: Re: Score mean versus probability
Post #5 Posted: Sun Nov 17, 2019 5:08 pm 
Lives with ko

Posts: 249
Liked others: 55
Was liked: 73
KGS: lepore
Forgive the question if it is a non-issue as I haven’t kept up with the details of this stuff…

I recall reading a while back that bots with a lead will sometimes play sub-optimally in the endgame to ensure their win. That is, trade down the expected margin of victory for an increase in probability of victory.

What assumptions does Katago make about expected margin of victory with respect to this issue (if it is actually a thing)? Is laying off the gas a bit baked into the score estimation? If so, is it just a point or so?

Top
 Profile  
 
Offline
 Post subject: Re: Score mean versus probability
Post #6 Posted: Sun Nov 17, 2019 9:21 pm 
Honinbo

Posts: 9174
Liked others: 2812
Was liked: 3099
mhlepore wrote:
I recall reading a while back that bots with a lead will sometimes play sub-optimally in the endgame to ensure their win. That is, trade down the expected margin of victory for an increase in probability of victory.


Well, the impression that many people, myself included, have is that top bots, going back to the MCTS bots before AlphaGo, typically win games by smaller margins than an amateur dan typically would, and maybe even weaker humans. The claim has been made in the bots' defense that they give up points in order to secure the win. To my mind, that claim has never been proven. OTOH, I am unaware of anybody coming up with a case where a top bot would have lost a game versus human play because of giving up a few points in the endgame.

There was a case a while back where a top bot lost a point at the end of play by unnecessarily filling in a point of territory, thus losing the game by ½ pt. But that was by territory scoring with a 6½ pt. komi, which is not the game the bot was playing.

The main problem with the defense of the bots, it seems to me is what is meant by a winrate. IIUC, a winrate estimate assumes that the bot is playing against itself. That weakens the defense argument, because a bot could well have a blind spot that it would share with itself as the opponent, but which a different opponent would exploit. The argument then becomes that the bots make objectively suboptimal endgame plays that increase their estimate of the odds of winning the game against a player that makes the same mistakes that it does. Hardly compelling.

We already know that strong amateurs are still better than the bots in certain situations such as those with long ladders and large semeai. Humans are good at depth first search in local situations, local being a fuzzy concept. By contrast today's top bots do a kind of best first search over the whole board. That can put them at a disadvantage versus humans. Because a game of go tends to divide into a number of local situations in the endgame, human play can approach perfection, because depth first local search pays off. There is still the question of which local region to play in, but humans have good heuristics and algorithms for that. Anyway, I doubt that any of today's top bots could solve every problem in Berlekamp and Wolfe's Mathematical Go, if they were amended for a 7½ pt. komi.

I have not been motivated to look for endgame mistakes by top bots because, well, who cares? And I am not at all sure that top bots of 2018 and 2019 make game losing endgame errors (Edit: simply by making small plays). For instance, I ran across an example in the Elf commentaries where Elf recommended filling a ⅓ pt. ko instead of nailing down the win by filling a larger ko, so that it would not matter whether it won the ⅓ pt. ko or not. What human would play that way? Well, as it turns out, Elf would have won the larger ko, as well, so no harm done. ;)

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

"Once in a very great while his eyes light up for a moment, and he says "Whee!" very quietly."
— Lion Miller

Top
 Profile  
 
Offline
 Post subject: Re: Score mean versus probability
Post #7 Posted: Mon Nov 18, 2019 6:08 am 
Gosei

Posts: 1431
Liked others: 720
Was liked: 475
Rank: AGA 3k KGS 1k Fox 1d
GD Posts: 61
KGS: dfan
mhlepore wrote:
Forgive the question if it is a non-issue as I haven’t kept up with the details of this stuff…

I recall reading a while back that bots with a lead will sometimes play sub-optimally in the endgame to ensure their win. That is, trade down the expected margin of victory for an increase in probability of victory.

What assumptions does Katago make about expected margin of victory with respect to this issue (if it is actually a thing)? Is laying off the gas a bit baked into the score estimation? If so, is it just a point or so?

KataGo cares about the margin of victory and tries to win by more (or lose by less) if possible, so it should make fewer slack endgame moves in the first place. It does this by using a reward function that dispenses a higher award for bigger wins. You can see the function in Appendix F of the paper. (Edit: I should add that the graph in the appendix looks like "the function" but in fact it is just one dynamic component of it.)

Of course you have to be careful with bonuses like this because if the bonus is too small, it can get lost in the noise, and if the bonus is too big, it can induce the bot to take unnecessary risks (e.g., getting excited about possibly winning by 1.5 instead of 0.5, even though that opens up the possibility of losing by -0.5).


This post by dfan was liked by: mhlepore
Top
 Profile  
 
Offline
 Post subject: Re: Score mean versus probability
Post #8 Posted: Mon Nov 18, 2019 10:06 am 
Honinbo

Posts: 9174
Liked others: 2812
Was liked: 3099
dfan wrote:
KataGo cares about the margin of victory and tries to win by more (or lose by less) if possible, so it should make fewer slack endgame moves in the first place. It does this by using a reward function that dispenses a higher award for bigger wins. You can see the function in Appendix F of the paper. (Edit: I should add that the graph in the appendix looks like "the function" but in fact it is just one dynamic component of it.)


Appendix F? I got 38 pp. up to Appendix D. :scratch:

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

"Once in a very great while his eyes light up for a moment, and he says "Whee!" very quietly."
— Lion Miller

Top
 Profile  
 
Offline
 Post subject: Re: Score mean versus probability
Post #9 Posted: Mon Nov 18, 2019 10:11 am 
Gosei

Posts: 1431
Liked others: 720
Was liked: 475
Rank: AGA 3k KGS 1k Fox 1d
GD Posts: 61
KGS: dfan
Bill Spight wrote:
Appendix F? I got 38 pp. up to Appendix D. :scratch:

I bet you are looking at an earlier version of the paper. The link I gave should have gone to the latest version by default, but here it is explicitly: https://arxiv.org/abs/1902.10565v3


This post by dfan was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: Score mean versus probability
Post #10 Posted: Mon Nov 18, 2019 2:45 pm 
Lives with ko

Posts: 249
Liked others: 55
Was liked: 73
KGS: lepore
Thanks dfan and Bill.

I was mainly wondering if an estimated 2 point win was really a 4 point difference with a few points of slack built in. That appendix seems to put my concern to rest.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group