It is currently Thu Mar 28, 2024 11:20 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 18 posts ] 
Author Message
Offline
 Post subject: A note on percentages
Post #1 Posted: Wed Aug 30, 2023 6:38 am 
Lives in sente

Posts: 724
Liked others: 1023
Was liked: 30
Rank: BGA 3 kyu
KGS: Elom, Windnwater
OGS: Elom, Elom0
Online playing schedule: The OGS data looks pretty so I'll pause for now before I change it.
I love intelligence shows. I remember an activity from the website of the Dara O'Brian's Scool of Hard Sums TV show. One question asked how much the water content in some jelly had been reduced if the percentage of mass had increased from 98% to 99%. That that I didn't figure it out then shows my dumbness, haha




































The answer is of course that the amount of water has reduced by 50%. Per cent quite literally means per 100 which quite literally means for every hundred--and in any case the expedient question is for every hundred of what? This is ignored by the silly way in which we use percentages. %=1/100 so whenever you say percentage the correct usage is to follow it up with 'of blah', and it grates on me when people leave it out.

The problem with the way people use AI percentages is that it's always the percentage of combined winrate of both sides, which is useless for both sides. What you actually want is the winrate of each side proportional to each other. If you are at 1% you could never make a move that loses more than that no matter what, however bad the move is. Yet in reality, a move that goes from 1% to 0.5% is just as bad as a move that goes from 50% to 33.6734694%, since in both cases the proportion of your winrate to your opponent's winrate nearly doubles.

Percentage derived from change in winrate proportional to combined winrate of all players becomes less the more biased to one side a game gets so in those situations points estimates are useful for judging how good or bad a move was. Percentage derived from the player with highest winrate it's not that different, but percentage derived from the player with lower winrate it's much better since that value will always be close to the proportional winrate between all players, however there's still a transfer of percentage that makes it slightly off. The winrate proportional to the other players' winrate or winrates is best, but this is best displayed not as a percentage but as a multiple, so in addition to the standard percentage derived from winrate proportional to combined winrate, a number that shows the multiple of how many times bigger the winrate of the player with the highest winrate's winrate is compared to the player with the lowest winrate should be shown, and this is the number that should be used to judge the quality of moves, of course . . .

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #2 Posted: Wed Aug 30, 2023 7:44 am 
Gosei

Posts: 1590
Liked others: 886
Was liked: 527
Rank: AGA 3k Fox 3d
GD Posts: 61
KGS: dfan
Elom0 wrote:
Yet in reality, a move that goes from 1% to 0.5% is just as bad as a move that goes from 50% to 33.6734694%, since in both cases the proportion of your winrate to your opponent's winrate nearly doubles.

If we play games with $100 at stake, a move that goes from 1% to 0.5% loses (on average) 50 cents, while a move that goes from 50% to 33.67% loses (on average) $16.33.

A similar argument holds if rating points rather than dollars are at stake.

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #3 Posted: Wed Aug 30, 2023 8:08 am 
Lives in sente

Posts: 724
Liked others: 1023
Was liked: 30
Rank: BGA 3 kyu
KGS: Elom, Windnwater
OGS: Elom, Elom0
Online playing schedule: The OGS data looks pretty so I'll pause for now before I change it.
dfan wrote:
Elom0 wrote:
Yet in reality, a move that goes from 1% to 0.5% is just as bad as a move that goes from 50% to 33.6734694%, since in both cases the proportion of your winrate to your opponent's winrate nearly doubles.

If we play games with $100 at stake, a move that goes from 1% to 0.5% loses (on average) 50 cents, while a move that goes from 50% to 33.67% loses (on average) $16.33.

A similar argument holds if rating points rather than dollars are at stake.


Yes, but I'm referring to judging the quality of a move rather than the from total winrate value of a move. In a situation where there are many moves on the board that can lose 16+1/3 %, where as.

Let's say there is a board position in which a move loses 16+1/3 %. Then let's say you give the other side enough komi points so that your winrate is so low that that same move now just loses 0.5%. It is exactly the same The only possible argument is that if the move was a slow or safe move, then when you fall further behind in winrate then the move itself is objectively a worse move for a non-perfect player, however if the move was a risky move then it just helps the argument, as the move has now objectively increased in quality for a non-human to play even though it's from total winrate value is less. Looking at a different kind of value, we shouldn't study late endgame much at all since the 2 and 1 and fractional point values are way less than the values of moves in earlier phases of the game! Move quality is how small a loss in value a move is compared to the correct move or moves which lose no value in from total winrate or points

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #4 Posted: Thu Aug 31, 2023 3:03 am 
Judan

Posts: 6087
Liked others: 0
Was liked: 786
May we even compare percentages for the positions before and after a particular move? Why? I think we may not because a) percentages rely on different sets of playouts and b) empirical scores are also taken into account.

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #5 Posted: Thu Aug 31, 2023 3:18 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
If your winrate is 1%, then a "perfect" move that maximizes your score may decrease your winrate to 0.5%: since you are so much behind, you need to complicate the game, so you may accept to play suboptimal moves (in terms of score) if this increases the likelihood that your opponent makes a blunder.

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #6 Posted: Thu Aug 31, 2023 8:15 am 
Oza
User avatar

Posts: 2408
Location: Ghent, Belgium
Liked others: 359
Was liked: 1019
Rank: KGS 2d OGS 1d Fox 4d
KGS: Artevelde
OGS: Knotwilg
Online playing schedule: UTC 18:00 - 22:00
This is why we talk about percentage and percentage points.

If your winning percentage is 99% and then 98% your chances have been reduced by 1/99 which is 1,0101 ... % while your opponent's chances have raised from 1% to 2% which is an increase of 1/1 which is 100%. It's more intuitive to say one has decreased by 1% and the other has increase by 1% but technically those are percentage points.

BTW, I don't know how the portion of water has changed if the mass of the jelly in the jar has gone from 98% to 99% because I don't know the relative density of jelly and water. I'm splitting hairs of course, what is meant is volume I assume.


This post by Knotwilg was liked by: Elom0
Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #7 Posted: Tue Sep 05, 2023 4:31 am 
Dies in gote

Posts: 52
Liked others: 2
Was liked: 12
"judging the quality of a move" based solely on percentages given by an AI program seems dubious at best..

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #8 Posted: Tue Sep 05, 2023 6:31 am 
Lives in sente

Posts: 724
Liked others: 1023
Was liked: 30
Rank: BGA 3 kyu
KGS: Elom, Windnwater
OGS: Elom, Elom0
Online playing schedule: The OGS data looks pretty so I'll pause for now before I change it.
Knotwilg wrote:
This is why we talk about percentage and percentage points.

If your winning percentage is 99% and then 98% your chances have been reduced by 1/99 which is 1,0101 ... % while your opponent's chances have raised from 1% to 2% which is an increase of 1/1 which is 100%. It's more intuitive to say one has decreased by 1% and the other has increase by 1% but technically those are percentage points.


I'm not quite sure what you're saying . . . Since in both cases, those are percentages of games projected to win over games played, and my entire point is we should include percentages of games won over games the opponent is projected to win!

Knotwilg wrote:
BTW, I don't know how the portion of water has changed if the mass of the jelly in the jar has gone from 98% to 99% because I don't know the relative density of jelly and water. I'm splitting hairs of course, what is meant is volume I assume.


probably

Yakago wrote:
"judging the quality of a move" based solely on percentages given by an AI program seems dubious at best..


Huh?

jlt wrote:
If your winrate is 1%, then a "perfect" move that maximizes your score may decrease your winrate to 0.5%: since you are so much behind, you need to complicate the game, so you may accept to play suboptimal moves (in terms of score) if this increases the likelihood that your opponent makes a blunder.


This is truer for AI neural nets than humans. Let's remember that alphago introduced the concept of playing endgame moves that lost points because they were safer moves that assured a half-point win. Therefore moves AI consider as losing points but or more risky and complicated would have a higher winrate. The only reason why the move you play would have a lower winrate is if at your skill level it would make the game fairly complicated but to AI it's still simple so it loses points for no benfit.

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #9 Posted: Tue Sep 05, 2023 9:47 am 
Oza

Posts: 3647
Liked others: 20
Was liked: 4626
Quote:
alphago introduced the concept of playing endgame moves that lost points because they were safer moves that assured a half-point win.


Since I am not a fan of the AlphaGo/DeepMind invented chewing gum, lobotomy surgery, Star Wars and the penny post school of thought that seems so popular now, I'd like to ask about this.

Was playing for a half-point win not a characteristic first of the Monte Carlo method (Remy somebody?) and was it not used in programs before (even on kgs)? I have never played go often enough in recent decades to have personal experience, but this claim does set off a tinkle in my head.

I don't wish to deny the AlphaGo achievement, but human progress is more often than not based on building on the shoulders of giants.


This post by John Fairbairn was liked by: Elom0
Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #10 Posted: Tue Sep 05, 2023 2:04 pm 
Lives in gote

Posts: 476
Location: Netherlands
Liked others: 270
Was liked: 147
Rank: EGF 3d
Universal go server handle: gennan
Who (other than pros) still looks at AI evaluation by winrate since KataGo started offering evaluation by score, more than 4 years ago?

If my winrate drops from 56% to 43% by a move I made, this is not very useful information for me, other than telling me it was probably a mistake.
However, if KataGo says this move changed my 2.5 point lead to a 2.5 point deficit, it gives me a much better feel for how big of a mistake that move was.

Also, when I see the score graph of my game staying within a 5 point band around even during the whole game, I understand that overall this was probably a decent game for players around my level.
At the same time, a winrate graph of the same game could still show huge swings (possibly going below 10% and above 90%), because (especially later in the game) a 5 point lead or deficit is pretty decisive to a strong AI (it's assuming an opponent of its own level).

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #11 Posted: Tue Sep 05, 2023 5:32 pm 
Lives in sente

Posts: 757
Liked others: 114
Was liked: 916
Rank: maybe 2d
John Fairbairn wrote:
Quote:
alphago introduced the concept of playing endgame moves that lost points because they were safer moves that assured a half-point win.


Since I am not a fan of the AlphaGo/DeepMind invented chewing gum, lobotomy surgery, Star Wars and the penny post school of thought that seems so popular now, I'd like to ask about this.

Was playing for a half-point win not a characteristic first of the Monte Carlo method (Remy somebody?) and was it not used in programs before (even on kgs)? I have never played go often enough in recent decades to have personal experience, but this claim does set off a tinkle in my head.

I don't wish to deny the AlphaGo achievement, but human progress is more often than not based on building on the shoulders of giants.


Yes, as far as I'm aware this is was a decently common feature of MCTS bots ever since MCTS was discovered to be effective in Go after 2007. I also wouldn't consider it an "achievement", just an unintentional and often actively unhelpful-to-human-users feature of how bots tend to behave. If you don't tell them to care about score, then they don't care about score. If anything, older MCTS bots found it was a minor weakness - you could sometimes improve the strength (very slightly) by adding a slight bonus for improving score, not too large, but more than zero, encouraging bots to prefer to increase the margin a little to give a safety buffer against things they couldn't see if it didn't expose them to more risk.


This post by lightvector was liked by: Elom0
Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #12 Posted: Tue Sep 05, 2023 10:31 pm 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
Elom0 wrote:
The only reason why the move you play would have a lower winrate is if at your skill level it would make the game fairly complicated but to AI it's still simple so it loses points for no benfit.


Some positions are still complicated for bots, i.e. require hundreds of thousands of playouts.

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #13 Posted: Wed Sep 06, 2023 5:49 am 
Lives in sente

Posts: 724
Liked others: 1023
Was liked: 30
Rank: BGA 3 kyu
KGS: Elom, Windnwater
OGS: Elom, Elom0
Online playing schedule: The OGS data looks pretty so I'll pause for now before I change it.
John Fairbairn wrote:
Quote:
alphago introduced the concept of playing endgame moves that lost points because they were safer moves that assured a half-point win.


Since I am not a fan of the AlphaGo/DeepMind invented chewing gum, lobotomy surgery, Star Wars and the penny post school of thought that seems so popular now, I'd like to ask about this.

Was playing for a half-point win not a characteristic first of the Monte Carlo method (Remy somebody?) and was it not used in programs before (even on kgs)? I have never played go often enough in recent decades to have personal experience, but this claim does set off a tinkle in my head.

I don't wish to deny the AlphaGo achievement, but human progress is more often than not based on building on the shoulders of giants.


Thank you for the correction! I am really not good at the technological fluffery, quite especially the game-playing bots. I've always much prefered the abstract conceptual fields of life than the practical.

jlt wrote:
Elom0 wrote:
The only reason why the move you play would have a lower winrate is if at your skill level it would make the game fairly complicated but to AI it's still simple so it loses points for no benfit.


Some positions are still complicated for bots, i.e. require hundreds of thousands of playouts.


Yes! And a bot may think a move that creates such a position which loses points also increase the winrate by making the game complicated and unpredictable even by bot standards

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #14 Posted: Wed Sep 06, 2023 10:11 am 
Judan

Posts: 6087
Liked others: 0
Was liked: 786
gennan wrote:
Who (other than pros) still looks at AI evaluation by winrate since KataGo started offering evaluation by score


I look at both AI percentages and AI scores as they do not correlate proportionally but, I suppose, KataGo considers both. By doing so, I learn well from this empirical information. However, it is often good to wait for the order of candidate moves to stabilise. Below 100k per top candidate move during the opening, I trust little. If necessary, I wait for (many) more visits for value stabilisation.

jlt wrote:
Some positions are still complicated for bots, i.e. require hundreds of thousands of playouts.


Quite a few positions do! In fact, I have seen positions requiring 5 ~ 50 million visits per top candidate move!

On one such occasion, the early "correct" candidate after ca. 200k per top candidate move turned out to be a mistake after some 10 million visits per top candidate move. With my human theory knowledge, I expected it to be a mistake so let the AI run for two or three hours.

On another occasion, the correct move had bad AI continuations (my judgement; AI claimed a close game but it was a fight with one player having a huge moyo advantage and the opponent already a weak adjacent group) until AI justified it by aforementioned many visits to find good early contination moves and avoid the bad fight.


This post by RobertJasiek was liked by: Elom0
Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #15 Posted: Thu Sep 07, 2023 7:02 am 
Dies in gote

Posts: 52
Liked others: 2
Was liked: 12
If you say that a position is complicated for an AI based on whether or not the 'best-move candidate' can change given a few million extra playouts, then you probably include a rather large amount of all positions.

For example, if you're at move 50-100, then the complexity of your position includes all variations until the end of the game.

I would cut the AI a bit of slack and say that it is not a complicated position for the AI, if its "immediate" candidate plays reasonably well into the position and doesn't encounter a lot of problems say in the next 20 moves of the game tree.

At least this would differentiate it from situations where there is a misjudgement of a semeai, or a missed tesuji or similar.

But sure, it's also an argument of scale, and what is considered 'complicated' can be a subjective thing..

Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #16 Posted: Fri Sep 08, 2023 2:17 am 
Lives in sente

Posts: 724
Liked others: 1023
Was liked: 30
Rank: BGA 3 kyu
KGS: Elom, Windnwater
OGS: Elom, Elom0
Online playing schedule: The OGS data looks pretty so I'll pause for now before I change it.
This sums up another aspect of what (2023-09-08 I think could perhaps be better considered when using AI win rates)



What I've been thinking of for the past couple of months, or maybe more than that, is that in a game between humans, the winrate is the. Which that human winrate is dependant on not just the skill level of the players, but also their style. That's why I strongly believe in the age of AI, oddly enough through the concept of winrates the human element is more glorified than ever since 바둑 teaching pros should emphasise their style and let potential students pay for the pro that matches their style, and the same is true for (象)将棋, chess and other mindsports..


Last edited by Elom0 on Fri Sep 08, 2023 6:44 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #17 Posted: Fri Sep 08, 2023 3:13 am 
Judan

Posts: 6087
Liked others: 0
Was liked: 786
He does not suggest what his paper title suggests but suggests to use both percentage and score, as I do.

Elom0, your message editing is mistaken. Please reveal the essential, lost contents of your message!


This post by RobertJasiek was liked by: Elom0
Top
 Profile  
 
Offline
 Post subject: Re: A note on percentages
Post #18 Posted: Fri Sep 08, 2023 6:58 am 
Lives in sente

Posts: 724
Liked others: 1023
Was liked: 30
Rank: BGA 3 kyu
KGS: Elom, Windnwater
OGS: Elom, Elom0
Online playing schedule: The OGS data looks pretty so I'll pause for now before I change it.
Winrate estimates that are given are normally AI winrates versus itself, and homosapien equivalents are your winrate versus yourself, and your opponent's winrate versus itself, which could be AI and we're back full circle. Between players of different skill levels, however, relative self-play winrate is what is required to ascertain the likehood of either player winning or losing, and there again, the concept of using winrate relative to proportion of projected won games is very relevant! Your winning winrate relative to you losing winrate in a position, all relative to the opponent's winning and losing winrates to itself, show the proportion to which the winrates of the player with the higher winning to losing self winrates should be multiplied to get the actual winrate between both players. If a position fits your style, then your winrate versus yourself in that position will be higher than that of a person who is generally the same skill level as you, however that position does not fit their style, and in that position you are the player with the higher skill level.

Both AI and generalised teaching lacks in that a move that would increase the teachers winrate may not increase the students; homosapien teachers account for this to some degree by offering simpler alternate moves to weak players, however the best way is too match the teacher to the style of the student.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 18 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: Bing [Bot] and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group