It is currently Mon Nov 11, 2019 1:42 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 672 posts ]  Go to page Previous  1 ... 30, 31, 32, 33, 34
Author Message
Offline
 Post subject: Re: This 'n' that
Post #661 Posted: Fri Oct 11, 2019 1:54 pm 
Honinbo

Posts: 8994
Liked others: 2726
Was liked: 3062
Elf's margin of error

What is the margin of error of Elf's winrates (near 50%)?

OC, there are two errors involved. One is the accuracy of Elf's preferences, i.e., is one play or other closer to Elf's style? The variance of the estimates gives us direct information about that. The second error is whether a play is a mistake or not. We have no direct evidence of that, but that is the more important question.

As I stated I have recently revised my guess about the second margin of error for Elf to 4% from 5%. I have now slightly revised that to 4½%. As if I can tell. :lol: OC, there is no hard and fast threshold, and the number of playouts matter. I am thinking of 10k playouts or more.

Anyway, here is a position to consider.

Click Here To Show Diagram Code
[go]$$Bc Large knight's enclosure
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 2 . . . . . . . . . 5 . . . . . |
$$ | . . . , . . . . . , . . . . . , 1 . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . b . . |
$$ | . . . . . . . . . . . . . . . . a . . |
$$ | . . 3 , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . 4 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

:b5: is not Elf's style. It prefers a, estimating Black's winrate as 50½% with 30.3k playouts. :b5:, OTOH, gets a winrate estimate of 46% with 19.3k playouts. (Actually, :b5: is not even on Elf's radar, getting 0 playouts. It inherits its winrate estimate from Elf's first choice for :w6:, the enclosure at b.) Anyway, I have no problem thinking that :b5: is not Elf's style, given the 4½% winrate difference and the 0 playouts, but I do have a problem thinking of it as an error. ;)

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

Everything with love.

Top
 Profile  
 
Offline
 Post subject: Re: This 'n' that
Post #662 Posted: Fri Oct 11, 2019 2:31 pm 
Lives in gote
User avatar

Posts: 588
Liked others: 50
Was liked: 209
:b5: is not on AlphaGo Teach either.

AlphaGo thinks that "a" has 46.8% winrate.


This post by jlt was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: This 'n' that
Post #663 Posted: Sat Oct 12, 2019 12:54 am 
Lives in gote

Posts: 444
Liked others: 1
Was liked: 127
Rank: KGS 2k
GD Posts: 100
KGS: Tryss
Elf is bad for reviewing games.

With 30k+ playouts (but the result is the same with much less) Katago think :b5: is a -0.4% winrate drop compared to a, (or a loss of 0.2pts), meaning it's purely stylistic

Top
 Profile  
 
Offline
 Post subject: Re: This 'n' that
Post #664 Posted: Sat Oct 12, 2019 2:34 am 
Honinbo

Posts: 8994
Liked others: 2726
Was liked: 3062
Tryss wrote:
Elf is bad for reviewing games.

With 30k+ playouts (but the result is the same with much less) Katago think :b5: is a -0.4% winrate drop compared to a, (or a loss of 0.2pts), meaning it's purely stylistic


Maybe so. But the problems with Elf are the same as with other bots, if perhaps more extreme. Elf has strong preferences and we are more certain that its preference reflects its style that that it indicates an error. But that is so for every top bot I know of. It's just that their preferences may not be so strong.

As jlt has noted, AlphaGoTeach also prefers a as its top choice, and does not report :b5: as an alternative. Let's take a look.

Click Here To Show Diagram Code
[go]$$Bc AlphaGoTeach's preferences
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 2 . . . . . . . . . . d . . . . |
$$ | . . . , . . . . . , . . . . c , 1 . . |
$$ | . . . b . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . a . . |
$$ | . . 3 , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . 4 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


It shows four possibilities, none including :b5:, in order of preference a - d. It winrate estimates are a - 46.8%, b - 45.5%, c - 45.3%, d - 45.0%. If we believe that these differences are a matter of style, then we want a winrate difference (near 50%) of more than 1.8% to think that it indicates an error.

But is it just a matter of style? Each top bot has its own developmental path dependency. How many of them pick a as its top choice? Does KataGo have a different top choice? If not, then that's three so far. :)

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

Everything with love.

Top
 Profile  
 
Offline
 Post subject: Re: This 'n' that
Post #665 Posted: Sat Oct 12, 2019 3:36 am 
Lives in sente

Posts: 1282
Location: Earth
Liked others: 467
Was liked: 209
KataGo:
Attachment:
Kata1.JPG
Kata1.JPG [ 157.12 KiB | Viewed 604 times ]


Last edited by Gomoto on Sat Oct 12, 2019 7:24 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: This 'n' that
Post #666 Posted: Sat Oct 12, 2019 4:29 am 
Lives in sente

Posts: 1282
Location: Earth
Liked others: 467
Was liked: 209
I forgot to close Lizzie in the background :-)

Attachment:
kata2.JPG
kata2.JPG [ 163.03 KiB | Viewed 594 times ]


This post by Gomoto was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: This 'n' that
Post #667 Posted: Sat Oct 12, 2019 1:30 pm 
Lives with ko

Posts: 214
Liked others: 23
Was liked: 13
Rank: IGS 3k
KGS: Bki
IGS: mlbki
What's interesting here is that all the bots seems to favor some kind of approach moves. If the difference between the evaluation was due to noise, you would expect that one bot might rank the enclosures (any of them) higher. But it doesn't, which suggest that while it can't be called a mistake, enclosing a corner might indeed be inferior in this position.

Does elf refusal to even consider the large knight move enclosure a matter of "style"? Or is this because in such a position this move is dominated by another move (i.e. at best equal) and so is never worth exploring?

Top
 Profile  
 
Offline
 Post subject: Re: This 'n' that
Post #668 Posted: Sat Oct 12, 2019 2:31 pm 
Lives in sente

Posts: 1282
Location: Earth
Liked others: 467
Was liked: 209
Oh, KataGo for example is considering lots of moves. All the red spots are considered by the bot. (I currently have not set up an ELF network.)

Top
 Profile  
 
Offline
 Post subject: Re: This 'n' that
Post #669 Posted: Thu Oct 17, 2019 4:31 pm 
Honinbo

Posts: 8994
Liked others: 2726
Was liked: 3062
My meeting with Takemiya

In the 1970s I spent one summer in Japan and made the cusp of 3 dan. For a few weeks I won every game as 2 dan but lost every game as 3 dan. ;) The last weekend before coming back to the US I went to a go weekend at a resort. The highlight on the last day was a game between Takagawa and Takemiya. Ishida Yoshio gave the running commentary. There was time to kill while the players got several moves ahead. Ishida warmed up the room. He was very good at it. (If you don't know, he is also a professional jazz guitarist. He used to perform solo at jazz coffee shops and kissaten in Tokyo.) At one point he asked who had come the farthest to be there. I raised my hand, which got a big laugh from the audience. "So it seems," said Ishida, and then said, "No, really. Who came the farthest to be here?" I was not amused.

Afterwards a doctor who was a 5 dan invited a couple of us to come to his rooms (rich guy!) that evening after his private teaching game with Takemiya. The doctor's wife met the two of us at the door. She was a very smiley and gracious roly-poly woman. After the four men were seated around a small table she retreated to what I supposed was the bedroom. Takemiya was very friendly. The conversation covered a lot of topics, including the practice of rich Turkish women in Istanbul cruising the streets looking for attractive young men to invite into their limosines. It turned out that Takemiya had arrived the night before and spent the evening in the resort's game room, which had a billiards table, but nobody else played billiards. If I had only known about that room, I might have played billiards with Takemiya! (Not that I am any good at billiards.) After almost two hours there was a lull in the conversation. At that point the adjoining door slid open and the doctor's wife appeared with a tray of fresh, hot tea and munchies. She earned my admiration. Whatever you may think about the paternalistic Japanese culture of the time and her apparent subservience, she was obviously on the ball. :bow: :bow: :bow:

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

Everything with love.


This post by Bill Spight was liked by 3 people: Applebaps, Gomoto, SoDesuNe
Top
 Profile  
 
Offline
 Post subject:
Post #670 Posted: Fri Oct 18, 2019 5:34 am 
Honinbo
User avatar

Posts: 8698
Location: Santa Barbara, CA
Liked others: 324
Was liked: 2026
GD Posts: 312
Hi Bill,

Do you happen to know how's Mrs. Takemiya these days?

Top
 Profile  
 
Offline
 Post subject: Re:
Post #671 Posted: Fri Oct 18, 2019 5:36 am 
Honinbo

Posts: 8994
Liked others: 2726
Was liked: 3062
EdLee wrote:
Hi Bill,

Do you happen to know how's Mrs. Takemiya these days?


No idea.

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

Everything with love.

Top
 Profile  
 
Offline
 Post subject: Re: This 'n' that
Post #672 Posted: Thu Nov 07, 2019 10:59 am 
Honinbo

Posts: 8994
Liked others: 2726
Was liked: 3062
A discussion of the opening of a game between Hattori Inshuku (W) and Yamamoto Genkichi on March 29, 1801 (GoGoD 1801-03-29a), based upon ELf's commentary. OC, it was a no komi game by territory scoring, so Elf's evaluation will be off for the actual conditions.

Hattori is the author of the handicap go classic, Okigo Jizai. :)

Click Here To Show Diagram Code
[go]$$Bc Popular start with minor errors
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 7 . . . . . . . . . . . . . . . |
$$ | . . 1 6 5 9 . . . , . . . . . , . . . |
$$ | . . . 8 2 . . 3 . . . . . . . 4 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


:w2: and :b3: had been popular for over a century, but are minor errors, according to Elf. Either may be OK sans komi, OC. :b5: and :w6: are also minor mistakes, as is :b9:.

Click Here To Show Diagram Code
[go]$$Wcm6 Elf variation for :w6:
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . 1 . . . |
$$ | . . . 2 . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . 7 . . . . . . . . . . . . . |
$$ | . . X . X 3 . . . , . . . . . , . . . |
$$ | . . . . O 4 5 X . . . . . . . O . . . |
$$ | . . . . . 6 8 . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


:w8: - :b11: are joseki. :w12: seems natural, but does not appear in Waltheri.

Click Here To Show Diagram Code
[go]$$Wcm14 Elf variation for :w6:, II
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . O . . . |
$$ | . . . X . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . 8 . 6 . . . . . . . . . . . . . . . |
$$ | . . . . 4 . . . . . . . . . . . . . . |
$$ | . . 7 3 2 O . . . . . . . . . . . . . |
$$ | . . X 1 X O . . . , . . . . . , . . . |
$$ | . . . 5 O X O X . . . . . . . O . . . |
$$ | . . . . . X X . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


:w14: and :w16: push through. Then :b19: and :b21: confine White to the corner.

Click Here To Show Diagram Code
[go]$$Wcm22 Elf variation for :w6:, III
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . O . . . |
$$ | . . . X . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . 1 . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . X . X . . . . . . . . . . . . . . . |
$$ | . . . . X . . . . . . . . . . . . . . |
$$ | . . O O X O . . . . . . . . . 4 2 . . |
$$ | . . X O X O . . . , . . . . . , 3 . . |
$$ | . . . O O X O X . . . . . 5 . O . . . |
$$ | . . . . . X X . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


:w22: encloses the top right corner. Then :b23: - :w26: play a joseki that is popular with the bots.

Click Here To Show Diagram Code
[go]$$Bcm9 Elf variation for :b9:
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 9 . . . . . , . . . . . 8 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 7 . . . . . . . . . . . . . . . |
$$ | . . . X 6 . . . . . . . . . . . . . . |
$$ | . . X O X 0 . 5 . , . . . . . , . . . |
$$ | . . 1 O O . 4 B . . . . . . . O . . . |
$$ | . 3 2 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


:b9: at 18 is joseki, but Elf regards it as a minor error and prefers the 3-3, claiming the corner and sandwiching White between the corner and the :bc: stone. :b11: does not appear in Waltheri.

Click Here To Show Diagram Code
[go]$$Bcm19 Elf variation for :b9:, II
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . 1 . . . . . |
$$ | . . . X . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . 2 . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . X . 3 . . . . . . . . . . . . . |
$$ | . . . X O . 4 . . . . . . . . . . . . |
$$ | . . X O . O . X . , . . . . . , . . . |
$$ | . . X O O . O X . . . 5 . . . O . . . |
$$ | . X O . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


Interesting how the play pingpongs back and forth between the bottom left corner and the rest of the board. :b21: - :b23: is a nice flow of the stones.

More later. :)

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

Everything with love.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 672 posts ]  Go to page Previous  1 ... 30, 31, 32, 33, 34

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group