Questions about a game

Bill Spight · Post by **Bill Spight** » Wed Jun 20, 2018 2:10 am

All positions are Black to play. The last play is marked.

First position:

Click Here To Show Diagram Code: [go]$$Bcm27 Move 27 $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . X O . . . . . . . . . . . . | $$ | . . X X X O . . . . . . . . . . X . . | $$ | . a O O X O . . . , . . . . . , . . . | $$ | . . . b W c . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O . O . . . . . . O . . X . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

It looks like Black has three choices, all in the top left corner. Two cuts and one hane.

Where would you play?

How difficult is this choice?

Click Here To Show Diagram Code: [go]$$Bcm27 Outer cut $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . X O . . . . . . . . . . . . | $$ | . . X X X O . . . . . . . . . . X . . | $$ | . . O O X O . . . , . . . . . , . . . | $$ | . . . . O 1 . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O . O . . . . . . O . . X . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

Not so easy, is it? Black took 1 min. 13 sec. He averaged 9.7 sec. per move.
Edit: White took 1 second for the previous play.

Black played

. That's my feeling, too.

Second position.

Click Here To Show Diagram Code: [go]$$Bcm33 Move 33 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | . . . . . X O . . . . . . . . . . . . | $$ | . . X X X O . . W . . . . . . . X . . | $$ | . . O O X O . O . , . . . . . , . . . | $$ | . . . . O X X . . . . . . . . . . . . | $$ | . . . . O . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O . O . . . . . . O . . X . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

No hint this time.

Where to play?

How hard is this choice?

Click Here To Show Diagram Code: [go]$$Bcm33 Invasion $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | . . . . . X O . . . . . . . . . . . . | $$ | . . X X X O . . O . . . . . . . X . . | $$ | . . O O X O . O . , . . . . . , . . . | $$ | . . . . O X X . . . . . . . . . . . . | $$ | . . . . O . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . 1 . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O . O . . . . . . O . . X . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

Black invaded after 2 min. 13 sec. This one appears to have been even harder for him.

Edit: White's previous play took 1 min. 4 sec. Maybe not so easy for either side.

As I have mentioned elsewhere, Botvinnik suggests that the time taken in a position is an indicator of its difficulty for the player. I propose to call this its subjective difficulty.

In case you haven't guessed, this game is the one with Carlo Metta playing Black vs. Reem Ben David, for which Metta was accused of cheating. In the rsgf file published by Bojanic, Leela's first choice is as follows.

Click Here To Show Diagram Code: [go]$$Bcm33 Leela $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | . . . . . X O . . . . . . . . . . . . | $$ | . . X X X O . . O . . . . . . . X . . | $$ | . . O O X O . O . , . . . . . , . . . | $$ | . . . . O X X . . . . . . . . . . . . | $$ | . . 3 . O . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O 1 O . . . . . . O . . X . . | $$ | . . . . . 2 . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

After the kikashi,

-

,

plays the eye stealing tesuji in the top left corner. Leela gives this a win rate approximately 4% higher than Metta's play. That is, it may be considered a mistake.

Stanislaw Frejlak provides an analysis of this game (see viewtopic.php?p=228924#p228924 ). He finds

"really strange". Leela is nondeterministic, and his Leela simply plays the eye stealing tesuji.

Edit: Alez Cieply also analyzed this game using Leela 11. (See https://docs.google.com/spreadsheets/d/ ... =468616376 ). His Leela also chose F-03. But it estimated a loss of only 2.6%, less clearly a mistake.

Position 3

Click Here To Show Diagram Code: [go]$$Bcm37 Move 37 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | a X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O . . . . . . . X . . | $$ | . W O O X O . O . , . . . . . , . . . | $$ | . . . . O X X . . . . . . . . . . . . | $$ | . . . . O . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O . O . . . . . . O . . X . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

Living at "a" is the obvious play. Or is it?

How hard is this position?

Click Here To Show Diagram Code: [go]$$Bcm37 Life $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | 1 X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O . . . . . . . X . . | $$ | . O O O X O . O . , . . . . . , . . . | $$ | . . . . O X X . . . . . . . . . . . . | $$ | . . a . O . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O b O . . . . . . O . . X . . | $$ | . . . . . c . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

Black made life in the corner, but only after almost 4 minutes, i.e. 3 min. 56 sec. This was the longest time he took for any play in the game. What made it such a subjectively difficult decision?

Edit: White's previous play took 3 seconds.

Frejlak says, "Probably many people would descend without thinking as Carlo did. But Leela suggests peeping!" OC, Carlo did not descend without thinking, he thought for a long time. (I would have made the same play as Carlo, but quicker.

) By the peep Frejlak means "a". Bojanic's Leela plays Black "b", White "c" first, then "a". It gives that play a win rate 7% higher than Carlo's play, so Carlo's play may be considered as a big mistake (and one that leaves White slightly ahead).

Edit: Cieply's Leela also recommends C-14, but gives a loss of only 3.08%.

Leela's play, like that of other top bots today, is very flexible. One possibility is to sacrifice the corner for strength on the left side.

This is twice Metta has taken a long time and then made mistakes, according to Leela.

Position 4

Click Here To Show Diagram Code: [go]$$Bcm39 Move 39 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O . . . . . . . X . . | $$ | . O O O X O . O . , . . . . . , . . . | $$ | . . . . O X X . . . . . . . . . . . . | $$ | . . . . O . . . . . . . . . . . . . . | $$ | . . W . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O . O . . . . . . O . . X . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

White has made life and pincered the Black stone on the left side. What now?

How difficult is this position?

Click Here To Show Diagram Code: [go]$$Bcm39 Black jumps out $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O . . . . . . . X . . | $$ | . O O O X O . O . , . . . . . , . . . | $$ | . . . . O X X . . . . . . . . . . . . | $$ | . . . . O . . . . . . . . . . . . . . | $$ | . . O c . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . X . 1 . . . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O a O . . . . . . O . . X . . | $$ | . . . . . b . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ ---------------------------------------[/go]

Carlo jumped. Frejlak says, "Every human jumps." His Leela plays "a". So does, Bojanic's, jumping after White "b".

Edit: Cieply's Leela recommends the attachment at "c".

Carlo took 52 seconds. What was he thinking about? Since the jump allows an underneath connection, perhaps he considered other plays in the vicinity.

Edit: White's previous play took 34 seconds.

Position 5

Click Here To Show Diagram Code: [go]$$Bcm51 Move 51 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O . . . . . . . X . . | $$ | . O O O X O . O . , . . . . . , . . . | $$ | . . . . O X X . . . . . . . . . . . . | $$ | . . . . O . . . . . . . . . . . . . . | $$ | . . O . . X . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . X . X . X . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . O . O . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . O . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O X O . . . . . . O . . X . . | $$ | . . . . X O . O . . . . . . . . . . . | $$ | . . . . . . W . . . . . . . . . . . . | $$ ---------------------------------------[/go]

White has just taken a stone with

. What now?
How hard is this decision?

Click Here To Show Diagram Code: [go]$$Bcm51 Move 51 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O . 1 b . . . . X . . | $$ | . O O O X O . O . a . . . . . , . . . | $$ | . . . . O X X . . . . . . . . . . . . | $$ | . . . . O . . . . . . . . . . . . . . | $$ | . . O . . X . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . X . X . X . . . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . O . O . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . O . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O X O . . . . . . O . . X . . | $$ | . . . . X O . O . . . . . . . . . . . | $$ | . . . . . . O . . . . . . . . . . . . | $$ ---------------------------------------[/go]

Carlo played

. So did both Leelas. Frejlak says he did not know where to play, maybe "a". I was also unsure where to play. Not Carlo. He took 2 seconds.

Edit: Cieply's Leela recommends the more distant play at "b". It considers

to have lost 1.35%.

Edit: White's previous play took 2 seconds, as well.

Position 6

Click Here To Show Diagram Code: [go]$$Bcm71 Move 71 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . . . | $$ | . . . . O W . . . . . . . . . . . . . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X . X X X . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . O . O . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . O . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O X O . . . . . . O . . X . . | $$ | . . . . X O . O . . . . . . . . . . . | $$ | . . . . . . O . . . . . . . . . . . . | $$ ---------------------------------------[/go]

White has just played

, cutting off two stones. Where now?
How hard is this position?

Click Here To Show Diagram Code: [go]$$Bcm71 Solidifying the center $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . . . | $$ | . . . . O W . . . . . . . . . . . . . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X . X X X . . . . . . . . . . | $$ | . . . , . . . . . , . . . . . , . . . | $$ | . . O . O . O . 1 . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . . . O . . . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O X O . . . . . . O . . X . . | $$ | . . . . X O . O . . . . . . . . . . . | $$ | . . . . . . O . . . . . . . . . . . . | $$ ---------------------------------------[/go]

surprised Frejlak. Me, too. His Leela agrees that it is slow. Bojanic's Leela thinks that it loses 2 percentage points. Frejlak thinks, "the game is undoubtedly good for black." Me, too. Bojanic's Leela thinks that Black is a 2:1 favorite after

.

Edit: Cieply's Leela thinks that

lost 4.09%. It also chooses Carlo's play.

Bojanic's Leela also chooses Carlo's play. Me, too. It looks obvious to me. Carlo took 30 seconds, however, over three times as long as his average.

Edit: White's previous play took 41 seconds.

Edit: Added the times for White's previous plays, in response to Javaness2.

Edit: Added choices and % losses from Ales Cieply's analysis with Leela 11.

Bill Spight · Post by **Bill Spight** » Wed Jun 20, 2018 2:11 am

Position 7

Click Here To Show Diagram Code: [go]$$Bcm79 Black 79 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . . . | $$ | . . . . O O . . . . . . . . . . . . . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X X X X X . . . . . . . . . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . . . . . . . . . . | $$ | . . . . . . . . . c . . . . . . . . . | $$ | . . . . . . W X X O . . . . . . . . . | $$ | . . . . . . O a O b . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O X O . . . . . . O . . X . . | $$ | . . . . X O . O . . . . . . . . . . . | $$ | . . . . . . O . . . . . . . . . . . . | $$ ---------------------------------------[/go]

White has just blocked with

. OC, Black may play elsewhere, but locally "a", "b", and "c" are possibilities.

How difficult is this choice?

Click Here To Show Diagram Code: [go]$$Bcm79 Push $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . . . | $$ | . . . . O O . . . . . . . . . . . . . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X X X X X . . . . . . . . . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . . . . . . . . . . | $$ | . . . . . . . . . c . . . . . . . . . | $$ | . . . . . . O X X O . . . . . . . . . | $$ | . . . . . . O 1 O . . . . O . . X . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . . X X X O . . , . . . . . , . . . | $$ | . . X O O X O . . . . . . O . . X . . | $$ | . . . . X O . O . . . . . . . . . . . | $$ | . . . . . . O . . . . . . . . . . . . | $$ ---------------------------------------[/go]

Metta pushed. Frejlak would not, and neither would either Leela. Nor would I. I like "c", which is a nice thick play.

Bojanic's Leela picks it, too. But

cannot be called a mistake, according to Bojanic's Leela.

Edit: Cieply's Leela chose Carlo's play.

White's previous move took 7 seconds. Carlo took 1 min. 10 sec.

Position 8

Click Here To Show Diagram Code: [go]$$Bcm97 Move 97 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . . . | $$ | . . . . O O . . . . . . . . . . X . . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X X X X X . . . . . . . O . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . . . . . . . . . . | $$ | . . . . . . . . . X . X . . . . O . . | $$ | . . . . . . O X X O . . . . . . . . . | $$ | . . . . . . O X O O . . . O . . X . . | $$ | . . X . . . . O . . . . . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . W O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

Black has just secured the bottom right corner with sente. What now?

How difficult is this position?

Click Here To Show Diagram Code: [go]$$Bcm97 Aji keshi? $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O b . . . . . . . | $$ | . . . . O O . . . . . . a . . . X . . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X X X X X . . . . . . . O . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . . . . . . . . . . | $$ | . . . . . . . . . X . X . . . . O . . | $$ | . . . . . . O X X O 1 . . . . . . . . | $$ | . . . . . . O X O O . . . O . . X . . | $$ | . . X . . . . O . . . . . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

Metta played

, which looks like aji keshi to Frejlak (me, too). Neither Leela plays it. Bojanic's Leela plays the double keima at "a", but thinks that

loses only 1½%, not enough to call it a mistake.

Edit: Cieply's Leela chose "b" and thinks the Carlo's play lost 2.18%.

Black took 56 seconds. White's previous move took 35 seconds.

Edit: Added plays and percentages from the analysis by Cieply's Leela 11.

Position 9

Click Here To Show Diagram Code: [go]$$Bcm5 Move 105 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . . . | $$ | . . . . O O . . . . . . X . . . X W . | $$ | . O O O X X . . O . . . . . . . . . . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X X X X X . . . . . . . O . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . O . . . . . . . . | $$ | . . . . . . . . . X X X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

White just played the probe,

. How to respond?

How difficult is this decision?

Click Here To Show Diagram Code: [go]$$Bcm5 Move 105 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X . . . , . . . | $$ | . . . . O X X . . . O . . . . . . 1 . | $$ | . . . . O O . . . . . . X . . . X W . | $$ | . O O O X X . . O . . . . . . . b a . | $$ | . X . O O X . O . . . . . . . . . . . | $$ | . . X X X X X X X . . . . . . . O . . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . O . . . . . . . . | $$ | . . . . . . . . . X X X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

Carlo played the defensive hane. Frejlak would have played at "a" to gobble up the stone. Me, too. Bojanic's Leela and Cieply's Leela would also play at "a", but not necessarily capture the stone; Frejlak's Leela would have played "b". (Bojanic's Leela does not consider plays after Black 105.)

Cieply's Leela thinks that Carlo's play is a mistake, losing 2.57%.

White's previous play took 3 min. 28 sec. Carlo took 2 min. 55 sec., his second longest time.

Position 10

Click Here To Show Diagram Code: [go]$$Bc Move 121 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X X O . , . . . | $$ | . . . . O X X . . . O . . . . . . X . | $$ | . . . . O O . . . . . X X O . . X . X | $$ | . O O O X X . . O . . . O W O . X . X | $$ | . X . O O X . O . . . . . X . . O X . | $$ | . . X X X X X X X . . . . . . . O O . | $$ | . . . , . O . . . , . . . . . , . . . | $$ | . . O . O . O . X . O . . . . . . . . | $$ | . . . . . . . . . X X X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

White has just connected at

. What now?

How difficult is this position?

Click Here To Show Diagram Code: [go]$$Bcm21 Keima $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X . . . X . . | $$ | . O O O X O . O . O O X X O . , . . . | $$ | . . . . O X X . . . O . . . . . . X . | $$ | . . . . O O . . . . . X X O . . X . X | $$ | . O O O X X . . O . . . O O O . X . X | $$ | . X . O O X . O . . . . . X . . O X . | $$ | . . X X X X X X X . . . . . . . O O . | $$ | . . . , . O . . . , . . 1 a . , . . . | $$ | . . O . O . O . X . O . . . . . . . . | $$ | . . . . . . . . . X X X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

Carlo played the keima. So did Uberdude's Leela. Frejlak seems to approve of Carlo's play, saying that his Leela "doesn't think of keima", but jumps to "a". Cieply's Leela also jumps to "a".

Cieply's Leela thinks that Carlo's keima lost 2.05%, another mistake.

White's previous play took 19 seconds. Carlo's play took 49 seconds.

Position 11

Click Here To Show Diagram Code: [go]$$Bcm29 Move 129 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X O . . X . . | $$ | . O O O X O . O . O O X X O . , . . . | $$ | . . . . O X X . . . O . . . . . . X . | $$ | . . . . O O . . O . . X X O . . X . X | $$ | . O O O X X . . O X . X O O O . X . X | $$ | . X . O O X . O . . . W O X . X O X . | $$ | . . X X X X X X X . . . . . . . O O . | $$ | . . . , . O . . . , . . X . . , . . . | $$ | . . O . O . O . X . O . . . . . . . . | $$ | . . . . . . . . . X X X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

White has just turned at

. How to respond?

How difficult is this position?

Click Here To Show Diagram Code: [go]$$Bcm29 Block $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X O . . X . . | $$ | . O O O X O . O . W W X X O . , . . . | $$ | . . . . O X X . . . W . . . . . . X . | $$ | . . . . O O . . O a . X X O . . X . X | $$ | . O O O X X . . O X . X O O O . X . X | $$ | . X . O O X . O . . 1 O O X . X O X . | $$ | . . X X X X X X X . . . . . . . O O . | $$ | . . . , . O . . . , . . X . . , . . . | $$ | . . O . O . O . X . O . . . . . . . . | $$ | . . . . . . . . . X X X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

Carlo blocked at

. Frejlak does not comment on that play.

Cieply's Leela chooses "a", which threatens the three

stones. Leela thinks that

loses 3.10%, a serious error. IMO, the block is simple and strong. Black is well ahead, why seek complications?

White's previous move took 1 min. 51 sec.

took 51 seconds.

Position 12

Click Here To Show Diagram Code: [go]$$Bc Move 131 $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X O . . X . . | $$ | . O O O X O . O . O O X X O . , . . . | $$ | . . . . O X X . . . O . . . . . . X . | $$ | . . . . O O . . O . . X X O . . X . X | $$ | . O O O X X . . O X . X O O O . X . X | $$ | . X . O O X . O . . X O O X . X O X . | $$ | . . X X X X X X X . W . . . . . O O . | $$ | . . . , . O . . . , . . X . . , . . . | $$ | . . O . O . O . X . O . . . . . . . . | $$ | . . . . . . . . . X X X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

White has just played the hane,

, threatening to cut Black in two. How to respond?

How difficult is this position?

Click Here To Show Diagram Code: [go]$$Bcm31 Block $$ --------------------------------------- $$ | . . . . X . . . . . . . . . . . . . . | $$ | X X . . . X O . . . . . . . . . . . . | $$ | . O X X X O . . O X X . X O . . X . . | $$ | . O O O X O . O . O O X X O . , . . . | $$ | . . . . O X X . . . O . . . . . . X . | $$ | . . . . O O . . O . . X X O . . X . X | $$ | . O O O X X . . O X . X O O O . X . X | $$ | . X . O O X . O a . X O O X . X O X . | $$ | . . X X X X X X X 1 O . . . . . O O . | $$ | . . . , . O . . . , . . X . . , . . . | $$ | . . O . O . O . X . O . . . . . . . . | $$ | . . . . . . . . . X X X . . . . O . . | $$ | . . . . . . O X X O X . . . . . . . . | $$ | . . . . . . O X O O O X . O . . X . . | $$ | . . X . . . . O . . . O . . O . . . . | $$ | . . . X X X O . . , . . . . O X . . . | $$ | . . X O O X O . . . . . . O X . X . . | $$ | . . . . X O . O . . . . O O X . . . . | $$ | . . . . . . O . . . . . O X X . . . . | $$ ---------------------------------------[/go]

Carlo replied with the simple block,

. Cieply's Leela chooses "a". But it considers

to be 0.60% better. (!)

is its second choice. Presumably it chose "a" because it got more Monte Carlo playouts.

White's previous play took 1 min. 34 sec.

took 49 seconds.

----
No more positions. These 12 positions include all 10 which took Carlo 49 seconds or more, plus a couple of others. OC, we don't know why they took so long, but Botvinnik's idea that they were relatively difficult for him is a good working hypothesis.

Javaness2 · Post by **Javaness2** » Wed Jun 20, 2018 2:34 am

Just to state the obvious, the kgb metric is fine up to a point. It would be nicer to have not just the time B spend on Move X but also the time W spend on the previous move (or moves if they were forced). What we can't tell is when B or W left the board to go to the bathroom, or nipped off to the kitchen to get a biscuit, or called up a professional on the telephone. As we don't know about such behaviour, we have to be careful about the kgb.

Bojanic · Post by **Bojanic** » Wed Jun 20, 2018 2:47 am

Bill,
nice topic for analysis.

I have to write again regarding time, since it is online game, we don't know what players were doing.
You might say that he was thinking for some time, but he could be reading news like I do, getting up, phone, etc.

Same goes for fast plays, it does not necessary mean that he didn't think on it.
If you run Leela in analysis mode (open empty sgf and start from beginning), you can go forward some moves. Also Leela will show you next moves on the bar, or in analysis window. Therefore, if opponent spent even little time thinking (10+ s), next moves could already be prepared.

During analysis of important moves I did almost similar as alleged cheater would do.

----

And finally, trying to go move by move and think would it be possible that 4d+ would play them is not good enough.
He could play most of them, but we have to take a look at bigger picture.
Main question is overall similarity of play to Leela, in suspicious and live games.

In 2 supsicious games, there is several important tenukis. He played all of them top Leela's choice (apart from one forcing sequence).
Theoretically, he could do it.
But how he could not hit one in WAGC games?

And finally, how all of his moves in 2 games were within 2% of Leela's top choice?
Human players rarely can do it, in short fighting games. Not in long games.

AlesCieply · Post by **AlesCieply** » Wed Jun 20, 2018 2:58 am

I think it is really worthwhile to pursue this approach. However, in the Italian appeal they demonstrate that Leela is not quite deterministic when deciding on its top move (or sometimes even on whether the move is among the top 3). Thus, my first question to your presentation would be - did you check whether in these chosen positions Leela "always" suggest the same move? I would run Leela let's say 5 times on each chosen position and with various playouts settings. We do not know for sure what computer CM used. We can just assume the information provided by the Italians (about 100k nodes in 30s) is OK. When playing with Leela I found that the estimated winrates are more settled than the Leela's top move suggestions, which also led me to choosing the type of analysis based on delta histograms rather than looking at selected significant moves. Both approaches have their merits and flaws ...

Bill Spight · Post by **Bill Spight** » Wed Jun 20, 2018 3:01 am

AlesCieply wrote:I think it is really worthwhile to pursue this approach. However, in the Italian appeal they demonstrate that Leela is not quite deterministic when deciding on its top move (or sometimes even on whether the move is among the top 3).

That's why I said what Frejlak's Leela said and what Bojanic's Leela said. Bojanic's rsgf file also gives win rate estimates.

Bill Spight · Post by **Bill Spight** » Wed Jun 20, 2018 3:08 am

AlesCieply wrote:When playing with Leela I found that the estimated winrates are more settled than the Leela's top move suggestions,

Good to know.

Looking at the rsgf files, it seems to me that the variability in the estimated win rates crucially depends upon the number of rollouts. (Big Duh!

) So to use Leela for evaluation, not for play, I would want to have the same number of rollouts for each possible choice. Preferably at least 10,000 each.

Edit: Today's top bots are optimized for play, not for evaluation.

Bill Spight · Post by **Bill Spight** » Wed Jun 20, 2018 3:15 am

Javaness2 wrote:Just to state the obvious, the kgb metric is fine up to a point. It would be nicer to have not just the time B spend on Move X but also the time W spend on the previous move (or moves if they were forced).

Interesting idea.

What we can't tell is when B or W left the board to go to the bathroom, or nipped off to the kitchen to get a biscuit, or called up a professional on the telephone. As we don't know about such behaviour, we have to be careful about the kgb.

Claro que si.

Bojanic wrote:I have to write again regarding time, since it is online game, we don't know what players were doing.
You might say that he was thinking for some time, but he could be reading news like I do, getting up, phone, etc.

Same goes for fast plays, it does not necessary mean that he didn't think on it.

Ditto.

If you run Leela in analysis mode (open empty sgf and start from beginning), you can go forward some moves. Also Leela will show you next moves on the bar, or in analysis window. Therefore, if opponent spent even little time thinking (10+ s), next moves could already be prepared.

Ditto.

Bill Spight · Post by **Bill Spight** » Wed Jun 20, 2018 3:18 am

Bojanic wrote:And finally, trying to go move by move and think would it be possible that 4d+ would play them is not good enough.

It is not good enough to demonstrate innocence. But that is not where the burden of proof lies.

Bojanic · Post by **Bojanic** » Wed Jun 20, 2018 5:53 am

Bill Spight wrote:
AlesCieply wrote:I think it is really worthwhile to pursue this approach. However, in the Italian appeal they demonstrate that Leela is not quite deterministic when deciding on its top move (or sometimes even on whether the move is among the top 3).
That's why I said what Frejlak's Leela said and what Bojanic's Leela said. Bojanic's rsgf file also gives win rate estimates.

For moves I considered important, I watched how analysis progressed, and noted when some choice first appeared.

Jan.van.Rongen · Post by **Jan.van.Rongen** » Wed Jun 20, 2018 11:38 am

There is so much to say about this analysis, so I am going to make some renarks in some random order.

1. We do know the specifications of that computer: unless you have reasons to think somebody is lying which I find disturbing. I even find it disturbing that it is suggested not knowing something that is known.

Anyway is has an NVIDIA GTX 960M, which is the dominant factor in speed. CPU and memory are far less important. The speed gain over the CPU is a factor of at least 10.

2. The time taken to move contains a wealth of information that has not been used in a thorough analysis. In the Italian appeal only the weaknesses of the original "98% method" were shown, but they failed to provide additional evidence that goes against the allegation of cheating. A statistical analysus can show that both players speeded up, resp. slowed down in the same periods: i.e. they could both experience the same situations as difficult or easy. This is IMO strong evidence of not cheating, because it is very difficult to simulate that behaviour.

3. It is an error to use Leela 0.11 to evaluate a situation when you investigate cheating with the same program because that will easily lead to circular reasoning. The win rate given by Leela 0.11 are very different from AQ (which existed at the time of the game) and from Leela Zero and Leela ELF.

4. Position 5: black 51 was played in 2 seconds, too fast to consult a computer. You could object that he could have peeked earlier, but both players were playing at a rather high speed in this sequence, so there was very little time to use an AI. Furthermore if you would use the Leela application in analysis mode it does NOT suggest this move until after white 50. In other words, there is firm evidence that B could not have cheated to play this move.

5. Position 6: Black 71 @ J9. Here we see the limits of the evaluation function of Leela 0.11, because it does not "see" the whole forced sequence that follows, where the difference increases from 2 to 10%. AQ sees a 8% difference immediately. So that classifies White 70 as a very bad move.

6. Edit again: sorry - I made an error in the move number.

Gobang · Post by **Gobang** » Wed Jun 20, 2018 12:00 pm

thanks.

Jan.van.Rongen · Post by **Jan.van.Rongen** » Wed Jun 20, 2018 12:25 pm

The following diagram shows AQ's evaluation after Blacks move. This diagram shows how Blacks "winning %" changes after his answer to Whites moves. So if it jumps up- White's move was not so good and Black replied took advantage from that.

The diagram is from 11-04 this year, when AQ was sill about as strong as LeelaZeo and stronger than Leela 0.11

: move_diagram.jpeg (65.14 KiB) Viewed 15655 times

Bojanic · Post by **Bojanic** » Wed Jun 20, 2018 1:01 pm

Jan.van.Rongen wrote: 3. It is an error to use Leela 0.11 to evaluate a situation when you investigate cheating with the same program because that will easily lead to circular reasoning. The win rate given by Leela 0.11 are very different from AQ (which existed at the time of the game) and from Leela Zero and Leela ELF.

It is wrong to use Leela in analysis of case where it is suspected that it was used?
Interesting reasoning. What is next, to use some random player's games instead of Metta's?

Jan.van.Rongen wrote:4. Position 5: black 51 was played in 2 seconds, too fast to consult a computer. You could object that he could have peeked earlier, but both players were playing at a rather high speed in this sequence, so there was very little time to use an AI. Furthermore if you would use the Leela application in analysis mode it does NOT suggest this move until after white 50. In other words, there is firm evidence that B could not have cheated to play this move.

Analysis window of Leela.
Please note that on bottom bar before W50 was played you can see L17 suggestion almost immediately.

Therefore, black had enough time, and your "firm evidence" is wrong.

Bojanic · Post by **Bojanic** » Wed Jun 20, 2018 1:04 pm

Jan.van.Rongen wrote:The following diagram shows AQ's evaluation after Blacks move. This diagram shows how Blacks "winning %" changes after his answer to Whites moves. So if it jumps up- White's move was not so good and Black replied took advantage from that.

The diagram is from 11-04 this year, when AQ was sill about as strong as LeelaZeo and stronger than Leela 0.11

The attachment move_diagram.jpeg is no longer available

And this is the diagram of game Metta-Ben David in Leela 0.11, which shows remarkably small differences.

This diagram is very remarkable on it's own - black made very few mistakes, that were rather small.
It is difficult how to explain not only such small deviations, but stream of Leela's top choices in human game of this length.

Life In 19x19

Questions about a game

Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game

Re: Questions about a game