Life In 19x19
http://lifein19x19.com/

Bad endgames by bots
http://lifein19x19.com/viewtopic.php?f=48&t=16484
Page 1 of 1

Author:  Bill Spight [ Sun Mar 03, 2019 6:05 pm ]
Post subject:  Bad endgames by bots

Title inspired by Saturday Night Live's Bad Theater for Children.

I could have posted just one game, but playing around with Leela 11 (Deep Leela) has convinced me that there are bad endgame plays by bots that cannot be explained by "bots think differently from humans". I also think that some bad plays by bots are the sort that humans would characterize as "looking for a place to resign", because the only hope of winning is bad play by the opponent, and the bad plays given the opponent the chance to do so.

So I offer at least one example game. Perhaps it will spark discussion of what went wrong and why. Perhaps others will offer examples.

Author:  Bill Spight [ Sun Mar 03, 2019 6:07 pm ]
Post subject:  Re: Bad endgames by bots

Example 1. Leela 11 lost a won game via Bent Four in the Corner.


Author:  moha [ Sun Mar 03, 2019 8:43 pm ]
Post subject:  Re: Bad endgames by bots

Leela Nonzero is known to be very bad in some corner L&D shapes, this seems like an example of that. I even recall seeing it missing simple moves that allowed a large group to be turned into direct ko. On the other hand, bad endgame would mean something different (at least to me), like intentionally giving up points for no apparent reason.

Author:  Bill Spight [ Sun Mar 03, 2019 11:26 pm ]
Post subject:  Re: Bad endgames by bots

Giving up points for no apparent reason can easily be explained by saying that bots don't think in terms of points. That's true, so we can't exactly say that that is bad. OC, the bot had some botty reason for its play, and we don't know what it is.

I remember years ago thinking about writing an endgame book for DDKs and took a look at several 10 kyu games. The main problem with their endgames was not seeing plays that arose when the dame were filled. Leela 11 has the same problem. Not as bad, OC.

Author:  John Fairbairn [ Mon Mar 04, 2019 3:57 am ]
Post subject:  Re: Bad endgames by bots

Bill:

In a series devoted to seeking the weak points of bots, Ohashi Hirofumi pointed out a mistake with Black 177 in the position below (White has just played the triangled stone).



This was a Golaxy self-play game where it is being trained on various komis, 6.5 here. Black played A and lost the game (after 304 moves) by half a point. Ohashi said (without explanation) that if Black had played B it would have won.

As you can see, the game has already been pretty wild, it's only halfway through and it got even wilder, with a huge trade. I don't think I've ever seen a wilder game, in fact. Yet Ohashi was pretty matter of fact about claiming that was a mistake, and he just added that this version of Golaxy mustn't have been perfected yet.

This was Part 3 of the series, with more to come, so you can infer the pros have found other alleged weak points, too. Ohashi covers Lizzie, Elf and Golaxy.

Author:  moha [ Mon Mar 04, 2019 9:38 am ]
Post subject:  Re: Bad endgames by bots

Bill Spight wrote:
Giving up points for no apparent reason can easily be explained by saying that bots don't think in terms of points.
This is the common explanation, but it is not the end of the story. Even in terms of winrates, giving up points for absolutely nothing is not advantageous. Also note that newer/stronger bots like golaxy do think in terms of points as well (even if only putting a minor weight on that), as this makes NN evaluations better and more reliable.

In pure winrate terms some compensation may be recognizable, like having a solid connection instead of a hanging connection somewhere, or leaving fewer ko threats. A position may also receive slightly higher evaluation purely as NN or MC noise. Or it may be close to 1% or 99% so the bot cannot distinguish between values of plays, etc.

Author:  mhlepore [ Mon Mar 04, 2019 10:05 am ]
Post subject:  Re: Bad endgames by bots

John Fairbairn wrote:
...Ohashi Hirofumi pointed out a mistake with Black 177 in the position below


This was a Golaxy self-play game where it is being trained on various komis, 6.5 here. Black played A and lost the game (after 304 moves) by half a point. Ohashi said (without explanation) that if Black had played B it would have won.
...


Since no explanation or proof was given, I'd be interested to see if the other strong AI machines agree with Ohashi or Golaxy. It seems like a complicated semiai, and my weak brain sees a downside to playing B as well.

Author:  gowan [ Mon Mar 04, 2019 7:30 pm ]
Post subject:  Re: Bad endgames by bots

John Fairbairn wrote:
Bill:

In a series devoted to seeking the weak points of bots, Ohashi Hirofumi pointed out a mistake with Black 177 in the position below (White has just played the triangled stone).



This was a Golaxy self-play game where it is being trained on various komis, 6.5 here. Black played A and lost the game (after 304 moves) by half a point. Ohashi said (without explanation) that if Black had played B it would have won.

As you can see, the game has already been pretty wild, it's only halfway through and it got even wilder, with a huge trade. I don't think I've ever seen a wilder game, in fact. Yet Ohashi was pretty matter of fact about claiming that was a mistake, and he just added that this version of Golaxy mustn't have been perfected yet.

This was Part 3 of the series, with more to come, so you can infer the pros have found other alleged weak points, too. Ohashi covers Lizzie, Elf and Golaxy.


Without trying to read it out, that situation seems to me to be possibly going to end as a seki (both w and b get an eye) but playing A might, in some way, allow white to initiate a ko to avoid connecting, while playing B will force white to connect, costing one point.

Author:  hyperpape [ Wed Mar 06, 2019 11:40 am ]
Post subject:  Re: Bad endgames by bots

If Black gives up the four stones, does he by any chance end up winning by .5? And if so, using what komi?

Author:  bernds [ Fri Mar 08, 2019 10:58 am ]
Post subject:  Re: Bad endgames by bots

Bill Spight wrote:
Example 1. Leela 11 lost a won game via Bent Four in the Corner.

Nonzero Leela isn't all that strong, is it? But this reminds me that I was impressed when I was watching Michael Redmond's recent video about the Genan-Shuwa game, where I had LZ analyzing alongside and it had no problem spotting the correct sequence in the corner leading to bent-four:

Click Here To Show Diagram Code
[go]$$Bc19
$$ . X . . O X . . . |
$$ . X . O . X . . . |
$$ X . X O O X . X . |
$$ O X . . O X O O 1 |
$$ . O X X O X 3 4 2 |
$$ O . O X X X O . 7 |
$$ O O O O X 0 O 6 5 |
$$ . . . X X O 9 . 8 |
$$-------------------+[/go]

Click Here To Show Diagram Code
[go]$$Bc19m11
$$ . X . . O X . . . |
$$ . X . O . X . . . |
$$ X . X O O X . X . |
$$ O X . . O X O O X |
$$ . O X X O X X O O |
$$ O . O X X X O . X |
$$ O O O O X O O O X |
$$ . . . X X O X 1 O |
$$-------------------+[/go]

Click Here To Show Diagram Code
[go]$$Wc19m12
$$ . X . . O X . . . |
$$ . X . O . X . . . |
$$ X . X O O X . X . |
$$ O X . . O X O O X |
$$ . O X X O X X O O |
$$ O . O X X X O . X |
$$ O O O O X O O O X |
$$ . . . X X O X X 1 |
$$-------------------+[/go]

Click Here To Show Diagram Code
[go]$$Bc19m13
$$ . X . . O X . . . |
$$ . X . O . X . . . |
$$ X . X O O X . X . |
$$ O X . . O X O O X |
$$ . O X X O X X O O |
$$ O . O X X X O . X |
$$ O O O O X O O O X |
$$ . . . X X O . 1 O |
$$-------------------+[/go]

Author:  Bill Spight [ Fri Mar 08, 2019 11:34 am ]
Post subject:  Re: Bad endgames by bots

hyperpape wrote:
If Black gives up the four stones, does he by any chance end up winning by .5? And if so, using what komi?


No, Black cannot afford to give up four stones. 7.5 komi, as that is what Leela was trained on.

Page 1 of 1 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/