It is currently Thu Mar 28, 2024 6:58 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 11 posts ] 
Author Message
Offline
 Post subject: Bad endgames by bots
Post #1 Posted: Sun Mar 03, 2019 6:05 pm 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
Title inspired by Saturday Night Live's Bad Theater for Children.

I could have posted just one game, but playing around with Leela 11 (Deep Leela) has convinced me that there are bad endgame plays by bots that cannot be explained by "bots think differently from humans". I also think that some bad plays by bots are the sort that humans would characterize as "looking for a place to resign", because the only hope of winning is bad play by the opponent, and the bad plays given the opponent the chance to do so.

So I offer at least one example game. Perhaps it will spark discussion of what went wrong and why. Perhaps others will offer examples.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Bad endgames by bots
Post #2 Posted: Sun Mar 03, 2019 6:07 pm 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
Example 1. Leela 11 lost a won game via Bent Four in the Corner.


_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Bad endgames by bots
Post #3 Posted: Sun Mar 03, 2019 8:43 pm 
Lives in gote

Posts: 311
Liked others: 0
Was liked: 45
Rank: 2d
Leela Nonzero is known to be very bad in some corner L&D shapes, this seems like an example of that. I even recall seeing it missing simple moves that allowed a large group to be turned into direct ko. On the other hand, bad endgame would mean something different (at least to me), like intentionally giving up points for no apparent reason.

Top
 Profile  
 
Offline
 Post subject: Re: Bad endgames by bots
Post #4 Posted: Sun Mar 03, 2019 11:26 pm 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
Giving up points for no apparent reason can easily be explained by saying that bots don't think in terms of points. That's true, so we can't exactly say that that is bad. OC, the bot had some botty reason for its play, and we don't know what it is.

I remember years ago thinking about writing an endgame book for DDKs and took a look at several 10 kyu games. The main problem with their endgames was not seeing plays that arose when the dame were filled. Leela 11 has the same problem. Not as bad, OC.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Bad endgames by bots
Post #5 Posted: Mon Mar 04, 2019 3:57 am 
Oza

Posts: 3647
Liked others: 20
Was liked: 4626
Bill:

In a series devoted to seeking the weak points of bots, Ohashi Hirofumi pointed out a mistake with Black 177 in the position below (White has just played the triangled stone).



This was a Golaxy self-play game where it is being trained on various komis, 6.5 here. Black played A and lost the game (after 304 moves) by half a point. Ohashi said (without explanation) that if Black had played B it would have won.

As you can see, the game has already been pretty wild, it's only halfway through and it got even wilder, with a huge trade. I don't think I've ever seen a wilder game, in fact. Yet Ohashi was pretty matter of fact about claiming that was a mistake, and he just added that this version of Golaxy mustn't have been perfected yet.

This was Part 3 of the series, with more to come, so you can infer the pros have found other alleged weak points, too. Ohashi covers Lizzie, Elf and Golaxy.

Top
 Profile  
 
Offline
 Post subject: Re: Bad endgames by bots
Post #6 Posted: Mon Mar 04, 2019 9:38 am 
Lives in gote

Posts: 311
Liked others: 0
Was liked: 45
Rank: 2d
Bill Spight wrote:
Giving up points for no apparent reason can easily be explained by saying that bots don't think in terms of points.
This is the common explanation, but it is not the end of the story. Even in terms of winrates, giving up points for absolutely nothing is not advantageous. Also note that newer/stronger bots like golaxy do think in terms of points as well (even if only putting a minor weight on that), as this makes NN evaluations better and more reliable.

In pure winrate terms some compensation may be recognizable, like having a solid connection instead of a hanging connection somewhere, or leaving fewer ko threats. A position may also receive slightly higher evaluation purely as NN or MC noise. Or it may be close to 1% or 99% so the bot cannot distinguish between values of plays, etc.

Top
 Profile  
 
Offline
 Post subject: Re: Bad endgames by bots
Post #7 Posted: Mon Mar 04, 2019 10:05 am 
Lives in gote

Posts: 389
Liked others: 81
Was liked: 128
KGS: lepore
John Fairbairn wrote:
...Ohashi Hirofumi pointed out a mistake with Black 177 in the position below


This was a Golaxy self-play game where it is being trained on various komis, 6.5 here. Black played A and lost the game (after 304 moves) by half a point. Ohashi said (without explanation) that if Black had played B it would have won.
...


Since no explanation or proof was given, I'd be interested to see if the other strong AI machines agree with Ohashi or Golaxy. It seems like a complicated semiai, and my weak brain sees a downside to playing B as well.

Top
 Profile  
 
Offline
 Post subject: Re: Bad endgames by bots
Post #8 Posted: Mon Mar 04, 2019 7:30 pm 
Gosei

Posts: 1625
Liked others: 542
Was liked: 450
Rank: senior player
GD Posts: 1000
John Fairbairn wrote:
Bill:

In a series devoted to seeking the weak points of bots, Ohashi Hirofumi pointed out a mistake with Black 177 in the position below (White has just played the triangled stone).



This was a Golaxy self-play game where it is being trained on various komis, 6.5 here. Black played A and lost the game (after 304 moves) by half a point. Ohashi said (without explanation) that if Black had played B it would have won.

As you can see, the game has already been pretty wild, it's only halfway through and it got even wilder, with a huge trade. I don't think I've ever seen a wilder game, in fact. Yet Ohashi was pretty matter of fact about claiming that was a mistake, and he just added that this version of Golaxy mustn't have been perfected yet.

This was Part 3 of the series, with more to come, so you can infer the pros have found other alleged weak points, too. Ohashi covers Lizzie, Elf and Golaxy.


Without trying to read it out, that situation seems to me to be possibly going to end as a seki (both w and b get an eye) but playing A might, in some way, allow white to initiate a ko to avoid connecting, while playing B will force white to connect, costing one point.

Top
 Profile  
 
Offline
 Post subject: Re: Bad endgames by bots
Post #9 Posted: Wed Mar 06, 2019 11:40 am 
Tengen

Posts: 4380
Location: North Carolina
Liked others: 499
Was liked: 733
Rank: AGA 3k
GD Posts: 65
OGS: Hyperpape 4k
If Black gives up the four stones, does he by any chance end up winning by .5? And if so, using what komi?

_________________
Occupy Babel!

Top
 Profile  
 
Offline
 Post subject: Re: Bad endgames by bots
Post #10 Posted: Fri Mar 08, 2019 10:58 am 
Lives with ko

Posts: 259
Liked others: 46
Was liked: 116
Rank: 2d
Bill Spight wrote:
Example 1. Leela 11 lost a won game via Bent Four in the Corner.

Nonzero Leela isn't all that strong, is it? But this reminds me that I was impressed when I was watching Michael Redmond's recent video about the Genan-Shuwa game, where I had LZ analyzing alongside and it had no problem spotting the correct sequence in the corner leading to bent-four:

Click Here To Show Diagram Code
[go]$$Bc19
$$ . X . . O X . . . |
$$ . X . O . X . . . |
$$ X . X O O X . X . |
$$ O X . . O X O O 1 |
$$ . O X X O X 3 4 2 |
$$ O . O X X X O . 7 |
$$ O O O O X 0 O 6 5 |
$$ . . . X X O 9 . 8 |
$$-------------------+[/go]

Click Here To Show Diagram Code
[go]$$Bc19m11
$$ . X . . O X . . . |
$$ . X . O . X . . . |
$$ X . X O O X . X . |
$$ O X . . O X O O X |
$$ . O X X O X X O O |
$$ O . O X X X O . X |
$$ O O O O X O O O X |
$$ . . . X X O X 1 O |
$$-------------------+[/go]

Click Here To Show Diagram Code
[go]$$Wc19m12
$$ . X . . O X . . . |
$$ . X . O . X . . . |
$$ X . X O O X . X . |
$$ O X . . O X O O X |
$$ . O X X O X X O O |
$$ O . O X X X O . X |
$$ O O O O X O O O X |
$$ . . . X X O X X 1 |
$$-------------------+[/go]

Click Here To Show Diagram Code
[go]$$Bc19m13
$$ . X . . O X . . . |
$$ . X . O . X . . . |
$$ X . X O O X . X . |
$$ O X . . O X O O X |
$$ . O X X O X X O O |
$$ O . O X X X O . X |
$$ O O O O X O O O X |
$$ . . . X X O . 1 O |
$$-------------------+[/go]

Top
 Profile  
 
Offline
 Post subject: Re: Bad endgames by bots
Post #11 Posted: Fri Mar 08, 2019 11:34 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
hyperpape wrote:
If Black gives up the four stones, does he by any chance end up winning by .5? And if so, using what komi?


No, Black cannot afford to give up four stones. 7.5 komi, as that is what Leela was trained on.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group