Life In 19x19
http://lifein19x19.com/

Guanzi-pu - Lizzie unreliable?
http://lifein19x19.com/viewtopic.php?f=18&t=16975
Page 1 of 1

Author:  John Fairbairn [ Tue Oct 15, 2019 3:07 am ]
Post subject:  Guanzi-pu - Lizzie unreliable?

I idly tested Lizzie with a problem from Guanzi-pu. She seemed to fluff it in a way. Maybe other bots can do better?

The problem I tried is below. You need to know that GZP is not a book of life & death but rather a book of boundary-play problems. Although L&D aspects can crop up, you are actually meant to be concerned with eking out extra profit, which you can do in various way - cutting off tails, fattening the slaughtered calf, encroaching, and so on. A tesuji (or two, or three - this is a hard book) is generally required rather than mere counting.



It is White to move, and a throw-in A makes a profitable capture. He cuts off the tail. But he can do better by fattening the calf.

With the position as shown, Lizzie gave the correct first move A (according to the original and backed by Go Seigen et al.), as below:



Lizzie also gave the correct follow-up as Black B. And then plumped for White C (after B was actually played). But the humans believe White D is the correct move.

I tried a variant position in which the rest of the board was filled up with absolutely secure territory for both sides, so that the only plays left were in the problem position. That altered Lizzie's perception but just seemed to make things worse initially. However, she did find the right answer after a few minutes.

That seems to suggest to me that bots can't be trusted in such positions unless you allow them to play slowly (much as for humans, I suppose :)), and the degree of trust may be much less early in the game. Also, how much time do you have to give them before you can trust them? If you have no human solution as a yardstick, we amateurs can be left high and dry. Pure L&D may be different because the stakes are higher and the outcome easier to understand.

The full human solution here is embedded in the position below.


Author:  Kirby [ Tue Oct 15, 2019 4:54 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

Just a side note - lizzie is not a bot - it's a graphical interface for analyzing games using Leela Zero. You can use various networks (e.g. different versions of Leela Zero) with Lizzie, and different versions will give different move recommendations.

Author:  dfan [ Tue Oct 15, 2019 5:06 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

Leela Zero (which is the engine I assume you ran using Lizzie as a graphical interface) does not care about the final score of the game, just who won. This is a common issue when using AlphaZero-inspired bots for analyzing - their endgame tends to be slack. So if you fill up the rest of the board, you have to take care to ensure that the correct sequence really is the difference between winning and losing.

Using Lizzie, I created a stable whole-board situation and asked KataGo, which does care about the margin of victory, rather than Leela Zero. I actually seem to be lucky in that the whole-board situation was such that the the correct sequence was the difference between winning and losing, and KataGo did find it immediately. It did also find it when I adjusted the komi so that playing at C and D both led to wins, which I imagine Leela Zero would not.

Author:  Gomoto [ Tue Oct 15, 2019 5:42 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

Go is a game about spooky actions at a distance. The local situations are always entangled to the whole board position.

The current bots are not simulating rationalizing about the optimal move in a local position, they are simulating intuition about the best way to win a game with the resources at hand (computing power in the AI case, reading abilities in the human case).

This is a post by me, so take it with a grain of salt as always.



(You could train a bot to get a good intuition about the "correct" solutions in these problems: The bot should not not play games against itself, but has to solve lots of such problems and compare the results to the known "optimal solution". Hope you have some millions of these problems with the "optimal" solutions available for this use case. Who decides what is the optimal solution with no info available about the entangled whole board position?)


It is possible to use current bots to analyze such problems, but I know some go related activities that are a hell of a lot more fun.

Author:  Gomoto [ Tue Oct 15, 2019 5:52 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

And now lets take this haphazard motion one daring step further:

We human beings developed by natural selection not only rational reasoning but also intuition, because our world is an entangled one as well like the cosmos we call goban ;-)

Let's go on!




And by the way analyzing the problem with KataGo did not add something usefull to the discussion in my opinion.

Author:  Tryss [ Tue Oct 15, 2019 6:28 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

Setting up a position where the correct sequence end with W+0.5 (with 7.5 komi), LZ and Katago gave me the correct answer.


Author:  Gomoto [ Tue Oct 15, 2019 6:50 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

Care to show KataGo thinking? My results differ :-)

(Edit: Perhaps, just a matter of letting it run for a while again ;-) see further down)

Author:  jlt [ Tue Oct 15, 2019 6:52 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

Unreliability of bots in some situations has been discussed already

https://lifein19x19.com/viewtopic.php?p=249425#p249425

P.S.

Tryss wrote:
Setting up a position where the correct sequence end with W+0.5 (with 7.5 komi), LZ and Katago gave me the correct answer.


(What an ugly board...)

Author:  Gomoto [ Tue Oct 15, 2019 6:52 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

A: (I do not care about the win rate, blue is blue :-) )

Attachments:
kat1.JPG
kat1.JPG [ 109.28 KiB | Viewed 8151 times ]

Author:  Gomoto [ Tue Oct 15, 2019 6:54 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

B:
Attachment:
kat2.JPG
kat2.JPG [ 197.93 KiB | Viewed 8150 times ]

Author:  Gomoto [ Tue Oct 15, 2019 6:56 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

C:
Attachment:
Kat3.jpg
Kat3.jpg [ 171.25 KiB | Viewed 8155 times ]

Author:  Gomoto [ Tue Oct 15, 2019 7:01 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

slightly off topic, or should I say slightly unreliable
Quote:
Unreliability of bots in some situations has been discussed already


Unreliability of bots opposed to what?

I am not used to anything reliable in my world. I take precautions to survive every day, allthough I know I will fail in the end without doubt :twisted: :D

Author:  jlt [ Tue Oct 15, 2019 7:11 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

@Gomoto: of course no entity, whether human or electronic, is 100% reliable. When I speak about unreliability, I am comparing bots with strong human players. While LeelaZero's moves are very often better than human ones, there are rare situations where an overwhelming majority of strong human players find a better move than LeelaZero.

Author:  Gomoto [ Tue Oct 15, 2019 7:30 am ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

Agreed. In every day situations I put more trust in bots. KataGo > LZ in every day situations aka ladder related positions. > as in reliability not as in strength.

(And jlt, thank you for explaining your view point in detail to me.)

Author:  Bill Spight [ Sun Oct 20, 2019 8:36 pm ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

I tried a different way, which may well suck, to create a board for such a problem. Here is a diagram. Does White make the Guanzi-pu play?

Click Here To Show Diagram Code
[go]$$Wc White to play
$$ ---------------------------------------
$$ | . . . X X . . . . . . . . . . . . . . |
$$ | . . O X O X X . X O . X . . . . . . . |
$$ | . . O X O O X X X O . . . . O O . . . |
$$ | . . . O O . O O X O . X . X X , O O O |
$$ | . . . . . . . O O X X . X O . . X X O |
$$ | . O O . . . . . . . . . O X X X X O X |
$$ | . X X O O X . X O . . . . O O O O O . |
$$ | . . . X O O X X O O O O . . . . . O . |
$$ | . . . . O . O X O X X X X . . . X X . |
$$ | . X X O O O O X . , . . . . . X . . . |
$$ | O O O X X X X X X X X O X . . . . . . |
$$ | O X . X . . . O O . O O X . . . . . . |
$$ | O X X . . O O . O . O O X . X X . . . |
$$ | O . . . . O X O O X X O O . O X O . . |
$$ | . O O O . O X O X . X X O O X . X O . |
$$ | . X X , . . X X X X O X O X . X O O . |
$$ | . . . X . X . . . X O O O . X X X X . |
$$ | . . . . . . X . O O . . . . X . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

Author:  Gomoto [ Sun Oct 20, 2019 9:11 pm ]
Post subject:  Re: Guanzi-pu - Lizzie unreliable?

KataGo:

Page 1 of 1 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/