It is currently Fri Mar 29, 2024 5:10 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 16 posts ] 
Author Message
Offline
 Post subject: Guanzi-pu - Lizzie unreliable?
Post #1 Posted: Tue Oct 15, 2019 3:07 am 
Oza

Posts: 3647
Liked others: 20
Was liked: 4626
I idly tested Lizzie with a problem from Guanzi-pu. She seemed to fluff it in a way. Maybe other bots can do better?

The problem I tried is below. You need to know that GZP is not a book of life & death but rather a book of boundary-play problems. Although L&D aspects can crop up, you are actually meant to be concerned with eking out extra profit, which you can do in various way - cutting off tails, fattening the slaughtered calf, encroaching, and so on. A tesuji (or two, or three - this is a hard book) is generally required rather than mere counting.



It is White to move, and a throw-in A makes a profitable capture. He cuts off the tail. But he can do better by fattening the calf.

With the position as shown, Lizzie gave the correct first move A (according to the original and backed by Go Seigen et al.), as below:



Lizzie also gave the correct follow-up as Black B. And then plumped for White C (after B was actually played). But the humans believe White D is the correct move.

I tried a variant position in which the rest of the board was filled up with absolutely secure territory for both sides, so that the only plays left were in the problem position. That altered Lizzie's perception but just seemed to make things worse initially. However, she did find the right answer after a few minutes.

That seems to suggest to me that bots can't be trusted in such positions unless you allow them to play slowly (much as for humans, I suppose :)), and the degree of trust may be much less early in the game. Also, how much time do you have to give them before you can trust them? If you have no human solution as a yardstick, we amateurs can be left high and dry. Pure L&D may be different because the stakes are higher and the outcome easier to understand.

The full human solution here is embedded in the position below.


Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #2 Posted: Tue Oct 15, 2019 4:54 am 
Honinbo

Posts: 9545
Liked others: 1600
Was liked: 1711
KGS: Kirby
Tygem: 커비라고해
Just a side note - lizzie is not a bot - it's a graphical interface for analyzing games using Leela Zero. You can use various networks (e.g. different versions of Leela Zero) with Lizzie, and different versions will give different move recommendations.

_________________
be immersed

Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #3 Posted: Tue Oct 15, 2019 5:06 am 
Gosei

Posts: 1590
Liked others: 886
Was liked: 527
Rank: AGA 3k Fox 3d
GD Posts: 61
KGS: dfan
Leela Zero (which is the engine I assume you ran using Lizzie as a graphical interface) does not care about the final score of the game, just who won. This is a common issue when using AlphaZero-inspired bots for analyzing - their endgame tends to be slack. So if you fill up the rest of the board, you have to take care to ensure that the correct sequence really is the difference between winning and losing.

Using Lizzie, I created a stable whole-board situation and asked KataGo, which does care about the margin of victory, rather than Leela Zero. I actually seem to be lucky in that the whole-board situation was such that the the correct sequence was the difference between winning and losing, and KataGo did find it immediately. It did also find it when I adjusted the komi so that playing at C and D both led to wins, which I imagine Leela Zero would not.


This post by dfan was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #4 Posted: Tue Oct 15, 2019 5:42 am 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
Go is a game about spooky actions at a distance. The local situations are always entangled to the whole board position.

The current bots are not simulating rationalizing about the optimal move in a local position, they are simulating intuition about the best way to win a game with the resources at hand (computing power in the AI case, reading abilities in the human case).

This is a post by me, so take it with a grain of salt as always.



(You could train a bot to get a good intuition about the "correct" solutions in these problems: The bot should not not play games against itself, but has to solve lots of such problems and compare the results to the known "optimal solution". Hope you have some millions of these problems with the "optimal" solutions available for this use case. Who decides what is the optimal solution with no info available about the entangled whole board position?)


It is possible to use current bots to analyze such problems, but I know some go related activities that are a hell of a lot more fun.

Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #5 Posted: Tue Oct 15, 2019 5:52 am 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
And now lets take this haphazard motion one daring step further:

We human beings developed by natural selection not only rational reasoning but also intuition, because our world is an entangled one as well like the cosmos we call goban ;-)

Let's go on!




And by the way analyzing the problem with KataGo did not add something usefull to the discussion in my opinion.


Last edited by Gomoto on Tue Oct 15, 2019 7:06 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #6 Posted: Tue Oct 15, 2019 6:28 am 
Lives in gote

Posts: 502
Liked others: 1
Was liked: 153
Rank: KGS 2k
GD Posts: 100
KGS: Tryss
Setting up a position where the correct sequence end with W+0.5 (with 7.5 komi), LZ and Katago gave me the correct answer.


Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #7 Posted: Tue Oct 15, 2019 6:50 am 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
Care to show KataGo thinking? My results differ :-)

(Edit: Perhaps, just a matter of letting it run for a while again ;-) see further down)


Last edited by Gomoto on Tue Oct 15, 2019 7:07 am, edited 2 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #8 Posted: Tue Oct 15, 2019 6:52 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
Unreliability of bots in some situations has been discussed already

https://lifein19x19.com/viewtopic.php?p=249425#p249425

P.S.

Tryss wrote:
Setting up a position where the correct sequence end with W+0.5 (with 7.5 komi), LZ and Katago gave me the correct answer.


(What an ugly board...)

Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #9 Posted: Tue Oct 15, 2019 6:52 am 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
A: (I do not care about the win rate, blue is blue :-) )


Attachments:
kat1.JPG
kat1.JPG [ 109.28 KiB | Viewed 7936 times ]


Last edited by Gomoto on Tue Oct 15, 2019 7:03 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #10 Posted: Tue Oct 15, 2019 6:54 am 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
B:
Attachment:
kat2.JPG
kat2.JPG [ 197.93 KiB | Viewed 7935 times ]

Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #11 Posted: Tue Oct 15, 2019 6:56 am 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
C:
Attachment:
Kat3.jpg
Kat3.jpg [ 171.25 KiB | Viewed 7940 times ]

Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #12 Posted: Tue Oct 15, 2019 7:01 am 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
slightly off topic, or should I say slightly unreliable
Quote:
Unreliability of bots in some situations has been discussed already


Unreliability of bots opposed to what?

I am not used to anything reliable in my world. I take precautions to survive every day, allthough I know I will fail in the end without doubt :twisted: :D

Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #13 Posted: Tue Oct 15, 2019 7:11 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
@Gomoto: of course no entity, whether human or electronic, is 100% reliable. When I speak about unreliability, I am comparing bots with strong human players. While LeelaZero's moves are very often better than human ones, there are rare situations where an overwhelming majority of strong human players find a better move than LeelaZero.


This post by jlt was liked by: Gomoto
Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #14 Posted: Tue Oct 15, 2019 7:30 am 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
Agreed. In every day situations I put more trust in bots. KataGo > LZ in every day situations aka ladder related positions. > as in reliability not as in strength.

(And jlt, thank you for explaining your view point in detail to me.)

Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #15 Posted: Sun Oct 20, 2019 8:36 pm 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
I tried a different way, which may well suck, to create a board for such a problem. Here is a diagram. Does White make the Guanzi-pu play?

Click Here To Show Diagram Code
[go]$$Wc White to play
$$ ---------------------------------------
$$ | . . . X X . . . . . . . . . . . . . . |
$$ | . . O X O X X . X O . X . . . . . . . |
$$ | . . O X O O X X X O . . . . O O . . . |
$$ | . . . O O . O O X O . X . X X , O O O |
$$ | . . . . . . . O O X X . X O . . X X O |
$$ | . O O . . . . . . . . . O X X X X O X |
$$ | . X X O O X . X O . . . . O O O O O . |
$$ | . . . X O O X X O O O O . . . . . O . |
$$ | . . . . O . O X O X X X X . . . X X . |
$$ | . X X O O O O X . , . . . . . X . . . |
$$ | O O O X X X X X X X X O X . . . . . . |
$$ | O X . X . . . O O . O O X . . . . . . |
$$ | O X X . . O O . O . O O X . X X . . . |
$$ | O . . . . O X O O X X O O . O X O . . |
$$ | . O O O . O X O X . X X O O X . X O . |
$$ | . X X , . . X X X X O X O X . X O O . |
$$ | . . . X . X . . . X O O O . X X X X . |
$$ | . . . . . . X . O O . . . . X . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.


This post by Bill Spight was liked by: Gomoto
Top
 Profile  
 
Offline
 Post subject: Re: Guanzi-pu - Lizzie unreliable?
Post #16 Posted: Sun Oct 20, 2019 9:11 pm 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
KataGo:


This post by Gomoto was liked by: Bill Spight
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 16 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group