It is currently Tue Jan 19, 2021 6:41 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 2 posts ] 
Author Message
Offline
 Post subject: Lizzie bug?
Post #1 Posted: Sat Jul 18, 2020 3:13 am 
Oza

Posts: 2814
Liked others: 16
Was liked: 3927
I was rather startled to review an even game this morning which started off with White ahead with a 57% win rate, then after the triangled move below, White was ahead with 91%.

The source of the problem lies in the fact that this was an old Chinese game, starting with the 4 star stones. The triangled move was therefore move 1.


However, as per old Chinese rules move 1 was played by White. I don't think that should make any difference for Lizzie, in the same way that it should make no difference to non-playing sgf programs, but I was after an AI evaluation of one specific move so I thought I would just swap the colours in the sgf file.

To my surprise that did not solve the problem. It actually made it more erratic. But I was able to solve the problem by switching the order of AB[] and AW[] in the file. When AB[] comes first and Black makes the first move, all works fine. It would seem, therefore, that the programmer has used the order of AB and AW to determine who plays first. This is definitely not sgf protocol.

This may have an impact if (as I do) you input a position by hand and "accidentally" start with White stones. There may also be odd results if you input a position without also adding captured-and-removed stones. I do include them, so haven't seen any strange results, and the reason I have included them was precisely to avoid strange results. But I had expected oddities only of the order of a few percentage points. It seems I was being optimistic, and the unexpected changes could have been at least as high as 50 percentage points. Gulp!

I haven't got the time or inclination to research the far end of a fart, but FWIW this behaviour appears on my Lizzie download and also on zbaduk's Lizzie. I haven't checked with KataGo.

Edit: diagram added


Last edited by John Fairbairn on Sat Jul 18, 2020 5:26 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: Lizzie bug?
Post #2 Posted: Sat Jul 18, 2020 6:02 am 
Honinbo

Posts: 9202
Liked others: 1547
Was liked: 1595
KGS: Kirby
Tygem: 커비라고해
When I download and open the SGF as is in lizzie backed by KataGo, it's black's turn to play. As such, it has around 90% win rate or something.

_________________
be immersed

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 2 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group