It is currently Wed May 22, 2019 4:23 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 11 posts ] 
Author Message
Offline
 Post subject: Learning Elfish
Post #1 Posted: Thu Aug 23, 2018 1:42 pm 
Judan

Posts: 5805
Location: Cambridge, UK
Liked others: 328
Was liked: 3106
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
A thread for miscellaneous musings on the opinionated Elf (Facebook's bot).

First up, the latest version (Elf v1 not v0) thinks black is better (52% +-1 I'd give as the error) on the empty board with 7.5 komi, which differs from other strong bots (AlphaGo 46.7%, Leela Zero #157 46.5%, FineArt 41% apparently, Elf v0 49.2%). It also believes diagonal 4-4s is good for black, so for white to play move 2 as 4-4 in an adjacent corner to black's first 4-4 is a slight mistake, -1%. But to play 3-4 in an adjacent corner is a huge mistake, -8% after 50k playouts (it started at -11%, I wonder how much after more); Kajiwara would be proud "move 2 lost the game"! (picture below shows 3-4 not facing the 4-4 which is more common in pro games whereas Kajiwara's was facing it, Elf thinks his 2 is basically the same). The reasoning seems simple enough (if not the size of the mistake), black plays diagonal 4-4, white takes last corner, and then black has sente to (low, of course) approach the 3-4 which is tedomari. Or after more playouts black uses his 5th move to 3-3 invade the 4-4 in sente, and then approach. Interestingly, a pattern search of human pro games also rates it poorly at only at only 45.6% (a real win%) despite being the 3rd most popular choice (15%), whilst the adjacent 4-4 is most popular (50%) and has a healthy 50.5% win. Diagonal 4-4 is just a little behind at 49.6%. Human pros keep playing this 3-4 move, and AlphaGo teach doesn't think it's bad, but I wonder if Elf and the wisdom of pro crowds are onto somthing.
Attachment:
Elf mv2.PNG
Elf mv2.PNG [ 581.61 KiB | Viewed 1809 times ]


If we have parallel 4-4s, Elf v1 likes to outside approach or 3-3 invade slightly more than inside approach, but its view of white 6 shocked me. Backing off with the knight move, AlphaGo Teach's best move, a perfectly decent move for Leela Zero #157 (though switches to lower right 3-3 after 30k playouts but knight is less than 1% minus) and the most common (and healthy win%) human pro move is barely considered and seen as a big mistake, -7%. Instead if thinks white should approach black back (and lots of ignored approaches follow).
Attachment:
Elf mv5.PNG
Elf mv5.PNG [ 595.97 KiB | Viewed 1809 times ]


So what should black do to take advantage of white backing off and giving black sente? (not needing to add a move to the approach stone after knight answer is the new normal). 3-3 invade of course, and this is something AlphaGo Teach and LZ agree on, though they don't share Elf's opinion it is good for black. If white does the common jump (as AG and LZ like, Elf thinks it's a mistake, that's for another post) instead of hane after push then Elf thinks white should keep extending and take gote, rather unusual these days to allow the 3-3 invasion in sente:
Attachment:
Elf mv13.PNG
Elf mv13.PNG [ 600.5 KiB | Viewed 1809 times ]


If white does tenuki then, unless it is a strongly sente almost contact move like 3-3 invasion (knight approach isn't sente enough) then Elf thinks it is worth spending a move on and good for black to continue with the atari and descend at the top left corner. By playing around with some other positions, Elf seems to think this shape isn't joseki but good for black.
Attachment:
Elf mv14.PNG
Elf mv14.PNG [ 597.11 KiB | Viewed 1809 times ]


Here's a recent game I played on Fox based on this idea, which thanks to my opponent's co-operation ended up working rather well for me.


This post by Uberdude was liked by 3 people: Bill Spight, Elom, yakcyll
Top
 Profile  
 
Offline
 Post subject: Re: Learning Elfish
Post #2 Posted: Thu Aug 23, 2018 3:20 pm 
Lives in sente

Posts: 1109
Location: Earth
Liked others: 359
Was liked: 181
By the way, did you run some tests ELF v1 versus ELF v0 (I managed to detect some setup errors with Lizzie on my side in the past like this.)

(The newer setup lost to the older, after check I had not setup correctly the new one)

Top
 Profile  
 
Offline
 Post subject: Re: Learning Elfish
Post #3 Posted: Thu Aug 23, 2018 3:41 pm 
Lives in sente

Posts: 1109
Location: Earth
Liked others: 359
Was liked: 181
Your setup seems correct (I setup a test with ELF v1 and get similar numbers).

Top
 Profile  
 
Offline
 Post subject: Re: Learning Elfish
Post #4 Posted: Thu Aug 23, 2018 4:57 pm 
Lives in sente

Posts: 1109
Location: Earth
Liked others: 359
Was liked: 181
By the way what network is the one with the Hash a30471. It is the recent strongest test network, but it is not ELF?

EDIT: It seems to be a 256x40 LZ version and it seems to be stronger than ELF :)

Top
 Profile  
 
Offline
 Post subject: Re: Learning Elfish
Post #5 Posted: Thu Aug 23, 2018 5:32 pm 
Judan

Posts: 5805
Location: Cambridge, UK
Liked others: 328
Was liked: 3106
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Stronger on equal playouts in test match not so surprising for twice as deep network, but how about equal time?

Edit: did it actually play elf or just elo calculated from match vs recent LZ which has inflated elo.

Top
 Profile  
 
Offline
 Post subject: Re: Learning Elfish
Post #6 Posted: Thu Aug 23, 2018 6:03 pm 
Lives in sente

Posts: 1109
Location: Earth
Liked others: 359
Was liked: 181
(I think it is probably a test version of the leela zero network combined with elf by an user named bjixyo.)
have a look at the data and the graph on the leela zero site: https://zero.sjeng.org/
It is the one with the highest + sign for now.

Top
 Profile  
 
Offline
 Post subject: Re: Learning Elfish
Post #7 Posted: Fri Aug 24, 2018 2:09 am 
Judan

Posts: 5805
Location: Cambridge, UK
Liked others: 328
Was liked: 3106
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Gomoto wrote:
(I think it is probably a test version of the leela zero network combined with elf by an user named bjixyo.)
have a look at the data and the graph on the leela zero site: https://zero.sjeng.org/
It is the one with the highest + sign for now.

It's one of the 40 block networks by bjixyo, trained on LZ mainline self-play game to version #165 and maybe some Elf games but definitely not Elf network merged in. It is, as I suspected, an Elo rating based on its winning record vs a 20b LZ which suffer from extra-inflated ratings and not on actually beating Elf:
- a304 85% win vs 8993cd24 (20b #168).
- 8993 has an Elo of 12133 calculated by lots of ~55% win vs previous 20b versions with an inflation factor of about 5 (based on my previous calculation by comparison to the fixed strength of Elf), first 20b was 11775 elo.
- a304 was only 50.6% vs e2be4815, another bijyxo 40b.
- e2be4815 won 42% vs Elf v1.
- win% isn't transtitive, but transitive with one jump is far more reliable than inflated self-play of 10 jumps
- So without a match I'm pretty confident to say Elf v1 could well be stronger than a304.
- Some more comparisons: Elf v0 got 74% and v1 81% vs 9c56ae62 20b #166 which is a little weaker than #168. So this is the first transitive evidence that a304 is slightly better at beating up 20b LZ than Elf v1 is (but if it's at equal playouts 40b is getting more time). But I suspect newer versions of LZ are relatively better at beating old versions of itself than they are at beating external reference players (afterall that's how you choose promotion) so would still bet on Elf v1 being stronger than a304 in head-to-head.


This post by Uberdude was liked by: Gomoto
Top
 Profile  
 
Offline
 Post subject: Re: Learning Elfish
Post #8 Posted: Fri Aug 24, 2018 9:45 am 
Lives in sente

Posts: 1109
Location: Earth
Liked others: 359
Was liked: 181
Super Lizzie has a much better grasp on this position than ELF:

Attachment:
superlizzie.jpg
superlizzie.jpg [ 160.81 KiB | Viewed 1651 times ]



(While ELF struggels with the ladders)

Attachment:
elf.jpg
elf.jpg [ 160.92 KiB | Viewed 1650 times ]

Top
 Profile  
 
Offline
 Post subject: Re: Learning Elfish
Post #9 Posted: Fri Aug 24, 2018 10:15 am 
Dies with sente

Posts: 74
Location: France
Liked others: 10
Was liked: 13
Rank: FFG 1d
ELF v1 on my computer has similar results to your super leela. Actually, it considers the double hane: I stopped at 7.1k playouts for the double hane and I have only 65 playouts for the inside move.

Top
 Profile  
 
Offline
 Post subject: Re: Learning Elfish
Post #10 Posted: Fri Aug 24, 2018 10:30 am 
Lives in sente

Posts: 1109
Location: Earth
Liked others: 359
Was liked: 181
Yes, Elf v1 handles the situation different to Elf v0

Top
 Profile  
 
Offline
 Post subject: Re: Learning Elfish
Post #11 Posted: Sun Aug 26, 2018 12:21 pm 
Judan

Posts: 5805
Location: Cambridge, UK
Liked others: 328
Was liked: 3106
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
About the 40 block LZ, a more recent one did have a match with Elf v1 (probably equal visits so more time for 40b) and won 47%. So Elf still King but getting close.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group