It is currently Thu Mar 28, 2024 3:25 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 18 posts ] 
Author Message
Offline
 Post subject: Elf teaches us how to play the black hole vs LeelaZero
Post #1 Posted: Fri Jun 15, 2018 9:15 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
baduk1 continues his interesting bot vs bot experiments with forcing Elf to play the black hole opening vs LeelaZero, Elf gave itself 4% winrate to start:





This post by Uberdude was liked by 3 people: Bill Spight, dsatkas, swannod
Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #2 Posted: Fri Jun 15, 2018 3:39 pm 
Oza
User avatar

Posts: 2777
Location: Seattle, WA
Liked others: 251
Was liked: 549
KGS: oren
Tygem: oren740, orenl
IGS: oren
Wbaduk: oren
What was the end result?

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #3 Posted: Fri Jun 15, 2018 4:21 pm 
Lives in gote

Posts: 502
Liked others: 1
Was liked: 153
Rank: KGS 2k
GD Posts: 100
KGS: Tryss
If I'm right, black win by 1.5 points

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #4 Posted: Fri Jun 15, 2018 4:26 pm 
Dies in gote

Posts: 39
Liked others: 40
Was liked: 10
Black wins by 1.5 points if I count correctly.

Edit: Tryss was quicker-

_________________
Couch Potato - I'm just watchin'!


Last edited by Baywa on Fri Jun 15, 2018 4:27 pm, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #5 Posted: Fri Jun 15, 2018 4:27 pm 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
#me too. :D

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #6 Posted: Fri Jun 15, 2018 8:28 pm 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Me too, but I didn't add it to the sgf so as not to spoil the ending. :)

Something I'd like to check here is did LZ fail to see Elf's good moves or see but misevaluate them?


This post by Uberdude was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #7 Posted: Sat Jun 16, 2018 6:00 am 
Oza
User avatar

Posts: 2401
Location: Tokyo, Japan
Liked others: 2338
Was liked: 1332
Rank: Jp 6 dan
KGS: ez4u
The question is what happens when you switch colors? Or else give LeelaZero 16k visits? :rambo:

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #8 Posted: Tue Jun 19, 2018 5:07 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Uberdude wrote:
Something I'd like to check here is did LZ fail to see Elf's good moves or see but misevaluate them?


So the first move of Elf's that stood out to me as a strong move was q7. Leela #145 doesn't consider this at all until 1k nodes (the 2 extends are main choices), around 2k it notices it is strong and examines it more and is #1 by 3k with a 27.4% winrate. It gave the inside hane a few moves prior 25.2%. I then updated to the latest #149 network and it found q7 within 300 playouts and was #1 in under 1k.


This post by Uberdude was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #9 Posted: Thu Jun 21, 2018 2:16 pm 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Next nice move of Elf, this cutting across knight's move in response to white j16. It's the natural shape weakness so I think plenty of strong humans could find it, but the timing is smart. Letting LeelaZero #150 analyse for ages she considers k15 practically the only move with 120k playouts and 28.4%, e17 gets 5 (not 5k). Once played LZ quickly realises it's a good move, black win up to 35.3%, a 7% swing which is pretty huge for 1 move. A severe blindspot, looks like LZ needs to be more exploratory. The continuation is as expected.

Click Here To Show Diagram Code
[go]$$Wcm54
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . X O . . . |
$$ | . . . O 2 . O . X . . . . . X . . . . |
$$ | . . O , . O . O 1 X . . . . . , O . . |
$$ | . . X X . X X . . . . . . . . X O . . |
$$ | . . . . . . . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . X . O . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . X . . |
$$ | . . O , . . . . . , . . . . . , O . . |
$$ | . . . . . . . . . . . . . . . . . O . |
$$ | . . . . . . . . . . . . . . . . O . O |
$$ | . . O . X . . . . . . . . . . X X O . |
$$ | . . . . . . . . . . . . . . . X . X . |
$$ | . . . . . . . . . . . . X . . X . X . |
$$ | . . O , . . . . . , . . . . X O X . . |
$$ | . . . . X . . . O . O . . O O O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


This post by Uberdude was liked by 2 people: Bill Spight, gamesorry
Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #10 Posted: Thu Jun 21, 2018 2:58 pm 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
Uberdude wrote:
Next nice move of Elf, this cutting across knight's move in response to white j16. It's the natural shape weakness so I think plenty of strong humans could find it, but the timing is smart. Letting LeelaZero #150 analyse for ages she considers k15 practically the only move with 120k playouts and 28.4%, e17 gets 5 (not 5k). Once played LZ quickly realises it's a good move, black win up to 35.3%, a 7% swing which is pretty huge for 1 move. A severe blindspot, looks like LZ needs to be more exploratory.

(Emphasis mine.)

Is that the result of self-play? Or is it just one of those things that we can expect and have to put up with? OC, no player is going to be perfect, but it seems like the current Zero bots have deep, if contained, weaknesses that they might not have if they had trained against a variety of opponents.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #11 Posted: Thu Jun 21, 2018 4:05 pm 
Lives in gote

Posts: 311
Liked others: 0
Was liked: 45
Rank: 2d
I think such blind spots are a tricky problem also involving the search code of the bot. Even in training it will only learn moves that its search can find within the low selfplay visit limit (in each position it is trained towards search results). So if there is a blind spot in the net, that can only get fixed if the bot doesn't rely too much on its policy during selfplay search - otherwise it will just keep reinforcing the oversight.

It seems this problem affects bots to different extent though, with LZ search being one of the most rigid / least exploratory. But it also uses ELF net for some selfplay games now, so some blind spots should start to close (ELF based search can see different moves).

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #12 Posted: Fri Jun 22, 2018 2:31 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
A relevant game I came across today, Alexander Dinerstein 3p shows us how not to use the black hole opening against the much stronger Mi Yuting 9p:


Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #13 Posted: Thu Jul 05, 2018 4:09 pm 
Dies in gote

Posts: 44
Liked others: 2
Was liked: 14
Rank: EGF 1 kyu
KGS: finity
Alexander is a great teacher to us all. I hope we'll see many more educational games in the future!

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #14 Posted: Thu Jul 05, 2018 5:28 pm 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
moha wrote:
I think such blind spots are a tricky problem also involving the search code of the bot. Even in training it will only learn moves that its search can find within the low selfplay visit limit (in each position it is trained towards search results). So if there is a blind spot in the net, that can only get fixed if the bot doesn't rely too much on its policy during selfplay search - otherwise it will just keep reinforcing the oversight.


IIUC, Monte Carlo bots are guaranteed to find the correct play in infinite time. I would be surprised if the Zero bots don't meet the same criterion. OC, that does not mean that they cannot develop blind spots that will last for millennia. :lol:

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #15 Posted: Fri Jul 06, 2018 7:26 am 
Lives in gote

Posts: 311
Liked others: 0
Was liked: 45
Rank: 2d
Bill Spight wrote:
moha wrote:
I think such blind spots are a tricky problem also involving the search code of the bot. Even in training it will only learn moves that its search can find within the low selfplay visit limit (in each position it is trained towards search results). So if there is a blind spot in the net, that can only get fixed if the bot doesn't rely too much on its policy during selfplay search - otherwise it will just keep reinforcing the oversight.
IIUC, Monte Carlo bots are guaranteed to find the correct play in infinite time. I would be surprised if the Zero bots don't meet the same criterion. OC, that does not mean that they cannot develop blind spots that will last for millennia. :lol:
This convergence you mention means the bot will find the best move if it is run in a given position for infinity (though even this only holds if all moves are guaranteed to get infinite further visits, which may not always be the case with NN-based pruning). But there is no infinite search during selfplay, and the training problem I wrote above may not show the same convergence.

In a given position 2-3k visits may give the same results as the current raw policy net (which can be an oversight, and, say, 100k visits could find a better move - but this will never happen). In this case no learning takes place - the network is just just trained towards its current output, reinforcing it. So the blind spot could only be fixed if the bot comes across a position where it manifests AND the visit limit is enough for the search to find the correct move (or at least slightly more correct evaluations) EVEN if started from the wrong policy distribution. This latter part, the level of exploration is where various bots differ: some only look at moves that have decent policy weights, some spend a few visits on less promising moves as well.

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #16 Posted: Fri Jul 06, 2018 8:36 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
Improvement through self play requires a different kind of search from game tree search, a search which explores variations of the self. I pointed out the possibility of self reinforcing blind spots long ago. In fact, I suggested the possibility that AlphaZero Chess reached a self play plateau that could not beat a version of Stockfish which used an opening book and endgame table base. I am unaware of anybody else suggesting that.

One possible response to plateaus and blind spots is to increase the search during self play games. I had that possibility in mind when I said that correct play could be found in infinite time. ;) (Not that I would take that approach to overcoming the currently identified blind spots. :))

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #17 Posted: Fri Jul 06, 2018 10:50 am 
Gosei

Posts: 1590
Liked others: 886
Was liked: 527
Rank: AGA 3k Fox 3d
GD Posts: 61
KGS: dfan
Bill Spight wrote:
I pointed out the possibility of self reinforcing blind spots long ago. In fact, I suggested the possibility that AlphaZero Chess reached a self play plateau that could not beat a version of Stockfish which used an opening book and endgame table base. I am unaware of anybody else suggesting that.

People used to be pretty worried about self-play plateaus until AlphaGo Zero came along and slaughtered all the top pros. :) But they may have been too quick to stop being concerned about it.

Top
 Profile  
 
Offline
 Post subject: Re: Elf teaches us how to play the black hole vs LeelaZero
Post #18 Posted: Fri Jul 06, 2018 1:16 pm 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
dfan wrote:
Bill Spight wrote:
I pointed out the possibility of self reinforcing blind spots long ago. In fact, I suggested the possibility that AlphaZero Chess reached a self play plateau that could not beat a version of Stockfish which used an opening book and endgame table base. I am unaware of anybody else suggesting that.

People used to be pretty worried about self-play plateaus until AlphaGo Zero came along and slaughtered all the top pros. :) But they may have been too quick to stop being concerned about it.


Well, if your goal is just to beat all humans, what's the problem? ;)

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.


This post by Bill Spight was liked by: dfan
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 18 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group