Leela Zero Stuck

[Aram]

Worth reading for those wondering about what GCP has planned for the future:

https://github.com/gcp/leela-zero/issues/591

[Uberdude]

LeelaZero having ladder troubles against a human on OGS:

[moha]

LeelaZero having ladder troubles against a human on OGS

I would expect the network to realize it's own vision range problem (from selfplay), and preemptively avoid unclear ladders. It's interesting this does not happen.

[Bill Spight]

LeelaZero having ladder troubles against a human on OGS

I would expect the network to realize it's own vision range problem (from selfplay), and preemptively avoid unclear ladders. It's interesting this does not happen.

Perhaps mutual blindness? Neither version of LeelaZero whether the ladder works, and so it is never played out, and LeelaZero never learns. (Or it will take a long time.) As for avoiding unclear ladders, not much is clear, is it? Especially in the opening.

[moha]

LeelaZero having ladder troubles against a human on OGS

I would expect the network to realize it's own vision range problem (from selfplay), and preemptively avoid unclear ladders. It's interesting this does not happen.

Perhaps mutual blindness? Neither version of LeelaZero whether the ladder works, and so it is never played out, and LeelaZero never learns. (Or it will take a long time.) As for avoiding unclear ladders, not much is clear, is it? Especially in the opening.

There is a point in playing out unread ladders though, for randomization. So during selfplay the side behind may play it.

[Bill Spight]

LeelaZero having ladder troubles against a human on OGS

I would expect the network to realize it's own vision range problem (from selfplay), and preemptively avoid unclear ladders. It's interesting this does not happen.

Perhaps mutual blindness? Neither version of LeelaZero whether the ladder works, and so it is never played out, and LeelaZero never learns. (Or it will take a long time.) As for avoiding unclear ladders, not much is clear, is it? Especially in the opening.

There is a point in playing out unread ladders though, for randomization. So during selfplay the side behind may play it.

How often will the next step of a ladder occur with quasi-random play?

[kwhyte]

I think the net will first learn to trust that all unclear ladders work. It is a much worse mistake to run out of a long ladder then eventually works than to chase stones in a long ladder that eventually fails. It is true that the next step of a ladder is always likely to be somewhat likely in quasi-randomized play since the net learns quickly that you should consider all ataris, so playing the ladder out for a few steps should still happen sometimes, but for ladders long enough to get the net into trouble that's a lot of coincidence (and all the random trials where it plays the ladder out a third of the way across the board and then plays away are going to end badly - which will it learn faster, don't play unclear ladders or don't stop playing a ladder that's already gone a few steps?).

[moha]

There is a point in playing out unread ladders though, for randomization. So during selfplay the side behind may play it.

How often will the next step of a ladder occur with quasi-random play?

Why quasi-random play?

[Bill Spight]

With probabilistic reasoning, perhaps there is a threshold, partly dependent upon circumstances, where the ladder is long enough for both sides to play out, because losing the long ladder will lose the game. Playing it out is the only chance to win.

[Bill Spight]

There is a point in playing out unread ladders though, for randomization. So during selfplay the side behind may play it.

How often will the next step of a ladder occur with quasi-random play?

Why quasi-random play?

Because I did not think that the choice of plays was completely random.

[Uberdude]

Is Bill's point that the quasi-random play of monte carlo rollouts will not be very good at playing ladders so it won't learn them well, but moha's counter point that Leela Zero doesn't use monte carlo rollouts?

[moha]

Is Bill's point that the quasi-random play of monte carlo rollouts will not be very good at playing ladders so it won't learn them well, but moha's counter point that Leela Zero doesn't use monte carlo rollouts?

I actually have trouble following the logic myself.

My original idea was that - as seen with AlphaGo vs. humans - a bot can understand something like "danger, unclear". And avoid it (even at some cost) when ahead but seek it when behind. With a smallnet Zero selfplay, the bot has no reason to fear being exploited (since the opponent cannot see/read the ladder either), so what remains is "random" probability (e.g. ladders are a bit more likely to work on empty boards than later stages) with huge variance on game result.

I guess Bill may hint at difficulties for the early network in collecting experience at all, thus reaching this point? At first stages with close to random play, ladders are not played. Deepmind also noted that - unlike humans - a bot learns about ladders relatively late.

[Bill Spight]

Is Bill's point that the quasi-random play of monte carlo rollouts will not be very good at playing ladders so it won't learn them well, but moha's counter point that Leela Zero doesn't use monte carlo rollouts?

No, my point is that if you have two players who share a blindness, it is difficult for them to learn from each other to overcome that blindness. It has nothing to do with AI per se. Humans have the same problem.

It was moha who brought up randomness in the play, and I thought that it was not pure randomness, hence "quasi-random".

[Bill Spight]

Deepmind also noted that - unlike humans - a bot learns about ladders relatively late.

Right. Even human beginners can apply logic. (Ladders are logical.) Could bots learn ladders using logic? Of course. But not the bots that are currently in vogue.

[Uberdude]

Something I found interesting to ponder about: Leela Zero is about 1 dan strength now I believe (+/- a few stones) and it likes to play early 3-3 invasions just like AlphaGo (both regular and Zero version) does. It's not very strong yet so it's not some respected oracle like AG that we would emulate, but does this mean it has also discovered some objective truth that early 3-3s are good (which is basically what we have surmised from AG), or is this more like some bias or quirk of how machine learning Go programs work and they get stuck playing them. AG never outgrew this trait and got really strong, but is it really strong because of or despite these 3-3 invasions?