How often will the next step of a ladder occur with quasi-random play?

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

I think the net will first learn to trust that all unclear ladders work. It is a much worse mistake to run out of a long ladder then eventually works than to chase stones in a long ladder that eventually fails. It is true that the next step of a ladder is always likely to be somewhat likely in quasi-randomized play since the net learns quickly that you should consider all ataris, so playing the ladder out for a few steps should still happen sometimes, but for ladders long enough to get the net into trouble that's a lot of coincidence (and all the random trials where it plays the ladder out a third of the way across the board and then plays away are going to end badly - which will it learn faster, don't play unclear ladders or don't stop playing a ladder that's already gone a few steps?).

Why quasi-random play?

With probabilistic reasoning, perhaps there is a threshold, partly dependent upon circumstances, where the ladder is long enough for both sides to play out, because losing the long ladder will lose the game. Playing it out is the only chance to win.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Because I did not think that the choice of plays was completely random.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Is Bill's point that the quasi-random play of monte carlo rollouts will not be very good at playing ladders so it won't learn them well, but moha's counter point that Leela Zero doesn't use monte carlo rollouts?

I actually have trouble following the logic myself.

My original idea was that - as seen with AlphaGo vs. humans - a bot can understand something like "danger, unclear". And avoid it (even at some cost) when ahead but seek it when behind. With a smallnet Zero selfplay, the bot has no reason to fear being exploited (since the opponent cannot see/read the ladder either), so what remains is "random" probability (e.g. ladders are a bit more likely to work on empty boards than later stages) with huge variance on game result.

I guess Bill may hint at difficulties for the early network in collecting experience at all, thus reaching this point? At first stages with close to random play, ladders are not played. Deepmind also noted that - unlike humans - a bot learns about ladders relatively late.

No, my point is that if you have two players who share a blindness, it is difficult for them to learn from each other to overcome that blindness. It has nothing to do with AI per se. Humans have the same problem.

It was moha who brought up randomness in the play, and I thought that it was not pure randomness, hence "quasi-random".

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Right. Even human beginners can apply logic. (Ladders are logical.) Could bots learn ladders using logic? Of course. But not the bots that are currently in vogue.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Something I found interesting to ponder about: Leela Zero is about 1 dan strength now I believe (+/- a few stones) and it likes to play early 3-3 invasions just like AlphaGo (both regular and Zero version) does. It's not very strong yet so it's not some respected oracle like AG that we would emulate, but does this mean it has also discovered some objective truth that early 3-3s are good (which is basically what we have surmised from AG), or is this more like some bias or quirk of how machine learning Go programs work and they get stuck playing them. AG never outgrew this trait and got really strong, but is it really strong because of or despite these 3-3 invasions?

I've also been wondering about the early 3-3 playing style. One thing that is on my mind, is the settings under which the network is trained.

LeelaZ/AlphaGo plays every training game with only 1600 playouts, added noise, and extra randomization in the opening.

The value which the network puts to each move, is based on games under these settings. If an opening is 47% winrate for black, it is because to the network, it looks like something where it would win 47% of the time as black under the above settings.

So you can ask - what kind of strategy can you develop when you and your opponent have limited reading ability, and you're pseudo drunk? If killing groups inside a moyo is harder in general and requires precise reading, then taking guaranteed territory sounds pretty tempting, but that's of course just speculation

So to expand upon the question of whether or not these moves are objectively good for any skill level, we can also ask if they're objectively good under any time setting/reading ability.

Would a network trained with 100.000 playouts in its training games play differently?

A quick note. The evaluation in terms of percentage of wins assumes errors in play. Otherwise the evaluation will be 100% or 0%. We cannot therefore identify objectively good or bad plays.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

This is good pedantry, but I think we can improve it .

The evaluation in terms of percentage of wins is the product of a process which incorporates errors in play. Otherwise the evaluation will be 100% or 0%. We cannot therefore identify objectively good or bad plays without some probability of error (and we do not know how big that error is).

Big picture: we don't know how far AlphaGo was from perfect play, so at some level, we can cast doubt on all of its evaluations. It's possible that the early 3-3 points are just a bias that were never exposed because humans aren't strong enough to take advantage of them. However, if we want to assess whether a move is objectively good, in most cases our best shot will be the AlphaGo oracle—for now.

_________________
Occupy Babel!

Also: there is no reason to assume AlphaGo is any more biased than we are. All we know is it's better than us, so, in principle, we should assume its biases are better than ours, too.

My point is this bias of liking early 3-3 invasions is not only present when it is better than us though. AG and Leela Zero like them early in their training when they are weak players, less than 1 dan. It seems odd to me that such a weak player could have discovered something closer to the objective truth of go than humans did collectively over millenia of play (well I suppose opening at 4-4 is relatively recent if we don't count the ancient chinese cross 4-4 starting position). So something like Bill suggested that a preference for corner over centre territory arrising from the way tromp taylor rules score incomplete positions seems a plausible avenue of investigation. Or maybe it's just because they are simple. Just thinking out loud AG Lee didn't do early 3-3s but Master did. Can we explain this as it needing a lot of self play to rid itself of the human bias of not playing them, whereas training from zero it can start liking them early. I suppose AG/L Zero also start doing knight move approaches to corner stones when they are weak, but as that agrees with established human knowledge I'm not surprised that they don't "grow out" of this habit or see it as remarkable.

Since a few days Leela Zero is on a 6x128 Network and since a fix yesterday with great improvements: http://zero.sjeng.org/

It will likely soon catch up the original Leela!