Why humans fail against AI

[Elom]

14 years ago, Charles Matthews wrote one of the Sensei's Library articles that still stands as it was: "The Articulation Problem" https://senseis.xmp.net/?ArticulationProblem

(giving you some time to read it)

Thinking of it, I am also reminded of my university days, where I would have a "Higher Algebra" professor teaching us about Lie groups. He was not a very good teacher and we had to read a lot of books from the library to get the theory. Incidentally, that poor teaching method, forcing self learning, proved more effective than some of the well conceived syllabi and teaching by professors better at articulating their craft. At the time I held the conviction that someone who could not well articulate their knowledge, probably didn't understand it very well either.

Today I no longer think that way. I now believe that beyond a certain level of expertise, it becomes very difficult to convey your understanding in plain language...

I heard that a study was conducted on which a four-year-old was told that a box contained sweets. Upon opening it, the child found that it actually contained pencils. And when asked on what another child who walked into the room with no way of knowing what was in the box would think, the answer was: pencils.

Minblindness is one of many names for this phenomenon, and we do not grow out of it entirely.

[Bill Spight]

Also remember that Leela Zero is just trying to give itself advice, despite the fact that it has allowed you to peek at its thoughts. It has learned that opening at the 4-4 and 3-4 points work well, and is reminding itself of that fact. 3-3 and tengen may be just a little worse, but there's no real need for it to spend a lot of time thinking about them when it has already learned through millions of games that they're not as good as the move it likes better.

"3-3 and tengen may be a little worse." Or not.

The Zero bots are trained by a form of reinforcement learning. I have some experience with different forms of reinforcement learning with animals and humans. The 4-4 and 3-4 points may be examples of what are called superstitions. B. F. Skinner ran an experiment in which he gave individual hungry pigeons pellets of food at slightly irregular, short intervals of time. After a while he noticed that each pigeon was engaging in different repetitive behaviors. One was turning around in circles, as I recall. What had happened was that each pigeon was doing something when it got fed. Then it tended to do the same thing, and, as it happened, got fed again when it was doing it. So it kept doing it. Skinner dubbed these irrelevant but reinforced behaviors superstitions. Similarly, if bots develop an early preference for 3-4 and 4-4 points, and other plays are not much better, then the 3-4 and 4-4 points will get preferentially reinforced. Even if, as it turns out, the other plays are slightly better.

That said, I don't really believe that the 3-4 and 4-4 plays are superstitions. IMO, the 3-3 may be an "inaccuracy", slightly inferior but not a mistake. Tengen may be a mistake, however.

If a lot of zero bots converge on the 3-4 and 4-4, that is good evidence that they are not superstitions.

----

There is also the personal equation. For instance, if you like the 3-3, you may well get better results with it than with the 3-4 or 4-4, even if they are perhaps "objectively" better. Dosaku liked the 3-5, but did not overcome the prejudice against the 4-4. It was centuries before humans did that.

[Calvin Clark]

If a lot of zero bots converge on the 3-4 and 4-4, that is good evidence that they are not superstitions.

Unfortunately a lot of Zero Bots are effectively clones of AlphaGo Zero.

So it's like we are studying white mice and thinking mice should be blind. Not that we can't learn something from these mice. We learn a lot. But we don't learn that they are the only possible mice.

[Bill Spight]

If a lot of zero bots converge on the 3-4 and 4-4, that is good evidence that they are not superstitions.

Unfortunately a lot of Zero Bots are effectively clones of AlphaGo Zero.

So it's like we are studying white mice and thinking mice should be blind. Not that we can't learn something from these mice. We learn a lot. But we don't learn that they are the only possible mice.

But they start off playing randomly, right? Perhaps there is something in the Tromp-Taylor rules that favors the 3-4 and 4-4, but otherwise they might converge to those two plus the 3-3 or maybe just the 3-3 and 4-4, or some other combination. No?

[Kirby]

Is it possible that there's some sort of characteristic of the AlphaGo Zero type algorithm that lends itself toward converging at 3-4 or 4-4, even if those moves aren't optimal?

Like a local maximum or something?

Like if all the pigeons pecked at their food, but it turns out that some other bird swallows it whole?

I don't know enough about REL to tell (even if I used it for a project).

[Bill Spight]

Is it possible that there's some sort of characteristic of the AlphaGo Zero type algorithm that lends itself toward converging at 3-4 or 4-4, even if those moves aren't optimal?

Like a local maximum or something?

Like if all the pigeons pecked at their food, but it turns out that some other bird swallows it whole?

I don't know enough about REL to tell (even if I used it for a project).

I don't know if we can talk about humans converging on anything, but certainly the 4-4 became popular in the 20th century.

Years ago I played around with a weak bot with a form of reinforcement learning. FWIW, it came to prefer the following plays for the first move in this order:

3-4
4-4
3-5
4-5
5-5
3-3
.

It was not strong, but it was different.

[Calvin Clark]

From MiniGo results

Worth observing: For nearly all of the generations, 3-3 was its ONLY preferred opening move. This drove a lot of the questions around whether or not we had adequate move diversity, sufficient noise mixing, etc. It is unclear whether the early errors (e.g., initial 50-100 generations with broken batch-norm) have left us in a local minima with 3-3s that D-noise & etc. are not able to overcome.

[Bill Spight]

The question I'm really asking here, continuing on Charles' great article, is: should we humans keep trying conveying the knowledge we draw from professionals (now AIs) in the carrier that has be en so successful for mankind, language, or should we express thoughts in a more effective way, one that is closer to professional/AI thinking? And what would such "language" look like?

There is already such a very expressive language, both for professionals and AIs: it is a deep and thick tree of variations. When professionals analyze a game, they don't just talk about it in abstract high-level ideas, but they lay down tons of very long variations. The public commentaries that one usually sees, which are meant for amateurs, are really just the tip of the iceberg compared to a pro-to-pro analysis of a game.

This language is supplemented by verbal language or tacit understandings about both the evaluations at the end of the variations and about pruning the search tree, that is, which variations not to consider. As a rule, amateurs cannot do either with much confidence.

However, when the final evaluations are clear, such as with tsumego and endgames, I think that amateurs can also benefit from such thorough analysis. In particular, I have been impressed by the thoroughness of Mr. K's pages on life and death. http://mrkigo.sakura.ne.jp/ksikatuindex.html

[Bill Spight]

From MiniGo results

Worth observing: For nearly all of the generations, 3-3 was its ONLY preferred opening move. This drove a lot of the questions around whether or not we had adequate move diversity, sufficient noise mixing, etc. It is unclear whether the early errors (e.g., initial 50-100 generations with broken batch-norm) have left us in a local minima with 3-3s that D-noise & etc. are not able to overcome.

It seems like they discovered overfitting in part because symmetrical positions produced quite different results. Did they not train over all symmetries? (Except Black and White, OC.)

[John Fairbairn]

I have been watching a game with LeelaZero playing itself via Lizzie, which means I get to see which moves it is considering.

That experience led me to thinking that AI is not going to teach us anything explicit, such as new proverbs, but not because bots can't speak (yet). I am moving towards the view that human pros are already doing what they do correctly - they are just not doing enough of it.

I was struck by how many times I'd look at the board where Lizzie showed the candidate moves, and I'd say to myself, "Yes, I'd consider those." I already usually have that experience when reading pro commentaries. Almost all the moves they talk about are on my radar. I think most amateur dan players, and probably a lot of kyu players, would be able to say the same thing. And of course pros would be able to say a fortiori.

The main problem we amateurs have is discriminating between each move. Evaluation, in other words. But I get the sense that this is not a problem for most pros. They can evaluate as well as bots, and conceivably better.

Where I sense the bots may be superior is that they come up with more candidate moves, and quite a few of these are unexpected to me, and I suspect they would be surprising to pros. After all, we have O Meien telling us pros just didn't think about early shoulder hits, as obvious as they seem now. We see pros pay the hanetsugi after the 3-3 invasion. There are many stories about even pros having blind spots.

It's easy to see how this syndrome may come about. We are creatures of habit (we are the rotating pigeon someone else mentioned here). We seek to prune the list of candidate moves so we can search deeper, or because we are under time pressure. We hunch over the board in concentration, losing our peripheral vision. We do not get short, sharp shock lessons very often because other players operate the same way.

But what I saw on Lizzie a significant number of times was a surprising move away from the main action, or a move that was surprising because the surrounding shape in which it was played was unfamiliar. Also, especially later in the game when you might expect a lot of forcing moves and one-way streets to be available, the bot was looking at a dozen or more moves and maybe in five different parts of the board.

I sense therefore the bots have a much richer palette of moves. If these were all shown to a pro, I suspect he could evaluate them all as well as the bots. But if he never sees them for himself, he can never use them. On a small scale, the problems amateur have with tsumego can be seen as an analogue of this.

If I'm right, pros need primarily to learn to widen their gamut of candidate moves.

Has anyone else had thoughts along these lines?

[Bill Spight]

Interesting observations, John.

As for new proverbs, I think we will get some. Coming up with proverbs is one thing humans are good at.

From what you say, I hope that you are optimistic about the ability of humans to learn from current bots. I am.

I do think that the bots are better than pros at evaluating influence. Pros will get better at that.

I am not so sure that bots have a richer palette of moves than pros. Much of human thinking is unconscious, so we don't really know what moves a pro has processed without bringing them to consciousness. Lizzie lets us see such plays. You can see this kind of thing in language processing. Humans are blissfully unaware of most of the ambiguity of everyday language. It's a good thing, too.

If I'm right, pros need primarily to learn to widen their gamut of candidate moves.

That's what I tell kyu players. Maybe that's what a bot would tell pros, if it could talk.

But what I saw on Lizzie a significant number of times was a surprising move away from the main action, or a move that was surprising because the surrounding shape in which it was played was unfamiliar.

Yes, it seems to me that the bots play tenuki more often than humans. A corollary to that is that shapes arise that are unfamiliar to us.

[Uberdude]

In middle game positions I agree that LZ can consider a wider variety of candidate moves, and make a high quality evaluation of them far more quickly than a pro (you can add more cores to a GPU, you can't add more brains to a pro) so choose the best one, but I disagree pros evaluation of choices they both think of are better. I think it's worse (ladders and other bot foibles on low playouts excluded), and considerably so in the kind of open positions without clearly defined goals in which you see pros says things like "both possible, too hard to say" (though maybe they have stronger private thoughts). This is illustrated by pros being 20% win (and feeling behind themselves too) after just 40 moves of opening where LZ doesn't play moves the pro wouldn't conceive of. LZ is just playing 20 moves in a row picking the right one every time (or at least only a tiny bit wrong), whereas the pro maybe gets half right, a handful of minus a few percent, and makes a 10% mistake every 20 moves or so.

Take this knight move answer as an example. Elf v1 thinks it's a big (7%) mistake. LZ thinks it's basically fine; in a few months/years maybe LZ will come to dislike it too (Facebook have lots more computing resources that the LZ volunteer contributors).

$$B
$$ +---------------------------------------+
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 6 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . X . . . |
$$ | . . . . . X . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ +---------------------------------------+

Click Here To Show Diagram Code

[go]$$B
$$ +---------------------------------------+
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 6 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . X . . . |
$$ | . . . . . X . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ +---------------------------------------+[/go]

Or take the recent Iyama vs Cho Meijin game, the post game comments

After the game， Iyama Meijin said that he had not been sure how well he was doing after the first day， but started to feel comfortable with his lead in territory after the ko on the right with which he captured several White stones. Cho 9P agreed， saying that he was happy with his position on the first day， but started to realize he was behind while playing the ko on the right. Cho said he put a good effort into the game， and would try his best in the following games of the match.

I think there's a lot of cultural modesty/politeness getting in the way here, but if Elf/LZ are to be believed the summary should be more like Iyama saying "I was doing great after Cho played move after move of mistakes in the lower left, I got a big lead, but then just towards the end of the first day I noobed it up and it went back to even again". Does he have the evaluation/judgement skills to see this (either live or on reflection with lots of time and analysis, and as a suspicion or a confident belief?) but is just too polite to say? Something I might try is playing out this game with the greedy LZ version (that doesn't throw away points so long as it wins) to see what these big leads translate to in point margin at the end of an LZ vs LZ game.

And yes, it is interesting to watch the bots play against each other. I recently started a Elf v0 vs Elf v1 game, it was interesting to see where they differed in their evaluations and expected sequences: v0 though it was winning and v1 more even for a while, but that was shown to be predicated on v0 having a blindspot for a good move v1 played later (and one of my top 5 candidate moves). I probably won't have time to finish it so I'll upload my annotated sgf in the Elf thread.

P.S. Here's a new proverb from bots:

When your 4-4 gets invaded at 3-3, block the wrong* side.

* according to the old advice of blocking the "wider" or one with more scope for development side. It's not as perverse as it sounds, with the jump and double hane or inside cut joseki you counter-intuitively end up blocking off the side you didn't block with the 3-4 stone after the 3-3.

[Bill Spight]

P.S. Here's a new proverb from bots:

When your 4-4 gets invaded at 3-3, block the wrong* side.

* according to the old advice of blocking the "wider" or one with more scope for development side. It's not as perverse as it sounds, with the jump and double hane or inside cut joseki you counter-intuitively end up blocking off the side you didn't block with the 3-4 stone after the 3-3.

[Bill Spight]

BTW, there was a bot called GoGod that showed its estimated winrates online, and presumably elsewhere, during the game. Perhaps doing so will make go more interesting to spectators who are not good enough to have much idea of who is ahead and by how much. It's like having a scoreboard for go.

[Gomoto]

During an EARLY 3-3 invasion the difference between the sides is usually <1% with several networks. (okay against sanrensei its 3% difference)

Blocking side does not matter. Choose what you like is my advice.