This 'n' that

[Bill Spight]

Long games

The AlphaGo vs. AlphaGo games tend to be long. Kasparov noted that about computer vs. computer chess, so there may not be anything specific to go in that. But I think that there is.

There are two possible explanations that come to mind. One is ko fights, where several stones are played on only a couple of points of the board; another is that AlphaGo against itself tends to leave positions unsettled, or to make less settled positions than humans to start with. I think that both are true.

Why ko fights? It is a theorem in the mathematics of go that if the mean value of the whole board position (the count) is in your favor and you have the move (sente), then with perfect play you will win the game — unless there is a ko fight. (That theorem shows one advantage of thinking in terms of points, BTW. Oh, yes. If the opponent takes a ko and you do not play a ko threat, that is still a ko fight. ) So if your opponent has the advantage in terms of the count and the move, your only chance of winning (with perfect play) lies in a ko fight. If we generalize to imperfect play and imperfect positional judgement, if you perceive yourself to be behind, a ko fight may be your best bet to win. A mistake by the opponent may be, as well, but if your opponent is very good, like AlphaGo, then he won't make many mistakes. So realistically, if AlphaGo perceives itself to be a bit behind its twin, it may play for kos, and then if, a few moves later, its twin perceives itself to be a bit behind, it may welcome the coming ko fight, as well. OC, if AlphaGo plays a human and perceives itself to be ahead, it may avoid kos. But a close match where each player is trying to maximize its chances of winning may well generate kos.

Another possible explanation is that AlphaGo leaves positions unsettled or produces relatively unsettled positions to start with. Leaving positions unsettled may have a similar basis to ko fights in a close match, and again, when AlphaGo plays humans it often seems to take local losses to settle the local positions. Against its twin, it may leave them unsettled to offer better chances to win when it perceives itself to be behind. And then, because the match is so even, its opponent may soon perceive itself to be behind and also leave positions unsettled.

But I don't think that that is quite enough to explain making unsettled positions to start with. Fan Hui may have had such plays in mind when he said that AlphaGo is concerned with efficiency over territory. I think that he is right in that, but I disagree with the suggestion that humans are not. However, in its self-play games AlphaGo's style does seem to me to be quite light, and I suspect that that does have something to do with best play.

In these regards AlphaGo's play reminds me of two giants of 20th century go, Go Seigen for light opening play, and Kitani Minoru for making and leaving unsettled positions. They were fathers of the New Fuseki, and I think that that is related. With the New Fuseki the pendulum swung away from territorial play, too far, I expect, but now perhaps the pendulum has swung too far in the other direction. AlphaGo may be telling us to think less about territory and more about efficiency.

Edit: OC, there is a refutation to the idea that in a close match, each player will leave positions unsettled in order to preserve the chance of winning, namely that each player will settle positions when it perceives itself to be ahead in order to reduce the opponent's chances of winning. The answer to that, I think, is that in go it is generally easier to leave positions unsettled than to settle them. Tenuki is always an option.

[EdLee]

Is there always a reason why a move is bad, besides "the computer said so"?

Currently, Go is unsolved (as is chess): we don't what's the "correct" komi, and we don't know the result with perfect play -- B wins, W wins, or no result ?
Assuming the final outcome (with perfect play) must be one of the three, then each move can be labeled 'good' if it's on a branch toward a win, 'bad' if a loss, and 'neutral' if no result.
Is this the question ? Or is the question about in the absence of the entire game tree ?

[moha]

Is there always a reason why a move is bad, besides "the computer said so"?

Currently, Go is unsolved (as is chess): we don't what's the "correct" komi, and we don't know the result with perfect play -- B wins, W wins, or no result ?
Assuming the final outcome (with perfect play) must be one of the three, then each move can be labeled 'good' if it's on a branch toward a win, 'bad' if a loss, and 'neutral' if no result.
Is this the question ? Or is the question about in the absence of the entire game tree ?

Kasparov criticised young players for not understanding the reasons behind a move, besides "the computer said so". I think this quote can be thought of meaning "leads to good outcome as revealed by deep minimaxing". So, should we always expect other, human-like reasons, strategy etc. behind a move? See the Alphago cases quoted above, where even a pro reviewer could only conclude "the computer says so" (in essence).

[EdLee]

Hi moha,

I think this quote can be thought of meaning "leads to good outcome as revealed by deep minimaxing". So, should we always expect other, human-like reasons, strategy etc. behind a move? See the Alphago cases quoted above, where even a pro reviewer could only conclude "the computer says so" (in essence).

Would it be fair to say it's similar to asking "Will humans always 'understand' every AlphaGo-class+ move?" (emphases added.)
The answer seems very clearly a no IMO.

OC, this will not stop (at least some) people to continue to search for "human-centric" reasons, strategies, and explanations behind future AI moves. And in some cases, these human reasons will be in the ball park. In other cases, they'll be 'wildly' off.

One example Kasparov mentioned was "mate in 492 moves".
It's not unthinkable in the future to see "mate in 1,600 moves" (he mentioned 16,000+ ). Humans can try to come up with the most "reasonable" explanations for why the computer plays the first of the 1,600 moves, but as long as they cannot read past 100 (what's the current peak depth of the top people like Carlsen? ), then any and all human reasons are at best educated guesses, and at worst, pure fantasy.

[Bill Spight]

Hi, guys!

I doubt if humans will ever understand why planets go retrograde, either.

[EdLee]

Hi Bill!

Sorry, I missed the joke/reference...

Oh.

IMG_0029.GIF (63.49 KiB) Viewed 14963 times

[Kirby]

Kasparov criticised young players for not understanding the reasons behind a move, besides "the computer said so". I think this quote can be thought of meaning "leads to good outcome as revealed by deep minimaxing". So, should we always expect other, human-like reasons, strategy etc. behind a move? See the Alphago cases quoted above, where even a pro reviewer could only conclude "the computer says so" (in essence).

My interpretation of Kasparov's point was that young players accepted the computer's "answer" just because the computer said that it was correct. The "human-like reasons, strategy etc." do not provide a means of producing a more effective move than the computer in terms of optimality, however, "human-like reasons, strategy etc." provide us a path toward understanding how to produce good moves for other board positions.

As an illustration, AlphaGo seems to like invading at the 3-3 point, even when it is contrary to traditional human reasoning and strategy. Human reasoning and strategy that suggests not to invade into the 3-3 point is not superior to AlphaGo - perhaps AlphaGo's invasion was correct in the positions where AlphaGo played it.

But to simply accept some idea that "3-3 point is always a good place to play because AlphaGo played it" is neither educational nor true in all cases.

Instead, we'd better apply human reasoning and strategy to try to identify the reason that the 3-3 point was a good place to play at the times that AlphaGo played it. Otherwise, we're just guessing.

[EdLee]

"human-like reasons, strategy etc." provide us a path toward understanding how to produce good moves for other board positions.

Yes.

Human reasoning and strategy that suggests not to invade into the 3-3 point is not superior to AlphaGo...

Yes; seems an odd way to put it -- AG's evaluation is most likely superior to the current best human understanding, and the gap will only widen.

But to simply accept some idea that "3-3 point is always a good place to play because AlphaGo played it" is neither educational nor true in all cases.

Yes ( who would think like that ?! )

Instead, we'd better apply human reasoning and strategy... Otherwise, we're just guessing.

Yes, and with any luck, humans may be able to improve from studying AG's moves. ( Caveat: post 289 ). One question is: how much ?
I seem to remember someone ( Mr. Hassabis ? ) mentioned that the human chess pros haven't improved so much since Deep Blue and the subsequent advancement in chess engines. ( Anyone has the reference ? ) Humans will continue to play human chess ( running ), regardless of future improvements in chess engines ( automobiles, planes, spaceships, teleports... ).

[dfan]

One example Kasparov mentioned was "mate in 492 moves". It's not unthinkable in the future to see "mate in 1,600 moves" (he mentioned 16,000+ ). Humans can try to come up with the most "reasonable" explanations for why the computer plays the first of the 1,600 moves, but as long as they cannot read past 100 (what's the current peak depth of the top people like Carlsen? ), then any and all human reasons are at best educated guesses, and at worst, pure fantasy.

I haven't seen the context of the Kasparov quote, but chess has been completely solved for all positions with only a total of 7 or fewer pieces on the board (including the two kings) and for a lot of the positions, mostly ones without pawns, the correct moves literally make no sense, in that there is no discernable strategy or narrative to them; they can only be memorized. Luckily, those positions basically never come up in actual play (almost all endgames in practice are, narratively, about queening a pawn while avoiding stalemate).

[EdLee]

Hi dfan, yes, Kasparov indeed mentioned the 7-piece endgame database;
that was the context of his quote "mate in 492 moves": 22:05 ... ~22:40...

[moha]

Kasparov criticised young players for not understanding the reasons behind a move, besides "the computer said so". I think this quote can be thought of meaning "leads to good outcome as revealed by deep minimaxing". So, should we always expect other, human-like reasons, strategy etc. behind a move? See the Alphago cases quoted above, where even a pro reviewer could only conclude "the computer says so" (in essence).

My interpretation of Kasparov's point was that young players accepted the computer's "answer" just because the computer said that it was correct.

This is not that different from what I wrote. I doubt his point was that they didn't manually verify the machine reading.

As an illustration, AlphaGo seems to like invading at the 3-3 point, even when it is contrary to traditional human reasoning and strategy. Human reasoning and strategy that suggests not to invade into the 3-3 point

This is not a good example IMO. Alphago's 3-3 is backed by humanly understandable points - it changed the joseki slightly (no hane-and-connect), which - in certain cases - leaves the invader with sente and a somewhat better result than the old joseki. The Master games also often shown things that are explainable well, as Redmond's excellent reviews demonstrated. But OC, in those cases the AI was usually ahead, so played nice and solid moves that aimed to simplify the game.

I think human strategy and reasoning works, just does not always work, and will always lose to reading in the end. This is why it is less reliable in 9p / Alphago games - at that levels there is simply no acceptable alternative to deep reading. Which is basically what Lee Sedol said after his match.

[EdLee]

I think human strategy and reasoning works, just does not always work, and will always lose to reading in the end.

Yes. Running works, but only up to a point; after that, horses, etc.

Garry Kasparov quotes Picasso to the effect that computers are useless, they only give answers.

This is not the first time he reveals his shortsightedness.
~09:50 ... 11:00 interview around the Deep Thought match:

When Letterman asked Kasparov whether he thought the computer would ever beat the top human pros, he said no with great certainty.
( I don't remember his words exactly, but his feeling was very clear:
he thought the computer would never beat the top chess pros. )
OC, shortly afterwards, Deep Blue won.

His current quote to Demis shows he's not looking far enough ahead when AI will provide not only original questions but superior questions, superior solutions, and superior ways to teach the human brain.

[Bill Spight]

I think human strategy and reasoning works, just does not always work, and will always lose to reading in the end. This is why it is less reliable in 9p / Alphago games - at that levels there is simply no acceptable alternative to deep reading. Which is basically what Lee Sedol said after his match.

However, it seems clear that in most of the Master games this winter, Master took an early lead, too early for even deep reading to have much effect. The lead was based upon strategic superiority. The version of AlphaGo that played Lee Sedol last year was rather weaker that the current version. Humans have a lot to learn about strategy from AlphaGo (and probably other programs, as they get stronger). And I am confident that humans will learn a lot.

[Uberdude]

I think human strategy and reasoning works, just does not always work, and will always lose to reading in the end. This is why it is less reliable in 9p / Alphago games - at that levels there is simply no acceptable alternative to deep reading. Which is basically what Lee Sedol said after his match.

However, it seems clear that in most of the Master games this winter, Master took an early lead, too early for even deep reading to have much effect. The lead was based upon strategic superiority. The version of AlphaGo that played Lee Sedol last year was rather weaker that the current version. Humans have a lot to learn about strategy from AlphaGo (and probably other programs, as they get stronger). And I am confident that humans will learn a lot.

And Lee's win came from AlphaGo's failure to read a sequence of only about a dozen moves, well within the abilities of a decent amateur dan player.

[moha]

I think human strategy and reasoning works, just does not always work, and will always lose to reading in the end. This is why it is less reliable in 9p / Alphago games - at that levels there is simply no acceptable alternative to deep reading. Which is basically what Lee Sedol said after his match.

However, it seems clear that in most of the Master games this winter, Master took an early lead, too early for even deep reading to have much effect. The lead was based upon strategic superiority. The version of AlphaGo that played Lee Sedol last year was rather weaker that the current version. Humans have a lot to learn about strategy from AlphaGo (and probably other programs, as they get stronger). And I am confident that humans will learn a lot.

On the latter I agree. But why strategic superiority? I'm pretty sure it's leads were based on wholeboard minimaxing. This is where it's strength lies, it's main innovation: with NN the tree can be pruned enough that deep minimaxing is possible, even in a non-local sense. And it makes MC effective for the first time. Even in the opening, reading dozens of moves ahead, without significant oversights, always taking the biggest points, best approach moves, rarely losing sente, and even then leaving the best followups, etc. - wholeboard minimaxing was never seen before, so most people underestimate it's power.

Fortunately, most of it's leads could be explained by human reasoning as well - these are the cases we can learn from. But there were also the cases I quoted above, where even the reviewer couldn't explain why it got ahead with a seemingly worse strategy. And actually the few cases where Master fell behind in the beginning, were because of the human's strategy and choice of direction was better (and he either didn't make big enough mistakes early on, or at least the reviewer could point out the correct sequence that would have left Master with a losing game). These were cases where the consequences of a poor strategic choice fell outside the reading horizon, I think.

And Lee's win came from AlphaGo's failure to read a sequence of only about a dozen moves, well within the abilities of a decent amateur dan player.

I don't think this is surprising. Heavy pruning always have the risk of a missing a critical move. Alphago's lines are nowehere near optimal (IIRC the NN's are low dan levels by themselves), they are just reasonably correct, so that they work well in cases where there is no single arcane keymove to be missed (opening and early middlegame). So it looks at most moves that humans would find somewhat interesting, and can read them very deep. And if you try millions of such lines they are informative about the leaf position as well (MC). No wonder it prefers solid positions, that won't suffer from overlooking a critical move.

BTW, these are also the reasons I believe it is nowhere near perfect play yet. Just much stronger than what we have seen so far.