AlphaZero paper discussion (Mastering Go, Chess, and Shogi)

[pookpooi]

Last word on Shogi world

Late last year, a film about shogi player Satoshi Murayama called Satoshi no Seishun hit Japanese cinemas. It shows Murayama, who died at 29 in 1997, delivering one of his most famous proclamations: “The day will never come when a computer defeats a pro shogi player.”

https://qz.com/906447/artificial-intell ... ese-chess/

[pookpooi]

no opening book, no endgame database, no heuristics, no nothing! full paper coming soon, will have things like early games.

https://twitter.com/demishassabis/statu ... 4462542849

So this is not the full paper yet? Maybe they'll come with the usual DeepMind blog post

[dfan]

[re contempt] I don't know if other engines do anything clever like evaluating opponent's previous moves to work out they are weaker so they can get away with overplays and go for a win.

This would be hard unless you had multiple games to go on (and no chess engines that I know of have a "meta-game" like that), since by the problem setup your opponent's moves so far in this game have been about as good as your own.

[jeromie]

Stockfish won the 2016 TCEC, but this year the match is currently in progress between the other 2 big names in computer chess, Houdini and Komodo (interestingly Stockfish was better against Komodo head-to-head in qualifying but didn't do so well beating up the weaker half of the 8-player field, not enough "contempt" in the lingo, I suppose this means it reads for both players so will play a draw line if it sees against itself that's the best it could do, but maybe it could overplay and go for a win and get away with it against weaker opposition, I don't know if other engines do anything clever like evaluating opponent's previous moves to work out they are weaker so they can get away with overplays and go for a win). Here's an interesting interview with the creators of Houdini and Komodo, which includes GM Larry Kaufman who is also a strong shogi and decent Go player who wrote an interesting comparison of the games I've posted before (he thinks shogi is the best game).

http://www.chessdom.com/interview-with- ... y-kaufman/

Some famous last words on November 27th

Robert (Houdini developer): Well, I think we are all waiting for artificial intelligence to pop up in chess after having seen the success of the artificial intelligence approach of Google for the Go game. And so basically what I would expect if some of these giant corporations would be interested is that in the next five years chess also might see that kind of development. For example the artificial intelligence for the evaluation of a position, it could produce some very surprising results in chess. And so, we’re probably waiting for that and then we can retire our old engines. Look at the AlphaChess engine that will be 4000 Elo. [chuckles]

Nelson (moderator): Yep, at that point we can all fade back into history. Larry, anything to add?

Larry (GM and Komodo developer): Well, I also followed closely the AlphaGo situation. The guy who is the head of it at Google Mind is a chess master himself, Demis Hassabis. Although Go is thought to be a much harder game than chess to beat the best humans at, and they have certainly proven that they can do that, it is so far yet to be proven that a learning program such as the latest one from DeepMind [can replicate that in chess]. Their latest learning program beat the pants off all other, previous Go programs. But that does not apply to chess. Nobody has a self-teaching chess program that can fight with Houdini or Komodo. That’s a fantasy. Maybe that’s the challenge, to get Google to prove that it applies to chess too. But who knows.

5 years turned out to be 12 days

It continues to surprise me how every field thinks that their problem will be too difficult for AI to have good performance right up until it does. Go players were no different before AlphaGo, but I would have thought chess’s relationship with strong computer programs would have changed their outlook.

[Monadology]

I've been quite interested in machine learning, and artificial intelligence in general, since college. But given the rate at which artificial intelligence is advancing, I have an odd combination of feelings. Ultimately, I suppose it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.

Maybe it's something else - like emotion or the human experience... Or maybe the bots will tell us someday

I'll try not to derail this thread into a philosophical discussion, so I'll keep my thoughts to this post (if you want to discuss further, feel free to PM me Kirby!):

All the artificial intelligence so far developed has intelligence that is directed. Human beings have the capacity to reflect on our aims and revise them in a very general way - this is evidenced both in the wide diversity of things human beings devote their lives to (not just careers or hobbies, look also at cultural values and practices) as well as the extremes to which humans will go to accomplish those (even to the point of death, consider martyrs of various stripes, such as Buddhist monks self-immolating in protest).

Maybe one day AI will develop this generalized reflective capacity, and begin making choices about whether it will devote its intelligence to mastering Go, or Shogi, or medical diagnosis, or what-have-you. But I give it as just one example of the rich array of cognitive abilities that human beings have that machines are nowhere near approaching.

That said, I think your other suggestions are important. I think tying the meaning of life to special cognitive capacities has its own pitfalls. Pookpoi mentioned cats, but we only have to look as far as humans with cognitive disabilities. An often overlooked implication of rooting the meaning of human life or the moral status of humans in our special intellectual capacities is that human beings lacking them have less meaningful (or non-meaningful) lives or do not have the moral status of unimpaired human beings. This is an implication that I certainly find problematic, and so it tends to undermine the idea that we should base our understanding of these concepts in the special intellectual capacities of human beings.

[Uberdude]

[re contempt] I don't know if other engines do anything clever like evaluating opponent's previous moves to work out they are weaker so they can get away with overplays and go for a win.

This would be hard unless you had multiple games to go on (and no chess engines that I know of have a "meta-game" like that), since by the problem setup your opponent's moves so far in this game have been about as good as your own.

Yeah, from one game would be hard, but if you could tell the bot who the opponent was and its known weaknesses or at least general level maybe. But look what I found wandering the internet:

Another approach its designers claim to use is 'opponent modeling'; Junior might play moves that are not objectively the strongest but that exploit the weaknesses of the opponent. According to Don Dailey ″It has some evaluation that can sting if it's in the right situation—that no other program has.″[2]

[Uberdude]

A few comments from a low-dan Go and somewhat better chess playing friend:

Boris, as a chess and go player can you give your impression of how much AlphaZero plays standard chess opening book lines versus going of into unchartered opening territory. I always thought human (helped by bots recently) knowledge of chess openings was closer to perfect play than for go and there was less chance to play novel early moves which weren't suboptimal (chess openings being whole board rather than just a corner like in go, and sharper). So in go AlphaGo Zero is playing 3-3 invasions before move 10 and surprising us (starting in corners and approaching with knight's moves we got right though!) and not playing many of our long josekis, is it similar in chess making innovations in the first few moves or does it do the normal book opening for the first 10 (or however many) moves (ply) and then start winning with new things after that? https://www.reddit.com/r/chess/comments ... a/dqubdxz/ suggests maybe some of the former but I don't understand chess enough.

That's a great question and I think your intuition is right. There doesn't seem to be any innovation in early moves. I'd argue that's a property of chess for being a simpler game and the ability to analyse much further than in go (all 3-3 alike moves would've been tried by strong players at some point). I've only liked through openings without a board - I'll update my answer once I've seen how it slowly grinds a win. On average the games seem to be longer than human games.

My impression of the style is that it has an extremely strong positional understanding. It evaluates and handles the most complex positions of unbalanced material very well. The way in which it keeps activity and slowly converts the superior placement of pieces in gambit situations (it sacrifices plans happily) is impressive and reminiscent of the most memorable games by top players from the past. Botvinnik vs Tal match comes to mind

[John Fairbairn]

He [Habu] also got to play with AlphaGo!

Not really - they only played 20 moves.

But I'm not sure how strong he is in Go.

Unless he's improved recently (and why would he even try?), he plays pros on 6 stones or more.

[John Fairbairn]

If you allow yourself to ponder on this, it just gets worse Kirby, I'm afraid. Huxley once said an intellectual is someone who has found something more interesting than sex. One might be tempted now, therefore, to say "sod it" and go back to prioritising sex. But David Levy, one of the big names in computer chess, recently did a late-in-life PhD at the Sorbonne on computers and sex, and this is now in book form as "Love and Sex with Robots." He even goes so far as to say that continuing advances in computers and robotics "will make legal marriages between Homo and Robo feasible by mid-century."

Best to take a detached Zen-like view, and recall that life, like go, is a game of co-existence.

[dfan]

But look what I found wandering the internet:

Another approach its designers claim to use is 'opponent modeling'; Junior might play moves that are not objectively the strongest but that exploit the weaknesses of the opponent.

Yeah, it seems I was wrong that people aren't looking at the metagame: more information here. It's difficult to implement in the current environment, though, because the standard interface for talking to a chess engine (UCI) doesn't tell it who it's playing. So any between-game hyperparameter tuning has to be done manually by an operator.

(Of course this sort of opponent modeling is totally essential for exploitative play in poker, say.)

Perhaps in the future there will be computer matches in an environment where the engines have the ability to modify themselves between games. That could be pretty cool.

[pookpooi]

Unless he's improved recently (and why would he even try?)

He's likely the best human candidate against AlphaZero, strongest Shogi professional ever, International Chess Master, Amateur dan Go player.

[johnsmith]

One thing in modern chess engines is that, to know who is better at the current position, you need to look at the estimate that shows how many pawns you are ahead. It's not percentages like in go.

For example: in all of these games Stockfish at one point thought it is 1.10 ahead, but in reality, Alpha Zero may have thought it is winning by 75% which is much more convenient. Isn't it? Maybe chess players were thinking in the wrong direction. People are saying that Alpha Zero is making a lot of long-term sacrifices.

[moha]

The chess part seems very interesting. Silver himself quoted chess and checkers as examples where search is absolutely essential (because of the tactical nature), and knowledge-based approaches are not appropriate. This is still about search of course, but I would have bet blindly that NNs are not a good deal here for their gain / slowdown ratio (I assumed 100 times, but the paper says 1000 times - and I thought the handcrafted evaluation are already reasonably good).

But in retrospect, there is some logic in this. The other day I peeked at the Houdini-Komodo match, and it seemed the current state of art is roughly 50M pos/s, for about 5G per move selection, with a typical depth or PV length of 30 plies. This would mean a searched branch factor of 2 with full minimax, so on average maybe 4 or more with perfect alphabeta (even more actually, because of selective depth). In any case, the 1000 times slowdown would be regained at a successful branch factor reduction of about 1.25 times, which is not unrealistic.

it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.

That is still in the future. IMO intelligence means ability to solve previously unseen tasks, so I'm not sure if these examples qualify, even as early birds. Unless matchboxes are intelligent .

[Bill Spight]

One thing in modern chess engines is that, to know who is better at the current position, you need to look at the estimate that shows how many pawns you are ahead. It's not percentages like in go.

For example: in all of these games Stockfish at one point thought it is 1.10 ahead, but in reality, Alpha Zero may have thought it is winning by 75% which is much more convenient. Isn't it? Maybe chess players were thinking in the wrong direction. People are saying that Alpha Zero is making a lot of long-term sacrifices.

In traditional go evaluation, if you are ahead on points and have the move, you will win with correct play, unless there is a ko that enables your opponent to win. Otherwise, even knowing the exact point value of the position is not enough to say whether you are winning or not. That is one reason that the win rate of quasi-random rollouts was more effective for MCTS bots than the average winning point margin. In go, if you know not only the point value of the position but also the temperature, then you can make a pretty good estimate of who is ahead. For instance, if your opponent is 1½ pts. ahead, you have the move, and the temperature is 3, the game is very close; if the temperature is 1, you are a goner without a ko or the opponent's error; if the temperature is 5, you are the favorite. These temperatures might translate to an estimated win rate of less than 5% for temperature 1, 50% for temperature 3, and 66% for temperature 5. Note the non-linearity. These are my estimates of actual win rates, not win rates with quasi-random play. Monte Carlo win rates might be more like 30%, 50%, and 60%. Quien sabe?

Anyway, it may be that go playing programs that made use of estimates of both point value and temperature would perform better than programs that make use of ill defined "win rates". AFAIK, no strong program has utilized both estimates, so who knows? In any event, unlike win rates, point values and temperatures are well defined in go, even if they may be difficult to calculate in many positions.

In chess engines, point evaluations in terms of hundredths of a pawn have been very effective. OTOH, knowledge of temperature is quite crude. Is there any more accuracy than quiescent vs. non-quiescent? It is interesting that AlphaZero's use of "win rate" has proved more effective than the use of point evaluations by Stockfish. Chess seems to have no exact theory of temperature, so is pretty well stuck with point evaluations or, now, win rates, neither of which is well defined.

Edit: "Is there any more accuracy than quiescent vs. non-quiescent?" Well, yes. Checkmate and stalemate precisely as cool as you can get.

[Bill Spight]

it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.

That is still in the future. IMO intelligence means ability to solve previously unseen tasks, so I'm not sure if these examples qualify, even as early birds. Unless matchboxes are intelligent .

Yes, I regard intelligence as the ability to do something well that you have never done before.

Fairly obviously, I think that there are different kinds of intelligence.