Life In 19x19

AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

Full paper at https://arxiv.org/abs/1712.01815







AlphaZero hasn't lose against Stockfish but is it because of so many draws that the Elo in the graph is very close
Meanwhile, AlphaZero training just 34 hours beats 60 out of 100 games against AlphaGo Zero 20 blocks 3 days that beat AlphaGo Lee 100 out of 100 games. But look like they're not gonna train for 40 days, not anymore.

Pookpooi, thanks.

Congrats to Deep Mind.

The paper includes 10 sample games for chess (v. Stockfish),
but none for shogi (v. Elmo) or Go (v. AGZ).

Elmo, maybe the final old-school (non-NN) shogi program,
still managed to beat AZ in 8% of the games.
(Re: game 4, AGL v. LSD)

Page 15: Pondering was disabled for all players. --
Does pondering mean thinking on the opponent's time ?

Ugh... the chess set is buried in 70+ storage boxes...

I've been quite interested in machine learning, and artificial intelligence in general, since college. But given the rate at which artificial intelligence is advancing, I have an odd combination of feelings. Ultimately, I suppose it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.

Maybe it's something else - like emotion or the human experience... Or maybe the bots will tell us someday

maybe the bots will tell us someday

If we're so lucky.

EdLee wrote: Does pondering mean thinking on the opponent's time ?

Yes, at least in this paper's operational definition.

Kirby wrote:Ultimately, I suppose it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.

I'm pretty sure my cat will choose to be born as a cat again in the next life even though it knows that human are smarter than it.

Impressive again from DeepMind! At one of his talks I got the impression from Demis he wasn't so interested in making a chess program as basically a solved problem already (and maybe a bit of a cop out in case theirs wasn't as strong as stockfish et al), but they went and did it anyway!

I have a few questions for the chess experts here:
- looking at the chess openings is AlphaZero playing the long standard opening book lines or has it found a way to diverge early without playing bad moves? My impression of chess was human knowledge of the opening was closer to perfect play than in go and is sharper so there was less scope for novelty into unexplored areas without playing suboptimal moves.
- Is the play of stockfish near its peak strength, i.e if it has more time or resources does it get significantly better (anyone try at home) and not play the moves that let AZ beat it? I wonder if perhaps neural networks bots are better at blitz than tree search bots (in training you essentially transfer the skill from tree search into one huge function which is quick to compute).*

Edit: Now I read the paper the 100 game match was not 1 seconds a move like for the Elo evaluation in the graph, but 1 minute a move with 64 threads and 1 GB hash which sounds better but still I'm not clear how far from peak strength and diminishing returns that is (and could be a lot smaller than the 4 TPUs AZ got). Looking at the kibitz on the TCEC match many chess players are dismissive of the conditions, saying the specs for stockfish engine are unfair/small.

EdLee wrote:Ugh... the chess set is buried in 70+ storage boxes...

This may be easier to browse: https://lichess.org/study/EOddRjJ8

(via /u/Spreek on /r/chess: https://www.reddit.com/r/chess/comments ... a/dqu9pah/ )

mistakenot wrote:(via /u/Spreek on /r/chess: https://www.reddit.com/r/chess/comments ... a/dqu9pah/ )

Yes, chess players rejoice because they see some games, DeepMind should release some game records for shogi community though. From what I browse in Twitter they're not impressed due to lack of game records and high computational usage.

Hi mistakenot, Thanks. Did you "upload" the AZ-stockfish games there ?

they're not impressed...

Man, give them an inch...!

pookpooi wrote:

mistakenot wrote:(via /u/Spreek on /r/chess: https://www.reddit.com/r/chess/comments ... a/dqu9pah/ )

Yes, chess players rejoice because they see some games, DeepMind should release some game records for shogi community though. From what I browse in Twitter they're not impressed due to lack of game records and high computational usage.

Yeah, DeepMind should publish some of the shogi games too. It would be interesting to see them, or rather read what other people think of them (seeing as I don't know much about shogi myself).

EdLee wrote:Hi mistakenot, Thanks. Did you "upload" the AZ-stockfish games there ?

Nah, I just found the link in /u/Spreek's comment, as mentioned. I assume they're responsible, unless they too got the link from someone else.

I also thought Spreek's comment and the replies to it in the /r/chess discussion were interesting: it sounds like they think AlphaZero's moves feel different than previous computer chess, possibly more "natural".

EDIT: also found another viewer for the chess games via twitter: https://chess24.com/watch/live-tourname ... -stockfish

I'm wondering why DeepMind choose Shogi out of many board games beside Go (chess is understandable) and finally found some clues



Yoshiharu Habu is the only person ever to hold seven Shogi titles simultaneously in 1996, even right now he's still holding two of it, and the only person ever to has all seven lifetime titles. He's also Chess Master.



He also got to play with AlphaGo! But I'm not sure how strong he is in Go.

from http://www.nhk.or.jp/special/ai/article1.html

I'm wondering why DeepMind choose Shogi out of many board games beside Go

Another usual suspect: Xiangqi ; ...Chaturanga.

EdLee wrote:

I'm wondering why DeepMind choose Shogi out of many board games beside Go

Another usual suspect: Xiangqi ; ...Chaturanga.

For diplomatic reason, choosing Xianqi to conquer by (non-Chinese) AI is a really bad choice.

Stockfish won the 2016 TCEC, but this year the match is currently in progress between the other 2 big names in computer chess, Houdini and Komodo (interestingly Stockfish was better against Komodo head-to-head in qualifying but didn't do so well beating up the weaker half of the 8-player field, not enough "contempt" in the lingo, I suppose this means it reads for both players so will play a draw line if it sees against itself that's the best it could do, but maybe it could overplay and go for a win and get away with it against weaker opposition, I don't know if other engines do anything clever like evaluating opponent's previous moves to work out they are weaker so they can get away with overplays and go for a win). Here's an interesting interview with the creators of Houdini and Komodo, which includes GM Larry Kaufman who is also a strong shogi and decent Go player who wrote an interesting comparison of the games I've posted before (he thinks shogi is the best game).

http://www.chessdom.com/interview-with- ... y-kaufman/

Some famous last words on November 27th

Robert (Houdini developer): Well, I think we are all waiting for artificial intelligence to pop up in chess after having seen the success of the artificial intelligence approach of Google for the Go game. And so basically what I would expect if some of these giant corporations would be interested is that in the next five years chess also might see that kind of development. For example the artificial intelligence for the evaluation of a position, it could produce some very surprising results in chess. And so, we’re probably waiting for that and then we can retire our old engines. Look at the AlphaChess engine that will be 4000 Elo. [chuckles]

Nelson (moderator): Yep, at that point we can all fade back into history. Larry, anything to add?

Larry (GM and Komodo developer): Well, I also followed closely the AlphaGo situation. The guy who is the head of it at Google Mind is a chess master himself, Demis Hassabis. Although Go is thought to be a much harder game than chess to beat the best humans at, and they have certainly proven that they can do that, it is so far yet to be proven that a learning program such as the latest one from DeepMind [can replicate that in chess]. Their latest learning program beat the pants off all other, previous Go programs. But that does not apply to chess. Nobody has a self-teaching chess program that can fight with Houdini or Komodo. That’s a fantasy. Maybe that’s the challenge, to get Google to prove that it applies to chess too. But who knows.

5 years turned out to be 12 days

Nobody has a self-teaching chess program that can fight with Houdini or Komodo. That’s a fantasy.

Nice if someone finds the clip when Letterman asked Kasparov, (paraphrasing) "Do you think the chess computer will ever beat the best humans?" and Kasparov said no. (This might be shortly after he'd beaten Deep Thought. ...Early 1990's?)

Famous last words, indeed.