AlphaZero paper discussion (Mastering Go, Chess, and Shogi)

For discussing go computing, software announcements, etc.
pookpooi
Lives in sente
Posts: 727
Joined: Sat Aug 21, 2010 12:26 pm
GD Posts: 10
Has thanked: 44 times
Been thanked: 218 times

AlphaZero paper discussion (Mastering Go, Chess, and Shogi)

Post by pookpooi »

AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

Full paper at https://arxiv.org/abs/1712.01815

Image

Image

Image

AlphaZero hasn't lose against Stockfish but is it because of so many draws that the Elo in the graph is very close
Meanwhile, AlphaZero training just 34 hours beats 60 out of 100 games against AlphaGo Zero 20 blocks 3 days that beat AlphaGo Lee 100 out of 100 games. But look like they're not gonna train for 40 days, not anymore.
Last edited by pookpooi on Wed Dec 06, 2017 8:10 am, edited 2 times in total.
User avatar
EdLee
Honinbo
Posts: 8859
Joined: Sat Apr 24, 2010 6:49 pm
GD Posts: 312
Location: Santa Barbara, CA
Has thanked: 349 times
Been thanked: 2070 times

Post by EdLee »

Pookpooi, thanks.

Congrats to Deep Mind.

The paper includes 10 sample games for chess (v. Stockfish),
but none for shogi (v. Elmo) or Go (v. AGZ). :)

Elmo, maybe the final old-school (non-NN) shogi program,
still managed to beat AZ in 8% of the games.
(Re: game 4, AGL v. LSD)

Page 15: Pondering was disabled for all players. --
Does pondering mean thinking on the opponent's time ?

Ugh... the chess set is buried in 70+ storage boxes... :oops:
Kirby
Honinbo
Posts: 9553
Joined: Wed Feb 24, 2010 6:04 pm
GD Posts: 0
KGS: Kirby
Tygem: 커비라고해
Has thanked: 1583 times
Been thanked: 1707 times

Re: AlphaZero paper discussion (not the same as AlphaGo Zero

Post by Kirby »

I've been quite interested in machine learning, and artificial intelligence in general, since college. But given the rate at which artificial intelligence is advancing, I have an odd combination of feelings. Ultimately, I suppose it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.

Maybe it's something else - like emotion or the human experience... Or maybe the bots will tell us someday :grumpy:
be immersed
User avatar
EdLee
Honinbo
Posts: 8859
Joined: Sat Apr 24, 2010 6:49 pm
GD Posts: 312
Location: Santa Barbara, CA
Has thanked: 349 times
Been thanked: 2070 times

Post by EdLee »

maybe the bots will tell us someday
If we're so lucky.
T.jpg
T.jpg (31.48 KiB) Viewed 25477 times
pookpooi
Lives in sente
Posts: 727
Joined: Sat Aug 21, 2010 12:26 pm
GD Posts: 10
Has thanked: 44 times
Been thanked: 218 times

Re:

Post by pookpooi »

EdLee wrote: Does pondering mean thinking on the opponent's time ?
Yes, at least in this paper's operational definition.
Kirby wrote:Ultimately, I suppose it makes me question the meaning in life for humans. If it's intelligence, then computers are beginning to have more meaning than humans.
I'm pretty sure my cat will choose to be born as a cat again in the next life even though it knows that human are smarter than it.
Uberdude
Judan
Posts: 6727
Joined: Thu Nov 24, 2011 11:35 am
Rank: UK 4 dan
GD Posts: 0
KGS: Uberdude 4d
OGS: Uberdude 7d
Location: Cambridge, UK
Has thanked: 436 times
Been thanked: 3718 times

Re: AlphaZero paper discussion (not the same as AlphaGo Zero

Post by Uberdude »

Impressive again from DeepMind! At one of his talks I got the impression from Demis he wasn't so interested in making a chess program as basically a solved problem already (and maybe a bit of a cop out in case theirs wasn't as strong as stockfish et al), but they went and did it anyway!

I have a few questions for the chess experts here:
- looking at the chess openings is AlphaZero playing the long standard opening book lines or has it found a way to diverge early without playing bad moves? My impression of chess was human knowledge of the opening was closer to perfect play than in go and is sharper so there was less scope for novelty into unexplored areas without playing suboptimal moves.
- Is the play of stockfish near its peak strength, i.e if it has more time or resources does it get significantly better (anyone try at home) and not play the moves that let AZ beat it? I wonder if perhaps neural networks bots are better at blitz than tree search bots (in training you essentially transfer the skill from tree search into one huge function which is quick to compute).*

Edit: Now I read the paper the 100 game match was not 1 seconds a move like for the Elo evaluation in the graph, but 1 minute a move with 64 threads and 1 GB hash which sounds better but still I'm not clear how far from peak strength and diminishing returns that is (and could be a lot smaller than the 4 TPUs AZ got). Looking at the kibitz on the TCEC match many chess players are dismissive of the conditions, saying the specs for stockfish engine are unfair/small.
User avatar
mistakenot
Beginner
Posts: 7
Joined: Thu Mar 10, 2016 1:23 pm
GD Posts: 0
Been thanked: 7 times

Re:

Post by mistakenot »

EdLee wrote:Ugh... the chess set is buried in 70+ storage boxes... :oops:
This may be easier to browse: https://lichess.org/study/EOddRjJ8

(via /u/Spreek on /r/chess: https://www.reddit.com/r/chess/comments ... a/dqu9pah/ )
pookpooi
Lives in sente
Posts: 727
Joined: Sat Aug 21, 2010 12:26 pm
GD Posts: 10
Has thanked: 44 times
Been thanked: 218 times

Re: Re:

Post by pookpooi »

mistakenot wrote:(via /u/Spreek on /r/chess: https://www.reddit.com/r/chess/comments ... a/dqu9pah/ )
Yes, chess players rejoice because they see some games, DeepMind should release some game records for shogi community though. From what I browse in Twitter they're not impressed due to lack of game records and high computational usage.
User avatar
EdLee
Honinbo
Posts: 8859
Joined: Sat Apr 24, 2010 6:49 pm
GD Posts: 312
Location: Santa Barbara, CA
Has thanked: 349 times
Been thanked: 2070 times

Post by EdLee »

Hi mistakenot, Thanks. Did you "upload" the AZ-stockfish games there ? :)
they're not impressed...
Man, give them an inch...! :-?
User avatar
mistakenot
Beginner
Posts: 7
Joined: Thu Mar 10, 2016 1:23 pm
GD Posts: 0
Been thanked: 7 times

Post by mistakenot »

pookpooi wrote:
mistakenot wrote:(via /u/Spreek on /r/chess: https://www.reddit.com/r/chess/comments ... a/dqu9pah/ )
Yes, chess players rejoice because they see some games, DeepMind should release some game records for shogi community though. From what I browse in Twitter they're not impressed due to lack of game records and high computational usage.
Yeah, DeepMind should publish some of the shogi games too. It would be interesting to see them, or rather read what other people think of them (seeing as I don't know much about shogi myself).
EdLee wrote:Hi mistakenot, Thanks. Did you "upload" the AZ-stockfish games there ? :)
Nah, I just found the link in /u/Spreek's comment, as mentioned. I assume they're responsible, unless they too got the link from someone else.

I also thought Spreek's comment and the replies to it in the /r/chess discussion were interesting: it sounds like they think AlphaZero's moves feel different than previous computer chess, possibly more "natural".

EDIT: also found another viewer for the chess games via twitter: https://chess24.com/watch/live-tourname ... -stockfish
Last edited by mistakenot on Wed Dec 06, 2017 2:28 am, edited 3 times in total.
pookpooi
Lives in sente
Posts: 727
Joined: Sat Aug 21, 2010 12:26 pm
GD Posts: 10
Has thanked: 44 times
Been thanked: 218 times

Re: AlphaZero paper discussion (not the same as AlphaGo Zero

Post by pookpooi »

I'm wondering why DeepMind choose Shogi out of many board games beside Go (chess is understandable) and finally found some clues

Image

Yoshiharu Habu is the only person ever to hold seven Shogi titles simultaneously in 1996, even right now he's still holding two of it, and the only person ever to has all seven lifetime titles. He's also Chess Master.

Image

He also got to play with AlphaGo! But I'm not sure how strong he is in Go.

from http://www.nhk.or.jp/special/ai/article1.html
Last edited by pookpooi on Wed Dec 06, 2017 2:28 am, edited 1 time in total.
User avatar
EdLee
Honinbo
Posts: 8859
Joined: Sat Apr 24, 2010 6:49 pm
GD Posts: 312
Location: Santa Barbara, CA
Has thanked: 349 times
Been thanked: 2070 times

Post by EdLee »

I'm wondering why DeepMind choose Shogi out of many board games beside Go
Another usual suspect: Xiangqi ; ...Chaturanga.
pookpooi
Lives in sente
Posts: 727
Joined: Sat Aug 21, 2010 12:26 pm
GD Posts: 10
Has thanked: 44 times
Been thanked: 218 times

Re:

Post by pookpooi »

EdLee wrote:
I'm wondering why DeepMind choose Shogi out of many board games beside Go
Another usual suspect: Xiangqi ; ...Chaturanga.
For diplomatic reason, choosing Xianqi to conquer by (non-Chinese) AI is a really bad choice.
Uberdude
Judan
Posts: 6727
Joined: Thu Nov 24, 2011 11:35 am
Rank: UK 4 dan
GD Posts: 0
KGS: Uberdude 4d
OGS: Uberdude 7d
Location: Cambridge, UK
Has thanked: 436 times
Been thanked: 3718 times

Re: AlphaZero paper discussion (not the same as AlphaGo Zero

Post by Uberdude »

Stockfish won the 2016 TCEC, but this year the match is currently in progress between the other 2 big names in computer chess, Houdini and Komodo (interestingly Stockfish was better against Komodo head-to-head in qualifying but didn't do so well beating up the weaker half of the 8-player field, not enough "contempt" in the lingo, I suppose this means it reads for both players so will play a draw line if it sees against itself that's the best it could do, but maybe it could overplay and go for a win and get away with it against weaker opposition, I don't know if other engines do anything clever like evaluating opponent's previous moves to work out they are weaker so they can get away with overplays and go for a win). Here's an interesting interview with the creators of Houdini and Komodo, which includes GM Larry Kaufman who is also a strong shogi and decent Go player who wrote an interesting comparison of the games I've posted before (he thinks shogi is the best game).

http://www.chessdom.com/interview-with- ... y-kaufman/

Some famous last words on November 27th
Robert (Houdini developer): Well, I think we are all waiting for artificial intelligence to pop up in chess after having seen the success of the artificial intelligence approach of Google for the Go game. And so basically what I would expect if some of these giant corporations would be interested is that in the next five years chess also might see that kind of development. For example the artificial intelligence for the evaluation of a position, it could produce some very surprising results in chess. And so, we’re probably waiting for that and then we can retire our old engines. Look at the AlphaChess engine that will be 4000 Elo. [chuckles]

Nelson (moderator): Yep, at that point we can all fade back into history. Larry, anything to add?

Larry (GM and Komodo developer): Well, I also followed closely the AlphaGo situation. The guy who is the head of it at Google Mind is a chess master himself, Demis Hassabis. Although Go is thought to be a much harder game than chess to beat the best humans at, and they have certainly proven that they can do that, it is so far yet to be proven that a learning program such as the latest one from DeepMind [can replicate that in chess]. Their latest learning program beat the pants off all other, previous Go programs. But that does not apply to chess. Nobody has a self-teaching chess program that can fight with Houdini or Komodo. That’s a fantasy. Maybe that’s the challenge, to get Google to prove that it applies to chess too. But who knows.
5 years turned out to be 12 days :D
User avatar
EdLee
Honinbo
Posts: 8859
Joined: Sat Apr 24, 2010 6:49 pm
GD Posts: 312
Location: Santa Barbara, CA
Has thanked: 349 times
Been thanked: 2070 times

Post by EdLee »

Nobody has a self-teaching chess program that can fight with Houdini or Komodo. That’s a fantasy.
Nice if someone finds the clip when Letterman asked Kasparov, (paraphrasing) "Do you think the chess computer will ever beat the best humans?" and Kasparov said no. (This might be shortly after he'd beaten Deep Thought. ...Early 1990's?)

Famous last words, indeed.
Post Reply