AlphaZero paper discussion (Mastering Go, Chess, and Shogi)

[djhbrown]

One strong point of this network + search approach seems that it's hard to imagine a game now, where this wouldn't dominate humans.

Imagine a game that does not terminate in a number of moves shallow enough that search to completion is practical (Monte-Carlo requires search to completion in the absence of a reliable evaluation function of non-terminal positions).

A game like football, or gambling on the stock market, for example.

[Revilo]

I've been out of the trade for a while but I'd say that the specs do not seem too shabby.

Where were they actually described? I only recall seeing things like "1G hash, 64 threads" - but 64 at which kind of hardware? (OC, we can roughly guess from the pos/s given.)

Honestly, I've made the exact educated guess you suggest. I've also assumed that the paper doesn't give evals per seconds but nodes per second - because Stockfish doesn't report its eval calls.

On my Galaxy S8, Stockfish is running at about 3000k nodes per second. We can guestimate from here that they deployed Stockfish on a nice little cluster to achieve 70000k. We can also try to guestimate from the other direction by comparing against the stats given by Stockfish's test cluster: http://tests.stockfishchess.org/tests. It is stated that 459 cores on 99 machines results in roughly 740.000k nodes per second - about 10 times more than what the setup Alpha played against reached. So let's say, 10 quads or something like that, roughly. I hope they are going to be more specific in the announced detailed paper.

Anyway, I judge this to be a quite ok matchup. Stockfish has a fast but rather dumb evaluation function, so it needs raw NPS to go deep into the search space. Alpha has a slow but sophisticated eval (neural network) so it should have proportionally less NPS because otherwise it would be an uneven contest.

Stockfish developer Marco Costalba has actually acknowlegded that this is a huge achievement:

I have read the paper: result is impressive!

Honestly I didn't think it was possible because my understanding was that chess is more "computer friendly" than Go....I was wrong.

It is true, SF is not meant to play at its best without a book and especially 1 fixed minute per move cuts out the whole time management, it would be more natural to play with tournament conditions, but nevertheless I think these are secondary aspects, what has been accomplished is huge.

(see here: http://talkchess.com/forum/viewtopic.ph ... 307#741307)

[pookpooi]

Criticism from Computer Shogi community (in English) http://www.uuunuuun.com/single-post/201 ... ogi-engine

[pookpooi]

DeepMind is the talk of the town at NIPS 2017 conference

David Silver presents AlphaZero. He didn't try AlphaZero vs. Strongest version of AlphaGo Zero yet, and it might take at least one year 'if' DeepMind decide to go opensource.

And this slide is basically a declaration of war to handcraft system


And DeepMind take full feedback/criticism from at least two presenters I found



In which Demis Hassabis defend his AI approach himself (video)
https://twitter.com/thinkmariya/status/ ... 3281185793

The last one, Oriol Vinyals presents DeepMind's progress on Star Craft AI (in which I wish them the best of luck), this is the slide comparison to Go


I think the paper itself is hotter (and controversial) than even AlphaGo in Nature paper (excluding Lee Sedol match hype) because it tackles computer chess, computer shogi, and computer go community at once. So the feedback/reaction is very strong.

[jeromie]

I don’t think the response is necessarily more heated because of the involvement of the chess and shogi communities; it’s more heated because they claim to have developed a generalizable AI algorithm. Since their examples thus far have used perfect information games, there is a question of how applicable this approach is to other types of problems. Naturally, Deepmind is more optimistic than some other researchers.

[djhbrown]

how applicable this approach is to other types of problems

a question beyond the narrow focus of a Go forum, but a key question for AI and its investors. The short answer is that dcnn has wideranging applicability, as it is a generic pattern discrimination technique (so, for example, one day dentists will use it to tell them whether a tooth needs repair); the long answer is that the scope of future enhancements to dcnn cannot be circumscribed because we do not yet know what those future enhancements will be - already there are people working on expanding dcnn functionality to tasks such as natural language and edge detection.

i imagine that one future development will be the synthesis of composite networks of dcnns that can embrace hierarchical conceptions, and another the development of learning techniques less regimented and more focussed than broadscale hill climbing through static arrays.

[John Fairbairn]

Here is what most of the world's top chess players think, including how they cope in a world already dominated by chess computers. Lessons for go pros?

https://www.chess.com/news/view/alphaze ... ish-author

From this and other articles it seems that chess and shogi players, while admitting the impressiveness of the achievement, are dissatisfied at the conditions chosen for their game's representative. DeepMind may have blundered a little there and unnecessarily taken some shine off their achievement. Artificial stupidity still rules, OK?

[Uberdude]

Artificial stupidity still rules, OK?

Nice demonstration from FineArt, which can beat up top pros but filled in its own territory instead of passing so lost to a weaker bot in the AI Ryusei: forum/viewtopic.php?p=225812#p225812

(and I agree about the match conditions taking the shine off).

[djhbrown]

If you post content, people should be free to criticize that content.

Below is my recent output. Feel free to criticise its content.

The Hierarchy of the Imagination
Conceptual structures are intrinsically hierarchical, regardless of whether the reality conceived is itself hierarchical. Examples are given in two different domains.

Learning to SWIM
Mechanisms are described by which a model of conceptual reasoning about Go can learn new techniques from its own analyses of expert moves and assimilate expert advice.

The second one contains an example of a conceptual reasoning machine learning from Alphago zero and from Michael Redmond.

[EdLee]

Tech review

[moha]

Tech review

So it seems AZ excels in the kitchen as well.

For the earlier discussion about intelligence, and whether these RL successes may count as such: I wrote that intelligence is ability to solve previously unseen problems, almost the opposite of what reinforcement learning does. This felt a bit weak argument then, but I now convinced myself further with this analogy.

Consider animal behaviour, instincts in particular. Animals can solve complex problems but fail at simple ones if only minor things change and the behaviour suggested by instict is not working anymore. IMO evolutional selection, genetics and mutation = reinforcement learning, where the reward function is survival and reproduction. And instinct vs intelligence = animal vs human behaviour = RL vs intellect. So:

1. random play = eons of failures before success, then eons of failures again
2. reinforcement learning = eons of failures before success, then repeating success (until the problem changes)
3. intelligence = success reasonably soon, using knowledge from other domains

[djhbrown]

instinct vs intelligence = animal vs human behaviour = RL vs intellect

Animals, plants and bacteria learn to improve the efficacy of their behaviour too. See for example pages 90 and 126 of http://sites.google.com/site/djhbrown2/LC1to5.doc

[EdLee]

The Economist: $ in AI.

[djhbrown]

...The most important is whether AI will always depend on vast amounts of data... A competing vision of AI stresses simulations, in which machines teach themselves using synthetic data or in virtual environments. Early versions of a program developed to play Go, an Asian board game, by DeepMind...

Trust the doublethinkThe Economist to mangle a couple of facts to spin a thinly-disguised corporate advertisement in the vein of "Persil Washes Whiter" to suck in more investors by claiming that Alfie Baby doesn't rely upon big data [so she must be really intelligent and will take over the outside real world without needing any real-world data as well as triumphing over the inside world of a board game of limited depth].

Propaganda Works - You Know This
So how about a little propaganda for understanding?

[Sneegurd]

Nobody has a self-teaching chess program that can fight with Houdini or Komodo. That’s a fantasy.

Nice if someone finds the clip when Letterman asked Kasparov, (paraphrasing) "Do you think the chess computer will ever beat the best humans?" and Kasparov said no. (This might be shortly after he'd beaten Deep Thought. ...Early 1990's?)

Famous last words, indeed.

I'm late to the party, just recognized AlphaZero --- I was not expecting that. At least I was surprised, Google continued their aggressive nuclear attacks on everything they could find
O-M-F-G