Life In 19x19
http://lifein19x19.com/

AlphaZero paper discussion (Mastering Go, Chess, and Shogi)
http://lifein19x19.com/viewtopic.php?f=18&t=15289		 Page 3 of 5
Author:  Uberdude [ Thu Dec 07, 2017 4:33 am ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
John Fairbairn wrote:
I have tried to warn against treating AlphaGo as a cargo cult, so welcome this reminder from an AI professional.

His recent output is not of professional quality.

John Fairbairn wrote:
I'm also constantly reminded of something that Michael Redmond said: go is a very simple game. It may be hard for humans but change the tool and Redmond's proven right.

In some ways I think Go is actually relatively easy for humans given its game tree/state complexity compared to chess and shogi: because the pieces don't move it makes it easier to visualise and chunk, and we have excellent neural networks for visual patterns. One of the points Demis made in the Q&A after a recent AlphaGo film showing was that he is pretty confident (but weren't the last failures, hah!) this latest generation of AI research with neural networks was on the right track to general AI is that it is informed and inspired by a system we know works, namely our brains. He reckoned now we were on maybe the 2nd rung of several dozens.
Author:  moha [ Thu Dec 07, 2017 6:50 am ]
Post subject:  Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Kirby wrote:
Bill Spight wrote:
moha wrote:
IMO intelligence means ability to solve previously unseen tasks, so I'm not sure if these examples qualify, even as early birds. Unless matchboxes are intelligent :).
Yes, I regard intelligence as the ability to do something well that you have never done before. :D

@Bill: I'd say that I'm not really intelligent at anything, then. I rarely do something well the first time I try it. Only after practice can I get any sort of competency.

This would still fit, at least by my definition. I didn't mean right at first try, just reasonably soon.

This is the biggest problem with RL today, which is why it doesn't seem intelligence - yet. Basically it is usable only where the environment is known and can be simulated, because of the insane amount of experimental data (trial and error) required. Humans - maybe animals in general - always needed to learn and to adapt fast, to stay alive. But this may change for AI in time, neural networks are still in their infancy. Perhaps real cognitive abilities will also be developed.

BTW, it's amazing Deepmind never tried anything that would be out-of-the box. The first time a programmer tries NNs, the first thoughts are often about potential changes (new activation functions / network structures). But they just took the state of the art (from the visual field), and applied it vanilla to go/chess. Perhaps as a demonstration that these things are already somewhat mature.
Author:  dfan [ Thu Dec 07, 2017 7:15 am ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
John Fairbairn wrote:
Quote:
Me too; in my case, that's since 1971; but i don't think that during that time AI has advanced much compared to advances in computer hardware, despite the current hullabaloo.

I have tried to warn against treating AlphaGo as a cargo cult, so welcome this reminder from an AI professional.

As another data point, I am an AI professional and think that AI has advanced hugely since 1971. Of course computer hardware has advanced a ridiculous amount too! Most (all?) current AI applications would not have been feasible on 1971 hardware but that doesn't cancel out the huge amount of progress that has been made.

To put it a different way: if you handed a bunch of 1971-era AI researchers 2017-level hardware, they wouldn't be able to create AlphaGo (until they had duplicated decades of research).

Meanwhile, on the subject of centipawns vs. winning percentage: one thing to keep in mind is that chess has three results. You can just treat draws as 1/2 (as tournaments do) and calculate expected value, but there really is a difference between positions that are 40%W 20%D 40%L and 5%W 90%D 5%L. And this difference is meaningful not just on the board but off of it; if you had the choice between those two positions, your choice would depend a lot on your position in the tournament (if you're half a point behind the leader and it's the last round, not much matters besides the %W bin). I haven't looked carefully at the AlphaZero paper yet so I don't know what they do here, though I would expect (no pun intended) that they just try to maximize expected value.
Author:  moha [ Thu Dec 07, 2017 1:37 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
dfan wrote:
if you handed a bunch of 1971-era AI researchers 2017-level hardware, they wouldn't be able to create AlphaGo
They weren't hopeless though. :)
Author:  dfan [ Thu Dec 07, 2017 2:39 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
moha wrote:
dfan wrote:
if you handed a bunch of 1971-era AI researchers 2017-level hardware, they wouldn't be able to create AlphaGo
They weren't hopeless though. :)

Indeed, though note that TD-Gammon was developed 21 years after 1971 and benefited from (and extended!) the theories of reinforcement learning developed during those decades. (I saw Tesauro talk at a conference last year about his experience creating TD-Gammon and it was quite interesting.)
Author:  Uberdude [ Thu Dec 07, 2017 4:37 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
I don't understand much of backgammon, bur found it interesting that TD-Gammon upended some opening theory and made humans start playing moves they previously thought were bad, rather like AlphaGo and the early 3-3 invasions.
Author:  djhbrown [ Thu Dec 07, 2017 5:59 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
Uberdude wrote:
His recent output is not of professional quality.
Please can someone who is competent to judge explain why Uberdude who has never manifested any evidence of professional competence in AI has licence to make repeated personal abuse attacks upon me in this forum over several years without ever receiving a reprimand from a moderator?

Being good at Go does not make one good at being human.
Author:  Kirby [ Thu Dec 07, 2017 7:30 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
dfan wrote:
moha wrote:
dfan wrote:
if you handed a bunch of 1971-era AI researchers 2017-level hardware, they wouldn't be able to create AlphaGo
They weren't hopeless though. :)

Indeed, though note that TD-Gammon was developed 21 years after 1971 and benefited from (and extended!) the theories of reinforcement learning developed during those decades. (I saw Tesauro talk at a conference last year about his experience creating TD-Gammon and it was quite interesting.)


I agree that advancements have been made from AI outside of hardware. But it's also hard to separate the two. For example, a number of research advancements may have been made possible with increased computing power and availability.

It's somewhat of a hypothetical argument, similar to how people talk about who would be stronger between a "modern day Shusaku" and, for example, Ke Jie. It's nice to think about as a thought experiment and make various reasoning, but the idea can't extend the limits of its hypothetical nature.

As to whether 1971 researchers could replicate AlphaGo, I'd say that dfan is probably right, but who really knows?
Author:  Kirby [ Thu Dec 07, 2017 7:35 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
djhbrown wrote:
Uberdude wrote:
His recent output is not of professional quality.
Please can someone who is competent to judge explain why Uberdude who has never manifested any evidence of professional competence in AI has licence to make repeated personal abuse attacks upon me in this forum over several years without ever receiving a reprimand from a moderator?

Being good at Go does not make one good at being human.


OK. As an admin, I'd like not to take sides on the matter. Let's remain cordial with one another. For all in the thread, please refrain from personal attacks, whether that means questioning an individual's professional credentials, or questioning their qualities as human beings.

Let's keep the discussion related to topics related to AlphaZero, AI, Go, and the like - not about individuals.
Author:  dfan [ Thu Dec 07, 2017 8:22 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
Kirby wrote:
I agree that advancements have been made from AI outside of hardware. But it's also hard to separate the two. For example, a number of research advancements may have been made possible with increased computing power and availability.

Oh, this is 100% the case. There was a lot of neural network research in the 1980s that seemed like a dead end because it would require too much computing power to be useful. Thirty years later...

Quote:
As to whether 1971 researchers could replicate AlphaGo, I'd say that dfan is probably right, but who really knows?

Maybe they'd get there eventually! But a lot of AI research has been done in the last 46 years, and I'm pretty reluctant to believe that it all could have been reproduced in a couple of years or something if only computers were faster. A lot of very smart people did a lot of very hard thinking for a long time to get where we are now.
Author:  Kirby [ Thu Dec 07, 2017 8:40 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
Probably correct, dfan.
Author:  pookpooi [ Thu Dec 07, 2017 9:57 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
In a rare occasion American Go Association post an article from chess.com on their E-Journal but IMHO they should add details on AZ outclass AGZ 20 blocks as well, otherwise the article is not related to Go at all.
Author:  djhbrown [ Thu Dec 07, 2017 10:02 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
dfan wrote:
There was a lot of neural network research in the 1980s that seemed like a dead end because it would require too much computing power to be useful.
That's an overstatement and a misattribution of cause, since it was not lack of computing power but lack of functionality that held NNs up - the breakthrough was Le Cann's idea of convolutions. Go-wise, the idea of trying out Monte-Carlo search was also a quantum leap, so putting 2 and 2 together was a smart move.

So far, so good...

Where to next? A Go program that can talk? Or one that can see eyes? NNs can separate images containing a cat from those that don't, but can they draw a line around the cat?
Author:  Gomoto [ Thu Dec 07, 2017 10:30 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
I for one do not care if AI is not there yet.

If it plays go and wins, I like it. :ugeek:
Author:  Gomoto [ Thu Dec 07, 2017 10:34 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
I learned some things about life, while playing go.

Therefore if it plays go, it knows some things about life too. ;-)
Author:  jeromie [ Thu Dec 07, 2017 11:28 pm ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
djhbrown wrote:
can they draw a line around the cat?

This looks pretty good to me. :)
https://github.com/s9xie/hed

On a side note, my focus area for my master's degree in computer engineering was intelligent systems. Unfortunately, neural networks were out of vogue at my school in the early 2000s. I wasn't thrilled with my research area; I sincerely wish I could have studied something akin to what is happening in machine learning now!
Author:  HermanHiddema [ Fri Dec 08, 2017 1:56 am ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
djhbrown wrote:
Uberdude wrote:
His recent output is not of professional quality.
Please can someone who is competent to judge explain why Uberdude who has never manifested any evidence of professional competence in AI has licence to make repeated personal abuse attacks upon me in this forum over several years without ever receiving a reprimand from a moderator?

Being good at Go does not make one good at being human.


Questioning the quality of someone's work is not a personal attack.

If you post content, people should be free to criticize that content.
Author:  Revilo [ Fri Dec 08, 2017 4:46 am ]
Post subject:  Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Uberdude wrote:
I have a few questions for the chess experts here:
- looking at the chess openings is AlphaZero playing the long standard opening book lines or has it found a way to diverge early without playing bad moves? My impression of chess was human knowledge of the opening was closer to perfect play than in go and is sharper so there was less scope for novelty into unexplored areas without playing suboptimal moves.


I've just browsed through the games (https://lichess.org/study/EOddRjJ8) and I have to say that I'm quite impressed. It actually hasn't really innovated in the opening, but what can be clearly seen is that it has no qualms about positional sacrifices, even large ones. This is something that the alphabeta bean counters do not tend to do just like that - the reason being that piece values are fix and positional bonuses and maluses rarely add up enough to compensate for a piece - a pawn or an exchange (i.e. of rook against bishop or knight) sometimes maybe, but a full piece? Doesn't happen.

Alpha apparently likes to play a gambit in the Queen's Indian Defense that has been around for a long time already, and which has become very popular in the wake of early games of Garry Kasparov, who used this as his main weapon in the early 80s before he became world champion.

So what does Alpha make of it? We'll have a look at game 10 (see above link).

White's gambit move is the 7th, pushing the pawn to d5. Alpha's first major deviation from established theory is the 12th move, with the 14th being the first completely new move. So what happened then? Only five moves later Alpha sacrifices its knight on h6 for no immediate material compensation - it's just that Black's rook and knight are still at home, the Black king looks vulnerable on h6 and every White piece is going to be efficiently developed about two or three moves later.

I will have to check what the latest Komodo or Houdini think about this sacrifice, but I'm confident that they are not going to like it much. Moreover, in the moves following the sacrifice, White just continues to develop calmly. It would take a lot of confidence and positional judgement for a human grandmaster to play like this, but it is conceivable. Human-style play for sure. Long lasting positional compensation isn't something the bean counters like very much, though.

A similar positional piece sacrifice can be seen in game 9, White's 30th move. The point is White's very nice follow up on the 32nd move, after which White ends up a piece down but Black is tied up nicely and its extra piece, the bishop at b7, doesn't do any relevant work. Alpha converts its advantage without any further fireworks 20 moves later. This sacrifice is probably also quite a leap for a traditional engine (alphabeta + hand crafted evaluation function).


Uberdude wrote:
- Is the play of stockfish near its peak strength, i.e if it has more time or resources does it get significantly better (anyone try at home) and not play the moves that let AZ beat it? I wonder if perhaps neural networks bots are better at blitz than tree search bots (in training you essentially transfer the skill from tree search into one huge function which is quick to compute).*
Edit: Now I read the paper the 100 game match was not 1 seconds a move like for the Elo evaluation in the graph, but 1 minute a move with 64 threads and 1 GB hash which sounds better but still I'm not clear how far from peak strength and diminishing returns that is (and could be a lot smaller than the 4 TPUs AZ got). Looking at the kibitz on the TCEC match many chess players are dismissive of the conditions, saying the specs for stockfish engine are unfair/small.


I've been out of the trade for a while but I'd say that the specs do not seem too shabby. Also, 70000k evals per second vs. 80k - well, how much additional hardware do they actually suggest to throw at Alpha? :) Of course, the reason is that multithreaded alphabeta search does not scale up so well and the search has to be deep enough to compensate for the relative dumbness of the evaluation function. So they would probably have rather had something like tournament time controls (2h for 40 moves and so on) instead of a fixed time per move - then Stockfish could have used its time management (make forced moves immediately, use saved up time when problems appear (a sudden "fail-low" because a deep resource was discovered by the search - in these cases engines often use additional time to search deeper and fix the variation).
Author:  moha [ Fri Dec 08, 2017 5:48 am ]
Post subject:  Re: AlphaZero paper discussion (Mastering Go, Chess, and Sho
dfan wrote:
Maybe they'd get there eventually! But a lot of AI research has been done in the last 46 years, and I'm pretty reluctant to believe that it all could have been reproduced in a couple of years or something if only computers were faster. A lot of very smart people did a lot of very hard thinking for a long time to get where we are now.
I think advances in theory also depend on hardware though. If it takes a week to try out something, progress will be slower than if answers appear in a minute or two. :)

Nonetheless, I still find it amazing how close TD-Gammon was to AGZ, 25 years ago (the biggest difference probably residual convnets - go would probably be impossible with the simple networks of that time). But since NNs started to move again recently (with some of their longstanding problems solved), we will probably see more drastic advances from now on.

One strong point of this network + search approach seems that it's hard to imagine a game now, where this wouldn't dominate humans. Even for games created especially as "tough for computers". A human facing a complex task does the same (apply some intuition, then think ahead and compare in a few promising lines - with less depth and accuracy OC). So if a human can do it, a program can probably do it too. Even Starcraft :) - but that will probably need the next advance in NNs.
Author:  moha [ Fri Dec 08, 2017 5:58 am ]
Post subject:  Re: AlphaZero paper discussion (not the same as AlphaGo Zero
Revilo wrote:
I've been out of the trade for a while but I'd say that the specs do not seem too shabby.
Where were they actually described? I only recall seeing things like "1G hash, 64 threads" - but 64 at which kind of hardware? (OC, we can roughly guess from the pos/s given.)
Page 3 of 5 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/