Life In 19x19

From the Wired article on the subject, it sounds like the bulk of the game data came from allowing the engine to play itself. The article as a whole is a bit sensationalist, but I found that an interesting tidbit.

Also interesting is that no opening book was used. Previously my understanding is all the other computer go attempts have an opening book.

yoyoma wrote:Some quote from the paper
https://storage.googleapis.com/deepmind ... ing-go.pdf

We trained the policy network pσ to classify positions according to expert moves played in the KGS data set. This data set contains 29.4 million positions from 160,000 games played by KGS 6 to 9 dan human players; 35.4% of the games are handicap games.

The final version of AlphaGo used 40 search threads, 48 CPUs, and 8 GPUs. We also implemented a distributed version of AlphaGo that exploited multiple machines, 40 search threads, 1202 CPUs and 176 GPUs.

The results of the tournament (see Figure 4,a) suggest that single machine AlphaGo is many dan ranks stronger than any previous Go program, winning 494 out of 495 games (99.8%) against other Go programs. To provide a greater challenge to AlphaGo, we also played games with 4 handicap stones (i.e. free moves for the opponent); AlphaGo won 77%, 86%, and 99% of handicap games against Crazy Stone, Zen and Pachi respectively. The distributed version of AlphaGo was significantly stronger, winning 77% of games against single machine AlphaGo and 100% of its games against other programs.

AlphaGo was using 48 CPUs + 8 GPUs. CrazyStone had 32 CPUs, Zen had 8 CPUs, Pachi had 16 CPUs. They were using 5 seconds per move.

The mention of GPUs intrigues me. Does anyone know whether this means using something like CUDA to distribute their computations. As far as I can see modern NVIDIA cards contain thousands of CUDA cores. So does 8 GPUs represent another 20,000 cores and 176 GPUs represent another 400,000 cores of some kind? Does anyone know how appropriate they would be for the types of computations made by AlphaGo?

GPU's are (apparently) particularly well suited to deep learning applications. NVIDIA even has a deep learning SDK. This does use CUDA as part of the library.

They didn't seem to be using cuda to distribute things(may have used it in running the neural nets though). The GPU's mostly seemed to be used to run the neural networks asynchronously while the CPU is doing the rollout. Since the networks results are used in the rollout they need to get those values quickly.

As a snippet while they are describing their search

The leaf positions are communicated to the worker CPUs, which execute the rollout phase of simulation, and to the worker GPUs which compute network features and evaluate the policy and value networks.

Also didn't see much mention that 5 informal games were played at the same time and the record for that was 3-2 in favor of alphago.
The informal games had 3 x30 byo-yomi time control. So when time gets tight it starts to struggle a bit.

Regarding the KGS 6-9d training games, in Deepmind's last paper they mentioned that a neural net trained on just the 9d games was better than on the 6-9d games so they probably use different subsets of the KGS games to make different versions of AlphaGo which then play each other and improve through reinforcement learning (a key advance).

This is so exciting! Enough to make me stop lurking here


I even tried my hand at blogging again after a five-year break. Hope I didn't misinterpret too much:
http://www.allaboutgo.com/articles/alph ... -champion/

Suddenly eager to learn more about this!

DrStraw wrote:How much did Google (a huge commercial venture) pay to use the KGS database (a private venture)? It should be enough to pay the KGS owner and developer (WMS) to retire and make all those requested improvements to the server. If Google can play Lee $1M, then it can pay KGS a portion of that for use of its data.

I wish KGS had more funding, but is the database not publicly accessible to some extent? I imagine it wouldn't be difficult to find a way to bulk download SGF files...

Or maybe WMS is not telling us something

Publicly available doesn't mean there aren't terms that restrict bulk use (though KGS doesn't have lawyers). I worked on a prototype project that made use of bulk downloaded Google Maps Streetview data that was abandoned partly over legal concerns over the use of that publicly available data. So although I expect Deepmind are legally in the right a degree of moral quid pro quo would be appreciated.

palapiku wrote:

This data set contains 29.4 million positions from 160,000 games played by KGS 6 to 9 dan human players

Is it just me or does that seem like a pretty bad dataset to try to beat Lee Sedol with?

those are all (mostly) just amateur games.
why didn't they use a collection of professional games, such as from GoGoD?

Five minute simplified explanation of neural networks and why you would use GPUs for them:

Neural networks can be visualized as a graph consisting of nodes and edges (connections between nodes). The nodes are some sort of mathematical function like f(x)=m*x+b. That node would have three input edges (x, m and b) it and one output edge. The inputs and outputs are connected to other nodes or can be inputs/outputs to things outside the network.

A neural net will consist of many nodes arranged in interconnected layers. For example, part of a neural net for a go playing program might be a 19x19 grid of nodes, each with an input representing a point on a go board (0, 1 and 2 for empty, black and white). Those 361 nodes performing a mathematical function on individual numeric inputs can also be represented as a single node performing the same function on a 19x19 matrix.

So, actual neural network programs boil down to massive amounts of math on vectors, matrices and higher-dimensional arrays of numbers called tensors. Video games graphics is also a bunch of linear algebra and math on vectors and matrices. And we already have commodity specialized processors (GPUs) for doing exactly that, and they're many times more efficient at it than traditional CPUs.

Nikolas73 wrote: but is the database not publicly accessible to some extent? I imagine it wouldn't be difficult to find a way to bulk download SGF files...

Perhaps they bought the BiGo database which already has loads of KGS games.
http://bigo.baduk.org/assistant_databases.html

xed_over wrote:

palapiku wrote:

This data set contains 29.4 million positions from 160,000 games played by KGS 6 to 9 dan human players

Is it just me or does that seem like a pretty bad dataset to try to beat Lee Sedol with?

those are all (mostly) just amateur games.
why didn't they use a collection of professional games, such as from GoGoD?

Their published work needed to be peer reviewed so it should be many weeks old. I supplied a copy of Go4Go collection to the team in December, which must be after the submission of the paper.

Marcel Grünauer wrote:By the way, the paper also said that the Fan Hui matches took place from 5th to 9th October 2015, so who knows what happened since then.

Things got "serious" in October - this I have from the horse's mouth.

Fairly clearly a three-month timescale to have the work written up and published with a big PR splash (done); a six-month timescale for a challenge match. I spoke to Demis about this on Monday: he said contacts with the KBA were very friendly.

To state the obvious, AlphaGo is being trained as we speak.

Well, I just got my 5 minutes of fame over this.

Yesterday morning I got a call at work from our national broadcasting company, saying they were doing a radio show on "man and computer" that evening and asking whether I'd be willing to be on it to say something about go and computers. I agreed, and they promised to call me back to hammer out the details later. They did so around 8 pm, when it turned out this was going to be about AlphaGo. Of which I had then been aware for all of an hour

Long story short, I appeared on a 10 minute segment together with a science journalist who filled in the parts about AI and Google and we had a nice radio chat about go, computers and what this means for the future. I told them that this is a very impressive result, and, when asked, that it does not "spoil the game" for me, because it takes nothing away from the fun and challenging nature of the game. I told them I expect the match with Lee Sedol to be very close and exciting, and that although I still slightly favor Lee, I expect it can go either way.

Really interesting all in all, and a great opportunity to get some much needed publicity for the game!

EDIT: For those who understand Dutch: http://www.radio1.nl/popup/terugluister ... 1-27/23:00 (skip to 41:43 in, it's about 8 minutes total)

British media appearances:

Charles Leedham-Green was interviewed on Go and AI on
The World Tonight on Radio 4.
http://www.bbc.co.uk/programmes/b06yfcpm 34:26 in.

(I can't help feel the obsessing over how to hold a stone is off-putting, likewise starting with "No, you're doing it wrong", but it gets better. Also they had a few inaccuracies, saying Deepmind was the name of the program rather than company, and said it beat a world (rather than European) champion).

News at 10 is at
http://www.bbc.co.uk/iplayer/episode/b0 ... n-27012016
until 18:30 today,

(Links might not work outside the UK without IP location spoofing)