Computers reach 5d on KGS

[yoyoma]

I seriously think that this is ALL there is to such programs. If they could really do more, they would have been 5d at ANY time limits since as with a human players - more time would translate into better play. But i don't think it does. This is why you mostly see the programs playing 15 sec per move games. Double this time and I suspect the 5d program would drop to a much lower grade.

I might be wrong, though... Hard to say when you can't really observe such games. On the other hand - this by itself is telling, I think.

Zen19D - 5d - playing with 9x:15
Zen19N - 3d - playing with 20minute main + 5x:30

Also Zen19N was playing on a less powerful machine, and it hasn't been around since April.

[oren]

See, this is why I don't allow general chat for computers that log in to KGS. If I did turn on that feature, the chat rooms would be full of their trolling and bragging.

This violates their computer rights of free speech!

[Bantari]

I seriously think that this is ALL there is to such programs. If they could really do more, they would have been 5d at ANY time limits since as with a human players - more time would translate into better play. But i don't think it does. This is why you mostly see the programs playing 15 sec per move games. Double this time and I suspect the 5d program would drop to a much lower grade.

I might be wrong, though... Hard to say when you can't really observe such games. On the other hand - this by itself is telling, I think.

Zen19D - 5d - playing with 9x:15
Zen19N - 3d - playing with 20minute main + 5x:30

Also Zen19N was playing on a less powerful machine, and it hasn't been around since April.

That sort-of supports what I said.
Double the time again, and Zen will be 1k. In real-life tournament conditions, it would be 3k. At home, under unlimited time - just for practice, of course - it will be 5k. Still the manufacturers can claim, and the fans can go Oh-and-Ah, about a computer program being 5d. I dunno...

I guess we will not know until we put the programs to some real test. Who knows, we might all get surprised... some of us more than others.

[Mnemonic]

@Bantari: sorry but NO! Anybody that would bash a human player of equal strength by saying: "I haven't played against him yet, but I ‘know’ he is not 5d" would be at least highly criticized. You have supplied some arguments why the bot shouldn't be doing so well (e.g. time limits), and I have accept them. Zen19d has said (itself?) that it only plays 4d under reasonable time limits. Other people have pointed out that a similar program is only 3d. But to compare it to a 5k is just disingenuous! You cannot possibly believe that a human gets miraculously 10 stones better with higher time limits.

Here are my main claims, and I believe you disagree with them:
I think that it was initially thought that chess required much more intelligence than it does, so that we overestimated the value of teaching a computer to play it well.

In contrast to your other issues I believe that this belief of yours is just plain wrong! Chess was NEVER used as a scientific benchmark to how smart computers were. There was significant research into chess between 1970 (after the AI researchers found out that general purpose machines where not practical) and 1990 (when chess computer beat grandmasters). The reason was that after “human-level intelligence” failed they tied the next best thing: chess. Now we know that chess (and go!) is a long way off of being any kind of benchmark. You can cheat chess, you cannot cheat speech recognition! This is why speech recognition was proposed in the 1950's as the benchmark! Since then no better benchmark has been proposed!

The problems computers have with language are not solely a matter of general purpose knowledge, but also a matter of understanding/knowing rules of language (syntax) and the meaning of words, neither of which I think computers have accomplished.

The thing is: understanding/knowing rules of languages or the meaning of words is not a matter of complex algorithms. It’s a matter of having a database and a logical programming language (both available since the 70's). There have been 40 years (or rather 5, then they gave up) of research into this subject. You simply CANNOT do it! And syntax rules have nothing to do with it. Name a syntax rule and I will program it into a computer, simple as that. The thing is, you CANNOT name every syntax rule. (Believe me, we've tried. Oh, we've tired) You CANNOT emulate speech from a top-down approach. The only way is bottom-up, and that is hard because it requires a fully functional human brain! That is why the Turing Test is universally accepted.

[iazzi]

The issue is that because human understanding of language is recursive, and current techniques for machine translation aren't, the computer can never match the human's competence.

Wow, I would never make that bet... Maybe I will not see it in my life (and I would be very sad of that) but from that to never... It looks a bit strong.

Consider an analogy: you give me a set of mathematical claims and have me test whether they are true or false. All of them have proofs or disproofs.

It may be offtopic, but the second statement is false in general

And I think there are algorithmic generators of correct theorems (they don't really prove anything we care about...) And as far as I know there are programs that check mathematical statements for correctness. Some times they may not find an answer fast enough, of course, but the same is true for humans. Again, the machines are quite weak in this regard, but theoretically they could do anything.

In correct go play there may be nothing even remotely similar to what we call "influence".

Now that I have defined influence pretty precisely (*) via its degrees of connection, life and territory, it has become clear that there is a lot of influence in correct go. Only the traditional light radiation model can be abandoned (although for rough "intuitive" approximation, quite some players might still find it useful).

I hope this does not look dismissive, but I doubt you have correct play at your hand (no one has figured correct chess play yet). Just isolating the longest game line would be a massive achievement. And proving it to be such would almost certainly require to have the winning strategy. Until you can express the winning strategy in human language, no one can really say to have an idea on how it would look like. To define a positional judgement you have to put strategies in normal form, and find the sets of discriminants for the winning strategy (as functions of the positions) and find the set that minimizes the sum of kolmogorov lengths of each discriminant.

I would rather expect that you have your set of mnemonics figured out and they contain something you call "influence" and you have a way to link this "influence" to your idea of good play. Or you could simply define "good play" as the one that maximizes a reward (length of play or number of moves to win) but then you have just implied all the work above

[snorri]

Seriously...
I have not played against the Zen 5d program, but I did a few games against some other program claiming to be 1d/2d and few months back and let me tell you - this programs were nowhere near dan! I think the trick is to play very fast games and capitalize of humans making really silly mistakes, especially kyu players and players not used to fast games. When I observe the Zen program play right now, it looks like it makes nothing more than average moves, but it makes them consistently, without major blunders - within the time limits. I would be very interesting to see how this software would fare against a reasonable dan player, in even game, with longer time controls.

So what? Can't the exact same comment be made about some human players? How many KGS 5ds hold those ratings entirely through blitz and would not be able to do so in OTB longer games? If so, it seems a little unfair to blame the people running these bots for adopting a strategy that's been used for years by humans.

As for winning while playing only average-looking moves, this is also what was said of Lee Changho. It can hardly be considered an insult.

[shapenaji]

I seriously think that this is ALL there is to such programs. If they could really do more, they would have been 5d at ANY time limits since as with a human players - more time would translate into better play. But i don't think it does. This is why you mostly see the programs playing 15 sec per move games. Double this time and I suspect the 5d program would drop to a much lower grade.

I might be wrong, though... Hard to say when you can't really observe such games. On the other hand - this by itself is telling, I think.

Zen19D - 5d - playing with 9x:15
Zen19N - 3d - playing with 20minute main + 5x:30

Also Zen19N was playing on a less powerful machine, and it hasn't been around since April.

That sort-of supports what I said.
Double the time again, and Zen will be 1k. In real-life tournament conditions, it would be 3k. At home, under unlimited time - just for practice, of course - it will be 5k. Still the manufacturers can claim, and the fans can go Oh-and-Ah, about a computer program being 5d. I dunno...

I guess we will not know until we put the programs to some real test. Who knows, we might all get surprised... some of us more than others.

I HAVE played it, and though I won it's really difficult to judge its strength. It has a very unusual style, but I (being somewhat familiar with how these codes operate) was playing anti-comp go.

It may not be 5d, but it's not weaker than 3d imo

(EDIT: also, I'm not sure how that would support your position, the reason why N was weaker was that the hardware was considerably weaker, not that the time controls were shorter. This is a powerful brute-force program tied to databases of human play, 26 cores makes a bigger difference than 15 seconds. Also, Humans don't get linearly stronger with time, and the coders have said it stabilizes at 4d in longer games)

[emeraldemon]

Someone mentioned trying to do a Malkovich style with a bot... I don't have Zen19, But Fuego is pretty decent and will produce playouts of its thought process (actually w/ GoGui you can watch it consider moves in real time, it's pretty interesting). If someone was interested we could try to set up a game where I just let fuego think about the move for 1 hour or so, then put its "thoughts" in hide tags. Playing short time games on my hardware, I've found it to be a bit weaker than me, but not much, maybe 4k or 3k. I don't know what would happen if there's a longer time limit.

Is anyone interested in challenging a bot to a Malkovich?

[RobertJasiek]

iazzi, of course I do not know the correct, perfect play. I do know though that each play (even perfect play) is related to connection, life and territory impact (because each play can be expressed in those terms). Therefore influence, which does consist of connection, life and territory impact, will be meaningful for explaining perfect play.

[Joaz Banbeck]

...
So, the hard part isn't coming up with reasons-- I could probably write a program right now with a built in set of possible reason fragments, give it a few simple rules, and it would generate moderately convincing reasons for any move...

You could program it to make comments for Magicwand's Malkovich games.

[hyperpape]

@iazzi Yes, but you'll see in my post that I was referring to a carefully chosen set of claims, each of which was provably true or false, where the counterexamples showed a certain structure. It's easy enough to wall off particular types of questions that can always be answered through a single method.

And yes, automatic proof engines exist, and are getting remarkably good.

[hyperpape]

Someone mentioned trying to do a Malkovich style with a bot... I don't have Zen19, But Fuego is pretty decent and will produce playouts of its thought process (actually w/ GoGui you can watch it consider moves in real time, it's pretty interesting). If someone was interested we could try to set up a game where I just let fuego think about the move for 1 hour or so, then put its "thoughts" in hide tags. Playing short time games on my hardware, I've found it to be a bit weaker than me, but not much, maybe 4k or 3k. I don't know what would happen if there's a longer time limit.

Is anyone interested in challenging a bot to a Malkovich?

This is a nice idea, but how far is Fuego from the top bots?

[iazzi]

Actually, I was hinting that they probably don't exist in the way people think they do. At the heart of it, it is pure reading.

With this I agree 100%. I said it badly, but what I mean is that the only objective way to say that a play is correct is to read until the end of the game and confirm that it is leads to a sure win. (you need disambiguations, but you can use any kind of ordering)

Incidentally this means that having a super strong computer in chess still has no bearing on how much we know of the correct play.

Even when chess programs say something like "the rook on the 7th makes this a winning position", they say it because they've done all the reading. A rook on the 7th in a slightly different position might *not* be winning.

However this is false. Computer programs are not able to do all the reading even when there are about ten pieces (including kings) on the board. They use the above heuristic to actually say that the position is good enough and needs no more evaluating, exactly like a human would do. Would you lose a queen blindly? No. You would consider it a lost position and not read the variation any longer. So does a computer, it uses heuristics to avoid reading.

Understanding a chess position is vastly simpler due to the discreet pieces. E.g., in chess the concept "a knight on a high row" means something; in go "a stone in a high position" means nothing without boatloads of context.

I agree with this too: Go is more complex. The only thing I want to point out is that there is no known qualitative difference and we can expect computers to beat the best humans at some point. I prefer to avoid delusions, especially after seeing the situation in chess.

@iazzi Yes, but you'll see in my post that I was referring to a carefully chosen set of claims, each of which was provably true or false, where the counterexamples showed a certain structure. It's easy enough to wall off particular types of questions that can always be answered through a single method.

Oh, I did not realise it was a condition, sorry.

I have to say that giving a set of problems chosen in a particular subset and get them solved by a general purpose machine makes me unsure of what we are actually checking. For example there are some papers where statistics of solutions to k-SAT problems are analysed. Since they have to define a probability distribution over the problems one (me but also a few colleagues raised the same objection) is left to wonder if the results depend on it and how natural the choice actually is. I have the same feeling about the test you propose, but I may be wrong.

[hyperpape]

iazzi: Yes, the test I mentioned was quite limited. I was constructing an analogy, and the domain was very artificially limited to make the analogy easy to grasp.

[Mike Novack]

Why go with opinion on questions that are essentially empirical?

We do have good evidence of the releative strength of these bots (they do often play each other).
We can somemtimes find out (will be told if we ask) the strength of the hardware of the bots.
We in some cases possess the programs and are able to run them on known hardware at whatever time control we choose.

Thus --- Fotland claims MFOG 12.022 running on a 2 core ~2 GHz machine at a time control of 60 minutes for all moves should be about 1 dan against humans. Now I'm not personally strong enough to test that but at our local go club a 7 dan gave it 6 stones and a 5 dan 4 stones and both lost to the computer and they judged, "yes, 1 dan". Interesting to note they felt an unbalanced 1 dan, weaker than that strategically but stronger tactically; not unlike what is typical of a go prodigy when a young child.