Computers reach 5d on KGS

[Bill Spight]

So, the hard part isn't coming up with reasons-- I could probably write a program right now with a built in set of possible reason fragments, give it a few simple rules, and it would generate moderately convincing reasons for any move in a pro game. This would be the digital equivalent of confabulation.

As in, "The market is off of its highs on profit-taking."

[daniel_the_smith]

...
So, the hard part isn't coming up with reasons-- I could probably write a program right now with a built in set of possible reason fragments, give it a few simple rules, and it would generate moderately convincing reasons for any move in a pro game...

I'd find even that fascinating.

I suspect it might be a bit harder than you say, depending on how convincing you have in mind. Plausible sounding sentences, yes, but ones that are even remotely related to the actual move/position? Consider yourself challenged!

Haha, alright, I'll give it a shot, but first I need to finish what I'm doing on my website (as I would do this by building off of it).

Edit: I already had some similar features planned...

So, the hard part isn't coming up with reasons-- I could probably write a program right now with a built in set of possible reason fragments, give it a few simple rules, and it would generate moderately convincing reasons for any move in a pro game. This would be the digital equivalent of confabulation.

As in, "The market is off of its highs on profit-taking."

Exactly. Although I think I could do a bit better than that...

[hyperpape]

I think when humans do this it's mostly confabulation. (IOW, your brain internally came up with a good/bad judgment, and then you verbally come up with reasons to support your feeling. You know this is what happened if you have ever started to explain something and realized halfway through that you were totally wrong!) Some are better at producing convincing confabulations than others...

This is true in a lot of cases, but depends on the domain. Go reading is probably pretty low in confabulation, because it's a mostly conscious process that, by its nature, requires that you remember it pretty well. But if you ask "why didn't you consider move x?" or "why do you think that capture is big enough", then you're moving into the domain of confabulation or "it just seemed that way."

[iazzi]

Show me where there are computers that understand what words mean. Where are there even computers that understand syntax for natural language?

IBM is trying to go there. Watson was a big step forward. It is debatable how much forward. But there are a couple of things that are usually disregarded in relation with computers.

First you can inspect them. When Watson said Toronto was an US city, you will laugh. but it can easily detail exactly how it arrived at such conclusion. And you may realise how to avoid such a wrong reasoning.

The second is that they can inspect themselves. After you build a monster like Deep Blue or Watson, after every answer you can tell them: "Good job! but... couldn't you have realised that faster?" They will optimise themselves, or put in a more human way, they will try to extract principles from raw data. The principles need not be the same that a human would use. In correct go play there may be nothing even remotely similar to what we call "influence".

The hard part is making those reasons correspond with reality. And if the bot's choice is based off of "in 100,000 positions, this move came out the best most often", there's not going to be a way to express that. The bot would have to examine all the failed positions, identify the commonalities between them, and then it could say something like, "If I play X, it's no good because of Y, if Z, then W, ... So, this move avoids most of the problems."

I am not sure what you are measuring here.

A chess playing program will show you the variations he considers best and confutations. It will also tell you that a position is better than another because of doubled/passed pawns, it will evaluate the strength of knights and bishops depending on how many pawns are still on the board etc... Incidentally they do not evaluate moves, only positions. In game theory there is no concept like doing the same move in different positions.

They will say: "I would like to go to this position because the advantage of a rook on the 7th row is good enough to win from here". I think it is exactly what you would call "general principle".

Of course there is a detail here. They will actually say "I would like to go to this position because there is no way to stop me from having a rook on the 7th line in ten moves and this advantage is good enough to win from here", i.e. they combine positional judgement with plain old reading ahead.

But even that isn't terribly useful in the way you guys might want. For the computer to produce a general principle for the situation would genuinely be impressive (unless it's via the confabulation route, in which case it's not impressive and it's questionable that it's based in reality).

You assume that general principles exist at all. It is very unlikely. What you call general principles are a set of mnemonics to estimate the relative winning probability. Something that the computer does very well, in fact

And if your bot is based on Bayesian weighting of a jillion small automatically tuned factors or something similar, you could have written the source code and still not have even the slightest inkling of how it works.

You overestimate the number of factors and their obscurity. E.g. if I remember correctly Rybka has an abnormally high opinion of a knight in a high row, something very easily understandable and translatable in human terms yet completely unexpected!

[Mnemonic]

Honestly, I don't know how super-early AI people thought they would tackle chess. I do know that they thought general purpose machines would have been developed by the 60's so that trying to conquer specific problems might be wasting time. However, my statement did not regard the methods of trying to solve chess, but if chess was ever considered as a test of intelligence (i.e. if a computer beats a human in chess a computer is as smart as a human) AFAIK this was never the case. The Turing Test was introduced very early (1950) and has garnered wide support over several decades of research.

This seems to rest on the idea that there can be only one test of intelligence, and that is not plausible.

I agree with you on a theoretical level, but usually when we are speaking of intelligence we are talking about our "human-level biological intelligence". If your goal is to build/program computers that emulate this intelligence the accepted test is the Turing Test (and variations). Are there disagreements over the exact implementation of the Turing Test, yes! If a computer passes one Turing Test does that automatically mean computers are as smart as humans, no! But the general principals of the Turing Test are accepted and used by AI researchers for over half a century and is the only known test that correlates human and machine intelligence reasonably well.

I will give you a historic example why speech is such a good approach. When programmers first set out to build Turing machines they asked linguist to provide them with a list of every imaginable grammatical rule English has. The computer returned garbage. So they invented thousands of more rules and still got garbage. Speech is more than just knowing a bunch of works and several grammar rules. And this more cannot be accurately described, not by programmers, not by linguists. You cannot cheat the Turing Test.

Even if you disagree with all the above, you can't possibly argue for replacing the Turing Test with chess.



As a reply to the rest of your post: If I can show you an algorithm or a program that can execute a task as good or better than a human, is that good enough or does the algorithm/program have to work exactly like a human? If last is the case then there has been no progress in AI since the invention of computer because we have nowhere near the hardware necessary for emulating the human brain or the technology for reengineering the brain, although that will change in the next few decades.

If you allow my first statement then I think my examples were reasonable. Spell checking and translating were my examples for word processing, not syntax. And I believe computer are quite good at it. They can, for example, give you synonyms and antonyms for a lot of words, and quicker than most human experts (accuracy is still a problem, but that is not a problem of understanding the word, but of understanding speech )

If you are doubting my claims that computers have reached a reasonable level: link

[daniel_the_smith]

They will say: "I would like to go to this position because the advantage of a rook on the 7th row is good enough to win from here". I think it is exactly what you would call "general principle".

Of course there is a detail here. They will actually say "I would like to go to this position because there is no way to stop me from having a rook on the 7th line in ten moves and this advantage is good enough to win from here", i.e. they combine positional judgement with plain old reading ahead.

I call that digital confabulation. It's easier for chess, but go bots don't work like that. At least not the current best ones (monte carlo). Their positional judgement amounts to "of 40,000 random games, this move won the most". With chess, even if you evolve an evaluation function, you can analyze it afterwards and see that it favors knights or something, so the confabulation is probably based on something real. A go eval function is likely to be very difficult to analyze in a similar way; for example, it likely would appreciate something that amounts to thickness, but it probably would have thousands of rules to determine how thick something is, and it will be very hard to turn that into something that makes sense to humans.

But even that isn't terribly useful in the way you guys might want. For the computer to produce a general principle for the situation would genuinely be impressive (unless it's via the confabulation route, in which case it's not impressive and it's questionable that it's based in reality).

You assume that general principles exist at all. It is very unlikely. What you call general principles are a set of mnemonics to estimate the relative winning probability. Something that the computer does very well, in fact

Actually, I was hinting that they probably don't exist in the way people think they do. At the heart of it, it is pure reading. Even when chess programs say something like "the rook on the 7th makes this a winning position", they say it because they've done all the reading. A rook on the 7th in a slightly different position might *not* be winning.

And if your bot is based on Bayesian weighting of a jillion small automatically tuned factors or something similar, you could have written the source code and still not have even the slightest inkling of how it works.

You overestimate the number of factors and their obscurity. E.g. if I remember correctly Rybka has an abnormally high opinion of a knight in a high row, something very easily understandable and translatable in human terms yet completely unexpected!

If anything, I am probably still underestimating the difficulty. Understanding a chess position is vastly simpler due to the discreet pieces. E.g., in chess the concept "a knight on a high row" means something; in go "a stone in a high position" means nothing without boatloads of context. IOW, a piece in chess is more analogous to a group in go than a single stone. A group in go is made of N stones in M squares, with Y enemy stones occupying Z liberties, etc. To get something on the same conceptual level as a piece in chess is a huge classification problem. Chess has 6 piece types; go has an enormous number of "piece types".

[hyperpape]

Mnemonic: to answer one particular point, in describing a human ability we often work by specifying a competence: the things that the human can do. In the case of language, it is to understand an infinite set of sentences (competence is opposed to performance--what we will achieve in real circumstances. There are grammatical English sentences that would take longer than my life to utter. And if I did hear them, I wouldn't remember enough to understand them). The task of AI is to match that competence of a human (again, performance may differ in various ways).

Exactly similar ways of achieving that competence are not at issue. But for many tasks an implementation can look impressive, but not come close to matching human competence.

The issue is that because human understanding of language is recursive, and current techniques for machine translation aren't, the computer can never match the human's competence.

Consider an analogy: you give me a set of mathematical claims and have me test whether they are true or false. All of them have proofs or disproofs. 99% of them are true or have a counterexample below 10,000, the rest do not have a counterexample until after 1,000,000,000,000,000,000,000. I write a program that tests all numbers up to a million, and I have 99% accuracy in finding false statements (nerds: you'll see that this is a sloppy statement. I know). That looks impressive, but I have obviously not made a dent in the real problem.

If I write an automatic proof assistant (these exist), then even if I can only prove 50% of the cases that a human can, I may have a real AI success story. That can be true, even if my program doesn't write proofs the way that a human would, and doesn't find proofs the way a human does. It all depends on the details, but the different of implementation is not a barrier in principle.

[RobertJasiek]

In correct go play there may be nothing even remotely similar to what we call "influence".

Now that I have defined influence pretty precisely (*) via its degrees of connection, life and territory, it has become clear that there is a lot of influence in correct go. Only the traditional light radiation model can be abandoned (although for rough "intuitive" approximation, quite some players might still find it useful).

(*) A reduction to axioms is now just a matter of hours.

[Mnemonic]

@hyperpage: I agree with your last post, but what is your argument. Or rather, what are we arguing about?

Are we arguing about the Turing Test? Or whether computers understand words and are capable of holding conversations? Or what role chess played in AI research?

[mdobbins]

This makes me wonder: are bots allowed on DGS?

See: http://senseis.xmp.net/?DGSBots

The page is dated 2009 so it may be outdated.

If you want to put one on DGS, there is a Robot interface (which I use heavily in the anDGS Android client)

[Solomon]

At this rate...

[hyperpape]

@hyperpage: I agree with your last post, but what is your argument. Or rather, what are we arguing about?

Are we arguing about the Turing Test? Or whether computers understand words and are capable of holding conversations? Or what role chess played in AI research?

Here are my main claims, and I believe you disagree with them

1. I think that it was initially thought that chess required much more intelligence than it does, so that we overestimated the value of teaching a computer to play it well.
2. The problems computers have with language are not solely a matter of general purpose knowledge, but also a matter of understanding/knowing rules of language (syntax) and the meaning of words, neither of which I think computers have accomplished.
3. An extension of the last point: statistical approaches to translation do not really involve understanding of language, where two earlier posts show what I think understanding syntax would require: viewtopic.php?p=65277#p65277 and viewtopic.php?p=65295#p65295.

[Bantari]

The option I was looking for was missing from the survey.
"Depends on how much the KGS rating system drifts (to accommodate computer players?)"

Seriously...
I have not played against the Zen 5d program, but I did a few games against some other program claiming to be 1d/2d and few months back and let me tell you - this programs were nowhere near dan! I think the trick is to play very fast games and capitalize of humans making really silly mistakes, especially kyu players and players not used to fast games. When I observe the Zen program play right now, it looks like it makes nothing more than average moves, but it makes them consistently, without major blunders - within the time limits. I would be very interesting to see how this software would fare against a reasonable dan player, in even game, with longer time controls.

The advantage of computers is that they can calculate faster and more consistently than humans. So it stands to reason that there is probably a time setting at which the game is pretty much degraded to such few-move-deep calculations, and nothing more. At such time limits, computers should have advantage... it seems right now they have an advantage up to about 5d. Which means they can calculate shallow sequences as well as a human 5d.

I seriously think that this is ALL there is to such programs. If they could really do more, they would have been 5d at ANY time limits since as with a human players - more time would translate into better play. But i don't think it does. This is why you mostly see the programs playing 15 sec per move games. Double this time and I suspect the 5d program would drop to a much lower grade.

I might be wrong, though... Hard to say when you can't really observe such games. On the other hand - this by itself is telling, I think.

[RobertJasiek]

There are more aspects like this. A human has to try extra-hard to get a game with Zen at all because everybody wants to play it. When you finally manage to play it, you are already exhausted from clicking super-fast, waiting and watching others play Zen. Contrarily the program never becomes tired.

(The programs have become impressive but their current rating is not so meaningful.)

[wms]

See, this is why I don't allow general chat for computers that log in to KGS. If I did turn on that feature, the chat rooms would be full of their trolling and bragging.