Can the AI be beat?

[Gobani]

Do you think the AI will ever be beaten?

I am optimistic. It happened once. Go is vast enough that even Alphago can't brute force calculate out every possible future... which means it must play based on a conception of things within a given paradigm, which is hard coded into it by a person. As humans, we can change the paradigm that we view things in as easily as we change our mind. We aren't algorithms. This makes me believe it must be possible to beat the AI.

Just the idea alone is worth thinking about. Yet another gift go gives us!!

[Knotwilg]

Do you think the AI will ever be beaten?

I am optimistic. It happened once. Go is vast enough that even Alphago can't brute force calculate out every possible future... which means it must play based on a conception of things within a given paradigm, which is hard coded into it by a person. As humans, we can change the paradigm that we view things in as easily as we change our mind. We aren't algorithms. This makes me believe it must be possible to beat the AI.

Just the idea alone is worth thinking about. Yet another gift go gives us!!

The paradigm that is "hard coded" by humans is not made of go concepts though: it's a particular construct of neural networks and machine learning, given a highly agnostic input (the rules of the game), which makes it equally applicable to Chess as to Go. That is the key difference with the legacy of computer Go, where a programmer was indeed feeding algorithms with go concepts. It's also a key difference with the early versions of AlphaGo, which used high dan games as input, rather than the rules.

So I don't share your "optimism" but neither am I despondent about the fact itself. I'm rather passionate about the flux of ideas AI has generated and what humans are doing with it.

[Gobani]

Great points, I'm less optimistic now after your reply

[seloran]

Do you think the AI will ever be beaten?

I am optimistic. It happened once. Go is vast enough that even Alphago can't brute force calculate out every possible future... which means it must play based on a conception of things within a given paradigm, which is hard coded into it by a person. As humans, we can change the paradigm that we view things in as easily as we change our mind. We aren't algorithms. This makes me believe it must be possible to beat the AI.

Just the idea alone is worth thinking about. Yet another gift go gives us!!

From Wikipedia: “AlphaGo Zero is a version of DeepMind's Go software AlphaGo. AlphaGo's team published an article in the journal Nature on 19 October 2017, introducing AlphaGo Zero, a version created without using data from human games, and stronger than any previous version. By playing games against itself, AlphaGo Zero surpassed the strength of AlphaGo Lee in three days by winning 100 games to 0, reached the level of AlphaGo Master in 21 days, and exceeded all the old versions in 40 days.”

In “AlphaGo the Movie” on YouTube, it’s mentioned that move 78 by Lee Sedol in game 4 was like a move that only about 1 in 10,000 human players would make I guess according to what’s fed to AlphaGo for training, and I presume AlphaGo did not expect it and failed to recover from the shock.

However, because AlphaGo Zero was not trained using data from human games but rather just the knowledge of the rules and an enormous amount of self play, it should not have the flaw of not being able to handle move 78, and I am inclined to think humans without computational enhancements or time to evolve much much further in brain power would not be able to beat even AlphaGo Zero, let alone further possible improvements of AI by the time those become a reality.

[CDavis7M]

Yes. And others have already beaten AIs.

In a non-handicap game? No. Never. Not against a true high-end AI.

Still, handicap games are a long tradition in Go. A win on the correct handicap is still a win.

[dust]

A win on the correct handicap is still a win.

It's not really the same when AI handicap settings are tweakable.

[dankenzon]

No. It won't be beated again. We can win a few ones in thrillions with handicaps, but the difference is beyond human capacity.

[dankenzon]

Do you think the AI will ever be beaten?

I am optimistic. It happened once. Go is vast enough that even Alphago can't brute force calculate out every possible future... which means it must play based on a conception of things within a given paradigm, which is hard coded into it by a person. As humans, we can change the paradigm that we view things in as easily as we change our mind. We aren't algorithms. This makes me believe it must be possible to beat the AI.

Just the idea alone is worth thinking about. Yet another gift go gives us!!

"which means it must play based on a conception of things within a given paradigm, which is hard coded into it by a person."

I know somebody already told you this is not true. But that's the difference. Alphago and self-learning AI engines are not programmed and they learn from its own experience, creating new filters and trying new moves. That's whay in few months AI created new josekis and strategies. More than all the contributions one person can do in his/her whole life.
They can accumulate thousands of years of experience in a few days. Never get tired. Never fail.
here you'll come with "but Lee Sedol won! it can happen again", yes, He won against a less perfect and less powerful and more "human-imitation" engine. The latest one has no such "human" errors

[Mike Novack]

Well .... we might want to ask the question in terms of power consumption.

We might want to compare an AI able to run on a computer limited to about 20-30 watts. That would make it fair.

[jlt]

What is the level of Katago on a powerful smartphone? Is it already superhuman?

[lightvector]

Likely yes. I would guess it would be heavily favored on average against any human player, although maybe it wouldn't be absolute and maybe might still lose occasionally, or be exploitable with some tricky joseki or something.

(I mean, with a big enough blind spot in a situation that swings the status of some huge dragon, even with a massive top GPU Kata could still lose, it's just that the bigger a blind spot you ask for, the rarer it gets.).

[lightvector]

And also, it's definitely more likely than one in "thrillions". Even though a human winning an even game against a top AI on good hardware nowadays is very unlikely, it's still massively, overwhelmingly more likely than that.

When trying to estimate things with low probabilities, a common mistake is to just start throwing out big numbers based on your feeling - which doesn't work because normal human brains don't have good feeling for big numbers. A better way is to anchor the number in a way that makes it more concrete.

For example, consider even "merely" an odds of 1 in a million. Well there are 365 days in a year. So 1000 days would be about 3 years. So a million days would be about 3 milleniums. So every day without fail for about three thousand years, we wave a magic wand and conjure up a new young sharp pro rivaling the likes of Shin Jinseo right now, and they get their shot at trying to beat the AI.

In all those lifetimes of trials, does the bot really never mess up with some blind spot severely enough to lose? Nobody ever stumbles upon a new Mi Yuting's flying dagger that the bot can't handle? The bot doesn't screw up some rare ladder like in Lee Sedol's game, some game-deciding capturing race, or something like that even once that the human player does luck into having happen? Think about all the reported blind spots of bots in the past (many of which you can still google around for), or some of the past discussion on this forum bot evaluation inconsistencies and margins of error and such.

Yeah. Current bots are overwhelmingly strong in even games, but there's no way they come even slightly close to having that level of consistency and infallibility.

[kvasir]

Often when people discus the limitations of game playing programs they start making up constraints for the computer instead of considering the inherit limitations of the human. They say use mobile phone, short time controls, long time controls, play 100s of games first to familiarize the human to the programs typical mistakes etc. However, it is in fact the human that would be at a great disadvantage with almost any EQUAL time controls even if the hardware is not great, not to talk about if they were forced to play 100s of games to prepare (exhaustion!). That is assuming you would choose appropriate weights, network size and software for the hardware and time controls, and one should allow for some iterations to get this choice right. If you are only willing to consider setups that could favor the human it might be a slippery slope and you could end up with a very strange match up.

Another fallacy is to make up win conditions. For example that it is sufficient for the human to have many tries and only win one game. Usually when two humans play we don't make much out of someone winning 1 in 50 games for example, winning by default and other anomalies. In my experience people are in general bad enough with numbers to think that if someone has 1 / 20 chance of winning a tournament game it is practically zero chance and people get really upset when it happens and think "what did I do wrong", "how can a (weak) 5 dan lose to a (strong) 1 dan, impossible!", "that guy must be getting stronger" and so on.

In all honesty, this game is played on the board, not in ones head, and there can always be a chance that the weaker player is the one playing better on the board or there is an upset.

[dust]

It depends on what you can mean by "be beat". A one off-win - maybe, but how significant is it and that would depend on what are the reasons for it.

Will AI "be beat" in terms of a human recognised as a champion over AI - no. Unless there's some kind of Elon Musk style go module implant.

[gennan]

No. It won't be beated again. We can win a few ones in thrillions with handicaps, but the difference is beyond human capacity.

In an even game, maybe Shin Jinseo would have odds lower than 1/10000 (maybe even lower than 1/100000).
But handicap changes things significantly. With Shin Jinseo getting a 3 stone handicap, I'd seriously consider betting on Shin Jinseo. With 4 a stone handicap, I think the odds would clearly favour Shin JinSeo.