The Shodan Go Bet

[jts]

Mildly amusing that someone would actually believe that a computer will not beat them in the next ~50 years..
(A computer 50 years ago..).

That "unless" is a very big unless. As they say on mutual funds, past results do not predict future returns. If you had predicted the future of the automobile in 1950 on the basis of progress since 1900, you would have reasonably predicted flying cars by 2000... And you would have been utterly wrong.

(I don't think we disagree on substance... Like you say, maybe we'll get quantum computers or maybe we'll get no progress in anything but financing new computer purchases. And if there is huge technical progress, maybe that will translate into 9p go bots, or maybe not. But to describe pessimists as slightly amusing is both patronizing and shows a lack of historical perspective, no?)

[Mike Novack]

Blitz is problematic.

I am close to certain that the reason many bots rank higher at blitz speeds is ........

a) Having less time means the bot must decide on the next move with less probability of that choice being the right one. But for the human not to have enough time to read anything out hurts worse.

b) Humans sometimes make an error in "time management" and so lose on time. Unless there is a bug in the program the bot will never get this wrong and so wins all of these games.

NOTE1: I would not doubt reports that this or that bot is somewhat weaker than its rating but would be suspicious of people reporting "by several ranks". One thing is very important. Need to disregard margins of victory one way or the other for some of these bots. Imagine playing under the conditions "the winner of the game will live; the loser goes before a firing squad". Humans have difficulty staying rational under a condition like that but the bot doesn't. Rational play (for humans too); if ahead, play so as to increase the certainty of staying ahead even if that decreases the margin of victory -- if behind, take any chance if that's the only way the game could conceivably be pulled out. The computer gets no satisfaction from "well I almost made it".

So disregard reports such as "If the bot was really 2 dan I should have needed 5 stones against it. I took only 3 stones and it beat me by just 9.5 points so it can't be 2 dan" or "I should have needed 5 stones against it. I took 6 and beat it by 40.5 points so it can't be 2 dan". That sort of thinking is Ok only against us irrational humans.

[hyperpape]

Idle thoughts:

If KGS ranks were analogous to a well functioning market, then there would be arbitrage. If bots were overranked, then players would know this and play them to boost their own rankings. This continues until the bots lose enough games to return to an accurate rank.

Why would this not happen?

1. The competition for games is sufficiently intense that arbitrageurs are crowded out by chumps who continue to lose to the bots.
2. There is too much of a stigma against playing bots--a 4d player who has played half his games against bots gets no respect, and has no incentive to arbitrage.
3. There are not enough players who believe bots are overranked to make the market clear.
4...?

[Bigstrongpolarbear]

4. It isn't analogous to a well functioning market? I don't know, but does it have to be?

[spacellama11]

Mildly amusing that someone would actually believe that a computer will not beat them in the next ~50 years..
(A computer 50 years ago..)

Computers will beat humans in Go much sooner than in 50 years, unless an unforeseen "wall" is hit. But it works both ways, we might make practical quantum computers, or maybe even something we can't even imagine yet, and solve the whole game for all I know.

And yet, I am rooting for the human in this bet. Woo.

LOL, why on earth does that computer have a giant steering wheel?

[danielfox]

Photoshop
http://www.snopes.com/inboxer/hoaxes/computer.asp

[Mike Novack]

A really interesting discussion.

I would like to understand more clearly the thinking/experience of those who believe the ratings of the bots available for play against on the servers is seriously infalted. One thing to claim "off somewhat" but quite another to believe off by several ranks.

a) Do you think it possible that this could be true only for some or would it necessarily be the case that if true for one then must be true for all of them? Since they play each other with reasonable frequency we do have evidence for their comparative rankings beng correct whatever their absolute ranks might be.

b) Depending on how you answer "a", how about the experience of those who get to play against (some) of these programs directly instead of on the servers? In many cases we do know whether the version on our own computers is the same as the bot is running and may even know the comparative power of the machines and how that could be expected to affect playing strength.

c) The time controls matter. As already discussed, the programs may play relatively stronger at blitz speeds. But not all bots are playing at blitz. But maybe that (slightly slower) speed is still too fast. But here we have to ask another question. At what speeds do humans choose to play other humans on the servers? In other words, we can't really object if the bot is playing at a popular speed.

[hyperpape]

It looks like the hardware has been decided for the first two games of the bet. Still no word on the software that will be used. http://dcook.org/gobet/details.html

[hyperpape]

John Tromp has won the first game, the opponent is Many Faces.

[yoyoma]

Game 1: Tromp won, summary from http://dcook.org/gobet/index.html :
It started off promising, but black played a few weak moves between about move 35 and 60; it still thought the game was close past move 80, but then its groups started dying. By move 140 or so its win estimate was around 15%, and it was probably behind by 100+ points.




Game 2: Tromp won, my own summary:
For a long time the game was peaceful, and then there was a huge trade of groups, black (Tromp) killing a group in the lower right, white (MFoG) killing a group in the upper left. After the trade White was leading, but White misplayed a fight in the upper right. After that Black had a huge lead and White played more and more desperate moves and eventually resigned.

[wms]

Both games are pretty convincing wins by Tromp. Would be interesting to change computer opponents and see if another strong computer player has better luck.

Edit: Clarifying that I meant changing *computer* opponents, not changing out Tromp for somebody else.

[yoyoma]

Both games are pretty convincing wins by Tromp. Would be interesting to change computer opponents and see if another strong player has better luck.

Ah yes I forgot to add that they are going to use a better hardware tomorrow. They are going to rent something from Amazon (some sort of cloud compute time rental? I don't know). They said Tromp has allowed this change from the original rule that the hardware must be present. I don't know if they are still keeping with the $5000 value limit for the hardware.

I wouldn't characterize game #2 as a convincing win by Tromp. The computer could have won if it played better in the upper right fighting.

[wms]

I think the computer made more mistakes - and stupider mistakes - than just the upper right fighting. After losing the group in the lower right, the computer kept coming back and making utterly stupid moves trying to bring that group back to life. Those moves didn't cost nearly as many points as the upper right loss, but they were moves that I'd expect from a 5k player, not a player that could give a 2d a decent game. In general the computer is making some good moves, but in both games it makes moves that do nothing except give John sente for free. That's why I call them both convincing wins - John's mistakes were too subtle for me to be sure of, but the computer made mistakes that jumped out at me.

Hopefully the better hardware will help the computer out. We'll see!

[tapir]

I can't imagine John Tromp plays moves like J14 and J16 in the second game or J18 in the first game against any human opponent. Basically the bot made a fool out of himself.

[hyperpape]

I'm not sure what you meant, tapir, but just to be clear, Tromp didn't play J14 and J16 in the latter game. The computer played as White.