The Shodan Go Bet

[yoyoma]

Zen19b [3d] plays 0+5x:10
Zen19 [3d] plays 3+3x:15
Zen19n [2d] plays 20+5x:30

So it looks like humans benefit more than Zen19 from longer thinking times.

[SpongeBob]

personally i dont think computer will beat me in my life time.
i am 43 years old and planning to live upto 100.
i guess you have 57 more years

As much as I would like your words to become true - you are too optimistic here. You (and I) have no idea about the progress that information technology will bring in the next decades.

THE SINGULARITY IS NEAR

[quantumf]

zen has a suspicious number of wins by time. something like 20% (no losses by time). at what time limits are the games played, why have the human players so much trouble with the time?

I would guess bots gain at least a stone (perhaps even 2) by always playing to their ability (i.e. they don't make blunders, they don't get bored, they don't get drunk, they don't get distracted, etc).

[shapenaji]

I dunno, I think the statistic "A computer gains one stone by doubling their time" is a little misleading, I don't think Zen gains that much with repeated doubling of computing time. Human players, otoh, can lose substantially more than one stone by halving their time.

If you include the sampling bias that players who beat a bot are less likely to play it in future, I suspect that the Zenbot is taking advantage of "Time-fish".

[quantumf]

I dunno, I think the statistic "A computer gains one stone by doubling their time" is a little misleading, I don't think Zen gains that much with repeated doubling of computing time. Human players, otoh, can lose substantially more than one stone by halving their time.

I think what the quote means is that the computer gains one stone by doubling its time, all other things being equal (in other words, the opponent doesn't get to double their time). Put another way, the program gains one stone compared to a known and stable set of fixed rank opponents, e.g. gnugo version x with thinking time y.

So yes, I agree it is misleading.

[gaius]

The KGS 3 dan rating is probably based largely on people just trying out a bot for fun, but the bot itself is always playing at maximum skill. If the humans would always play at maximum skill, I doubt whether the bot would even get up to 1 dan.

[lorill]

Well, the same could be said for any kgs player, wether bot or human.

[shapenaji]

Well, the same could be said for any kgs player, wether bot or human.

Eh, I think there's a difference, what I would LOVE to see, is a bot in a blind test, people don't know right off the bat if they're playing a bot or not. A sort of Turing rank test.

There are a number of things at issue with its rank:
A) People are looking for something different when they play a bot, hence they play differently
B) If bots make mistakes, they usually make them systematically, hence if you can beat a bot once, you can usually beat it indefinitely, likewise with losing.

[lorill]

I believe this was already done, by a manual operator inputing all the moves. I don't remember where I read this, though.

[Aeneas]

I know people who play bots because they they feel they can leave the game when something comes up without annoying somebody. Such players would tend to boost the ratings of the bots. It looks like Zen wins many games on time or resign...

Theodor

[hyperpape]

Surely someone exists who is both curious about how a bot would fare in a blind competition and has available hardware (out of all the bots currently playing on KGS, none of their controllers is curious?). Of course this is akin to a collective action problem--I'm saying this but don't have a computer that I could use to run a bot.

[yoyoma]

The KGS 3 dan rating is probably based largely on people just trying out a bot for fun, but the bot itself is always playing at maximum skill. If the humans would always play at maximum skill, I doubt whether the bot would even get up to 1 dan.

That's what is so interesting about this bet! You can be sure John Tromp will be trying.

I have mixed feelings about the whole thing. I like to track the progress of the computers, and I see it as progress of humankind, through our ability to get more and more things done through technology. But on the other hand it's sad to see another game slowly getting overtaken by bots (not that Go has reached that point yet).

So for humankind's progress, my heart is behind the computer (strange thing to say! ). But I think Tromp will win this time. In a few years I think it will swing the other way.

[Mike Novack]

Some misconceptions

a) The bots win a disproportionate number of games on "time".
Well yes of course, this is what you should expect to see, because unless there is a bug in the program the computer shouldn't ever lose on time. It "knows" how much time it can afford to spend "thinking" before making a move.

b) Until "weaknesses discovered"
Well this is more or less dated. An AI playing based on "go knowledge" might not be able to "see" something (has no code for that) but the strongest programs of the last couple years are all using MCTS to select moves and that algorithms doesn't depend on "go knowledge".

c) Why relatively stronger with less time?
This is actually deceptive and true only in a relatively narrow region of time. But it seems that way because these are the time ranges at which we usually play. The programs using MCTS are stronger in absolute terms given more time and would be very strong given unlimited time. At "blitz" speeds too short for the human to read anything out the bot does better against the human compared to speeds slow enough to allow the human reasonable to read. Were the time greatly increased above that errors in reading by the human would limit improvement with the additional time so this trend would reverse.

d) This rating based on playing against whom?
Well naturally selection is involved (who is willing to play the bot). But it isn't quite fair for any of us to say "not that strong" if we aren't willing to back that up by our own trial. Also very important to remember that our experience doing this form a year or two back might not be very relevant and that our experience playing against program A says little about what program B can or cannot do.

3) The machine matters
Since time is a factor, so is the power of the machine on which the program is running. This is very significant in challenges like this and significant when people say "I don't have a machine capable of runnign one of these bots" (odd, what are you using to access this forum -- well if a borrowed machine, one at the library, etc. where you aren't allowed to install software, I guess that would apply).

So this does affect what you would experience as the strength of the program running on your own machine at home and at least for the programs that are being sold you should be told "on what power machine at what time settings". Thus for example while MFOG 12.021 running as a bot on KGS has at the moment a rating over 2 dan that's on a machine far more powerful than you would have -- but Fotland is honest about that and calls this version 1 dan on a "standard machine" and will explain what that means. That program would not play at 1 dan on this old desktop from which I am writing this (but would on my laptop which is about 4 times more powerful).

[quantumf]

b) Until "weaknesses discovered"
Well this is more or less dated. An AI playing based on "go knowledge" might not be able to "see" something (has no code for that) but the strongest programs of the last couple years are all using MCTS to select moves and that algorithms doesn't depend on "go knowledge".

I'm not sure this is true. My understanding is that the playouts are directed to a degree by go knowledge. MCTS doesn't pick a random move from the 19x19 board, it picks from a set of suggested moves based on the current position. In fact, I'm under the impression that this is one of the main areas of research - finding effective ways to apply go knowledge to make useful suggestions.

[flOvermind]

Just because the bot doesn't use go knowledge, that doesn't mean a bot can't have a specific exploitable weakness.

For example, a naive implementation of MC playout will have a blind spot involving ladders, because it is extremely unlikely to randomly play out the ladder far enough to actually hit the ladder breaker. For that you at least need to make sure the bot always tries capture, atari and extend-from-atari moves, which is application of specific go-knowledge, and in that case it *removes* a specific weakness .