Zen19 had made 3d on kgs!

[Bantari]

The playing style of Monte-Carlo bots (Many Faces is one)
is completely different from human playing style. Therefore,
it is very normal that sometimes

(A) humans think that such a bot plays very poor moves from time to time

Well... it was more than just 'thinking' - these WERE retarded moves.

And while I grant you that it is certainly possible (and probably true) that programs view the game differently than humans, it would stand to reason that a 1d programs making retarded moves so much, would also have to make a fair number of really good moves which would balance out all the blunders. I am not sure I have seen any such moves, although it is certainly possible that they were there I am just not strong enough to notice them.

From my observation, the programs have had two advantages:
1) fast pace of the game
2) the ability (inherent to all programs, even the weakest ones) not to be psyched-out by the moves

Other than this - I have watched a lot of 1d games, and MFoG did not seem to play at that level.

It might be as you say, the program views the game differently than humans, but then the question is - what is its value as a sparring and teaching partner?

Anyways... I did not really wanted to harp about the weaknesses of computer programs in Go.
Just wondering if anybody has noticed stuff like that.

For example - has any dan player purchased MFoG and really gave it a workout at different time settings to see how it plays?

[Mike Novack]

This is tricky.

Time controls -- as far as I know we have only one example of a single bot on the same hardware playing at under two different time controls. Manyfaces is playing at 30 minutes for all the moves and Manyfaces1 at 10 sec per move. Note that these aren't actually very different average times per move. If you look at the stats, MFOG is stronger against a human opponent under the 10 sec time control. Why? Probably because the human player sometimes needs longer to read out a position, using time saved by making other moves faster.

Bad moves, "not 1 dan moves", and similar comments. I think you need to go by the stats. If you have a large number of games between a bot and human 1 dan players (and stronger or weaker human players with the appropriate handicaps) and the bot wins half of them then it must be playing at 1 dan.

[Bantari]

This is tricky.

Time controls -- as far as I know we have only one example of a single bot on the same hardware playing at under two different time controls. Manyfaces is playing at 30 minutes for all the moves and Manyfaces1 at 10 sec per move. Note that these aren't actually very different average times per move. If you look at the stats, MFOG is stronger against a human opponent under the 10 sec time control. Why? Probably because the human player sometimes needs longer to read out a position, using time saved by making other moves faster.

Bad moves, "not 1 dan moves", and similar comments. I think you need to go by the stats. If you have a large number of games between a bot and human 1 dan players (and stronger or weaker human players with the appropriate handicaps) and the bot wins half of them then it must be playing at 1 dan.

True.
And I am not saying it is not.
Except that the skill level a 1d displays at 10sec/move is slightly different that that of a 1d in slow games.
One of the reasons I think that ratings on servers should be adjusted for time controls, but that's another discussion.

What interests me is not if a MFoG with rating 1d plays as 1d in 10sec/move - I know that, this is what the stats say.
Does it also play as 1d in slower games? As I said - I don't want to trash computers here. I am just interested in how these ratings from KGS translate into real world under real tournament (or even club-game) conditions.

[Ingo Althofer]

The playing style of Monte-Carlo bots (Many Faces is one)
is completely different from human playing style...

Well... it was more than just 'thinking' - these WERE retarded moves.

That is normal. Monte-Carlo bots rely on a parameter called
"win rate". It is given in % and takes on values between 0 and 100.
The meaning of certain ranges of value, in the opinion of the bot:

50 % -- the game is about even

above 50 % -- bot has advantage
below 50 % -- bot is behind

above 70 % -- bot will very likely win (probability > 95 %)
below 30 % -- bot will very likely lose
(attention: these %-values are not exactly probabilities of win)

In the normal settings (like in MF on KGS), the bot will resign
when down to 30 %. Often it will start playing desperate moves
already when at 40 % or below.

The reason for this desperation strategy is that a loss with 0.5 points
is not better than a loss with 100.5 points.

... it would stand to reason that a 1d programs making retarded moves so much, would also have to make a fair number of really good moves which would balance out all the blunders.
...
It might be as you say, the program views the game differently than humans, but then the question is - what is its value as a sparring and teaching partner?

I use Many Faces for instance to analyse games of strong players.
As an example, see my analysis of a 7-dan game from the recent
European Go Congress.

http://www.althofer.de/computer-analysi ... rates.html

In this game, Shikshin (who finally became winner of the Congress) was
clearly behind after move 130 (being at 30 % in the opinion of Many Faces).
In this hopeless position he played on until move 247.
Without Many Faces I would not have seen this so clearly.

Ingo.

One question: You claim to have account "Bantari" on KGS.
But there is no user with this name (and games) in the current list.

[topazg]

I have posted in a few places that Monte Carlo make very poor sparring partners. The amount of nonsense they play both ahead and behind makes them impossible to learn from, and a lot of their early midgame moves are very strange, designed to maximise chances in fights as opposed to actually find the best opening moves.

I also have not played any engine on normal hardware that is consistently stronger than I am. This includes Fuego, Mogo and MFoG (although I haven't played ZenGo). I believe the 1k/1d strength of ZenGo is much more likely than 3d against human opponents, and the time controls explain a lot - at 10 sec/move I will play a much much much worse game. It's not just reading, it's the confidence and time to look over the board and decide on flow and direction, which are often choices it is hard to have in the middle of a big fight unless you have the time to think.

Don't get me wrong, I still think it is impressive that bots have come so far, but I do not buy 3 dan for a second.

I also have scepticism (but less so), about the analysis feature.

Firstly, I would be interested to compare, over say 300 top games (7d vs 7d upwards), the bot evaluation at moves 50, 100, and 150, against a pro's opinion.

Secondly, I would be interested if it compared "who's going to win?" in games that were very close all the way up to the end - say, for example, pro games that didn't end in resignation - at the 100 and 150 move marks.

If it can do both of these with an 80%+ accuracy, I will change my opinion, but until then I doubt MC predictive algorithms, simply because if its evaluation was that strong, it would be a higher rank itself.

[John Fairbairn]

When chess computers for the mass public first came out, I bought one that was highly rated (in both senses) and thoroughly enjoyed it at first, as it gave me a good game. Then I discovered its quirks, and soon it became possible to demolish the machine easily in every single game and even give it a large handicap. In go terms, if it was 1-dan when I bought it, it was 6-kyu once I got used to it.

I later went on to work on a shogi computer with some of the best brains in the computer chess business. I learned that my experience was normal, and indeed I learned further tricks so that my chess computer was more like 9-kyu now.

When our shogi computer was finished and we took it to Japan, claiming tongue in cheek that it was close to 1-dan, we even beat a pro on 4 pieces and got a front-page coup. But we knew the pro was being very nice to us - even we knew how to beat the machine on even bigger handicaps.

Back in the chess world, huge advances were later made, and of course Kasparov lost a famous match. But Garry complained bitterly at the time and ever after that the Deep Blue team refused to let him see examples of its previous games. The assumption is that the team knew that they could be beaten once its quirks were known.

I expect the same pattern to apply to computer go. I don't accept the MoGo and Zen ratings except as short-term indicators (and, yes, impressive ones at that). Once they go commercial I expect their grades to drift down far and swiftly, just like my first copy of MFOG.

Looking at the chess experience, though, should we now expect vast improvement? I'm not sure about that. Computers are good at storing vast opening lines. Great for chess, but not a major factor in go, perhaps? Computers are good at analysing fights. This is where they really dominate at chess nowadays, but in go you can often avoid fights, and the board is big enough to allow a lost fight to become a sacrifice with compensation elsewhere. (Calculating ability might, hwoever, work well in go for the endgame, where most of us are weak.)

On the whole, therefore, I would expect that people who buy a commercial copy of Zen or Mogo would be disappointed with their purchase in a matter of weeks, and I don't think we are going to see a genuine challenge to even a strong amateur for a very long time. I actually hope I'm wrong, but experience seems to count for something.

We may get a clearer view in London this Christmas. There is to be a ?ten-game match between John Tromp (a 1 or 2-dan, I think) and the best computer program as chosen by a computer person - for money! I think John Tromp's money is safe, although he may get bitten in the bum in the first game or two if he hasn't seen that program before.

[Mike Novack]

Except that the skill level a 1d displays at 10sec/move is slightly different that that of a 1d in slow games.
One of the reasons I think that ratings on servers should be adjusted for time controls, but that's another discussion.

What interests me is not if a MFoG with rating 1d plays as 1d in 10sec/move - I know that, this is what the stats say.
Does it also play as 1d in slower games? As I said - I don't want to trash computers here. I am just interested in how these ratings from KGS translate into real world under real tournament (or even club-game) conditions.

I thought that I discussed that already by referring to the two stats. Though I did so only by name (and not by rating). ManyFaces1 (at 10 seconds/move) has a somewhat higher rating than ManyFaces (30 minutes for the moves). A strong 1 dan vs a weaker 1 dan. The program can't benefit from "more time to think" as well as a human can. The algorithm does do better with more time but not proportionally so (it doesn't "scale" with time). Related to "sample size" in probability where the lay person often asks "why didn't they use a larger sample?" (they used a size such that it would take a gigantic increase in the size of the sample to increase the certainty of the result by a meaningful amount -- the point iof diminishing returns).

BUT (a very big but) it seems that from the point of view of the creators/vendors of such software optimizing for relatively quick play is crucial. The customers don't seem to want slow playing. In direct answer to your question, it would play weaker compared to the human if the human made good use of the additional available time (able to recognize should stop and think at this point -- one of my current problems).

In regard to the broader question, can software like this be used as a learning tool I think how used may be crucial. If grossly ahead will play overly safely. If grossly behind overly recklessly. So you want to establish initial conditions where the program will be slightly ahead (take a slightly inadequate handicap if weaker or give is a slightly excessive handicap if you are stronger).

What John just posted is relevant but more a matter of whether critical deficiencies can be remedied as fast as discovered or not. While game playing software not my "line of country" I have been watching the rate of improvement over time and it has been impressive. The issue isn't whether a release quickly jumps up a level and then begins declining as some loophole or other is discovered by its opponents as much as whether the developer analyzing games of this period of decline can identify the deficiency and find some "fix" for it. We humans "learn" from our mistakes; the program needs to be altered to do so (although not applicable here, that could be automatic if an AI of the "neural net" sort which can "learn" and "forget").

[palapiku]

A lot of their early midgame moves are very strange, designed to maximise chances in fights as opposed to actually find the best opening moves.

Sounds like the bots are Korean.

[Bantari]

One question: You claim to have account "Bantari" on KGS.

This is correct.

But there is no user with this name (and games) in the current list.

Is this a question?

[Ingo Althofer]

One question: You claim to have account "Bantari" on KGS.

This is correct.

But there is no user with this name (and games) in the current list.

Is this a question?

Not exactly. Here comes the corresponding question:
How do your claim and the non-existence of "Bantari"
in the KGS archives fit?

I am just wondering why a KGS participant (like you)
writes long postings with questions about the strength
of a bot instead of simply playing this bot on KGS.

Ingo.

[Bantari]

Please, Ingo, or whatever your real name is - do not make this thread about me.
I will try to answer your questions this time, out of politeness, but I do not see why do you expect me to justify myself to you. I don't know you, and you come across as being slightly too aggressive in your attempts to redirect the topic of this thread and focus on me rather than the actual subject. Actually, I do have some suspicions as to who you are... funny, really.

Not exactly. Here comes the corresponding question:
How do your claim and the non-existence of "Bantari"
in the KGS archives fit?

I am not sure what archives you are referring to.
I have an account Bantari on KGS, as anybody can verify by just checking the user info.
I do not play with that account, its just for socializing.
Sad precaution from my IGS days, you understand.

Not sure what else you are fishing for.

I am just wondering why a KGS participant (like you)
writes long postings with questions about the strength
of a bot instead of simply playing this bot on KGS.
Ingo.

The bot I have seen only plays 10sec/move games, which I am not very interested in.
If you read my posts carefully, you would have realized that my questions was how this speed-play rating translates into slower game rank. This question is mostly targeted at dan players owning MFoG. Do you belong to that group?

[Ingo Althofer]

Hello Bantari,

Please, Ingo, or whatever your real name is

My handle is my name from real life (except for o-Umlaut).
Now, I have also given the address of my website
in my profile here.

I will try to answer your questions this time, out of politeness,

Thanks.

but I do not see why do you expect me to justify myself to you. I don't know you, and you come across as being slightly too aggressive

Perception is subjective, on all sides.

I got a feeling of aggressiveness on your side, when
I read your first posting in this thread. Here is
the passage which increased my blood pressure.

Bantari wrote:

... I have seen ManyFaces, ranked at 1d...
It seemed to be making plenty of completely retarded
moves - and I mean REALLY RETARDED!
... the overall level was rather low, certainly not a
1d stuff. Not even a 5k stuff, most of the time ...

To give you my observation on the strength of Many Faces:
* It is KGS 1-dan at the 10sec/move level.

* It is slightly lower (perhaps by half a degree)
at slower levels like 30sec/move.

* It would be slightly stronger (perhaps also by half a degree)
without the ladder crap it produces from time to time.
When playing against ManyFaces you will realize that
it is not so easy to provoke "strange ladder" situations.

I am not sure what archives you are referring to.

I meant
http://www.gokgs.com/gameArchives.jsp

... my questions was how this speed-play rating translates
into slower game rank.

At slower times Many Faces is slightly weaker, as seen
from other MF accounts where it plays (sometimes) at
30 sec/move or 30 min/game.

This question is mostly targeted at dan players owning MFoG.
Do you belong to that group?

No. But over the years I have collected a good
amount of experience, concerning strengths and
weaknesses of go bots.

Ingo.

[Mike Novack]

The bot I have seen only plays 10sec/move games, which I am not very interested in.
If you read my posts carefully, you would have realized that my questions was how this speed-play rating translates into slower game rank. This question is mostly targeted at dan players owning MFoG. Do you belong to that group?

I do not belong to that group (I'm not a dan player). But I will try (again) to explain some things about the behavior of MFOG and other programs using the same basic algorithm.

You question is too specific to how you want to use the program and too specific to your particular situation for answers to be general statements about the usefulness of this sort of go playing software. The performance of algorithms of this class varies* with the "state of the game". Will do very well if the state of the game favors the computer (it's chances are >50%) and poorly otherwise. I think that is what makes it less useful to you, but wouldn't necessarily make it unuseful to people willing to use it under the proper conditions.

So when you ask "is it really 1 d" you mean specifically "will it usually play as well as a 1 d if playing against a human 1 d opponent even?" That is a different question from whether it will play as well as a 1 d human would against an opponent needing 3-4 stones agaist a 1 d or whether it will play as well as a human 1d would when taking a handicap against a stronger human opponent. Especially if in the former case the handicap is just a tad too little or in the latter a tad too much. And I stress that usually (for the "playing even" situation). Some of the games would be satisfactory (in the way you want) but others not.

You also ask about the effect of different time controls. Compared to the human it will do better at uniformly short time provided that's enough to get the "sample size" adequate. It isn't playing objectively weaker when given more time but can't benefit as much as the human can and so comparatively weaker. Not by a huge amount.

So to the general question "is this sort of software useful?" I'd say depends on who you are and how are you trying to use it. I think it's very useful for a player weaker than 1 d as a learning tool, strong enough to punish mistakes. ROFLOL but how it plays provides me with a fairly reliable indication of the state of the game. If I see it begin to make some unnecessary safety plays then I know that perhaps contrary to my own evaluation, it is safely ahead and will win the game. If I see it making senseless overplays the reverse (but sometimes I've missed something vital and those aren't actually overplays!). So I find it useful, but I'm not a 1 d player (need 4 stones against MFOG12 to have any chance, 5 to have good chances, crush it at 6) --- btw, that's playing at 1 hour for all the moves using a 2 core 2+ gHz machine.

* That is the way in which most different from a human player whose performance level isn't going to be so drasticly dependent on "the state of the game".

[lindentree]

I got a registration key for MFOG 12 as a prize for winning a local tournament. I've only played a handful of games with it, time settings 15 minutes, 5/30. In the one even game, at first it seemed to be a reasonable facsimile of AGA 2-3 kyu, but somewhere in the middle game (neither side having a significant advantage in my view), it started playing bizarre responses to my moves, and completely collapsed. The other games I gave it 9 stones, just to see if I could win (I couldn't), and 3 stones, where I might have won if I hadn't been playing late at night and just messing around.

[prokofiev]

I wonder whether they'd be more fun to play against if there were a "play to maximize expected points" option (instead of "play to maximize expected winning probability"). Should that be easy for developers to implement?

It might even make them stronger in general: waiting for an opponent to make a mistake can be better than making ridiculous overplays, at least for sufficiently ridiculous values of ridiculous.