Have we been duped by AlphaGo?

[Mef]

I received an email from David Ormerod of GoGameGuru today, which may be more relevant to the specifics of the time settings used in the AlphaGo vs. Lee Sedol match. He is a little busy with his baby at the moment, but indicated that it would be alright to quote him.

It appears that Younggil and David were concerned back in February that the members of the Korean camp weren't taking the match seriously enough, and might not be briefing Lee Sedol as well as they could. They contacted Lee Sedol's sister, Sena (she used to live in Sydney, incidentally), and requested that she pass on information about computer Go and to have Sedol consult with a computer expert. David indicated that Sena later explained the reasoning that Lee Sedol had in selecting the two hour time limit.

Here's a quote from David's email to me:

Anyway, I don't think anyone has mentioned that longer time in a human vs computer match fatigues the human, but not the computer. This was a key factor in Lee's decision to choose 2 hours each instead of a longer time limit. He was worried that with both players using 3 hours or more he would get too tired and the computer would gain an advantage -- as you know, the match with Gu was 4 hours. He may have miscalculated this, but he thought he could play well enough with 2 hours and not get too tired.

So while it is possible that Lee Sedol miscalculated the optimal time settings for the match, it appears that he believes that fatigue would affect the quality of his play, whereas it would not affect the computer. With a longer time setting, like with his match against Gu, presumably he felt that his game quality would be worse with the increased time.

Of course, this is not any sort of definitive proof as to the best time setting, but it at least provides some evidence for this theory:


If nothing else, I'd think that we can see that game quality is not so simple to analyze, just by looking at the time limits alone.

Alternatively, it supports this theory (where a human, while still making better moves with increased time, does not utilize this advantage to the extent a computer would):

[wineandgolover]

... so, just how many angels can dance on the head of a pin?

[Uberdude]

Kirby, remember JF only called 30 second byo yomi (as in the non-captain games of the Korean Baduk league I believe) Mickey Mouse time limits, not 3 hour games. He called those 'rapid' but not many others agreed with that adjective. Interesting point about the Lee Sedol match though. Does Lee still feel that way after the result?

@seigenblues, about a 5 second game benefiting LSD I disagree if you allow Google Deepmind so throw more computing resources at the problem, see http://lifein19x19.com/forum/viewtopic. ... 63#p204263.

[Kirby]

Alternatively, it supports this theory (where a human, while still making better moves with increased time, does not utilize this advantage to the extent a computer would):

No, not really. A fatigued human probably does not make better moves with increased time. Had Lee Sedol picked two hours because he felt that a computer would benefit more from increased time, it could support your theory.

But the fact that fatigue was the concern suggests that, at least to Lee Sedol, he felt there was potential that his play would be degraded due to fatigue.

Lee Sedol may not be an expert on game quality, but I think you are stretching it to try to suggest that LSD thinks he plays better quality moves when fatigued.

Now, I suppose your argument is that Lee Sedol felt his relative increase quality over time was less than the computer's due to fatigue, but this is adding to what we heard from him.

Specifically, we heard that he thinks he'd be fatigued from a longer time setting. That alone seems more likely to suggest that he thinks he'd play worse with too much time.

Anyway, I still admit that other graphs and models are possible. But we should not be so quick to assume that more time means a better quality game.

[Kirby]

Kirby, remember JF only called 30 second byo yomi (as in the non-captain games of the Korean Baduk league I believe) Mickey Mouse time limits, not 3 hour games. He called those 'rapid' but not many others agreed with that adjective. Interesting point about the Lee Sedol match though. Does Lee still feel that way after the result?

The OP suggests a Disney relationship with the AlphaGo match:

AlphaGo has simply confirmed that the real Voldemort was Mickey Mouse.

Later in this thread, the games were called rapid, so the classification was said to be "rapid", but the OP at least suggests inluence from Mickey Mouse.

Regarding LSD's feeling after the match, I am not sure. This is just what David told me about their interaction with Lee Sedol's sister. If we want to get his current opinion, we'd have to ask him or his sister, again.

[Kirby]

Alternatively, it supports this theory (where a human, while still making better moves with increased time, does not utilize this advantage to the extent a computer would):

To further elaborate on this, Mef, your graph suggests that, given additional playing time, game quality continues to improve indefinitely (potentially, with diminishing returns).

Do you really believe that there is no point in time at which a human being could become fatigued enough to lose the benefit of additional time completely, and start playing worse?

Based on Lee Sedol's comment, we don't know whether he thinks a game longer than 2 hours would be disadvantageous to his absolute game quality, or simply his relative quality to the computer. But this thread is about whether longer time limits could have led to a better result against AlphaGo, so it's somewhat irrelevant.

[dfan]

Do you really believe that there is no point in time at which a human being could become fatigued enough to lose the benefit of additional time completely, and start playing worse?

In theory, if a player was given X time and thought that they would play better if given a smaller amount of time Y, they can just play as if they were given Y time. The reverse strategy is obviously not true. Therefore in theory there is no downside to being allotted more time.

Of course, the opponent may make better use of their additional time, making it a bad idea for our player to play a game at higher time limits. (I've encountered this to my benefit in chess games, where an opponent who would give me a good fight in a 30-minutes-each game doesn't have the concentration to use their time effectively in a 2-hours-each game.) Or our player may feel compelled to use all the time they have been given, on principle, and thus perform worse due to fatigue.

[Uberdude]

Do you really believe that there is no point in time at which a human being could become fatigued enough to lose the benefit of additional time completely, and start playing worse?

In theory, if a player was given X time and thought that they would play better if given a smaller amount of time Y, they can just play as if they were given Y time. The reverse strategy is obviously not true. Therefore in theory there is no downside to being allotted more time.

Note true, if they are expected to sit at the board whilst their opponent takes a long time to play their move. If they are allowed to leave and have a nap and pause the clock it might help, but then after adjournments you might not be "in the zone" and take some time to get your mind back into the game (speaking from personal experience of lunch pauses in the British Championship title match). But yes if AlphaGo were given 2 hours and Lee Sedol 4 hours, then in theory he should equal or better to 2 hours each because he can choose to use the extra time or not (of course there is a danger he uses the extra time thinking it will make him play better, but then he gets fatigued and blunders so actually plays worse).

I find the Lee Sedol wanting 2 hours to avoid fatigue angle particularly interesting given he pushed for 4 hours in the match with Gu Li when Gu wanted shorter limits. Perhaps this is a reflection he is playing another human who also fatigues, probably more so than he does (Lee often catches up in the endgame when Gu Li makes mistakes).

To answers Kirby's question, yes I believe there is a time where humans fatigue and play worse, though it will differ from person to person. I would expect pros to in general have more stamina than amateurs, and older players less than young (but not very young inexperienced ones, maybe early 20s with a good few years of experience is the best age?). It seems games with more than 4 hours each are split over multiple days, in the super-long games of old like the game of the century Shusai vs Go (24 hours each) their were numerous adjournments (to Shusai's advantage, he studied the game in the breaks with others, hence Kitani ensuring sealed moves in their match). I think when JF is advocating long time limits to create high-quality masterpiece games he may be thinking of this format to avoid fatigue problems.

The longest real-time games I have played are the British Championship games, which are 3 hours each with 10 stones in 10 minutes overtime. The first game of last year's championship started at 10:00 am and didn't finish until around 7:30 pm iirc (with ~1 hour lunch break). I fatigued and made many basic errors towards the end (losing a game I had been winning by dozens of points by 2.5 in the end), but if I had played faster I probably wouldn't have played as well in the beginning which was a high-stakes fight. The subsequent games weren't as long (about 6-7 hours), and I was largely successful in maintaining my concentration. I would expect pros to have better mental stamina than me (but they presumably also think more intensely), so Lee's choice of 2 hours is a bit lower than I expected, but then he was playing a tireless computer.

P.S. Have a chuckle at my expense below, it was an exciting game!

[Kirby]

I agree with Uberdude's point that you have to wait for your opponent to play, which can result in additional fatigue. Also, pros are trained to play according to their time limits. In fact, I believe AlphaGo even had a neural network to train itself to divide its time up. Even when pros are playing an obvious move, they often wait until the last minute to play. This can be mentally draining. Pros are human, too, so while they are doing their best to utilize their time effectively, there may be other factors (e.g. respect for the opponent, being "in the zone", etc.). It's kind of like komi. Komi has changed over time, and players have adjusted their playing style to accommodate komi. Same is true for time settings, I think.

Sometimes, the "optimal" use of your time is not straightforward.

I'm reminded of a time back as an undergrad in my operating systems class. There was some multi-threaded programming assignment that I had to do, which was due right before Thanksgiving Day at 10:00pm on Wednesday (Thanksgiving is always a Thursday in the USA). It was around Monday, and I wanted to leave home early for Thanksgiving to go home for the break. So starting Monday evening, I started programming. Things progressed somewhat smoothly, but I had a subtle bug in my program, which was not correct. I really wanted to leave home early for the break, so I kept working on it. I stayed up all night until Tuesday morning, and still hadn't finished. I kept trying to solve the bug, and it got to be Wednesday. Still, the bug was there, and I couldn't figure it out. I was exhausted, and felt defeated. But my ride was leaving, and I had to go home for the break. So I gave up at that point, got in the car, and slept as my friend drove us across the state. I hadn't slept much at all since Monday, so it felt great. Nonetheless, it was somewhat sad that I couldn't figure out my program. Finally, I got home, met with my family, and had dinner. Then, I took another nap - I was still exhausted. I woke up something like 9:20pm that Wednesday. I was somewhat defeated about the program I had been working on for countless hours, but decided to give it another try before the assignment was due at 10:00pm. Within about 5 or 10 minutes, I saw a block of code that I didn't quite understand that I had written. It seemed kind of funny. So I just removed it and rewrote it the way I thought it should be. Well, as luck may have it, that totally fixed the bug. And I was able to submit my program in time before the 10:00pm deadline. After getting some rest, I was able to solve in about 5 or 10 minutes what I had been working on for hours across the past couple of days. It was a lesson for me and taught me that sleep is valuable, and the break was necessary. My mind can work more efficiently, and produce a better result than when I am exhausted.

Despite this, I think as humans, we have some motivation to keep focused on a task. People stay late at work to get that last bit of the project done. They stay up all night working on homework assignments like I did. Sometimes, the extra time helps and is necessary. But sometimes, as in the case of my operating systems assignment, the extra time doesn't help as much as a little bit of rest.

---

Anyway, I want to emphasize that I don't know for sure if play degrades after a particular limit. I'm just saying that it's possible, and Lee Sedol's comment about fatigue means that it could be an issue. It's one piece of evidence that points to the possibility that a 2 hour game can still be a quality game. And I have faith that a game played under those limits can be a quality game - not of the "Mickey Mouse" variety.

Most of this thread is just speculation. We don't have real, substantial data indicating better quality of a given type of time control. All we have are a few quotes from Lee Sedol, some speculation from game commentaries, and then our own opinions and preferences.

Personally, I have faith that games using "modern time limits" are high quality games - and in a lot of cases, just as high quality as games played under longer time settings. But it's dangerous to make generalizations. I think this should be done on a game-by-game basis. If nothing else, I hope to convey that the idea that high quality of games played under today's time settings is a viable possibility.

[Bill Spight]

A note about so-called mental stamina

If you are talking about the ability to maintain concentration and effectiveness of mental tasks, I think that stamina is all or nearly all physical. One study had subjects perform mental tasks (math, IIRC) for several hours straight -- at least 6 hours, as I recall --, while lying down. They did not find any diminution in speed or accuracy over that time. (I don't know if any recent studies have found any difference.)

When I went to bridge tournaments in my twenties I used to play three sessions per day, for a total of about 10 hours of play with about 5 hours total time between sessions (8:00 a.m. to 11:00 p.m.). When I was in my forties I started skipping the morning game and going to bed for at least one hour of rest and relaxation after the afternoon session. For serious competition you need to train for both physical fitness and relaxation. Tension is physically taxing. (At one national tournament I moved so little that my self-winding watch stopped. My partner offered to chase me around the table. )

[Bantari]

Really? So you would call a game with plenty of mistakes and a game without any mistakes to be of the same quality?

Sometimes, sure. I enjoy the game of go. Moves may be bad for boring reasons, or really interesting reasons. Mistakes just have little to do with quality.

If so, have you accoplished this my lowering the amount and severity of your mistakes?

Yes, I have grown stronger by making fewer mistakes, or at least, fewer mistakes of certain kinds. But some majority of my games are such that I found them to be quality games, or else I would not continue playing. If I disliked all my games because of my mistakes, why would I continue to play for all these years? If you don't like pickles, you don't eat pickles. I mean, if micky-mouse-timing is causing a drop in the quality of professional games, I have to wonder what enjoyment amateurs possibly could get of their own games.

I am not really sure what you are trying to say here. That more interesting games are automatically of higher quality? That you only continue playing because you make mistakes? Or that the level of pro game quality should somehow impact the enjoyment amateurs take from their own games?

I think this is all unrelated, but maybe I just don't understand you.

To me the equations are simple:
1. fewer and less severe mistakes = stronger play
2. stronger play (on both sides) = higher quality game

You seem to agree with #1. Do you have a problem with #2?
This might be your issue then, you might be defining "quality" in some weird personal way which is not really apparent to me.

[gowan]

A note about so-called mental stamina

If you are talking about the ability to maintain concentration and effectiveness of mental tasks, I think that stamina is all or nearly all physical. One study had subjects perform mental tasks (math, IIRC) for several hours straight -- at least 6 hours, as I recall --, while lying down. They did not find any diminution in speed or accuracy over that time. (I don't know if any recent studies have found any difference.)

When I went to bridge tournaments in my twenties I used to play three sessions per day, for a total of about 10 hours of play with about 5 hours total time between sessions (8:00 a.m. to 11:00 p.m.). When I was in my forties I started skipping the morning game and going to bed for at least one hour of rest and relaxation after the afternoon session. For serious competition you need to train for both physical fitness and relaxation. Tension is physically taxing. (At one national tournament I moved so little that my self-winding watch stopped. My partner offered to chase me around the table. )

In the Japanese pro literature tairyoku, literally body strength, is said to be important in pro go. This supports your comments above.

[erislover]

I am not really sure what you are trying to say here. That more interesting games are automatically of higher quality?

It's hard to state it more plainly.

To me the equations are simple:
1. fewer and less severe mistakes = stronger play
2. stronger play (on both sides) = higher quality game

You seem to agree with #1. Do you have a problem with #2?

I don't have a problem with 2. I agree that stronger play on both sides is one way in which a game can be higher quality. I do not agree that strong play is identical with higher quality. I don't agree that the existence of mistakes must mean a game is lower quality. The addition of dill to stuffed grape leaves is one way someone might attempt to improve their flavor. On a dill scale, of course it would be the only way, it's common sense. But why should we use a dill scale? It seems unreasonably restrictive and doesn't reflect our intuitions regarding cooking.

My intuition regarding go means both good and bad play is instructive. It defines the space of play. It shows us how to hone our intuitions around various styles of play. "We used to play this way, but no longer." Because it is bad? "We can't say it is bad. But..." White's strategy in no-komi go was to try to develop quickly and disrupt black's coordination. If this was a reasonable way to play, because the games were SO LONG and therefore must be higher quality, then the existence of komi has ruined go, as white's entire strategy has shifted to slower play. In these long-time games we all seem to love so much, white's play must have been very good; then, giving white seven points couldn't possibly have made it worse. And yet.

Well, we all have our opinions. In my opinion mistakes can be at least as interesting as good play, if for no other reason than to see an excellent refutation, devised by the opponent, or later in review. Then there is no way that mistakes must lead to a lower quality game.

[gowan]

I am not really sure what you are trying to say here. That more interesting games are automatically of higher quality?

It's hard to state it more plainly.

To me the equations are simple:
1. fewer and less severe mistakes = stronger play
2. stronger play (on both sides) = higher quality game

You seem to agree with #1. Do you have a problem with #2?

I don't have a problem with 2. I agree that stronger play on both sides is one way in which a game can be higher quality. I do not agree that strong play is identical with higher quality. I don't agree that the existence of mistakes must mean a game is lower quality. The addition of dill to stuffed grape leaves is one way someone might attempt to improve their flavor. On a dill scale, of course it would be the only way, it's common sense. But why should we use a dill scale? It seems unreasonably restrictive and doesn't reflect our intuitions regarding cooking.

My intuition regarding go means both good and bad play is instructive. It defines the space of play. It shows us how to hone our intuitions around various styles of play. "We used to play this way, but no longer." Because it is bad? "We can't say it is bad. But..." White's strategy in no-komi go was to try to develop quickly and disrupt black's coordination. If this was a reasonable way to play, because the games were SO LONG and therefore must be higher quality, then the existence of komi has ruined go, as white's entire strategy has shifted to slower play. In these long-time games we all seem to love so much, white's play must have been very good; then, giving white seven points couldn't possibly have made it worse. And yet.

Well, we all have our opinions. In my opinion mistakes can be at least as interesting as good play, if for no other reason than to see an excellent refutation, devised by the opponent, or later in review. Then there is no way that mistakes must lead to a lower quality game.

I think the word strong is used in different ways in go. Players could be said to be strong if they win most of their games. A move can be called strong if it threatens a bigger followup. A game with no strong fighting though but consists of delicate, subtle maneuvers could be a beautiful, great game. I think many of Lee Chang-ho games are like that.

[gowan]

My intuition regarding go means both good and bad play is instructive. It defines the space of play. It shows us how to hone our intuitions around various styles of play. "We used to play this way, but no longer." Because it is bad? "We can't say it is bad. But..." White's strategy in no-komi go was to try to develop quickly and disrupt black's coordination. If this was a reasonable way to play, because the games were SO LONG and therefore must be higher quality, then the existence of komi has ruined go, as white's entire strategy has shifted to slower play. In these long-time games we all seem to love so much, white's play must have been very good; then, giving white seven points couldn't possibly have made it worse. And yet.

The large komi has, in a way, switched the roles of Black and White. In no-komi go Black could win by playing slow, comfortable moves while White had to make up the deficit of getting no-komi. In, say, 7.5 komi WHite can take it easy and Black has to scramble. I don't know if it is still true but in the early years of large komi-go some players chose to play white stones rather than black when winning the nigiri.