AlphaGo and tengen

John Fairbairn · #1

I've been a bit puzzled as to why AlphaGo doesn't play tengen. And only one player (Zhou Junxun) has tried it on the human side.

But young Japanese pro Mine Yasuhiro has just tried a combination of tengen and an AlphaGo idea:

White did not respond to the 3-3 invasion, though (is that an idea as to how to play against AG?). But then Mine continued with the weirdness as below:

This has had a few outings even among top players, but never this early (with the exception of course of Daniel Barry in the first space game).

gowan · #2

Supposedly AG plays a fast, influence-oriented game, except for early 3-3 plays. Starting on tengen it can be difficult to make useful influence if the opponent plays a fast development erasing the influence.

Another thought: AG is based on taking in many many many pro games. Tengen as first move happens in pro games but very infrequently so, perhaps, AG's early learning failed to include enough tengen starts. Or maybe, perish the thought, starting on tengen is a bad move

?

Meretrix · #3

John Fairbairn wrote:

I've been a bit puzzled as to why AlphaGo doesn't play tengen.

Hi John,

David Silver of DeepMind partially addressed this when Michael Redmond and the AGA's Andrew Jackson interviewed him before game 2 of the AlphaGo / Ke Jie match. The interview starts at 15:10 and ends at 24:37. At 18:06, AJ and MR ask about the value network's opinions on various moves for black 1. Silver's response was

"Well the value network would certainly say if you played something crazy--for example, to play on the 1-1 point--it would certainly, you'd see a very big dent in the evaluation of the position. But amongst the standard opening moves which are played, AlphaGo actually evaluates them quite closely. So we're talking about very slight, just, you know, decimal point advantages to one move over another in the opening."

Michael Redmond then immediately followed up by asking about black 1 in the center or on the sides (and specifically mentions tengen). Silver's reply (at 19:19) was

"At least from what we've seen so far, AlphaGo does tend to prefer moves around the corner area to opening right in the center."

This doesn't address why AlphaGo prefers not to play tengen, of course. Although in the interview Silver does explain a) why DeepMind believes AlphaGo doesn't have nearly the bias people assume it does given the initial training dataset of human games; and b) how DeepMind addressed issues like AlphaGo missing the correct response to Lee Sedol's move 78 in game 4.

Oh, as a bonus--since you mentioned in another thread that pros "seem to be coming round to the view that 7.5 komi is a mite too much", Silver had this to say (at 17:43) in response to a question from Michael Redmond about komi and how AlphaGo evaluates the win percentage of the empty board:

"AlphaGo interestingly actually thinks that the game is really balanced with a 7.5 point komi, but it thinks there's just a slight advantage to the player taking the white stones."

As you are well aware, some people have suspected for a while now that a komi of 7 is perfectly balanced. In Chinese rules, at least, 5.5 komi seems more strongly advantageous to black than 7.5 komi is to white. But I remember Ke Jie saying, back when he had that phenomenal win streak with white in 2015, that he thought 7.5 komi was too good for white. (can't remember the source for this last offhand, will edit if I find it later)

Anyway, the interview is only ten minutes long and is full of interesting tidbits. Worth a watch if you haven't seen it already.

Edit: Still can't find the source for that Ke Jie quote, but Kirby apparently also saw it at the time and posted about it.

John Fairbairn · #4

Quote:

The interview starts at 15:10 and ends at 24:37. At 18:06, AJ and MR ask about the value network's opinions on various moves for black 1.

Thank you. I would love to know more but even with the volume set at maximum and earphones on I just can't hear this (or many other videos). Your partial summary is therefore a blessing

Knotwilg · #5

On komi: I've always felt that amateur komi should be less than pro komi. Komi is an estimation of the initial advantage black has and makes up for it by actual points. Exploiting an advantage to the full requires a very good player, while the player receiving komi can sit and wait for the actual game result and add their points to it. The less skilled the players are, the higher cost their mistakes will have, the wilder the scores will vary and the less precise a 7,5 komi will be, so the more advantageous to White.

We can probably best understand this unfair effect in beginner play. We probably agree beginner moves are close to random, with occasional pass moves or self killing moves. The average outcome of beginner games hence is jigo. Add komi to that and it becomes an unfair game in White's favor.

If amateur komi should be smaller than pro komi, then logically pro komi is smaller than God's komi. If Alphago thinks 7,5 is only slightly unfair to Black, then it's probably quite unfair for humans, even pros.

oren · #6

Knotwilg wrote:

On komi: I've always felt that amateur komi should be less than pro komi. Komi is an estimation of the initial advantage black has and makes up for it by actual points. Exploiting an advantage to the full requires a very good player, while the player receiving komi can sit and wait for the actual game result and add their points to it. The less skilled the players are, the higher cost their mistakes will have, the wilder the scores will vary and the less precise a 7,5 komi will be, so the more advantageous to White.

If this were true, you would expect black to have a significantly higher advantage on online games between kyu players, but my experience doesn't show this. I think really you overlook the fact that amateur players make so many mistakes greater thank komi that it's just easier to have everyone play by similar rules.

Meretrix · #7

John Fairbairn wrote:

Quote:

The interview starts at 15:10 and ends at 24:37. At 18:06, AJ and MR ask about the value network's opinions on various moves for black 1.

Thank you. I would love to know more but even with the volume set at maximum and earphones on I just can't hear this (or many other videos). Your partial summary is therefore a blessing

Oh. My apologies, I didn't know that. I tried to hit the highlights in my previous post, but for what it's worth here's the full interview transcript (I left out some "umm"s and "uh-huh"s etc. to improve readability, since they don't add any content, but it is otherwise complete):

Quote:

15:13 AJ: Hello out there everyone, I'm Andrew Jackson with the American Go Association. I'm joined today with Michael Redmond 9-dan professional--

15:20 MR: Hello.

15:20 AJ: --from the Nihon Ki-in, and David Silver, lead researcher of the AlphaGo team. David, thanks for joining us.

15:25 DS: No problem, nice to be here.

15:27 AJ: I think in the pre-match commentary here we'd like to ask you a few questions about AlphaGo, you know, where it's come in the last year. I guess we should start with if you'd like to summarize for those of us who weren't able to see the talk yesterday, how has AlphaGo changed since last year's version that we saw against Lee Sedol?

15:46 DS: Well, we've worked very hard to improve the algorithms that are inside AlphaGo, and in fact people tend to assume that when you do something like machine learning, it's all about the amount of computation and the amount of data that you actually use. But in fact, often it's the algorithms that really make the difference, and so this is really where we've focused on AlphaGo. And in fact, now the new version, AlphaGo Master, actually uses a lot less computation power. It uses about a tenth of the computation power of the version that played against Lee Sedol last year--

16:14 AJ: Shocking.

16:15 DS: --and it trains in a matter of weeks rather than months. And so why does this happen? Well, it's actually down to the methods that we use. And the main difference inside AlphaGo Master is that it's actually become its own teacher. So the way to understand this is that we really want to train AlphaGo on the best quality data that we could possibly find. And in our case, the best data that we can get our hands on is actually games played by AlphaGo itself. And so what we do is we actually get AlphaGo to play itself millions of times, and we use this extremely high-quality data--where it's kind of searched really deeply in each of these positions and done that in each position in the game and played all the way to the end--we use that data to train its neural networks.

16:54 AJ: To retrain it, in other words. Sort of like distilling its previous understanding.

16:58 DS: That's right.

16:59 AJ: Got it.

16:59 DS: And because of this it's much less reliant on human data and human knowledge than previous versions.

17:04 AJ: Got it. Awesome.

17:05 MR: That's very interesting.

17:06 AJ: Very interesting. So with the new value network, I think-- [gestures to MR]

17:10 MR: I have some questions about--well, I have zillions of questions, but just to keep it to some safe areas--I'd like to ask you, like with an open board like this, with komi--there's various komis that we play with throughout the world, like 6.5 or 7.5, so in this match it's going to be a 7.5 komi--so how does AlphaGo sort of, it always gives itself a certain winning percentage. So like, for instance, with this open board, with no stones on the board, it would give a winning percentage to black or white. How would that be?

17:43 DS: So AlphaGo interestingly actually thinks that the game is really balanced with a 7.5 point komi, but it thinks there's just a slight advantage to the player taking the white stones.

17:52 MR: Ah yes, ok. So today, let's see, Ke Jie has white so he has a slight advantage at this point.

17:58 AJ: And Ke Jie is famous for playing very well with white.

18:00 MR: Oh yes, it's going to be interesting, yes.

18:02 AJ: A very good winrate with white. I guess a followup question to that would be, of black's possible first moves, is there any that makes a dent in that slight advantage? Is there any move that seems to be-- [pauses]

18:15 MR: Or are there any moves that are maybe not as good as--

18:18 AJ: Or not as good, sure. Does the value network have any opinions on the first black move?

18:22 DS: Well the value network would certainly say if you played something crazy--for example, to play on the 1-1 point--it would certainly, you'd see a very big dent in the evaluation of the position. But amongst the standard opening moves which are played, AlphaGo actually evaluates them quite closely. So we're talking about very slight, just, you know, decimal point advantages to one move over another in the opening.

18:41 AJ: Just really small differences.

18:43 MR: Could I just ask, well, maybe you won't be able to know about this, but--since AlphaGo plays star points and 3-4 points mostly, and in the 60-game series we were seeing that--but of course, naturally a 1-1 point is obviously bad to a human, too. But we don't really know about moves in the center of the board, like, for instance, or on the sides, even, and very few games of professionals have been played like with tengen--

19:09 AJ: [simultaneously] --start at Tengen--

19:10 MR: --there's just a handful of games played by top pros. So does AlphaGo have any evaluation of that kind of unusual position with a move--

19:16 AJ: [simultaneously] --any of the unorthodox openings?

19:18 MR: --that is more ambiguous?

19:19 DS: At least from what we've seen so far, AlphaGo does tend to prefer moves around the corner area to opening right in the center.

19:26 MR: Oh, yes.

19:28 AJ: That's quite an interesting tidbit there, thank you very much. All right, well that--is there any other, perhaps future work or plans for AlphaGo that you can mention at this time?

19:41 DS: Well, I mean we're really just focused on what's happening this week. We're really excited to see what happens during the remaining matches and I can't wait to see the game played today. You know, as one of the developers, actually I think the second game is where you can start to actually relax and enjoy it a little bit--

19:56 AJ: Oh, good--

19:56 DS: --because the first game we're just watching to make sure everything goes as it should from a technical perspective--

20:02 AJ: Right, right--

20:03 DS: --and now we can all just sit back and--

20:05 AJ: --enjoy the games--

20:05 DS: [simultaneously] --perhaps enjoy the game a little more than the first one.

20:07 AJ: Proud parent, I know the feeling. All right, well that's wonderful, thank you very much. [pauses] I think we have five more minutes here of our pre-game commentary--

20:20 MR: Ok, well--

20:22 DS: Well, perhaps it--maybe it's worth clarifying, because you mentioned the value network--

20:25 AJ: Yes! Yeah--

20:26 DS: --and so would it be worth explaining what the value network is to some of the audience?

20:29 AJ: Sure, by all means.

20:30 MR: Oh yes, certainly.

20:32 DS: So AlphaGo inside it has two different neural networks, and these neural networks are representations of go knowledge. And so this is really what gives AlphaGo its brain, if you like; it's kind of its way of understanding what's happening in the position. And so it takes in something like the board position and it passes it through this brain, through the value network, to come up with some estimate of who's winning in that position. And that's just a number--if that number's high, then it says that AlphaGo thinks it's going to win in this position, and if it's low it thinks the opponent's going to win in this position.

21:04 AJ: Is that number directly translatable to a percentage?

21:07 DS: Yes. That number actually tells you the probability that AlphaGo assigns to winning the game from that point onwards. And then there's this second part of the brain which we call the policy network, and the policy network looks at the position and essentially recommends a move to play in this position. And again you can translate the final output of this part of the brain into its preferences over all of the moves, and, again, these you can think of as probabilities if you like. And what's different in AlphaGo Master is actually the way that these networks are trained. So, in AlphaGo Master these are trained from games that it's played against itself. And this means that it's played using this very high quality data that was played with these very long--the full power of AlphaGo playing searches, you can imagine produces very high quality moves. And those very high quality moves produced by AlphaGo's searches provide the training data which we use to train the policy network. In other words we try and get the policy network to predict the move that AlphaGo itself would have played at its full power of search and look-ahead.

22:07 AJ: Which sort of builds all of that previous knowledge into the new policy network that is being trained on this [unintelligible]

22:12 DS: That's right. And similarly the value network then trains on these very high-quality outcomes we have at the end of the games played by AlphGo against itself. You can imagine that if you're in a certain position, and you want to know who's ahead in that position, then, you know, a very good form of training data is actually to get AlphaGo to play out the game from that point all the way to the end. And so that's the training data we use to train the value network. We ask it to predict what would have happened in games against itself from that point onwards. And then this process gives us a new policy network and a new value network that we plug back into AlphaGo's search, and we iterate the whole process many many times.

22:46 MR: Yes.

22:47 AJ: And I know yesterday Demis [Hassabis] and yourself I think mentioned the concern that self-play might lead to gaps in that knowledge, and that Lee Sedol was brought in last year to maybe help find out those gaps, the same way Ke Jie--you know, is a way to see if there is any sort of self-fit. Is there some concern that the self-play would lead to almost an overfitting, where it has mutual blind spots? Or is there steps taken to address that?

23:12 DS: So that's a very interesting question, but in fact what we've seen is the opposite. That in fact, the kind of gaps in the knowledge that we saw previously in the match against Lee Sedol--we actually see that by continuing to play games against itself, and to train further and further on those games, it actually starts to address some of those issues.

23:30 AJ: That's very interesting.

23:30 DS: And so progressing further with the self-play actually appears to correct some of these misconceptions that AlphaGo used to have. Now of course this doesn't guarantee that it doesn't still have some gaps--

23:40 AJ: --that there might still be some out there--

23:42 DS: --and for us it's very hard to evaluate those additional gaps in knowledge without playing a match of this type.

23:48 AJ: Without actually have an evaluation.

23:49 DS: And so that's one of the reasons we're all so excited to be here, because we want to try and explore these kinds of amazing games against a top pro who's at the pinnacle of his game and the pinnacle of the go world. And then we have a chance to really find out if it has gaps. And if it doesn't, well, the beautiful thing is, we, you know--you can think of this as like two sculptors combining to make some kind of work of art together, and I'm just really excited to see what that sculpture ends up looking like today.

24:16 AJ: That's wonderful, that's wonderful.

24:17 MR: Yes.

24:19 AJ: All right, well we'll hope that Ke Jie can be a willing partner.

24:21 MR: Oh yes. [laughs]

24:23 AJ: All right, looking forward to it. Ke Jie with the white stones will be coming right up, and we'll see you soon.

24:29 MR: Yes.

24:29 DS: Thank you.

John Fairbairn · #8

Fantastic (and eye opening)! Thank you again.

Bill Spight · #9

Knotwilg wrote:

On komi: I've always felt that amateur komi should be less than pro komi. Komi is an estimation of the initial advantage black has and makes up for it by actual points. Exploiting an advantage to the full requires a very good player, while the player receiving komi can sit and wait for the actual game result and add their points to it. The less skilled the players are, the higher cost their mistakes will have, the wilder the scores will vary and the less precise a 7,5 komi will be, so the more advantageous to White.

We can probably best understand this unfair effect in beginner play. We probably agree beginner moves are close to random, with occasional pass moves or self killing moves. The average outcome of beginner games hence is jigo. Add komi to that and it becomes an unfair game in White's favor.

If amateur komi should be smaller than pro komi, then logically pro komi is smaller than God's komi. If Alphago thinks 7,5 is only slightly unfair to Black, then it's probably quite unfair for humans, even pros.

Well, having played around with Monte Carlo random play, I think that even human beginners' play is decidedly better than random.

On komi:

Let's take an easy to understand example of coupons with values that the player who takes the coupon gets. The values are evenly spaced between 0 and T, the top value.

If there are an even number of coupons, correct komi is T/2. For an odd number of coupons, n = 2m+1, correct komi is T/2 + T/2n. OC, if n is large, correct komi ≃ T/2. If we do not know whether n is even or odd, we can estimate correct komi as T/2 + T/4n.

What is correct komi for random play? If there are an even number of coupons, it is obvious that for any sequence of play, there is an equally likely sequence where each player gets the other's coupons. So the correct komi is 0. For an odd number of coupons, after the first player has taken a coupon, the correct komi for the rest of the coupons is 0. So correct komi for an odd number of coupons is the average value of the first coupon chosen, which is T/2 + T/2n. If we do not know whether n is even or odd, we can estimate correct komi as T/4 + T/4n. If n is large, correct komi ≃ T/4. Random play basically halves komi for perfect play.

One important difference between the coupon example and go is that there are no negative coupons. Human beginners at go are quite capable of making plays that lose points. (And not just beginners. :lol:

) How much of an effect that has is unknown, but what would correct komi for 15 kyus be? We still call them beginners, and they still make huge blunders, but I would not be surprised if komi of 2.5, or even 3.5, would give close to even chances for each player.

Even 40 years ago it was plain to me that komi of 6.5 for Japanese pros was appropriate. How about amateur dans? 5.5? SDKs? 4.5? A one point difference in komi affects only a small percentage of games, even for pros. It would not surprise me if a perfect player could give 5 or 6 stones to today's pros. That might translate to a 7.5 komi by territory scoring, but I doubt if it would mean a 9.5 komi by area scoring.

lightvector · **#10**

I pretty much agree with Bill's guess. To take a different approach from Bill's analysis that gives similar results:

Komi compensates the value of a half-move advantage. Therefore a non-crazy first-order guess would be to say that it should scale roughly with the average value of a player's moves relative to passing, divided by the average value of the optimal move relative to passing.

In Go, some possible moves have negative value, but even 20k players play moves that are on average far better than passing and extract a significant fraction of the value of a move. By the time you get into the amateur dan ranks, the amount you on average lose per move is very small, probably much less than 20%. Consider that losing a mere 2 point per moves for the first 100 moves (the opening and much of the midgame) would be 200 points, already much more than a 9 stone handicap's worth, and in the opening and midgame, pretty much every move is worth more than 10 points relative to passing.

So an extremely rough guess would be that komi for weak amateur dan players should be well within 20% lower than for pros, i.e. at most a point or so, and that maybe each 8 to 10 ranks might drop another point or so. That puts us not far from Bill's numbers - 15k players might easily balance at komi of 3.5.

Some data would be pretty cool to check this. I doubt that 15k players typically have good enough counting to adapt their play in close games based on whether they are a few points ahead or a few points behind, so if there were an online server that randomized who played each color in even games without any bias towards giving white to the player it thinks is stronger, and if one could cleanly exclude games where the players themselves chose the colors rather than going along with the randomization, then looking at the distribution of actual score margins in these games would give pretty good evidence.

pookpooi · **#11**

Last week I ask DeepZenGo co-creator Hideki and he said that in Japanese rule, Zen thinks 6.5 komi is a little bit favor white.

moha · **#12**

Bill Spight wrote:

I would not be surprised if komi of 2.5, or even 3.5, would give close to even chances for each player.

Komi 2.5 seems quite low, even compared to random players, if we assume komi 7 is correct otherwise.

Interesting topic, particularly komi for nonperfect play. Should it equal to the average board result, or should it be a value with closest to 50% winrate? The specific error distribution also seems important, and the token game is quite a rough model only. I did some quick sims with it and a few simple types of errors / suboptimal players, and it seems there are even rare cases where correct komi for weaker players is higher than for perfect players. Otherwise my guess would be similar: 1-2 points komi decrease for amateurs.

daal · **#13**

Isn't everything data-driven these days? It seems one could test the hypothesis that komi should be lower for weaker players by letting some weakish bots play each other a few zillion times with different komis, then examine the results.

Bill Spight · **#14**

daal wrote:

Isn't everything data-driven these days? It seems one could test the hypothesis that komi should be lower for weaker players by letting some weakish bots play each other a few zillion times with different komis, then examine the results.

Why not use real data? 40 years ago, a study of Japanese pro komi was published in the AGA Journal, using 1200 games with 4.5 komi and 1200 games with 5.5 komi. Terry Benson, AGA president and AGA Journal editor, sent me the article, along with the data, to review before publication. Both the games with 4.5 komi and the games with 5.5 komi pointed to a komi of 6.5. The author stated that it was "plain as a pikestaff" that correct komi was 7. There is plenty of online data for players of all levels. If small differences in komi had little to no effect in indicating proper komi for pros, they shouldn't make much difference for weaker players, either.

BlindGroup · **#15**

Bill Spight wrote:

If small differences in komi had little to no effect in indicating proper komi for pros, they shouldn't make much difference for weaker players, either.

Bill, I'm not sure I follow. Are you saying that the small differences in komi had no effect on how the pros in the experiment played? If so, how did they determine that?

Additionally, I don't see how they could conclude that the correct komi should be 7 if all of the evidence is from games with komi of less than 7. Wouldn't one need either evidence that the winning probabilities were balanced with a komi of 7 or that white had an advantage with a komi of 7.5 or 8? Or is the upper bound inferred from other data -- such as the experience under Chinese rules?

moha · **#16**

daal wrote:

It seems one could test the hypothesis that komi should be lower for weaker players by letting some weakish bots play each other a few zillion times with different komis, then examine the results.

I don't think SOME komi decrease could be doubted (for DDK levels at least), but the details are very vague. First, there is the "komi_ev" or "komi_wr" question. The two values that neutralize the EV (average board result) advantage or the winrate of the first player can differ somewhat, and the game can remain a bit unfair (either EV=0 or WR=50%, but not necessarily both). Also komi_ev is usually fractional, while komi_wr makes little sense in smaller than half point steps (so exactly 50% winrate cannot be expected anyway). Then there is the error distribution, IOW the kind of mistakes your weakish bots make - you may get slightly different results with different bots (or other groups of players).

One example is the token game above. With an odd number of tokens, consider two simple cases:
1. the players' error is that they sometimes pick a random token instead of the biggest one (with certain probability)
2. they sometimes pick the second-biggest instead of the biggest (with the given probability)

In the first case B's EV drop significantly (together with his winrate from 100%). But in the second case B's winrate drops but his EV increases, and both komi_ev and komi_wr will be higher than for perfect play.

Bill Spight · **#17**

BlindGroup wrote:

Bill Spight wrote:

If small differences in komi had little to no effect in indicating proper komi for pros, they shouldn't make much difference for weaker players, either.

Bill, I'm not sure I follow. Are you saying that the small differences in komi had no effect on how the pros in the experiment played?

It was not an experiment. IIRC, they were data published by the Nihon Kiin.

Quote:

If so, how did they determine that?

I observed that with the 4.5 komi games, if you subtracted 2 points from Black's point of view, making komi in effect 6.5, you got the closest to a 50-50 split. Subtracting 1 point from the 5.5 komi results also gave the closest to a 50-50 split. OC, that does not mean that a komi of 6.5 would give the closest split, because the players might play differently under that condition. I made that last point in a footnote to the article.

Quote:

Additionally, I don't see how they could conclude that the correct komi should be 7 if all of the evidence is from games with komi of less than 7.

Well, the author of the article argued that since net go scores integers and the integer komi that would give the closest to a 50-50 split for the data was 7, that was obviously correct komi. IIRC, he was not arguing for a statistical split, but inferring that correct play would yield a result of 7.

Quote:

Or is the upper bound inferred from other data -- such as the experience under Chinese rules?

No, he just used those 2800 Japanese pro game results. Actually, he has a better argument with Chinese scoring, because the vast majority of net scores are odd.

Much later edit: Corrected the number of games.

luigi · **#18**

Would 7 komi be equally balanced if the button were used to break ties?

Javaness2 · **#19**

Petr Baudis already made such experiments over a variety of board sizes. I won't adjust your anticipation by mentioning where to see the results.

It is very exciting to me how fresh AlphaGo's play makes the game feel. So many times you see people just trotting out the same opening - they will continue to do so now with the additional option to imitate AlphaGo. Every time I see some insight from AlphaGo it beings new life into the game for me. However this 4-3 approach to 3-4 deserves an ear wigging.

hyperpape · **#20**

In theory, we should get data, but I'm not doing that, so I'll add to the speculation. I won't be surprised if komi is smaller for dan level amateurs, SDKs or DDKs, but I won't be surprised if it's the same either.

Everyone seems to ignore the possibility that between certain sorts of weak players, komi is larger. Bill used the models of coupon-taking, but it only works if both players expect to take coupons of equal value on a given turn. Maybe weak players are prone to making unusually bad moves at some points as White.

Here's one way that could happen: in some kyu games, both players will sort of mindlessly build a large framework, but black tends to be a step ahead. Once that happens, it's an open question whether bad play leads to White invading and Black doing a bad job killing, or White playing passively and giving the game away.

Another fun variant: there are kyu players who immediately approach on move 2. They have a decent chance of coming away with sente and a decent position, so it's not always a loss, but once you're past the truly random level, you expect them to lose points on average.

AlphaGo and tengen

Who is online