“Decision: case of using computer assistance in League A”

[ben0]

Hello, hello,

Well, Carlo beat me in WAGC and he didn't even have to lift a finger. That's some leela sh*t right there!

On a more serious note, more than stats and analysis, I trust (strong) players instincts and feelings (mine too). There are ways of winning, a 4d winning against a 6d isnt that crazy, the 4d can be in a good day, the 6d can make some blunders, etc. But the way Carlo won was like a complete mastering of the games.
The way he performed in WAGC was pretty bad, I saw the games, sure he is a copycat of leela in the fuseki then he collapses. This middle game's collapse didn't happen online.

On a personal note I believe he is guilty but in the end it doesnt matter, something has to be done for online tourneys, if we want to keep the pandanet which is a great tournament, some regulations have to be implemented. If it happens not to be enough to make sure there are no cheating then it will be the end of pandanet (and online tourneys in general)

I always believed that such behavior wouldn't happen in League A because we basically all know each other, we are friends or acquaintances, there is a mutual respect of the game and of the players. Italy goes up to league A and all goes to sh*t ? (ok, no joke about italians cheating in sports...)

Anyway, let's not be blind, the cheating is pretty obvious but we just move on, make some new rules then we will see.

[MircoF]

Hello, hello,


I always believed that such behavior wouldn't happen in League A because we basically all know each other, we are friends or acquaintances, there is a mutual respect of the game and of the players. Italy goes up to league A and all goes to sh*t ? (ok, no joke about italians cheating in sports...)

Hello Ben, this must be not what you really think, too much insulting, a team and a Country. I'll never belive a go player could write something like this.

Am I wrong?

Mirco.

[Schachus]

The problem I see is well illustrated by the following two quotes:

his next games since the investigation started not look really like the same player's games (he would be clearly stupid, if he continues cheating after).

and

Sixth and the last impression came during Tuesday's (29th May) relegation game United Kingdom vs. Italy - game Chris Bryant vs. Carlo Metta. I have never heard about the British player before, but for a 3-dan, he performed very well and bravely. However, he couldn't simply face the power of white's play. And this time it was not Leela 11, but Leela Zero behind it.

Just assume for a moment, an innocent player played 2 or 3 games that were very strong for his level and suspicions are raised because of that. But now, no matter how he performs the next games, it will always be perceived as evidence towards him cheating before. If he plays "badly"/his usual level, people will think "of course he stopped cheating now, it is such an obvious differnce to before". If he continues to perform exceptionally, it will just be further evidence that he is cheating, cause it's now even more "suspicious games".


This means, no matter how the players' next games go, they will be evidence. But just the fact that he is still playing cant be evidence, can it?
Therefore I suggest, one should stop at the moment were there was a suspicion, collect all evidence up to that point and judge that. Dont judge what happens after if no matter what happens you could always use it against him.

[figgitaly]

"FiggItaly", may I ask you, what is your rank? If you are a dan player, it's hard to believe you made such a wrong judgement. White played a bit slack in this game, so it was quite a normal close game. White lost it, because of a clear blunder at move 108 - o15 was a greedy overplay and simple L7 geta would simply parry black's effort in separating white's two groups and would give white a comfortable lead. Please, don't try to fanatically find arguments for your side...

Oh, I did not see the entire game, I just observed the final part after the death of Dragos' group.

Cheers.

[Bill Spight]

@ How did I personally reach my opinion? (mainly for "Uberdude" and "Bill Spight")

There was a selection of the types, which aspects we are using for being convinced in Carlo's guilt. I don't think I would place myself in any of them, but probably the closest I'm to some mix between type 2 and 3.
My first suspicion came, when Carlo defeated my teammate Ondrej Kruml, but there was no strong point for protesting at that time.
Second time my suspicions rised after that legendary game and I got to know, that Israel is going to protest. To me, it made sense, but I was not really convinced, that the cheating really happened, but I started going through the games on my own and discussing them with other strong players.

Going over the games is important, and necessary when, as here, there is no physical evidence and only weak statistical evidence of cheating. (As my statistics and research methods prof told us, statistics proves nothing. Despite the impressive 98% figure, the statistics was poorly done.) Ken Regan has done a good bit of work about cheating in chess. This note gives some links: viewtopic.php?f=10&t=15538&p=228848&hilit=regan#p228848

Then the league manager's investigation (third change in my opinion) started and those famous 98% had been revealed, so I was like: "Hm, maybe my suspicions were not completely off."

Unfortunately, people give far too much weight to confirmatory evidence, and give the 98% figure far too much credit. This is not their fault. Statistical reasoning is only a couple of hundred years old, and is still in flux. Most people are unused to it.

So, I started to look into Carlo's games more in detail and his next games since the investigation started not look really like the same player's games (he would be clearly stupid, if he continues cheating after).
Fourth time my opinion was affected after seeing the WAGC games. Since then I'm 99,9% sure about Carlo's guilt.

I was amazed to hear that the original verdict of cheating had been based upon a single game, and only on the similarity of Carlo's play to Leela's. You are right to look at multiple games, and to contrast Carlo's play in games where he is suspected of cheating with his play in games where he is not. The contrast in his play between the two types of games is strong evidence, not the similarity of his play to Leela's, as far as cheating is concerned.

Fifth time, it hit me when "the decision" has been reversed and I became very angry... In other words, it basically allowes people to cheat, if they are sure not to be catched.

From what I have heard, the original verdict was based upon weak statistical evidence alone, of only 50 moves a single game. Your instincts were right, to examine the play in more than one game. Had the case been built properly, the verdict may well have stood. Or a different verdict may have been reached. You are a better player than I, but in that game it seemed to me that Carlo took the lead without making any remarkable plays, as his opponent showed poor judgement. So there may not be expert agreement that the result was due to the weaker player playing far above himself.

But yes, given the lack of physical or behavioral evidence of cheating, analysis of game records is necessary to render a verdict. The fact of the matter is, a good case was not built for cheating, and it was right to overturn the verdict. A better case might have been built. I feel sure that future cases of possible cheating will be better investigated.

Sixth and the last impression came during Tuesday's (29th May) relegation game United Kingdom vs. Italy - game Chris Bryant vs. Carlo Metta. I have never heard about the British player before, but for a 3-dan, he performed very well and bravely. However, he couldn't simply face the power of white's play. And this time it was not Leela 11, but Leela Zero behind it.

Perhaps you can post your game analysis that leads you to that conclusion.

[figgitaly]

@ Dragos vs. Carlo game:

What amazed me the most about white's play in this game was the combo starting from 74 etc. No way an ordinary human 4-dan would be able to find such a sequence (and it wouldn't probably even hit my mind either).

Sorry Lukan, but regarding this sequence. I have just done a quick analysis with Leela and Zen and they do not suggest that sequence at all (Is my pc not powerful enough?). It seems to me just a try to use aji / create confusion move by move and let the ladder work, usually AI's play more direct plan and they do not look so hard for some aji like humans do. Am I wrong?

[dfan]

I feel like, that many of you, who are defending Carlo can't imagine there could be any possibility of his guilt and rejecting any kind of suspicions.

I don't believe I've seen anyone here acting like that. I could have missed a comment or two though.

[Charlie]

I feel like, that many of you, who are defending Carlo can't imagine there could be any possibility of his guilt and rejecting any kind of suspicions.

I'm pretty happy to accept that Carlo might have cheated. In fact, I'd go so far as to say that *someone* in the near future certainly will cheat. The Go community needs to confront cheating in a mature and considered way and sooner is better than later. Whether Carlo is innocent or guilty does not change this fact.

The evidence on which the original judgement was formed and the way in which it was handed down were neither mature nor considered and the arguments flying to and fro in this very thread and the potential impact on future online tournaments as well as real-life tournaments, not only the EGC, serve to illustrate the consequences of mishandling cheating.

Do we want to play in a world of witch hunts and kangaroo courts?

The appeal succeeded and the whole matter should be consigned to the annals of history. Carlo should be treated as if he is and always was innocent and, as a community, we should learn from the incident and open the debate on cheating and how to handle it.

No further judgements should be passed until a proper framework for detecting cheating is put in place. I agree with Bill Spight (and other contributors to this thread) that this will need to involve analyses of several games, comparison to more than one robot player and some sort of analyses of the contrasts and trends within the suspect's play, online and offline. Luckily, the Chess world has already walked this path and we can learn a lot from them.

In the interim, we must endure a dark age of suspicion and, yes, some game results will be marred by cheating. Let this serve as an impulse to bring about the required changes.

Even those who are adamant that Carlo cheated -- whatever their reasons -- are best served by this course of action because, if they are indeed right, two possibilities exist: the new, mature and considered strategy will catch Carlo in the future, should he persist, and, should he never cheat again, we will avoid losing one of our number.

I don't know Carlo at all. I have likely never even met him. I do know that Go players aren't all that numerous and that strong dan players are valuable in our community. I also believe in rehabilitation.

[Javaness2]

Just a note - somebody asked who the appeals commission was. From the post it appears to be Victor Bogdanov, Ales Cieply, & Frank Janssen.

I think it is more productive now to respond to this part of their statement "We advise the EGF to enforce more strict rules for EGF sanctioned tournaments played on internet"

The EGF AGM is not so far away, time is short.
https://docs.google.com/document/d/1qYqtlX12Ml3pxs5HIInk9bhFdh9_c58h76Etp0gnIgI/edit

[Uberdude]

@ Dragos vs. Carlo game:

"Uberdude", you are grilling Dragos for a move 47 (which is indeed wtf), but isn't it normal that human 6-dan make mistakes like this time to time?

Yup, that was my point that this game was more Dragos lost than Carlo won. So although he is obviously a strong player and used to be almost 2700 this game he didn't play so well so cheating is not necessary to beat him. Maybe he doesn't play so well online? I noticed he resigned early in a fight with Tanguy and losing to Serbia was a big upset, but also he beat Csaba and Fionin so good wins there. About playing online, some people (probably more likely those who learnt offline) find it hard to play full-strength online, concentrate, take it seriously etc. Others, such as myself who mostly learnt online, actually find playing on a real board harder in some ways, e.g. particularly with play in the centre I find it easier to see the "big picture" online which is why in real tournaments I often kneel on my chair or stand up to look at the board from above.

It's very easy to argue and criticize having an AI by your side... And yes, as far as I know Dragos, he plays very routine territorial style and doesn't start fights intentionally early on.

That move 47 sequence I thought was really bad even on my own, but yes having a strong bot to review makes me more confident to criticize it and not think he had some genius plan I didn't understand. Maybe it wouldn't have been so bad if Dragos tried to fight, e.g. play a knight's move instead of one point jump to put more pressure on white (seems like white can get out not too hard though, plus b on left is weak) or not take gote with the thick connect on left side? Ben said Carlo collapsed in middle-game in WAGC, this game Dragos didn't put much pressure on him so few chances to collapse (only m9 cut). [One of my 3d friends has a boring territory style, his "favourite joseki" is 3-4, high approach, attach under, hane, pull-back, connect, one-space jump , and Dragos was his teacher many years ago, maybe that's where he got it from!].

What amazed me the most about white's play in this game was the combo starting from 74 etc. No way an ordinary human 4-dan would be able to find such a sequence (and it wouldn't probably even hit my mind either).

Well, we disagree there as although it is a nice strategy I think some human 4ds can find it. Maybe it's because I watch a lot of pro games so am relatively good at advanced strategies like this (but don't do tsumego so am bad at reading hence only 4d). That's if it is actually a good move, could Dragos have taken sente at some point to extend at m10, suffer a local loss in the upper right but gobble up the 2 stones now cut by m10 or even attack the lower white group? But the important issue is not is this sequence credible for a 4d to play, a 4d who's quickly improved to 5/6d or whatever, it's does Leela suggest this sequence so by playing it we think he cheated? And when I try Leela 0.11 on my PC it effectively doesn't consider it (after 200k simulations it's choice #18 with 5 sims).

P.S. Your English is fine!

I feel like, that many of you, who are defending Carlo can't imagine there could be any possibility of his guilt and rejecting any kind of suspicions.

I'm not convinced that Carlo didn't cheat, but am fairly sure that the evidence used (assuming it was 98% similarity statistic-based and not that the EGF hired a drone/ninja to peer through his window and spotted Leela on screen during the game) did not justify the conviction in which case the appeal decision is correct. I'm not defending Carlo per se, I'm defending due process and rigorous, quality analysis (misuse of statistics is a particular bee in my bonnet). I do also take the view that incorrectly punishing an innocent is much worse than failing to punish a guilty, so if there is reasonable doubt then not punishing is correct. It's hard to quantify, but at the moment my view is maybe 10-20% chance that he cheated. An example of a move I found suspicious was a reduction move he played against Kulkov, so I posed it as a problem here to see how common it was for dan-level humans to want to play around there.

Sixth and the last impression came during Tuesday's (29th May) relegation game United Kingdom vs. Italy - game Chris Bryant vs. Carlo Metta. I have never heard about the British player before, but for a 3-dan, he performed very well and bravely. However, he couldn't simply face the power of white's play. And this time it was not Leela 11, but Leela Zero behind it.

It's rather funny that Carlo ended up playing Chris, as Chris is actually one factor why I have taken such an interest in this case. He has improved very rapidly: last time I saw him he called himself 1d but was probably 1k, so when he turned up saying he was 5d I was sceptical, and the last tournament he went to he lost to a 1-2d (who I was recently embarrassed to lose to for the first time after 12 straight wins so I teased him a bit about that). But then he beat Daniel Hu 4d (also fast improver, going on 5d?) 4-0 in some practice games online before Daniel beat me in last year's British championship. He's had great results as our "secret weapon" in the league this season winning all his games, somewhat helped by being on a low board with his sandbaggy EGF rating: 2d+ 2d+ 2d+ 5d+ 5d+ 3d+ 3d+ 5d+ 4d+ Carlo-. So given his poor real-life results (2 years ago, he lacks tournament and playing under pressure/overtime experience) one might think his online results are suspicious and suggest cheating. I don't think he is (he's one of these autistic-spectrum maths genius types, he's won mental calculation championships), plus his style is kinda wacky and creative but also careless. This season he's often been winning a lot but then gets complacent and makes a mistake but still wins in the end (like that dumb liberty wasting push against Carlo). So I want to avoid a badly designed cheat detection process that could produce a false positive.

[figgitaly]

Sixth and the last impression came during Tuesday's (29th May) relegation game United Kingdom vs. Italy - game Chris Bryant vs. Carlo Metta. I have never heard about the British player before, but for a 3-dan, he performed very well and bravely. However, he couldn't simply face the power of white's play. And this time it was not Leela 11, but Leela Zero behind it.

I have done also a quick analysis with zen / leela / leela zero of this game. It seems that he did a lot of mistakes (or at least not suggested moves) according to all of them, especially on top right corner fight.

Do you have different analysis?

Cheers.

[Bill Spight]

Luckily, the Chess world has already walked this path and we can learn a lot from them.

Worth repeating.

[Bill Spight]

I feel like, that many of you, who are defending Carlo can't imagine there could be any possibility of his guilt and rejecting any kind of suspicions. We are not interested in destroying Carlo and Italy itself. We are only searching for justice and fair conditions for everyone.

As you know, I agree that suspicions are justified. But I also believe that the process that reached the verdict of cheating was flawed, and that the reversal of that verdict was fair and correct.

I also believe that you are right to analyze Carlo's game records. If there is a case to be built, that is where to look. That is why I am interested in your analysis of the Bryant-Metta game.

[Javaness2]

The Chess World learnt

• that Stewpot likes cruises
• never to hold qualifiers online
• to ban mobile phones
• to ban headphones
• When things get bad, use the toilet

[EdLee]

Hi Javaness,

that Stewpot likes cruises

Noob question: what's this reference ?