“Decision: case of using computer assistance in League A”

[Simba]

I'm confused about the reddit post that was linked. Does that thread refer to the game from Nov 2017? Back then, LZ was about 20 kyu at best. Also, I think Leela 0.11 was not out yet? If that's right, then the reddit post is clearly a troll.

No, it's referring to http://pandanet-igs.com/system/sgfs/736 ... 1527628060 (played 29th May 2018), with Carlo as white against myself as black.

[Bill Spight]

I am glad to see the points mentioned by the anonymous player were verified.

You mean that Leela 11 and Leela Zero evaluated a specific position differently and found different plays for it? Was anything else verified?

I still find it strange that Carlo confessed to someone specifically pointing out the point from your game and that person made it public.

Me too.

I still think your game and the particular point adds to the whole picture. Once again it is something that occurs with a small probability. We have a move that is a potential game decider, is evaluated differently by the two bots and it happens in one of CM's important games played on internet.

Yes, the odds against all of those things happening are quite large. That is true of many coincidences, as well.

It can be easily refuted by saying that a 4d player would have no problem to find the correct move that was played, so it proves nothing in itself.

What can be refuted? (I can guess, but. . . .) What difference does it make that a 4 dan could find the correct move without cheating?

Can it be judged as another indirect evidence.

Please spell out the argument, at least to yourself. I am unable to do so in any way that makes sense to me. What is relevant?

[Simba]

Yes, the odds against all of those things happening are quite large. That is true of many coincidences, as well.

What do you feel are acceptable odds against which an innocent player must not be found guilty? 10:1? 100:1? We're so far above those numbers at this point.

http://www.duhaime.org/LegalDictionary/ ... ities.aspx makes it clear that for civil cases, it only needs to be that the defendant is more likely to be guilty than innocent to rule against them.

For what it's worth, considering all the independent evidence in combination, mathematically I'm sure there is proof beyond reasonable doubt (i.e. guilt in the criminal definition), never mind proof on balance of probabilities (the civil definition). You can only multiply 40%, 10%, 1%, 0.05% and so on so many times before you end up getting something ridiculously unlikely.

[jlt]

You can only multiply 40%, 10%, 1%, 0.05% and so on so many times before you end up getting something ridiculously unlikely.

So far no one has proposed a convincing calculation. I would be convinced if someone came up with a precise figure like "there is a probability less than 1/100000 that he did not cheat". Why 100000? Because that number is higher than the number of participations in european tournaments during a year (if a tournament has 100 players I count 100 participations).

Please note that you cannot multiply percentages blindly. If you roll a die 10 times, and you get a 6 three times, one cannot conclude that you cheated because (1/6)x(1/6)x(1/6) = 0.0046 is small.

[Simba]

You can only multiply 40%, 10%, 1%, 0.05% and so on so many times before you end up getting something ridiculously unlikely.

So far no one has proposed a convincing calculation. I would be convinced if someone came up with a precise figure like "there is a probability less than 1/100000 that he did not cheat". Why 100000? Because that number is higher than the number of participations in european tournaments during a year (if a tournament has 100 players I count 100 participations).

Please note that you cannot multiply percentages blindly. If you roll a die 10 times, and you get a 6 three times, one cannot conclude that you cheated because (1/6)x(1/6)x(1/6) = 0.0046 is small.

Well, okay then - what would you accept as a 'convincing' calculation? The odds of any of these things can only be estimated. Are you happy enough to accept estimates with backing from statistics where possible?

And lol, no, one can't, not least because (1/6) * (1/6) * (1/6) isn't the chance of getting three sixes if you roll a die ten times, even if the rolls are independent... My background is as a mathematician, don't worry about that side of things from my perspective.

[jlt]

Well, okay then - what would you accept as a 'convincing' calculation? The odds of any of these things can only be estimated. Are you happy enough to accept estimates with backing from statistics where possible?

Yes, I would accept estimates.

And lol, no, one can't, not least because (1/6) * (1/6) * (1/6) isn't the chance of getting three sixes if you roll a die ten times, even if the rolls are independent... My background is as a mathematician, don't worry about that side of things from my perspective.

OK, I just wanted to make sure because I see too much nonsense from non-mathematicians.

[Charlie]

What do you feel are acceptable odds against which an innocent player must not be found guilty? 10:1? 100:1? We're so far above those numbers at this point.

Please read this: http://frmor.net/downloads/statistical_ ... _David.pdf . Specifically, read appendix "A.4 Multiple tests fallacy".

Whether you agree with the tests that frmor used or not is irrelevant. The statistics in the appendix are axiomatically true.

Since we don’t want unjust condemnations to happen too often, the probability of false positive should be at most
of the order of 2.5·10−5
. This way, among 1000 innocent amateur players, there would be no more than 2-3 wrongly
convicted in 10 years, so the test would be reasonably solid.

In other words, that's odds of 1 in 40000 against which an innocent player must not be found guilty.

Always remember that we want to create a world in which people want to play Go. Who will dedicate hours, days, years and decades of hard work to a game when it can all be ruined by a turn of poor luck, a kangaroo-court or an obscenely overweight duck?

[Bojanic]

[*] On the other hand, your analysis doesn't provide a convincing proof. What is missing is a statement like "the probability that C.M. could find these moves is x" with a sufficiently small x (like 10^-5 or 10^-6), with calculations explaining how you get this value.

And what does percentages means?
First two analysis were statistical, what was difference between 98% and 93%, and what at all they represent?

In those two internet games, Metta played almost every move like program.
In live ones, far from that. If this is not conclusive enough, what do you more need? Some calculation that says, it is 99% clear?

BTW when we were analyzing suspicious games, we spoke what was most suspicious in them.
Moves? No, it was possible he found them, they were not moves from god.
But one thing was heavily felt: one side had no chances. They did not have a chance during entire game, and even against a pro you will have a chances (at least to narrow his lead). And this is pretty good description of all those strings of A and B moves. He could play them, but for every 6d move, he should play one 2d move, and most of moves should be 4d. And there was none of weak moves in internet games (but there were in the live).

[AlesCieply]

I am glad to see the points mentioned by the anonymous player were verified.

You mean that Leela 11 and Leela Zero evaluated a specific position differently and found different plays for it? Was anything else verified?

Yes, and that it was likely at a decisive point in the game.

Can it be judged as another indirect evidence.

Please spell out the argument, at least to yourself. I am unable to do so in any way that makes sense to me. What is relevant?

Actually, I am having a trouble with it myself. I am still not sure what to make out of it.

[Simba]

BTW when we were analyzing suspicious games, we spoke what was most suspicious in them.
Moves? No, it was possible he found them, they were not moves from god.
But one thing was heavily felt: one side had no chances. They did not have a chance during entire game, and even against a pro you will have a chances (at least to narrow his lead). And this is pretty good description of all those strings of A and B moves. He could play them, but for every 6d move, he should play one 2d move, and most of moves should be 4d. And there was none of weak moves in internet games (but there were in the live).

It's so different playing the game to just watching. I don't really know how else to describe it, but playing against Leela Zero (which is what I think he used, not standard Leela) doesn't feel like playing against a person. I have no previous experience playing against it. I felt horrifically oppressed and helpless, like there was nothing I could even try, never mind pull off. Not fun in the slightest, I hope never to have to play a game like that again. I've played several professionals, none of whom made me feel like that or anything close to it.

^ this. What you wrote there matches exactly how I felt. It was extremely uncomfortable. No one here can understand what that feels like until they're put in a competitive situation against something like that. It's totally different watching it. I've never played a game like that before and I hope I never will have to play a game like that again.

[Bojanic]

Okay, I've kept quiet for long enough. I was hoping not to have to get involved, but I will speak up clearly at this point because I feel I need to.

Good to hear from you.
I watched your game in Leela 0.11, I did not saw similarities.
It did not expect to see results after the alarm was sounded.
This game should be analyzed in details, but it is far from being such a clear case like two games I analyzed.

If they dare to pull that line, even after the EGC is complete, get in touch with me directly (via email if necessary) and I will start a separate case against him for the game we played.

No problem.
You should try to examine on which program and set of weights could match your game, if you suspect it.
I cannot, I have old machine for LZ.

If this crap goes unpunished then the league is going to be in complete disrepute. The Mitics already have stepped aside because they don't want to deal with this sort of online cheating. Many other top players are furious, including myself.

Just to clarify, Mitics did not stepped aside, but entire team Serbia. Mitics did not have objections on their games, and we found no suspicious games on board 1-2.
If those caught cheating are not sanctioned, more investigation conducted and prevention policy introduced - which is minimum required for fair play - we don't want to play in such circumstances.
Regarding top players, for them it is easier to see difference in play.

[Uberdude]

Simba, I addressed your moderation/censorship concerns here: viewtopic.php?p=232823#p232823. The other points will have to wait (this time I have been summoned to do the washing up), but I will just quickly say

I am glad to see the points mentioned by the anonymous player were verified.

You mean that Leela 11 and Leela Zero evaluated a specific position differently and found different plays for it? Was anything else verified?

Yes, and that it was likely at a decisive point in the game.

This wasn't decisive, Carlo had clearly won already. In fact he had just tenukid for his previous move, in a rather human "thumbing my nose" way IMO though of course bots also tenuki when you play a threat which ends up gote (though I note my LZ wouldn't have played where Carlo did but one adjacent: extend not hane), so I would presume he had read out that it was indeed safe to tenuki and had this reply planned. But even if he played the weaker Leela 0.11 move he would still be winning plenty, this one was just more stylish and made more points.

I will share my analysis of the game (as a live kibitzer and LZ analysis) later.

[John Fairbairn]

John, I find this little bit quite funny. The cheating case is a hotly debated topic here, you also contribute to it, I guess you know Italy played UK in a qualification match and you did not bother to notice who Carlo Metta played against from your team.

Ales, servus! Actually the UK team is not my team. I am not a member of the BGA. I have not played a tournament in (guessing) over 20 years. I meet a tiny handful of UK players only a very few times a year in connection with T Mark Hall's legacy (though when he was alive I did get some news relayed from him). I know nothing about the EGF event, or the EGF personalities.

None of that strikes me as peculiar behaviour either individually or in combination, so I find the fact that you find it odd... well, odd. And odder yet that you seem to think it matters enough to post here. I'm not at all offended BTW - I'm amused, so thank you for this morning's smile!

My own contributions here (like others such as Bill Spight's, I believe) are not concerned with this specific case of ALLEGED cheating - everyone, please try to remember that word - so much as concerned over a rush to judgement using very tacky methods based on suspicion.

Simba has raised the concept of assessing guilt "beyond reasonable doubt." That could indeed be applicable in cheating cases, but in real legal cases in the UK it is just part of a huge apparatus with judges, counsel, solicitors and a watchful fourth estate, all operating under clear rules. And it not just one "furious" and suspicious person doing the assessing. It is a body of a dozen people, able to confer, for days if necessary, and to seek further advice, and they have to be unanimous or very close to in order to find guilt. Not a perfectly safe system but a tad better than an EGF committee meeting, and miles better than an L19 bear pit.

[bugsti]

There is censorship going on here by L19 admins which is quite frankly disgusting. I seriously expect better from you, knowing some of you in real life. Step up. The original poster of https://www.reddit.com/r/baduk/comments ... _admitted/ has contacted me by email claiming that he tried to post twice to L19 to share this, and that the admins/moderators declined to let his post go live both times. It is absolutely not your place to censor that. It's totally irrelevant whether you own the forum or not; if you wish to have an open community, then you absolutely must allow users to make their own decisions and openly discuss things like this. By deliberately censoring things like that, and not even allowing users to make up their own mind, you are taking a side. That is not the job of an administrator, moderator or community owner. Shape up. You might think yourselves able to silence some poor anonymous player who barely speaks our language, knowing that he can't stand up and fight against that, but don't try to do that to me. I can and will make one hell of a fuss if forced to. Allow this to play out from a neutral standpoint, as is your duty. Both sides have their defenders and detractors. Only stand in if personal attacks occur. No one has any right to censor information. I likely wouldn't have come in and presented all of this if you had have behaved appropriately towards that person.

In the above-linked thread, one very specific detail stands out: "He brag about move 156 in League A Qualification game being his perfect proof of not cheat because Leela 11 says it is very bad move but Leela Zero says it is super so he plays it."

In that thread, Uberdude performed analysis on that specific move. He concluded (https://www.reddit.com/r/baduk/comments ... d/e0c309j/ and https://www.reddit.com/r/baduk/comments ... d/e0c45dl/) that Leela 0.11 thinks that the sequence played is losing until explicitly shown the refutation (and doesn't find it after 100k simulations). As the original poster of that thread claimed, Leela Zero quickly finds and verifies its correctness.

PF137 in that thread goes on to explain (https://www.reddit.com/r/baduk/comments ... d/e0c509f/, and https://www.reddit.com/r/baduk/comments ... d/e0c7qfv/) how this is a very typical P versus NP sort of situation; it's easy to verify what the original poster says as being true, but it would have been extremely difficult for the original poster to come up with it if he hadn't been told it by Carlo. The severeness and unlikeliness of such a situation (game changing, and endgame) is discussed in https://www.reddit.com/r/baduk/comments ... d/e0cd338/.

PF137 takes things apart thoroughly in that thread: everything from psychological profile matching for Carlo, to the above P versus NP stuff, to social analysis of the original poster's requirement for anonymity. Make yourselves entirely familiar with it.

The best defence that (amusingly, mostly Italians, who'd have thought) people on the thread seem to come up with is "[the original poster] doesn't speak like an Italian learning English". Which is quite frankly stupid. I'm deeply involved in language learning and teaching; let me be very clear that there are (with the ever-presence of the Internet in today's world) infinitely many paths to learn a foreign language, and assuming people will follow the same one is nothing short of absurd.

Regardless, Uberdude's verification of the original poster's Leela 0.11 vs Leela Zero claim on move 156, and PF137's analysis throughout from multiple angles, is completely damning.

Simba, that post on reddit is clearly scam!

[Kirby]

I agree that the post is likely a scam. It makes a serious accusation, and the user refuses to provide evidence to back up his claim. The IP address is not from Italy. Ok, sure - he's using a proxy or something. But there's a lot of doubt cast onto the authenticity of the claim.

The admins are willing to work with this guy if he really wants his story told, but he needs to better back up his claims. It doesn't need to be in public. If the OP really contacted you by email, Simba, have him work with the admins and give us some more proof that he is who he said he is - or at least that his story has some shred of merit.

We are not in the business of censoring opinions here. But we are in the business of alleviating the forum from trolls and spam. Without working with us, this "Carlo Cheating" user's post seems to fall into this category.