ad hominem

Yes. But a fallacy?

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Isn't the whole idea of neural networks that AI can learn to recognize patterns that humans might not be aware of? Feed it a bunch of games where savvy players cheat and let it make the call. For a server like KGS, the combination of automated cheater detection and an automatic temporary ban would certainly serve as a deterrent without ruining anyone's reputation. It doesn't have to always be right, it just has to be the authority.

_________________
Patience, grasshopper.

I apologize for throwing oil on the fire, as we say. Let the discussion continue on topic.

NNs can only find things that are actually there, for example, what could also be found by reasonable human effort. How well would humans do here, even after a long study? With blunder-check method a cheater leaves minimal traces. As mentioned, unusual lack of big blunders may be seen as evidence, but only if it persists in dozens or hundreds of games. (And it is also easily possible to develop a cheating logic that mimics stronger humans' style and errors for example.)

The kind of "automated detection" in some people's mind is prolly something that marks a player after a few suspicious games already (maybe even showing a "botness percentage" besides every player's rank or rating ). But this is simply not realistic for the more clever types of cheating, only for the simplest type.

I wonder if a way forward, balancing the desires of people not wanting innocents incorrectly found guilty of cheating against those who worry about the decline of online Go if nothing is done out of fear of those false positives, is 2 stages of identifying/punishing suspected cheaters:
1) suspicious of AI use: such accounts could be flagged (new icon on KGS) after a not too massive investigatory effort by admins, say a 50/50. This would be sufficient grounds for disqualifying an anonymous account from a KGS+ tournament
2) very likely AI use: this is a tougher test, where it's ok to say even against a named individual, such as in the PGETC. Also with a new flag/icon on KGS.

I would also think updating the KGS TOS to outright ban use of accounts using AI is too restrictive; if someone uses AI and is honest and open about it, and the server has a way to filter such accounts out of game negotiation (new feature required), I'm fine with it (indeed I once made a YALZA (Yet Another Leela Zero Acount) account to test my bot against some on KGS, put that I was LZ on user info). Many people seem to also think it's ok to use AI if you clearly do so on a separate account (can't find old poll thread now to link). I would instead prefer "If you use AI at all during your games, you must flag yourself with the icon of 2 above and include a note that you use AI in your user profile."

jann, "unusual lack of big blunders may be seen as evidence": It is not.

1) Strong amateur players in real world games with sufficient thinking times should have a lack of big blunders. Quite a few players would achieve the same also in fast online games. For strong amateurs and stronger, it is not about big blunders but about reducing relatively small(er) blunders / mistakes.

2) Peaceful playing styles can be without big blunders. A player may have such a style in all his games. He might play endgame-like throughout the game. His biggest "blunders" might ca. 3 points.

Uberdude, I disagree to suspicision flags because they hurt player reputation countrary to his honest play.

For official tournaments, such as PETC, automatic detection with a high percentage of reliability is not good enough. Such tournaments need 100% correct assessment because any arbitration must follow the standards of the tournament's tournament rules incl. the EGF General Tournament Rules and its standards for play and arbitration: no cheating is required in each game; penalties presume evidence - not guesswork.

daal, "an automatic temporary ban would [not ruin] anyone's reputation": I disagree. It would ruin reputations because many would believe in the flags regardless of whether their indication is correct for particular players.

Moves that may be suspicious for a 6k are much less suspicious for a 7d. So a bot detection method that is "good enough" for 6k level may not be good enough for high dans. And high dans have more reputation to lose when they get wrongly flagged for cheating. It may even affect them financially (losing paying students). So it's very important that the detection method is accurate for all levels, not just for some obvious cases at 6k level.

Again, a lot of testing would be neccessary to validate bot detection methods.

On KGS, disabling the rank of an account for 6 months shouldn't be problematic since other people have no way of determining if it was done by an admin, or if you un-checked the box yourself.

Question: isn't focus on catching those who cheat in official tournaments, where prize money and/or near-pro status (attracting pupils) is at stake?
Because the effect of a 6k / 1d cheater on me as a KGS "customer" is not very different from the abundance of transparent bots: rank deflation.

On KGS (*) by far the most irritating behavior is players setting up a challenge and not responding when challenged. Declining without reason is a close second. Anti-social behavior, like dropping out of a lost game and returning after you dropped out, to stage a forfeit ... the list goes one. These are bigger nuisances to me than the chance of a human opponent using AI.

As this discussion shows, the work needed to find out AI cheaters will probably be massive and require thoughtful regulation, which is already absent in the above mentioned issues.

(On Fox: accepting then dropping out because of the win rating, forfeiting the game, so that you have to start all over again)
(On OGS: the absence of players, the changing of time settings, more anti-social behavior)
(BTW, overall experience is relatively fine.)

In other words: we have to protect the Antti Tormanens and keep them in the semi-pro scene. Not the Knotwilgs.

If these would be true, there would be no incentive to use bots for blunderchecking at all. As long as there are some effects of their use there are differences, but even I wrote that these are too subtle in clever use. Profiling in 100 games may be possible but not in 10 games or less.

I just want to understand what you are saying: between or during games?

Pros may have solved the problem already. Don't look for cheats. Just make cheating impossible. Use Zoom.

There was a story on the BBC yesterday to the effect that Zoom's sales have skyrocketed during the lockdown, and big IT companies have also joined the arms race. I have a little personal experience of Zoom myself, as we have been using it for Scottish country dancing lessons. Teachers from at least Scotland, USA and Japan have entertained and taught students from literally all around the world - there are huge numbers in all of North America, Australia, New Zealand Hong Kong, Eastern, Europe and so on. I expect many people here have used it for other things. At any rate, the message is clear: it works.

More to the point here, the Nihon Ki-in has been using it to run tournaments. They even call them the Zoom Challenges.

Obviously the main point was to allow activity during lockdown (live games are due to re-start very soon), and it seems to have all been set up easily so that the players see each other in a whole-body form.

That seems to wipe out some of the biggest problems also for fan servers - not just using bots but anti-social behaviour under the guise of anonymity. Your opponent (and mod?) can see what you are up to, and can see who you are. And you can chat in a quasi face-to-face social way. World wide.

Plus, it's all free. It may not be free if implemented on a large scale like KGS, but it must be workable there, surely?

In any case, I expect servers may fade away to some degree if people start using Zoom to hook up with friends and teachers on the other side of the world for face-to-face one-on-one sessions or mini-tournaments.

I therefore don't think Antti and his ilk need special protection (and don't forget pros cheat, too - see The Incident Room). We just need cheating to become bad enough to make the rest of us overcome our inertia to do something about it, something more practical than advanced level maths/statistics.

It's the lesson of history. When the pilfering of prisoners became a bad enough problem, the ancient Chinese changed from territory to area counting.

During games, ie. the smart kind of cheating, only eliminating significant mistakes.

Video for prevention appears attractive at first, but I don't think it's enough or effective even. Local human observers different oc.

Video chat software may have some use but only detects naive cheating and presumes video hardware (my PC has none).

Video chat software, however, and Zoom in particular is malware (even in many respects) or (other than Zoom) at the very least violates EU and German data privacy laws. One must not solve a small problem (some cheat) by a larger problem (malware).

Typical video chat software restricts the number of viewers.

Hence, it would need a new video chat software that abides by the laws and scales to go servers.

Using Zoom doesn't guarantee that an AI doesn't show moves on the screen (or on another screen).

Of course it doesn't. It's an aid. You can use it watch a player's eyes move, and so on. Humans are incredibly sensitive to anything that happens on the face.

It's not perfect but it's practical and free and it's here now.

And other one point the numbers guys keep overlooking is that a mathematical modelling solution, apart from being also imperfect while being impractical, expensive and not available yet, is that for people who don't understand the maths or statistics, the imposition of such a system creates an uncomfortable atmosphere - almost an invasion of privacy. It's a kind of Google/Facebook/Twitter approach to life. You have to trust something you don't really understand. Of course you have to do that sometimes in life - but for a mere board game? It's an anti-social solution to a social problem.

First, I'm not sure there is a subtle pattern here. There's just so much data the machine doesn't have. Which algorithm might the cheater be using? During which of there games were they hungry, or drinking, or are playing after a great night sleep? Which tsumego have they done lately? Which AI games have they reviewed that they might have picked up a move from? What were there goals for this game? I tend to open strong and then blunder away the middle game. But sometimes I set out to work on playing simply enough to not blunder bad. Those games look very different from my normal games. Sometimes I want to play an unfamiliar opening. Sometimes I miss a really obvious move, but 20 moves later find a clever way to escape my fate. Was that a blunder-recover pair, or professional level reading depth?

I think this is a common misconception with AI: if there's relevant data it doesn't have access to, that's a hard limit on what it can achieve. Add in the number of variations of AI networks the cheater could be using, and just the huge variation in styles and relative strengths among humans, plus the reality that everybody is after the same goal, and there's only so many plausible moves in most situations: I strongly suspect any signal from savvy cheating is just swamped by the rest of the variance in the world.

And even if there is a signal, subtle signals require lots of training data. AlphaGo Zero self-played ~5 million games. ImageNet (detecting images) is 14 million labels. On the one hand, binary "are they cheating" classification is an easier space than "choose every move in a Go game". On the other, I still think this is a very subtle thing that will require nuanced understanding of human styles. Maybe it's only 50,000 cheating players we need to record? Just 1% of learning to play? I'd still think that's low. I think it'd tend to learn subtle spurious correlations among the players who were tested. In particular, I'm sure it'd be very suspicious of anybody stronger than most of the 50,000 players we sampled. If you review too many games with bots, maybe you can't play anymore. But maybe I'm wrong and that's enough.

Of course, we'll need to resample those players every couple years as our understanding of Go improves...

In the chess world where cheating is more mature, I recall cases where people got caught with various settings in offline tournaments (and likely more cases without getting caught). It is much easier to do this under video.

If "anything happens on the face", what's next? You have suspicions - again. Offline you may walk there, ask questions, check physically.

In a tournament you need physical evidence on the playing site. But for casual online games, doesn't video provide a measure of protection, for both players? As an Arab saying goes, in part, "Tie your camel." OC, your camel could still get stolen, but it's a measure of protection.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.