On handling online cheating with AI

Bill Spight · Post by **Bill Spight** » Tue Jun 02, 2020 8:25 am

Example 8.

Click Here To Show Diagram Code: [go]$$B Move 131 $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . X . O . . X X X . X O O . . . | $$ | . . . X X O . X X O O O X X X O O O . | $$ | . . X , O . O O O , O X . . O X X O a | $$ | . . X O . X X . O O O X . . O . . X . | $$ | . . X . . X O O . . . O X X . X X . . | $$ | . X O X . X X . . . . O . . . . . . . | $$ | . X O O O . . . . . . . . . . . . . . | $$ | . X O . O X . . . . . . . . . . O . . | $$ | . . X O O . . . . , . . . . . , . . . | $$ | . . X . . . . . . . . . . . . . . . . | $$ | . . X O . O . . . . . . . . . . O . . | $$ | . . . O . . . . . . . . . . X . . . . | $$ | . . . . . X . X X X . . . . X O . . . | $$ | . . O . O X X O X O X X . . . O . . . | $$ | . . . O . X O . O O O X . X O , . . . | $$ | . . O X X X O O O . X O . X O X O . . | $$ | . . O . . X O . O . O X X . X O . . . | $$ | . . . . . . . . . . O . . X . b . . . | $$ ---------------------------------------[/go]

Who played the hane, who played the atari?
Both are sente, OC.

The winrate difference is 0.1%. Elf's play got 57k rollouts, the human play got 433.

Elf played the atari.

For the record, here is Elf's main variation.

Click Here To Show Diagram Code: [go]$$B Moves 131 to 131 $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . X . O . . X X X . X O O . . . | $$ | . . . X X O . X X O O O X X X O O O . | $$ | . . X , O . O O O , O X . . O X X O . | $$ | . . X O . X X . O O O X . . O . . X . | $$ | . . X . . X O O . . . O X X . X X . . | $$ | . X O X . X X . . 9 . O . . . . . . . | $$ | . X O O O . . . . . . . . . . . . . . | $$ | . X O . O X . . . . . . . . . . O . . | $$ | . . X O O . . . . , . . . . . 5 6 . . | $$ | . . X . . . . . . . . . . . . . 8 . . | $$ | . . X O . O . . . . . . . . . 7 O . . | $$ | . . . O . . . . . . . . . . X . . . . | $$ | . . . . . X . X X X . . . . X O . . . | $$ | . . O . O X X O X O X X . . 3 O . . . | $$ | . . . O . X O . O O O X . X O 4 . . . | $$ | . . O X X X O O O . X O . X O X O . . | $$ | . . O . . X O . O . O X X . X O 2 . . | $$ | . . . . . . . . . . O . . X . 1 . . . | $$ ---------------------------------------[/go]

Black plays kikashi against the bottom right corner and right side before attacking the White group on the top. The top right corner was not on Elf's radar.

RobertJasiek · Post by **RobertJasiek** » Tue Jun 02, 2020 8:28 am

Adin,

"What a human investigator is looking for is mostly plays that are highly unusual for the rank of the player (or for any human players)."

Such an investigator would be prejudiced by his own incompetence. Some human players that do not cheat can and do (even frequently) play moves highly unusual for humans or for humans of certain ranks.

"And other stuff which I will not detail here in case cheaters are reading."

If "justice" relies on secrecy, it is oppression.

Adin · Post by **Adin** » Tue Jun 02, 2020 8:40 am

A few months ago I played a game on another server than KGS. I got crushed. Looking at opponent user info his latest about one hundred games were all wins. He had skyrocketed from a very stable 2k to 4d and showed no sign of stopping. I contacted the server administration and they won't do anything since there is no policy regarding AI cheating. And meanwhile instead of complaining or taking action people get into the most abstract philosophical discussions involving 0.5% win rate. Oh well.

Knotwilg · Post by **Knotwilg** » Tue Jun 02, 2020 8:45 am

6/8 so far;
heuristics:
- moves with more visual appeal are human; moves that require reading to understand what they could accomplish in the first place, are bot's
- ajikeshi is human; sente is bot's (assuming I can see the difference)
- development is human; stability (or undermining it) is bot's

If I had consistently applied 2 & 3, I'd be closer to 8/8

Eventually this kind of study would lead to "better move's are bot's"

I won't get there I'm afraid, but a pro might. Which is the point of this thread.

Bill Spight · Post by **Bill Spight** » Tue Jun 02, 2020 8:46 am

Example 9

Click Here To Show Diagram Code: [go]$$Wc Move 186 $$ --------------------------------------- $$ | . . . . . . . . . . . . . . . . . . . | $$ | . . . . X . O . O X X X . X O O . O . | $$ | . . . X X O O X X O O O X X X O O O . | $$ | . . X , O X O O O , O X . . O X X O X | $$ | . . X O X X X . O O O X . . O . . X . | $$ | . . X X O X O O . . . O X X X X X . . | $$ | . X O X O X X . . X . O . . . . . X . | $$ | . X O O O . . . O X . X . . O X O X . | $$ | . X O . O X . . O O X . . . . X O X . | $$ | . . X O O . . . . , X . . . . X X O O | $$ | . . X . . . . . X . . . . . O O O X . | $$ | . X X O . O . a . X . . b O . . O O . | $$ | X O O O . . O . . . O . . . X . . . . | $$ | . X . . . X X X X X O . O . X O . . . | $$ | . . O . O X X O X O X X . . X O . . . | $$ | . . . O . X O . O O O X . X O O . . . | $$ | . . O X X X O O O O X . X X O . O . . | $$ | . . O . . X O . O . O X X . X O O . . | $$ | . . . . . . O . . . O X . X . X . . . | $$ ---------------------------------------[/go]

Who played the peep, who played the narabi?

The winrate difference is 0.3%. Elf's play got 28k rollouts, the human's play got 97.

Bill Spight · Post by **Bill Spight** » Tue Jun 02, 2020 8:50 am

Adin wrote:A few months ago I played a game on another server than KGS. I got crushed. Looking at opponent user info his latest about one hundred games were all wins. He had skyrocketed from a very stable 2k to 4d and showed no sign of stopping. I contacted the server administration and they won't do anything since there is no policy regarding AI cheating.

I'm with you on that.

And meanwhile instead of complaining or taking action people get into the most abstract philosophical discussions involving 0.5% win rate. Oh well.

It's not philosophical on my part, it's scientific. If we are going to base the question of cheating with AI on whether plays look like AI plays or humans plays, we ought to have some idea what we are talking about.

Comparing plays with small winrate differences is a way to control for the quality of the plays. That's standard scientific practice.

Adin · Post by **Adin** » Tue Jun 02, 2020 9:14 am

Knotwilg wrote:Which is the point of this thread.

The point of this thread is to hopefully make people realize that AI cheating in online is already a very real problem and it gets worse every day. And that action is very much needed but right now very little is done. We really need automated detection but I heard of nobody working on that.

Meanwhile admins and tournament organizers are usually afraid to take action even in obvious cases because of that 1% chance they might be wrong, which they certainly should accept and do take action! And the community of players instead of rallying against cheaters would rather endlessly and uselessly debate all the fine details and philosophical subtleties. There are many cases of blatant cheating that just get away with it.

Bill Spight · Post by **Bill Spight** » Tue Jun 02, 2020 9:20 am

Example 10.

Click Here To Show Diagram Code: [go]$$Wc Move 212 $$ --------------------------------------- $$ | . . . . . . . . . . . . . X X O . . . | $$ | . . . . X . O . O X X X . X O O . O . | $$ | . . . X X O O X X O O O X X X O O O O | $$ | . . X , O X O O O , O X . . O X X O . | $$ | . . X O X X X . O O O X . . O . . X O | $$ | . . X X O X O O . . . O X X X X X . X | $$ | . X O X O X X . O X . O . . . . . X . | $$ | . X O O O . . . O X . X . . O X O X . | $$ | . X O . O X X . O O X . . . a X O X . | $$ | . . X O O . . X . , X . . . O X X O O | $$ | . . X . . . O . X . X . . . O O O X . | $$ | . X X O . O . O X X O . O O . . O O . | $$ | X O O O . . O . . . O . . . X X O . . | $$ | . X b . . X X X X X O X O . X O . . . | $$ | X O O . O X X O X O X X . . X O . . . | $$ | . X . O . X O . O O O X . X O O . . . | $$ | . O O X X X O O O O X . X X O . O . . | $$ | . . O X . X O . O . O X X . X O O . . | $$ | . . . . . . O . . . O X . X X X O . . | $$ ---------------------------------------[/go]

Who played atari against four stones, who made the ko?

The winrate difference is 0.4%. Elf's play got 23k rollouts, the human's play got 221.

Elf played the atari against 4 stones. Neither player made the other play any time soon.

Game continuation.

Click Here To Show Diagram Code: [go]$$Wcm 12 $$ --------------------------------------- $$ | . . . . . . . . . . . . . X X O . . . | $$ | . . . . X . O . O X X X . X O O . O . | $$ | . . . X X O O X X O O O X X X O O O O | $$ | . . X , O X O O O , O X . . O X X O . | $$ | . . X O X X X . O O O X . . O . . X O | $$ | . . X X O X O O . . . O X X X X X . X | $$ | . X O X O X X . O X . O . . . . . X . | $$ | . X O O O . . . O X . X . . O X O X . | $$ | . X O . O X X . O O X . . . . X O X . | $$ | . . X O O . . X . , X . 3 . O X X O O | $$ | . . X . . . O . X . X . . . O O O X . | $$ | . X X O . O . O X X O . O O . . O O . | $$ | X O O O . . O . . . O . . . X X O . . | $$ | 2 X 1 . . X X X X X O X O . X O . . . | $$ | X O O . O X X O X O X X . . X O . . . | $$ | . X . O . X O . O O O X . X O O . . . | $$ | . O O X X X O O O O X . X X O . O . . | $$ | . . O X . X O . O . O X X . X O O . . | $$ | . . . . . . O . . . O X . X X X O . . | $$ ---------------------------------------[/go]

After

filled the ko,

made an eye without the atari against the four stones.

Click Here To Show Diagram Code: [go]$$Wcm12 Elf's mainline $$ --------------------------------------- $$ | . . . . . . . . . . . . . X X O . . . | $$ | . . . . X . O . O X X X . X O O . O . | $$ | . . . X X O O X X O O O X X X O O O O | $$ | . . X , O X O O O , O X . . O X X O . | $$ | . . X O X X X . O O O X . . O . . X O | $$ | . . X X O X O O . . . O X X X X X . X | $$ | . X O X O X X . O X . O . . . 2 . X . | $$ | . X O O O . . . O X . X . . O X O X . | $$ | . X O . O X X . O O X . 5 . 1 X O X . | $$ | . . X O O . . X . , X . 6 . O X X O O | $$ | . . X . . . O . X . X . . . O O O X . | $$ | . X X O . O . O X X O . O O . . O O . | $$ | X O O O . . O . . . O 3 . . X X O . . | $$ | . X . . . X X X X X O X O . X O . . . | $$ | X O O . O X X O X O X X 4 . X O . . . | $$ | . X . O . X O . O O O X . X O O . . . | $$ | . O O X X X O O O O X . X X O . O . . | $$ | . . O X . X O . O . O X X . X O O . . | $$ | . . . . . . O . . . O X . X X X O . . | $$ ---------------------------------------[/go]

played the atari against four stones, and then

took kikashi agains the Black group in the bottom right. Then

was a different jump in the center, allowing

. The ko was not on Elf's radar.

Bill Spight · Post by **Bill Spight** » Tue Jun 02, 2020 9:27 am

Adin wrote:
Knotwilg wrote:Which is the point of this thread.
The point of this thread is to hopefully make people realize that AI cheating in online is already a very real problem and it gets worse every day. And that action is very much needed but right now very little is done. We really need automated detection but I heard of nobody working on that.

It's not an easy task to find and validate an algorithm to do that.

Meanwhile admins and tournament organizers are usually afraid to take action even in obvious cases because of that 1% chance they might be wrong, which they certainly should accept and do take action! And the community of players instead of rallying against cheaters would rather endlessly and uselessly debate all the fine details and philosophical subtleties.

If we are going to deal with online cheating we have to develop correct methods and procedures. What the hell do you think I am doing here? Just because you do not realize the difficulty of the task does not make the effort useless.

Adin · Post by **Adin** » Tue Jun 02, 2020 9:42 am

Bill Spight wrote:What the hell do you think I am doing here?

I'd guess you're the only one who knows that. Perhaps proving why Reddit replaced traditional forums.

RobertJasiek · Post by **RobertJasiek** » Tue Jun 02, 2020 9:54 am

"his latest about one hundred games were all wins. He had skyrocketed from a very stable 2k to 4d and showed no sign of stopping."

This might be a case of AI cheating or something else. A different possible case: He was 4d but a victim of the rating / ranking system and stuck at 2k. Then he got a series of opponents whom he could always beat with dusciplined play.

Without further investigation, one cannot know which case of explanation applies.

For reference, when stuck at a too low KGS rank (ca. 1 or 2 ranks below my supposed skill rank of ca. 50% wins), disciplined playing and expected to beat opponents yield winning series of 10 or 20 games. Those series stop when eventually I get tired after too many games on a day, lose my discipline out of frustration of too slowly changing rating, out of forgetting about the fast time setting (when underrated, a long time setting would be unbesrably frustrating), meet the exceptionslly motivated usually weaker but then very well playing opponent or meet the hard to beat opponent (e.g. another victim of the rating system). Otherwise, such series could last much longer.

Bill Spight · Post by **Bill Spight** » Tue Jun 02, 2020 10:44 am

Example 11

Click Here To Show Diagram Code: [go]$$Wc Move 226 $$ --------------------------------------- $$ | . . . . . . . . . . . . . X X O . . . | $$ | . . . . X . O . O X X X . X O O . O . | $$ | . . . X X O O X X O O O X X X O O O O | $$ | . . X , O X O O O , O X . . O X X O . | $$ | . . X O X X X . O O O X . . O . . X O | $$ | . . X X O X O O . . O O X X X X X . X | $$ | . X O X O X X . O X . O X . . . . X . | $$ | . X O O O . . . O X . X . . O X O X . | $$ | . X O . O X X . O O X . . . . X O X . | $$ | . . X O O . . X X , X . O . O X X O O | $$ | . . X . . . O O X . X . . . O O O X . | $$ | . X X O . O . O X X O O O O . . O O . | $$ | X O O O . . O . . . O X . . X X O . . | $$ | X X O . X X X X X X O X O O X O . . . | $$ | X O O . O X X O X O X X . X X O . . . | $$ | X X O O . X O . O O O X . X O O . . . | $$ | a O O X X X O O O O X . X X O . O . . | $$ | . . O X . X O . O . O X X . X O O . . | $$ | . . . b . . O . . . O X . X X X O . . | $$ ---------------------------------------[/go]

Who played the atari, who played the hane?

The winrate difference is 1.6%. Elf's play got 18k rollouts, and the human's play got 255.

Elf played the 1 pt. sente at a. Locally, it is plainly superior to the hane, which gains less than 1 pt., since Black has less than 1 pt. of territory at stake locally. In any event, the sente is not a mistake.

First, let's look at Elf's main variation.

Click Here To Show Diagram Code: [go]$$Wcm26 One pt. plays $$ --------------------------------------- $$ | . . . . . . . . 5 3 4 . . X X O . . . | $$ | . . . . X . O . O X X X . X O O . O . | $$ | . . . X X O O X X O O O X X X O O O O | $$ | . . X , W X O O O , O X . . O X X O . | $$ | . . X W X X X . O O O X . . O . . X O | $$ | . . X X O X O O . . O O X X X X X . X | $$ | . X O X O X X . O X . O X . . . . X . | $$ | . X O O O . . . O X . X . . O X O X . | $$ | . X O . O X X . O O X . 6 . . X O X . | $$ | . . X O O . . X X , X . O . O X X O O | $$ | . . X . . . O O X . X . . . O O O X . | $$ | 2 X X O . O . O X X O O O O . . O O . | $$ | X O O O . . O . . . O X . . X X O . . | $$ | X X O . X X X X X X O X O O X O . . . | $$ | X O O . O X X O X O X X . X X O . . . | $$ | X X O O . X O . O O O X . X O O . . . | $$ | 1 O O X X X O O O O X . X X O . O . . | $$ | . . O X . X O . O . O X X . X O O . . | $$ | . . . . . . O . . . O X . X X X O . . | $$ ---------------------------------------[/go]

White takes the 1 pt. sente with

and then takes the 1 pt. gote on the top side. Note that B 231 connects three Black dragons, so that Black does not have to capture the two

stones at the cost of 1 pt.

Now let's look at the actual game.

Click Here To Show Diagram Code: [go]$$Wcm26 Actual game $$ --------------------------------------- $$ | . . . . . . . 5 4 6 . . . X X O . . . | $$ | . . . . X . O . O X X X . X O O . O . | $$ | . . . X X O O X X O O O X X X O O O O | $$ | . . X , W X O O O , O X . . O X X O . | $$ | . . X W X X X . O O O X . . O . . X O | $$ | . . X X O X O O . . O O X X X X X . X | $$ | . X O X O X X . O X . O X . . . . X . | $$ | . X O O O . . . O X . X . . O X O X . | $$ | . X O . O X X . O O X . . . . X O X . | $$ | . . X O O . . X X , X . O . O X X O O | $$ | . . X . . . O O X a X . . . O O O X . | $$ | 8 X X O . O . O X X O O O O . . O O . | $$ | X O O O . . O . . . O X . . X X O . . | $$ | X X O . X X X X X X O X O O X O . . . | $$ | X O O . O X X O X O X X . X X O . . . | $$ | X X O O . X O . O O O X . X O O . . . | $$ | 7 O O X X X O O O O X . X X O . O . . | $$ | . . O X . X O . O . O X X . X O O . . | $$ | . . 3 1 2 . O . . . O X . X X X O . . | $$ ---------------------------------------[/go]

After

-

in the bottom left, Black gets the 1 pt. play on the top side and the White takes the sente in the bottom left. This takes us up to the next example, so I will continue from there.

jlt · Post by **jlt** » Tue Jun 02, 2020 10:48 am

RobertJasiek wrote:"his latest about one hundred games were all wins. He had skyrocketed from a very stable 2k to 4d and showed no sign of stopping."

(...)

For reference, when stuck at a too low KGS rank (ca. 1 or 2 ranks below my supposed skill rank of ca. 50% wins), disciplined playing and expected to beat opponents yield winning series of 10 or 20 games.

(...) such series could last much longer.

Adin didn't say on which server the player had a 100-game winning streak, but if it's on Fox, Tygem or IGS, winning 100 games in a row would result in being promoted or double-promoted many times, and I think this is almost impossible to achieve without cheating.

Knotwilg · Post by **Knotwilg** » Tue Jun 02, 2020 11:06 am

Adin wrote:
Bill Spight wrote:What the hell do you think I am doing here?
I'd guess you're the only one who knows that. Perhaps proving why Reddit replaced traditional forums.

There have been very thoughtful responses to your question. No flaming, not hating. Bill has started a tangent to explore if individual moves can be easily found out as bot-copied. I must admit I find the tangent somewhat overtaking the main discussion but still trying to help out with it. It doesn't warrant the enmity with which you met it.

Both of you are helpful volunteers in the Go community. There's not enough of us to engage in heated debate - though I've been guilty of that myself.

Adin · Post by **Adin** » Tue Jun 02, 2020 11:18 am

I'm sorry if I've been a bit blunt about it. But this belongs to another thread. Posting 12 messages with diagrams and then maybe getting responses to them really disrupts the main discussion.

Life In 19x19

On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI

Re: On handling online cheating with AI