How Would You Use AI to Cheat?

[BlindGroup]

A few weeks ago, Bill brought up a very important point about detecting cheating in the thread about the Pandanet decision. We have no observed games of a known cheater playing. This complicates the use of statistical analysis to determine the likelihood of cheating significantly. So, I thought I would ask a related question to try to generate information about this: If you were trying to use an AI program to cheat online, how would you do it? And to be clear, I'm specifically referring to how would one change their play using the output from a given AI program -- not how would you get around the various strategies designed to make sure that one does not have access to that output. (Although that might also be a useful discussion

The original metric for comparing Carlos's play to Leela is succinctly described here: http://frmor.net/misc.html
"1. We analyze moves 51-150, that is, 50 moves for Black and 50 moves for White.
2. For every move, we say it is similar if the move is within Leela’s top 3 moves and no farther than 5% from
Leela’s top move.
3. All the moves excluded by the point above are called missed."

One way to interpret this is that it is predicated on a model in which the cheater is running each move through Leela and choosing among Leela's top choices as long as the options are within 5 percent of each other. This strikes me as an inefficient way to cheat. It would require letting Leela spend the same amount of time on each move. It seems more likely to me that someone would prefer to save computation time for moves with clear strategic importance or moves where the player is unsure how to proceed. My initial thought is that it would make more sense to just play "normally" and then refer to Leela only when it seems most valuable. Is that a viable strategy at the high dan level? Or are there better ways to use her output?

And along these lines, I also wonder if it might be useful for a few high-level dans to experiment with cheating using Leela (obviously not in official games!) just to gain experience and relay it back to decision makers. One would probably need to think about how to do this ethically, but it seems like it might yield useful information.

[Javaness2]

My initial thought would be just to keep Leela open during the game and basically just observe its analysis.
Play normally, but right before you make your move, check Leela. Does your choice appear there? Quite possibly not, but is there a game changing alternative?
If so pick it, but never more than 3 times in 1 game.
That way I would play my own game, and deviation would be within standard error.

[John Fairbairn]

Chess experimented with humans using computer assistance. That idea got little traction there but go has taken it up, at least in China, at pro level. It would be relevant to know why chess gave it up and also how go players use it. Anyone know?

There is one easy and useful way to cheat that is probaby undetectable online and that id to use the AI only for blunder checking. Even basically honest people could justify that to themselves

[dfan]

Chess experimented with humans using computer assistance. That idea got little traction there but go has taken it up, at least in China, at pro level. It would be relevant to know why chess gave it up and also how go players use it. Anyone know?

Chess gave it up mostly because it was boring. It has survived in correspondence chess, which now almost everywhere explicitly allows engine assistance. Most people find this boring too.

(Edit: I should note that a large reason that it's boring is that most games of this sort end in draws. So perhaps go wouldn't suffer as much.)

There is one easy and useful way to cheat that is probably undetectable online and that is to use the AI only for blunder checking.

Agreed. It is certainly true at my level, and I bet it is true even up to mid-dan levels, that just not making any stupid mistakes is sufficient to win (the definition of stupid mistake changes as you improve, of course). And this way you don't have to make any moves that are very out of the ordinary.

Even basically honest people could justify that to themselves.

Yeah. I think a lot of cheating (and I'm talking about the real world too, not just games) comes not from people setting out to scam others "unfairly", but from people thinking "I deserve this (win, money, etc.), let's just use a computer or whatever to ensure that I get what I should have coming to me and remove the small probability that it doesn't happen."

[Kirby]

Chess experimented with humans using computer assistance.

I think this is a good point. I don't know about the "boring" aspect, but cheating with computer assistance can be eliminated by allowing computer assistance as a part of the rules. Online tournaments can still be held with the expectation that computer assistance may be used (and could be encouraged).

Chess gave it up mostly because it was boring.

Maybe it would be a bit boring to play under the assumption that computers will be used. But at least the cheating aspect is eliminated... There would be nothing forcing a player to use computer assistance, so I guess he could avoid using the computer if he found it too boring (though, he'd probably lose).

I heard somewhere that, in chess, these "cyborg players" of human + computer are stronger than computer alone. That is to say, there are humans that can play in coordination with the computer such that their human contribution to play doesn't detract from the strength of the computer, but rather enhances it.

Is this true in the chess world? Or are pure computer players universally stronger than computer + human teams?

[Bill Spight]

Like a lot of people, my first thought was to use the bot to avoid blunders. Hopefully at most one per game. But I have had another thought.

Instead of me picking one of the bot's top three choices, let the bot pick one of my top three choices. So instead of me playing like the bot, let the bot play like me. To put it differently, let me play my best game. Assuming the bot is good at evaluating my plays, OC.

[hyperpape]

Like a lot of people, my first thought was to use the bot to avoid blunders. Hopefully at most one per game. But I have had another thought.

Instead of me picking one of the bot's top three choices, let the bot pick one of my top three choices. So instead of me playing like the bot, let the bot play like me. To put it differently, let me play my best game. Assuming the bot is good at evaluating my plays, OC.

Clever.

I'm not so smart..when I saw the question, my thought was "I guess I'd just input my opponent's moves and play what the bot tells me?" But I've never cheated like that, so perhaps it's too hypothetical?

[RobertJasiek]

Disclaimer: I would never use AI etc. to cheat because of ethics and the joy of my own thinking.

Below is my thought experiment how I would use AI to cheat supposing I would have some suitable, undetectable hardware, such as implant, glasses with display or projection on a wall outside the view of the control camera.

- use different AIs during one game,
- use AI only for critical decisions,
- only use support for a subset of all critical decisions,
- use AI only for complicated tactics or key strategic decisions (I do not need support for positional judgement but can imagine that other players might also need this),
- only consider a program's top 1 candidate first move because the quality of its other candidates is unclear,
- for a tactical decision, only view the AI's representative 1 top variation so that I might understand its first move and am not overloaded with more variations than I can handle,
- keep an open mind,
- keep my playing style,
- increase the AI impact over a period of a few years to simulate honest improvement,
- also study honestly to have quite something to show when pretending that it is all my own effort's result.

Without direct evidence (such as an unexpectedly noisy implant), I see no chance to prove any cheating.

[Kirby]

Maybe it's off into the future, but is it feasible that devices for monitoring brain activity will be more accessible in coming years? For example, I guess "electroencephalography" is a way of measuring electrical activity within the brain.

Is it possible that certain patterns of brain activity are detectable when playing go, coming up with the moves on our own?

If so, hypothetically in the future, it may be possible to require some sort of device to record brain activity. Presumably, there could be a model to identify differences between brain activity of a cheater and brain activity of someone not getting computer assistance.

Seems kind of scifi-ish to me, but the only way I can think of better identifying if someone is cheating or using their brain to come up with moves - is to use some sort of sensor or device to detect brain activity and observe the difference in pattern.

[Bojanic]

You are complicating things too much.

Step 1:
Apply for organization of important tournament.
Step 2:
Use latest version of strongest available AI during entire game.
It seems to be working fine so far...

[Gomoto]

Best way to cheat with AI is to play your game without and review it after the game with the AI.

[gowan]

Playing to win is not the real reason people play go. Pros reach a point where their ability to win tournaments declines but they keep on playing even though they aren't winning tournaments, why? I think fundamentally it is because they want to understand the game. They can work at this even without winning tournaments. So can we. Maybe we can win tournaments by using an AI assist but what would be the point? We would know that we ourselves didn't win so what satisfaction would winning that way give us? Money you say? Well, if we win money by using an undetectable AI assist, certainly others would be doing it and we wouldn't be as likely to win tournaments. If these AI assists were developed I predict people who weren't focused on understanding the game would give up playing. Notice that most of what we see on these fora is trying to understand the unusual moves the AI's like Alpha Zero and the like play, i.e. understanding the game.

[Kirby]

Playing to win is not the real reason people play go. Pros reach a point where their ability to win tournaments declines but they keep on playing even though they aren't winning tournaments, why? I think fundamentally it is because they want to understand the game.

I like the sentiment that you're brining to the table, but let me play devil's advocate and ask: If you don't care about winning, what is there to understand about the game? I know the rules - put stones on the intersections, and so on... Anything past that seems to be an objective toward winning the game.

Let's take it to the extreme: you keep playing the game, despite not winning any games. Do you get enjoyment from claiming some understanding? If you're not winning, what's the difference between now and when you were 20k? You're putting stones on a board in both cases. Is it just because you're thinking about more stuff while you're playing? 20ks might think a lot while they're playing, too...

I'm not trying to be snarky; I'm genuinely curious.

[EdLee]

{ blank } is not the real reason people { blank }.

Rather than and , it's multi-dimensional ( ~07:00 ... ~10:00 ):

[lowiq60]

Let's take it to the extreme: you keep playing the game, despite not winning any games. Do you get enjoyment from claiming some understanding? If you're not winning, what's the difference between now and when you were 20k? You're putting stones on a board in both cases. Is it just because you're thinking about more stuff while you're playing? 20ks might think a lot while they're playing, too...

I'm not trying to be snarky; I'm genuinely curious.

This might be slightly off-topic, but I can envision a scenario where you get enjoyment from understanding despite losing all of your games. Say you are a beginner and you play Leela Zero day and night, and lose 1000 consecutive games. In the process of losing 1000 games, your skill level improve from 20k to 1d. Even though you are still losing badly to Leela Zero at the skill level of 1d, your way of thinking will be quite different from your way of thinking at 20k level.