It is currently Sun Nov 18, 2018 9:33 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 720 posts ]  Go to page Previous  1 ... 7, 8, 9, 10, 11, 12, 13 ... 36  Next
Author Message
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #181 Posted: Thu Apr 12, 2018 4:26 am 
Judan

Posts: 5287
Location: Cambridge, UK
Liked others: 295
Was liked: 2776
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Dmytro wrote:
Uberdude wrote:
Although I load the whole game sgf into Leela, when I ask it for what it wants to play for move X I haven't done any analysis for moves after X (I used a separate sgf replayer to know what the human played) so I don't think the fact the sgf contains that information is used by Leela, but I will check with a truncated sgf. (It's a manual position-by-position analysis rather than bulk analysis of the game like go review partner does). If you go forward from X and do analysis then these simulations of the game tree are used if you move back to X and continue analysis.

I do not know much about Leela interface. But, logically, your way for game analysis looks good. Still, I would prefer to use truncated sgf to be 100% sure that there is no influence from next moves.

I checked with a truncated sgf, the results are essentially (but not exactly, see Herman's point below) the same. The Leela interface (or my ineptitude with it) makes it difficult to input both black and white moves to an unfinished sgf and have Leela only offer analysis instead of actually playing moves of the opposite colour in reply.

HermanHiddema wrote:
So, given that Leela's preferred moves are non-deterministic like this, it is possible that the same move might on one run be Leela's top choice, and on another be outside the top 3 or outside the 5% margin?

Definitely possible, though my feeling from the analysis I've done so far is it would be rare for a #1 to drop so far. Shuffling around of #2/#3/#4, and win% crossing the 5% mark more common.

HermanHiddema wrote:
Given one of your test games, for every position between moves 50-150, let Leela analyse the position five times, independently (i.e. close and reopen the position between runs). Then record if the human move played was ever Leela's top choice.

Too much work for me to do manually though! As a little test, here's a pic of 3 runs (50k, 50k, 150k) on the same position on a full sgf and 3 on a snipped one to also test Dmytro's point. This position Leela has a strong preference for the #1 of d15 and didn't put much effort into analysing the other choices. In other positions I've seen much flatter distribution of the effort so I'd expect more variance between runs (and also with #nodes). In all 6 d15 is #1 and has by far the most simulations. d14 and d16 are always taking the next 2 positions, but d14 is #2 in 4 of 6, and is always within 5% of #1, even when in 3rd. In 2 of the 4 where d16 is 3rd it is more than 5% worse than #1. The order of moves outside the top 3 changes a bit, but with so few simulations is basically noise.
Attachment:
snipped samples.png
snipped samples.png [ 81.53 KiB | Viewed 2372 times ]


John Fairbairn wrote:
Maybe we could try an electronic vote here, too.

I don't think that's a good idea, unless you can first ensure voters understand the evidence. Otherwise a lot of naive people will think "98% similarity to Leela => 98% is big, almost 100% => he cheated with Leela". If you can only vote after reading a detailed report on the evidence, doing a mini-course in statistics, reading an essay from Bill on Bayesianism etc and passing a mini-exam on them then I'd be happier with a vote. Then again we let uninformed people vote in much more important matters :).

Kirby wrote:
* I don't think punitive measures can fairly be taken without absolute proof of cheating.

I think "absolute" is too strong, "beyond reasonable doubt" is good enough for me in this case (but I have oodles of doubt). For less important things like regular KGS games even less strong evidence is ok.
Edit: skim reading some of drmwc's links from the bridge case I see "comfortable satisfaction" as an intermediate level of proof between "balance of probabilities" and "beyond reasonable doubt".

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #182 Posted: Thu Apr 12, 2018 4:34 am 
Lives in gote
User avatar

Posts: 427
Liked others: 59
Was liked: 89
Rank: 4 Dan European
In bridge, Fantoni and Nunes were accused of cheating. The site BridgeWinners was involved in analysing the evidence and briging the case. They were world champion level players.

Bridge is attempting to become Olynpic, and so the ulitimate arbitration for European bridge is the Court of Arbitration for Sport. Their appeal ultimately ended up at CAS, and they won the appeal earlier this year.

The EBL press release is here:
http://www.eurobridge.org/2018/01/10/4798/

The Bridgewinners thread on the CAS decision is here:
https://bridgewinners.com/article/view/ ... and-nunes/

Go at one point was seeking Olympic accreditation, and so it's possible that Go appeals could also end up at CAS.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #183 Posted: Thu Apr 12, 2018 5:37 am 
Oza

Posts: 2216
Liked others: 14
Was liked: 3175
Quote:
(Answering: Maybe we could try an electronic vote here, too.)

I don't think that's a good idea, unless you can first ensure voters understand the evidence. Otherwise a lot of naive people will think "98% similarity to Leela => 98% is big, almost 100% => he cheated with Leela". If you can only vote after reading a detailed report on the evidence, doing a mini-course in statistics, reading an essay from Bill on Bayesianism etc and passing a mini-exam on them then I'd be happier with a vote. Then again we let uninformed people vote in much more important matters :).


But that's my point. In so many areas of our lives all of us (even those who think they are rational) makes decisions and assessments in naïve, uninformed way - and that is usually how we expect other decisions to be made, as you seem to concede.

I think most of use realise the process is flawed and does (not just "is likely to", but "does") lead to rather serious mistaken consequences: criminal convictions, bombing Iraq, destroyed reputations, etc, etc. Still, we accept that as a necessary compromise for practical reasons.

Drug-busting in sport has tried to go the other "scientific" way at the most enormous cost and major inconvenience to the lives of athletes, and terminal boredom or frustration for sports fans, yet it still doesn't work - unless you're a greedy or hungry lawyer.

A game like go can't afford much in the way of anti-cheating and anti-drugs measures anyway, so the most sensible regime seems to be that we tolerate unscientific suspicions but we try to alleviate the inevitable but occasional malign consequences by (1) not doling out treatments as harsh as Carlo's and (2) making people aware that they have to take their own steps to appear to be above suspicion.

A hard-nosed extension of the last point, for example, might be to say that Carlo was unwise to limit his study to copying Leela and unwise to tell other people about it, so he was the victim of his own mistakes even if he didn't cheat. The phrase "justice needs to be seen to be done" works two ways, after all.

It could be instructive to see how an average amateur go audience would vote on this, I.e. to see what the normal real-world expectations would be. It may not be a "good idea", but is it the "best idea"?

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #184 Posted: Thu Apr 12, 2018 12:04 pm 
Judan

Posts: 7812
Liked others: 2143
Was liked: 2742
John Fairbairn wrote:
Maybe we could try an electronic vote here, too.


How did that work out for Socrates? ;-) :sad:

Uberdude wrote:
I don't think that's a good idea, unless you can first ensure voters understand the evidence. Otherwise a lot of naive people will think "98% similarity to Leela => 98% is big, almost 100% => he cheated with Leela".


Leaving aside my opposition to matching (confirmatory) evidence as well as matching in only one game, matching one of the top three Leela choices (unless it is not a good play) is a lousy metric. Why? Because, as Uberdude has shown, non-cheaters at that level also match quite frequently. What you would like is a more sensitive test, such as one by which non-cheaters match around 50% of the time. Matching Leela's top choice looks like it fits the bill pretty well. In Uberdude's admittedly non-random, insufficiently large sample, the matching to Leela's top pick among supposed non-cheaters has a median value of 48%. Using that metric Carlo matched 72% of the time.

Tell people this:
Quote:
In this one game, out of 50 moves in the middle (Black 51 - Black 99), 36 of Carlo's plays matched Leela's top choice for that play.


Research has shown that people are not so good at judging percentages or fractions, they are better at judging integers. :)

My guess is that most people would say, 36 matches out of 50? That's suspicious, all right, but not convincing.

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #185 Posted: Fri Apr 13, 2018 5:09 am 
Lives with ko
User avatar

Posts: 287
Location: Deutschland
Liked others: 231
Was liked: 118
Rank: EGF 4 kyu
Uberdude wrote:
If you can only vote after reading a detailed report on the evidence, doing a mini-course in statistics, reading an essay from Bill on Bayesianism etc and passing a mini-exam on them then I'd be happier with a vote.


Would it suffice to restrict the vote to amateur dan players? A "jury of his peers", as it were.

After all, *those* players are the stakeholders, here. Those are the players that will be being paired against Carlo in future matches and *those* are the players who stand to lose the most should this incident lead to either an atmosphere of mistrust and suspicion, to lax anti-cheater policies, or to overly draconian anti-cheater measures that lead to many false positives in the future.

Personally, I feel that this decision and the way it was handed down was far too focused on the local situation -- a single game in a single, online tournament -- than the whole board position. In my opinion, even if the evidence for cheating had been much, much stronger than it really is, one could *still* argue that this incident should serve only to open the conversation about anti-cheating measures and proper, responsible punishments for convicted cheaters.

The reason why cheating is against the rules is to prevent players from gaining an unfair advantage, after all, not to dole out punishments. If a player is suspected of cheating, surely future scrutiny and long-term observation achieves this far more elegantly than wrecking that player's reputation and tarring that player with a conviction?

Sure, you might never get the chance to convict because the player might never cheat again, foiling your efforts to catch him or her. I'd call that a resounding success, not a failure.

Even if I wasn't (also) a Bayesian, even if I didn't know enough about statistics to know what that means in the first place, I would still vote not-guilty, should I be asked, simply because the outcome of the vote has consequences and an unqualified "guilty" verdict must be the worst outcome for everyone. It does not create an environment where cheating is not a dominant strategy because cheaters will, ultimately, be caught. It creates a minefield: don't, whatever you do, resemble a bot or your Go-life is over.


This post by Charlie was liked by 4 people: Bill Spight, goTony, jeromie, Knotwilg
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #186 Posted: Fri Apr 13, 2018 5:26 am 
Judan

Posts: 5287
Location: Cambridge, UK
Liked others: 295
Was liked: 2776
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Charlie wrote:
It creates a minefield: don't, whatever you do, resemble a bot or your Go-life is over.

So to avoid being accused of cheating, maybe for my PGETC game next week I should run Leela to make sure I don't play too many moves similar to her suggestions :lol: . (I'll likely be playing Gilles van Eeden 6d, who scored 82% similarity to Leela's top 3 vs Viktor Lin; my best results against stronger players typically happen when I succeed in playing a solid honte style, which probably has a relatively high Leela match rate).


This post by Uberdude was liked by 3 people: Bill Spight, Bonobo, Charlie
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #187 Posted: Fri Apr 13, 2018 12:20 pm 
Lives in gote

Posts: 376
Liked others: 71
Was liked: 118
Rank: igs 3d+
Meanwhile in the chess world...
https://www.chess.com/news/view/gm-solo ... ms-protest

It seems that even with chess engines being much more advanced than anything we can do with Leela or Crazystone conclusions are hard to reach. Even tournament perfomances that are worlds apart (2000 elo to 2500 elo in this case, although one was blitz the other rapid) are apparently no smoking gun.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #188 Posted: Fri Apr 13, 2018 1:18 pm 
Lives in sente

Posts: 1261
Liked others: 568
Was liked: 368
Rank: AGA 4k KGS 2k
GD Posts: 61
KGS: dfan
Shenoute wrote:
Meanwhile in the chess world...
https://www.chess.com/news/view/gm-solo ... ms-protest

It seems that even with chess engines being much more advanced than anything we can do with Leela or Crazystone conclusions are hard to reach. Even tournament perfomances that are worlds apart (2000 elo to 2500 elo in this case, although one was blitz the other rapid) are apparently no smoking gun.

Ken Regan found zero evidence of cheating in this case.


This post by dfan was liked by 2 people: Bill Spight, Uberdude
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #189 Posted: Fri Apr 13, 2018 1:43 pm 
Lives in gote

Posts: 376
Liked others: 71
Was liked: 118
Rank: igs 3d+
dfan wrote:
Ken Regan found zero evidence of cheating in this case.

Thanks!

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #190 Posted: Tue Apr 17, 2018 12:56 pm 
Judan

Posts: 5287
Location: Cambridge, UK
Liked others: 295
Was liked: 2776
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Uberdude wrote:
I'll likely be playing Gilles van Eeden 6d, who scored 82% similarity to Leela's top 3 vs Viktor Lin; my best results against stronger players typically happen when I succeed in playing a solid honte style

Ooops, I played the taisha! :D

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #191 Posted: Wed Apr 18, 2018 1:03 pm 
Oza
User avatar

Posts: 2107
Location: Germany
Liked others: 7599
Was liked: 864
Rank: OGS SDK
Universal go server handle: trohde
Continuation: http://pandanet-igs.com/communities/eur ... rounds/416
Pandanet wrote:
League A 2017/2018 - Round 8
Italy vs. Poland
Tuesday 10 April 2018, 20:00 CET/CEST
[..]
3rd and 4th board temporarily postponed until an appeal is concluded.

_________________
“The only difference between me and a madman is that I’m not mad.” — Salvador Dali ★ Play a slooooow correspondence game with me on OGS? :)

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #192 Posted: Thu Apr 26, 2018 3:53 am 
Oza
User avatar

Posts: 2107
Location: Germany
Liked others: 7599
Was liked: 864
Rank: OGS SDK
Universal go server handle: trohde
UPDATE (and sorry for not having a better, more direct source)

According to Ingo Althöfer’s comment in the German DGoB forum, the Italian player in question will be allowed to play in rounds 8 and 9 until the appeal is concluded.

_________________
“The only difference between me and a madman is that I’m not mad.” — Salvador Dali ★ Play a slooooow correspondence game with me on OGS? :)


This post by Bonobo was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #193 Posted: Thu Apr 26, 2018 12:32 pm 
Lives in gote

Posts: 323
Location: Washington State
Liked others: 241
Was liked: 58
Rank: KGS13kyu
KGS: gotony
OGS: nghtstalker
Knotwilg wrote:
I agree the game should be forfeited and replayed. The punishment is too harsh for what's merely confirmed suspicion.


But if it is an internet game how does one confirm there is computer assistance?

_________________
Walla Walla GO Club -(on FB)

We play because we enjoy the beauty of the game, the snap and feel of real stones, and meeting interesting people. Hope to see ya there! お願いします!

Anthony

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #194 Posted: Thu Apr 26, 2018 1:08 pm 
Tengen

Posts: 4676
Liked others: 0
Was liked: 633
goTony wrote:
But if it is an internet game how does one confirm there is computer assistance?


forum/viewtopic.php?p=228864#p228864

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #195 Posted: Thu Apr 26, 2018 2:54 pm 
Lives in gote

Posts: 323
Location: Washington State
Liked others: 241
Was liked: 58
Rank: KGS13kyu
KGS: gotony
OGS: nghtstalker
RobertJasiek wrote:
goTony wrote:
But if it is an internet game how does one confirm there is computer assistance?


forum/viewtopic.php?p=228864#p228864


Reading over some of the posts in this topic it seems that we expect someone to use a computer for much of the game. All it takes to get an unfair advantage by cheating is to use it once in a difficult position to come up with a move or combination that you would not come up with or verify your thoughts. It seems with the technology here now that serious online tournaments will have to become a thing of the past. Because of the difficulty of monitoring the integrity of the players.

_________________
Walla Walla GO Club -(on FB)

We play because we enjoy the beauty of the game, the snap and feel of real stones, and meeting interesting people. Hope to see ya there! お願いします!

Anthony


This post by goTony was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #196 Posted: Thu Apr 26, 2018 9:57 pm 
Judan

Posts: 7812
Liked others: 2143
Was liked: 2742
goTony wrote:
Reading over some of the posts in this topic it seems that we expect someone to use a computer for much of the game.


It turns out that that has been, and still is, a pattern of cheating in chess. You have players whose every move is, according to top chess engines, neither a blunder nor a mistake, and who play only a few inaccuracies per game. Their rating never topped 2200 and suddenly they are playing like super grandmasters.

Such moves will typically be among the top three choices of any given chess engine, which is where I suppose the match one of the top three choices heuristic came from.

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #197 Posted: Fri May 04, 2018 2:35 pm 
Dies in gote

Posts: 34
Liked others: 0
Was liked: 6
Rank: FFG 5k
If i am not wrong Carlo just won against a european 6d in World Amateur GO Championship.
He was table 3 of Italy, now represent his country in WAGC and win against a 6d.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #198 Posted: Fri May 04, 2018 10:06 pm 
Judan

Posts: 5287
Location: Cambridge, UK
Liked others: 295
Was liked: 2776
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Fenring wrote:
If i am not wrong Carlo just won against a european 6d in World Amateur GO Championship.
He was table 3 of Italy, now represent his country in WAGC and win against a 6d.

Yes, he beat Ben DG 6d, the current French champion, a good win. Ben also lost to a 4d from Thailand so maybe he's jetlagged ;-). I will be following Carlo in the thread about the WAGC. (maybe people missed it in the usually quiet Tournaments subforum)

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #199 Posted: Sat May 05, 2018 12:34 am 
Dies in gote

Posts: 34
Liked others: 0
Was liked: 6
Rank: FFG 5k
Benjamin just say he missed the first round because the alarm clock was not powerful enough.
I didnt missed the tournaments forum,but think its related to this topic too.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #200 Posted: Sat May 05, 2018 4:38 pm 
Beginner

Posts: 1
Liked others: 0
Was liked: 0
Rank: 4dan
I think bannishing Carlo was mistake.

I like to play with Leela and i learn a lot with it.
When i am playing with Leela i use analysis windows to choose one of the best move. But it is my choice not Leela choice.

Playing Go is expression of my spirit, sometimes i prefer to play moves that is slightly under Leela best choice, because i want to be human not superbot.

So i think Carlo is an artist, and we have to let him express his art with Leela tools.

Carlo is much more in advance than normal go players, he already understand that human is nothing with no AI

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 720 posts ]  Go to page Previous  1 ... 7, 8, 9, 10, 11, 12, 13 ... 36  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: Bing [Bot] and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group