It is currently Tue Dec 10, 2019 8:39 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 720 posts ]  Go to page Previous  1 ... 10, 11, 12, 13, 14, 15, 16 ... 36  Next
Author Message
Offline
 Post subject: Re:
Post #241 Posted: Fri Jun 01, 2018 1:25 am 
Lives in sente

Posts: 1259
Liked others: 102
Was liked: 265
EdLee wrote:
Hi Javaness,
Quote:
that Stewpot likes cruises
Noob question: what's this reference ?

A certain FIDE official

_________________
North Lecale

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #242 Posted: Fri Jun 01, 2018 4:29 am 
Judan

Posts: 6229
Location: Cambridge, UK
Liked others: 356
Was liked: 3375
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Lizzie (LeelaZero UI) version 0.5 was just released and has a new feature to bulk analyse the game (like go review partner) and give a winrate graph so I used this on a few games. It agrees that Carlo dominated Dragos, the big drop being the bulge and empty triangle connection. Middle line is 50%, black winning is at top. In the endgame LZ identifies a few of Carlo's mistakes but Dragos usually didn't punish them (e.g. Carlo plays a move which isn't sente enough so Dragos should make his own sente yose gain somewhere else but instead he just meekly answered) so these are little upward spikes in the graph.
Attachment:
Carlo vs Dragos winrate.PNG
Carlo vs Dragos winrate.PNG [ 273.77 KiB | Viewed 2856 times ]

When humans are getting smashed by stronger bots (e.g. Lee Sedol vs AlphaGo or Haylee vs LeelaZero) the winrate graph is often like this, a one-way descent to defeat, whereas human games are typically more of a to-and-fro affair with mistakes from both making the leader change throughout the game. Of course sometimes one human just dominates the other and is in control the whole game, so whilst a descending graph like this could arouse suspicion it is far from proof that a bot was used (particularly when the good moves played by the winning player are not those suggested by the bot he is suspected of cheating with, as with the move 74 sequence).

Here are somewhat counterexamples of me beating/doing well against a stronger player. First vs Victor Chow 6/7d in the online league in Jan 2015 (so it should be easy to believe I didn't cheat with Leela as it didn't exist then!). At the end it thinks white won but that's because LZ assumes 7.5 komi but the league is 6.5 and I won by 0.5.
Attachment:
Andrew vs Victor.PNG
Andrew vs Victor.PNG [ 338.35 KiB | Viewed 2871 times ]

Or me vs Kim Seongjin 7d at the London Open 1.5 years ago. I was white so doing fairly well, see if you can guess where I went into byo-yomi after my 90 minutes main time (he'd used 10 minutes)!
Attachment:
Andrew vs Seongjin winrate.PNG
Andrew vs Seongjin winrate.PNG [ 270.27 KiB | Viewed 2871 times ]

The "98%" game vs Reem was interesting, as many commentators though Carlo was leading from fairly early so could play simply to win (Reem too soft in surrounding lower side moyo said Stanislaw), and that could explain the high similarity metric. Actually LZ thinks the lead switched a few times in middlegame and it wasn't a domination like vs Dragos. One of Carlo's moves it is most critical of was the empty triangle connection for move 117, instead of the wedge tesuji which Leela 0.11 also found (but his move still counted as a top 3 match as it was #3 and not >5% worse) .
Attachment:
Carlo vs Reem LZ winrate.PNG
Carlo vs Reem LZ winrate.PNG [ 342.8 KiB | Viewed 2871 times ]

Lukan mentioned the game vs Csaba (round after vs Reem) looked like a different player. LZ (#145, 500 playouts/move for the graph) agrees that it was a much more to-and-fro game than vs Dragos, and a quite a bit more than vs Reem. Csaba was black and won by 2.5 so the 7.5 vs 6.5 komi difference will be a distortion near the end. I'm planning to do a top 3 Leela (0.11) similarity analysis on it later (mostly for interest, we already know this is a poor metric to use as evidence of cheating).
Attachment:
Carlo vs Csaba winrate.PNG
Carlo vs Csaba winrate.PNG [ 282.76 KiB | Viewed 2793 times ]


[Edit: added missing graphs]


This post by Uberdude was liked by 2 people: Charlie, Shenoute
Top
 Profile  
 
Online
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #243 Posted: Fri Jun 01, 2018 4:52 am 
Honinbo

Posts: 9193
Liked others: 2813
Was liked: 3101
Uberdude wrote:
The "98%" game vs Reem was interesting, as many commentators though Carlo was leading from fairly early so could play simply to win (Reem too soft in surrounding lower side moyo said Stanislaw), and that could explain the high similarity metric.


That's similar to what I said, but not exactly. I did think that Carlo took an early lead, and that a lot of the matching moves were simple, or part of one lane road sequences. Therefore, many of the matches with Leela did not mean much, if anything.

Quote:
Actually LZ thinks the lead switched a few times in middlegame and it wasn't a domination like vs Dragos. One of Carlo's moves it is most critical of was the empty triangle connection for move 117, instead of the wedge tesuji which Leela 0.11 also found (but his move still counted as a top 3 match as it was #3 and not >5% worse) .
Attachment:
Carlo vs Reem LZ winrate.PNG


I guess this graph suggests that cheating by picking one of Leela's top three moves is not such a great strategy. :lol:

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

"Once in a very great while his eyes light up for a moment, and he says "Whee!" very quietly."
— Lion Miller

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #244 Posted: Sun Jun 03, 2018 11:17 am 
Judan

Posts: 6229
Location: Cambridge, UK
Liked others: 356
Was liked: 3375
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
I took a look at the Dragos (white) vs Mijodrag game with LeelaZero/Lizzie, it was rather interesting both from this cheating case and a simple studying go perspective. Mijodrag's current rating is now 3d, but he used to be 5d so should have some stronger ideas but probably lost sharpness over the years as many older players have. Still, a bad loss for Dragos so weakens the "Dragos is super strong 6d so highly unlikely a 3/4d could beat him without cheating" idea (that's not to say the manner of the win was more dominating in Carlo's case). As Lukan said Dragos's o15 was a massive blunder (-59%) and he would be winning with the net, but LZ also thought black messed up earlier in the fight so could have been winning if he played it better. Here's the game and win graph; replaying the game quickly I didn't have a feeling for who was winning as both played such a boring territory style and I couldn't be bothered to count, but LZ actually thought the general trend was upwards towards a black lead. However, there was a lot of zigzags, and that's because at nearly every move for quite some stretches of the game LZ thought both players were missing the urgent point concerning the strength of the white wall on the right, q11 for black and p12 for white (I checked a few times with LeelaElf and it agrees). So everytime black doesn't play q11 his winrate drops a load, because it expects white to play the best move at p12, and then when Dragos doesn't black's win goes back up by about the same amount. It's like when you are watching DDKs playing and both fail to live/kill a 50 point group, but on a higher level.



Attachment:
Dragos vs Mijodrag.PNG
Dragos vs Mijodrag.PNG [ 445.66 KiB | Viewed 2626 times ]


To hammer home how important LZ considers this area, here's the times it thinks it's better than the move they played (better move with a 1, game move marked with X):
Black kicked (a joseki) and white descended, LZ says should q11, if kick white should punish with hane and if black blocks then (crude?) atari and p12, so black should ignore hane to q11 and allow white poke into corner.
Click Here To Show Diagram Code
[go]$$Bc -5%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . , X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . 1 X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O . . |
$$ | . . O , X X . . . X . . . X . , Z . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


This m4 even I could tell was slack (if you want to play there LZ considers n5), but with q11 you need not fear m4 invasion as wall weak.
Click Here To Show Diagram Code
[go]$$Bc -7%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . , X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . 1 X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . Z . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


White approached, LZ says knight move key point.
Click Here To Show Diagram Code
[go]$$Wc -7%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . P . . . . |
$$ | . . . O . . . . . , . . . . . , X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . 1 . . . . |
$$ | . . . . . . . . . . . . . . . . X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


Black pincer, should push.
Click Here To Show Diagram Code
[go]$$Bc -9%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . O . . . . |
$$ | . . . O . . . . . , . Z . . . , X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . 1 X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


When white ignored pincer to make top left shimari (-12%) although initially LZ said p12 was best for white it then decided q15 press was best. Then when black split side q11 push still best.
Click Here To Show Diagram Code
[go]$$Bc -16%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . . . . . . O . . . . |
$$ | . . . O . . . . . , . X . . . , X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . Z . . . . . . . . . . . . 1 X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


White played checking extensions, should knight move or q15 press (very similar).
Click Here To Show Diagram Code
[go]$$Wc -17%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . . . . . . O . . . . |
$$ | . . . O . . . . . , . X . . . , X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . 1 . . . . |
$$ | . . X . . . . . . . . . . . . . X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . P . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


Black extended/approached to make base, push still more important (it thinks white shoulder answer push, if white ignored push to c14 pincer on left then black simply o11 jumps to attack wall and grow top left and is better.
Click Here To Show Diagram Code
[go]$$Bc -11%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . . . . . . O . . . . |
$$ | . . . O . . . . . , . X . . . , X . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . Z . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . 1 X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . O . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


White kicked, should knight or press still.
Click Here To Show Diagram Code
[go]$$Wc -8%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . . . . . . O . . . . |
$$ | . . . O . . . . . , . X . . . , X . . |
$$ | . . P . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . 1 . . . . |
$$ | . . X . . . . . . . . . . . . . X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . O . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


Black extended, a natural and locally very urgent move, but even now LZ says push is bigger and doesn't mind if white tigers on top and black gets to o11 jump.
Click Here To Show Diagram Code
[go]$$Bc -10%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . . . . . . O . . . . |
$$ | . . . O . . . . . , . X . . . , X . . |
$$ | . . O . . . . . . . . . . . . . . . . |
$$ | . . X Z . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . 1 X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . O . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


I'm repeating myself...
Click Here To Show Diagram Code
[go]$$Wc -13%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . P . . . . O . . . . |
$$ | . . . O . . . . . , . X . . . , X . . |
$$ | . . O . . . . . . . . . . . . . . . . |
$$ | . . X X . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . 1 . . . . |
$$ | . . X . . . . . . . . . . . . . X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . O . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


Ditto.
Click Here To Show Diagram Code
[go]$$Bc -14%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . O . . . . O . . . . |
$$ | . . . O . . . . . , . X . . . , X . . |
$$ | . . O . . Z . . . . . . . . . . . . . |
$$ | . . X X . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . 1 X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . O . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


White answered, in fact at this time (probably because of k17 now there) LZ thinks press is 0.3% better than the knight move, but both massively better than bump white played.
Click Here To Show Diagram Code
[go]$$Wc -16%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . O . . . . O . . . . |
$$ | . . . O . P . . . , . X . . . , X . . |
$$ | . . O . . X . . . . . . . . . 1 . . . |
$$ | . . X X . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . . X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . O . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


Again we see a local contact situation (like kick extend) so respond seems urgent, but big picture q11 push still much more important. (After extend it again slightly prefers for white press to knight move, both 12% better than o16 kosumi played).
Click Here To Show Diagram Code
[go]$$Bc -14%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . O . . . . O . . . . |
$$ | . . . O . O . . . , . X . . . , X . . |
$$ | . . O . . X Z . . . . . . . . . . . . |
$$ | . . X X . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . 1 X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . O . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


B answered the o16 kosumi with this knight, still should push, because...
Click Here To Show Diagram Code
[go]$$Bc -9%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . O . . . . O . . . . |
$$ | . . . O . O . . . , . X . O . , X . . |
$$ | . . O . . X X . . . . . . . . . . . . |
$$ | . . X X . . . . . . . . . . . Z . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X . . . . . . . . . . . . 1 X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . O . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


White should still p12, now it doubles as the nozomi shape I'd previously noticed AlphaGo likes to play.
Click Here To Show Diagram Code
[go]$$Wc -3%
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . O . . . O . . . . O . P . . |
$$ | . . . O . O . . . , . X . O . , X . . |
$$ | . . O . . X X . . . . . . . . . . . . |
$$ | . . X X . . . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . 1 . . . . |
$$ | . . X . . . . . . . . . . . . . X . . |
$$ | . . . , . . . . . , . . . . . O . . . |
$$ | . . O . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . . O X . . |
$$ | . . . . . . . . . . . . . . O X X . . |
$$ | . . O . . . . . . . . . . . O . . . . |
$$ | . . . . . . . . . . . . . . O . O O . |
$$ | . . O , X X . . . X . X . X . , X . . |
$$ | . . . O O X . . . . . . . . . X . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]


Once that top right shape settles the urgency of the q11/p12 points drops. However, there is a similar zigzag later in which it thinks white should be playing the n9 shape-making point and black should be playing the o8 shape-destroying peep which they both repeatedly miss.


This post by Uberdude was liked by: Bill Spight
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #245 Posted: Sun Jun 03, 2018 1:43 pm 
Dies in gote

Posts: 54
Liked others: 48
Was liked: 7
Rank: EGF 5k
DGS: 1k
Universal go server handle: dsatkas
just noticed, Carlo did pretty well in the world amateur tournamnent beating stronger opponents, perhaps food for thought?

http://europeangodatabase.eu/EGD/Tourna ... n=14713996

Top
 Profile  
 
Online
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #246 Posted: Sun Jun 03, 2018 1:49 pm 
Honinbo

Posts: 9193
Liked others: 2813
Was liked: 3101
Uberdude wrote:
there was a lot of zigzags, and that's because at nearly every move for quite some stretches of the game LZ thought both players were missing the urgent point concerning the strength of the white wall on the right, q11 for black and p12 for white (I checked a few times with LeelaElf and it agrees).


One nice thing about playing over ancient games was discovering that, even when play was generally quite territorial, top players often played for central strength. I think that even 300 years ago top players would have found Black Q-11 or White P-12. :)

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

"Once in a very great while his eyes light up for a moment, and he says "Whee!" very quietly."
— Lion Miller

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #247 Posted: Sun Jun 03, 2018 2:03 pm 
Gosei

Posts: 1432
Liked others: 722
Was liked: 475
Rank: AGA 3k KGS 1k Fox 1d
GD Posts: 61
KGS: dfan
dsatkas wrote:
just noticed, Carlo did pretty well in the world amateur tournamnent beating stronger opponents, perhaps food for thought?

http://europeangodatabase.eu/EGD/Tourna ... n=14713996

For what it's worth, the win against France was a forfeit.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #248 Posted: Mon Jun 04, 2018 7:19 am 
Beginner

Posts: 12
Liked others: 0
Was liked: 8
Rank: 1k
IGS: 2d
Hello, after 13 pages of posts over this argument, I think the
statistical analysis of the "method" used to accuse Carlo Metta can
help. You can find it here: http://frmor.net/misc.html. Hoping it
stops this unhelpful loosing of time, with kind regards, Mirco.


This post by MircoF was liked by 6 people: Bill Spight, Bonobo, Charlie, HermanHiddema, Uberdude, zermelo
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #249 Posted: Mon Jun 04, 2018 11:42 am 
Lives in sente
User avatar

Posts: 836
Liked others: 180
Was liked: 150
Rank: 3d
GD Posts: 422
KGS: komi
That's a pretty compelling document, and my interpretation is that Carlo is owed a huge apology from the go community.

It's very readable and amazingly clear, given how advanced the statistics involved are.

My only concern is that the author is very close to Carlo (his PhD adviser).

It would definitely be good to get some independent corroboration/replication of the results posted in the document.


This post by quantumf was liked by 2 people: Charlie, HermanHiddema
Top
 Profile  
 
Online
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #250 Posted: Mon Jun 04, 2018 1:21 pm 
Honinbo

Posts: 9193
Liked others: 2813
Was liked: 3101
I found the following paragraph to be of interest:

Francesco Morandin wrote:
We analysed another number that Leela provides: the probability that a strong amateur player would play this move. This number is called network probability. We put this in a table [12], where the choices of Metta are underlined and in red. We found that Metta’s choices were 49 out of 50 times in the human top 3 (and 46 out of 50 times in the human top 2). The only time his choice was “not human” (move 121) was a mistake according to Leela (Leela ranked this move in position 7 after indefinite time).


In my earlier criticism I pointed out that similarity (confirmatory evidence) is very weak evidence, and that contrast (disconfirmatory evidence) was necessary. I was unaware that Leela herself provided contrasting evidence, the contrast being between Leela's choices and the choices of strong amateurs. Picking one of Leela's choices which is not one of the human choices would be evidence of cheating. (Such evidence seems to be more available in chess, where top engines often play non-human moves.) There are other ways of finding disconfirmatory evidence, but they involve analyzing a large number of games. Here was disconfirmatory evidence that Leela put on the plate. However, that evidence was nil. There was no case where Metta picked a choice of Leela's that was not also a human choice.

To be sure, other evidence of cheating might have been developed, but was not.

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

"Once in a very great while his eyes light up for a moment, and he says "Whee!" very quietly."
— Lion Miller

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #251 Posted: Mon Jun 04, 2018 2:01 pm 
Lives in gote
User avatar

Posts: 634
Liked others: 51
Was liked: 224
MircoF wrote:
I think the statistical analysis of the "method" used to accuse Carlo Metta can help. You can find it here: http://frmor.net/misc.html.

The document says:
Francesco Morandin, page 2 wrote:
The average similarity from those 105 runs was 93.6%

I am surprised that Uberdude found 98% in his (single) test, whereas Francesco Morandin found 98% in only 2 runs out of 105.
Uberdude wrote:
I independently analysed the game to find the Leela similarity metric: I got 98% for Carlo (aka 49 out of 50 matched) and 80% for his opponent Reem (aka 40 out of 50 matched)

Anyway, the whole discussion in this thread shows that evidence of cheating is very weak. In post #234, it was observed that the sequence that a strong player found the most remarkable in the most remarkable win of Carlo Metta during PETGC was actually not suggested by Leela.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #252 Posted: Mon Jun 04, 2018 2:25 pm 
Gosei

Posts: 1354
Location: Earth
Liked others: 511
Was liked: 218
I do not make an argument for or against evidence of cheating with this post.

I just want to mention the fact, that a cheating player can choose to not play the highest win probability moves, but just moves recommended by the network probablity.

(So moves similar to network probability are no argument that no cheating happened.)


Last edited by Gomoto on Mon Jun 04, 2018 2:50 pm, edited 1 time in total.
Top
 Profile  
 
Online
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #253 Posted: Mon Jun 04, 2018 2:28 pm 
Honinbo

Posts: 9193
Liked others: 2813
Was liked: 3101
Gomoto wrote:
I do not make a argument for or against evidence of cheating with this post.

I just want to mention the fact, that a cheating player can choose to not play the highest win probability moves, but just moves recommended by the network probablity.

(So moves similar to network probability are no argument that no cheating happened.)


True. But that is not the claim. The claim, which bears the burden of proof, is that cheating happened.

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

"Once in a very great while his eyes light up for a moment, and he says "Whee!" very quietly."
— Lion Miller


This post by Bill Spight was liked by 2 people: dfan, Gomoto
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #254 Posted: Mon Jun 04, 2018 11:04 pm 
Judan

Posts: 6229
Location: Cambridge, UK
Liked others: 356
Was liked: 3375
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
jlt wrote:
MircoF wrote:
I think the statistical analysis of the "method" used to accuse Carlo Metta can help. You can find it here: http://frmor.net/misc.html.

The document says:
Francesco Morandin, page 2 wrote:
The average similarity from those 105 runs was 93.6%

I am surprised that Uberdude found 98% in his (single) test, whereas Francesco Morandin found 98% in only 2 runs out of 105.
Uberdude wrote:
I independently analysed the game to find the Leela similarity metric: I got 98% for Carlo (aka 49 out of 50 matched) and 80% for his opponent Reem (aka 40 out of 50 matched)


Me too. I included the excel spreadsheet I used earlier on the thread for analysis/verification. I could check again but I think it may be explained by my analysis being with 20-30k rather than 50k playouts (which I only settled on in later runs) and Leela starts off liking the high network probability human moves and switches to better less human moves with more playouts. Also I didn't record the winrate from top deviation in the spreadsheet to start so might have been a bit sloppy there in including some moves as matches which should have been rejected by the 5% clause. Plus as Francesco noted by doing the analysis manually there is the possibility for the human operator (even unintentionally) to introduce some bias in letting the sims run a bit longer/shorter than intended to make a move a match or not.

jlt wrote:
Anyway, the whole discussion in this thread shows that evidence of cheating is very weak. In post #234, it was observed that the sequence that a strong player found the most remarkable in the most remarkable win of Carlo Metta during PETGC was actually not suggested by Leela.

Now that I have a PC with gpu I let Leela analyse for 2.5m sims and q15 did rise from choice #18 to #5. So Leela did start to think about it more but was still far off local moves around the cutting stone (net and p7 peep top iirc).

Btw, this evening (7pm GMT) I'll be playing 1st board in the UK vs Italy league A/B promotion/demotion match against Matias Pankoke 5d. Italy have already won 2 boards, including Carlo beating Chris (our strongest but not highest ranked player), of the 5-game match so it'll be tough with us outranked on all 3 boards...


This post by Uberdude was liked by: Charlie
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #255 Posted: Tue Jun 05, 2018 2:10 am 
Lives with ko

Posts: 141
Liked others: 26
Was liked: 89
Rank: 5 dan
Hello,

I have read almost entire discussion, and i would like to point out to some facts that seems to be omitted here.

- In A league are strong amateur players, who are also very intelligent.
- results, and superficial analysis, can confirm that players accused of cheating did not use illegal software in all games.
- if we assume that some player decides to cheat, he will probably do it in a way not to arouse too much suspicion. After all, those are players with lot of playing experience, and, as we mentioned, are very intelligent.
- Conclusion would be that player who wants to cheat, will do it only in games where he would not arouse suspicion, i.e. 100 rating points around him.
- Also, since all players are strong players, they can use Leela only in critical moments, and continue to play by themselves after achieving advantage.
- Monitoring Leela for position analysis and some variations could help player to maintain enough lead.

Therefore, with minimal assistance, player can gain enough assistance to win a game.
And worst of all - it would be almost impossible to prove without a doubt if someone used help.


Analysis given in this topic focus on entire game (after opening, of course). There is a lot of moves that Leela could play, and lot of moves that human would play.
Taking into account points given above, it is not surprising at all. It would be surprising is someone used program for entire game, which would be idiotic to say the least.

-----

We must make several observations:
- After appearance of strong programs, PGETC management did nothing to prevent their usage.
- After suspicion that some players used it, very little investigation was done.
In current situation, regularity of games played on internet cannot be guaranteed.

After preliminary analysis of all games, I am afraid that even more players use computer assistance in a manner that would be difficult to prove.

-----

PS for those who are more versatile in analyzing, could you take a look at this game:
http://pandanet-igs.com/system/sgfs/627 ... 1496176456
Especially white's mistake.


This post by Bojanic was liked by 3 people: AlesCieply, Bill Spight, Javaness2
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #256 Posted: Tue Jun 05, 2018 3:12 am 
Oza
User avatar

Posts: 2165
Location: Germany
Liked others: 7961
Was liked: 896
Rank: OGS SDK
Universal go server handle: trohde
»New evidence for re-opening the Carlo Metta case«:

https://psv4.userapi.com/c848124/u63838 ... idence.pdf


(again, I don’t understand any of this)

_________________
“The only difference between me and a madman is that I’m not mad.” — Salvador Dali ★ Play a slooooow correspondence game with me on OGS? :)

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #257 Posted: Tue Jun 05, 2018 3:46 am 
Dies in gote

Posts: 65
Liked others: 31
Was liked: 55
Hello, I guess you know I was involved in dealing with the case as a member of the PGETC appeals committee. Since about that time I started to look into the matter also on my own trying to devise a better and statistically more sound method to check if someone used AI in internet games or not. My analysis is based on comparing the player performance in internet and live games. While working on it I found what I believe is a new evidence and last Friday I informed the EGF executive and involved parties. As it looks it is becoming a public knowledge you may better have it directly from me, the supplied document is here,
https://drive.google.com/file/d/1NaWwHx ... sp=sharing

The analysis itself evolved a bit since then (in particular, the Kulkov-Metta PGETC game was added), the current version is here
https://docs.google.com/spreadsheets/d/ ... sp=sharing
I would very much appreciate if it was reviewed and checked by others, I am also open to any critic how to improve it or what mistakes you find in it. I still do not consider it as an end product. I would like to add few more games and make comparison with games analyzed for different players. I would also like to repeat the analysis for some sequences where Leela might not be sufficiently precise or consistent. However, the work on it is rather slow and tedious. Feel free to contribute to it.

On the first sheet you also find a Pearson's chi-square test to compare the compatibility of the two histograms, for Carlo's internet and regular games. I am not an expert on statistics but I was told that the p-value represents the probability that the two sets could be results for the same population. In our case it means that the probability of both sets of games being played by one person is now about 0.0001 (=0.01%).

Finally, I would very much appreciate if Carlo Metta came out and explained why he presented an apparently fabricated game record to the league manager. I do believe he is in principle an honest man who has done a lot for the go community and can continue to do so. I just think he made a mistake with using AI in his internet games and now is afraid of admitting it.
EDIT: Here I refer to a game record from the Shakhov-Metta game Carlo himself suplied (among several other records) claiming it was played at regular tournament and contained also many moves "similar to Leela". In fact, the game was played at KGS and the record was edited to look as played "live", see the report for more details on it.


Last edited by AlesCieply on Tue Jun 05, 2018 6:25 am, edited 1 time in total.

This post by AlesCieply was liked by 3 people: Bill Spight, Hidoshito, zermelo
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #258 Posted: Tue Jun 05, 2018 4:23 am 
Lives in sente

Posts: 1259
Liked others: 102
Was liked: 265
I think that the first point in your summary is quite debatable right now.
Quote:
Carlo Metta’s performance in the first 7 PGETC league games was so exceptional that such a feat may occur[e] once in about 3000 tournaments.

The basic idea, that he did very well in this year's PGETC is of course relevant as an initial starting point. However, the winning percentages from Go Rating are probably not very reliable. Thus the figure you quote (1/3000) is probably best left out. Andrew Simon's already mentioned two players with similar 'super' performances this year.

Has it really not been already the subject of analysis to look at every internet tournament he played; show performance rating there, relative to his offline rating performances? Especially in light of this third point, I would say that Carlo's overall performance in the KGS event - I see it is http://www.europeangodatabase.eu/EGD/To ... n=16762284 (but there is also) http://www.europeangodatabase.eu/EGD/To ... y=T171018A - would be interesting.

Coming back to Bojanic's point, it is hard to believe smart guys are going to deliberately disguise their internet games. So I imagine that there is some explanation there.

_________________
North Lecale


Last edited by Javaness2 on Tue Jun 05, 2018 4:46 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #259 Posted: Tue Jun 05, 2018 4:32 am 
Lives in gote

Posts: 450
Liked others: 1
Was liked: 128
Rank: KGS 2k
GD Posts: 100
KGS: Tryss
A possible source of biais is that you're comparing games he won with games he mostly lost.

The game against Vasquez is closer to the online games than the other regular games, but that is also the one he won.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #260 Posted: Tue Jun 05, 2018 4:50 am 
Lives in gote
User avatar

Posts: 308
Location: Deutschland
Liked others: 268
Was liked: 126
Rank: EGF 4 kyu
AlesCieply wrote:
Finally, I would very much appreciate if Carlo Metta came out and explained why he presented an apparently fabricated game record to the league manager.


Just to be clear: the allegedly fabricated game record is *not* the kifu from the game that raised accusations of cheating but another game, between Carlo Metta and Kim Shakhov. You really should be specific in this instance.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 720 posts ]  Go to page Previous  1 ... 10, 11, 12, 13, 14, 15, 16 ... 36  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: EdLee and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group