Questions about a game

[bugsti]

Analysis window of Leela.
Please note that on bottom bar before W50 was played you can see L17 suggestion almost immediately.

Move50.jpg

Therefore, black had enough time, and your "firm evidence" is wrong.

My Leela 0.11 suggests L17 as third choice after M17 (1st) and E13 (2nd).

Is there any chance that Leela 0.11 has some hardware optimization? If I am right its code is not open source.

[Bojanic]

Analysis window of Leela.
Please note that on bottom bar before W50 was played you can see L17 suggestion almost immediately.

Move50.jpg

Therefore, black had enough time, and your "firm evidence" is wrong.

My Leela 0.11 suggests L17 as third choice after M17 (1st) and E13 (2nd).

First, we are discussing if this move could be known in advance. And it could.
Second, choice of moves varies during analysis. Now it was there from the start, then it fell out for some time, and when I checked it was there at 50k.

[Jan.van.Rongen]

It is wrong to use Leela in analysis of case where it is suspected that it was used?
Interesting reasoning. What is next, to use some random player's games instead of Metta's?

As I said before, your analysis is sloppoy and in this case you just confirmed what I wrote.

I said that "It is an error to use Leela 0.11 to evaluate a situation when you investigate cheating with the same program because that will easily lead to circular reasoning." It seems you just dont understand that or don't read carefully enough. I was mentioning evaluation and that has nothing to do with the case.

The second part of your remark is clearly meant as an insult. Oh well, I am immune to that.

Analysis window of Leela.
Please note that on bottom bar before W50 was played you can see L17 suggestion almost immediately.
...

Your "analysis window" is not an analysis window that a cheater would have used because it uses "no ponder". And (2) 3000- someting evaluation that your window shows is nothing on the GTX 960M that Carlos has. That's a fraction of a second on that machine and then it immediately continues to better alternatives.


Then about my AQ diagram

...
And this is the diagram of game Metta-Ben David in Leela 0.11, which shows remarkably small differences.

No it does not. In the AQ evalaution Black gains a plus after white 70, gains more and then gives it away again. About 5% gain is left. In your diagram, the Leela evaluation it is still at 15% at that point.

So thats why you should not rely on the evaluation function of that 1 AI. This diagram shows that maybe B had an advatage, but it was not big at move 124. Just the same 5% he had becuse of the bad white 70.

Now, which cheater would take his score up 15% and then down 15% again within 40 moves? This is very much evidence agaisnt cheating.

And then, how can a game that see-saws like this raise suspicion of cheating? Are high dan players really capable of positional evaluation to detect cheating? Why and when did Fragman think something was fishy?

[bugsti]

Now, which cheater would take his score up 15% and then down 15% again within 40 moves? This is very much evidence agaisnt cheating.

And then, how can a game that see-saws like this raise suspicion of cheating? Are high dan players really capable of positional evaluation to detect cheating? Why and when did Fragman think something was fishy?

As explained by someone here or in other threads, they fill a complain against that particular game beacuse that was the game with the most similarity rate according to a measure took by some self-called "experts". So maybe it was just an excuse for presenting a formal appeal by one of the team captain.

[AlesCieply]

Now, which cheater would take his score up 15% and then down 15% again within 40 moves? This is very much evidence agaisnt cheating.

Maybe because Carlo used Leela and not the AQ that came out in March 2018? According to Leela there are not so big jumps over those 40 moves, so Carlo could not have realized that his winrate goes up and down that much. He might also not used Leela for all his moves.

[bugsti]

Maybe because Carlo used Leela and not the AQ that came out in March 2018? According to Leela there are not so big jumps over those 40 moves, so Carlo could not have realized that his winrate goes up and down that much. He might also not used Leela for all his moves.

AQ was first released on 17 Semptember 2017, second release 15 October 2017, third release 22 January 2018.

I think it was already super strong since the 1st release.

Maybe you are confusing with the android version of AQ. That is more recent, 2 months ago.

[Bill Spight]

Analysis window of Leela.
Please note that on bottom bar before W50 was played you can see L17 suggestion almost immediately.

Move50.jpg

Therefore, black had enough time, and your "firm evidence" is wrong.

My Leela 0.11 suggests L17 as third choice after M17 (1st) and E13 (2nd).

Is there any chance that Leela 0.11 has some hardware optimization? If I am right its code is not open source.

Aren't the moves shown possible moves for White, not Black?

[Bojanic]

Aren't the moves shown possible moves for White, not Black?

With shades of red are marked possible moves for white.
On bottom bar, on left, you can see sequence of moves for best suggestion:
Nodes... Win.... PV: G1, L17, J14, J11...

[Bojanic]

Analysis window of Leela.
Please note that on bottom bar before W50 was played you can see L17 suggestion almost immediately.
...

Your "analysis window" is not an analysis window that a cheater would have used because it uses "no ponder". And (2) 3000- someting evaluation that your window shows is nothing on the GTX 960M that Carlos has. That's a fraction of a second on that machine and then it immediately continues to better alternatives.

This window is exactly what cheater could use.
In Leela, open empty SGF file. You can put or remove moves at your will and analyze them.
It is better option than to simply play with Leela, since it gives you more liberty for analysis and to play own moves.

And more important:

the GTX 960M that Carlos has

Italian team has OFFICIALLY sent info on configuration Metta used to PGETC enquiry, and stated it is capable of 100k variations in 30seconds.
Now, since you obviously have info that Metta has stronger machine and that he used it, would you be so kind to forward this information to Ales Cieply?
To remind you, your information means that Italian team either lied or withheld information of this to Inquiry, which would be second time.

Then about my AQ diagram

...
And this is the diagram of game Metta-Ben David in Leela 0.11, which shows remarkably small differences.

No it does not. In the AQ evalaution Black gains a plus after white 70, gains more and then gives it away again. About 5% gain is left. In your diagram, the Leela evaluation it is still at 15% at that point.

Remarkably small differences to Leela's play, not to AQ.
Of course it will show bigger differences in other program.

PS could you put here link to AQ program?
It would be good idea to analyze other games with it.

[Javaness2]

I said that "It is an error to use Leela 0.11 to evaluate a situation when you investigate cheating with the same program because that will easily lead to circular reasoning." It seems you just dont understand that or don't read carefully enough. I was mentioning evaluation and that has nothing to do with the case.

When you say evaluate, what do you mean exactly? Are you talking only about judging who is ahead, or only about what the correct moves to play next are, or are you talking about both, or are you talking about something else?

I still think it would be nice to have a supercheater program, which takes your move, checks it against the recommendations of N bots, returns a goodness metric for your move, and suggest alternatives.
Example

Code: Select all

So far you cheated 3 times in 67 moves
SuperCheater rated K10 at 0.8, black winrate is 0.48
AQ suggests L11
Leela 0.11 suggests F3
Leela 0.10 suggest F3
Pachi suggests P4

I mean, this is really going to f--k up life for the cheating detectors.

[Bojanic]

Javaness,
it can not be done automatically.
I will put here game which looks positive to Leela (same deviations histogram), but on the second look...

[AlesCieply]

Maybe because Carlo used Leela and not the AQ that came out in March 2018? According to Leela there are not so big jumps over those 40 moves, so Carlo could not have realized that his winrate goes up and down that much. He might also not used Leela for all his moves.

AQ was first released on 17 Semptember 2017, second release 15 October 2017, third release 22 January 2018.

Thanks for the additional information. I am really not familiar with AQ, must have messed up somehow with the release date. I only noted JanvanRongen tags the diagram he posted as made in April this year, so I assumed the latest AQ version. Anyway, my point stands. As far as Carlo is using Leela he does not know that his winrate goes up and down that much over those 40 moves. There is up and down seen in the Leela analysis of the game I presented as a part of my New evidence report but not that drastic as AQ suggests. However, it is worth noticing that even my Leela immediately realizes that move 70 was a mistake (about 4%), in a reasonable agreement with the 5% observed in the AQ graph.

[AlesCieply]

Now, since you obviously have info that Metta has stronger machine and that he used it, would you be so kind to forward this information to Ales Cieply?

Ehm, I am no longer involved with EGF/PGETC dealings concerning the CM case. As far as I know, it was not re-opened despite my report. And I also feel too biased now so I already informed EGF that I want to remove myself from judging it provided it is re-opened and reaches the appeals committee again.

Concerning the computer Carlo used in his PGETC games we only know what we were told. It does not necessarily mean that Carlo really used it. He could have stayed longer at work and use a stronger PC there, only he knows.

[Javaness2]

Javaness,
it can not be done automatically.
I will put here game which looks positive to Leela (same deviations histogram), but on the second look...

Of course you need several computers in communication with each other for using multiple bots, but you can make a simple version with just 1 bot. It's off-topic here
On topic - here is AQ https://github.com/ymgaq/AQ
There were other choices, like Ray (?) https://github.com/zakki/Ray

[AlesCieply]

As explained by someone here or in other threads, they fill a complain against that particular game beacuse that was the game with the most similarity rate according to a measure took by some self-called "experts". So maybe it was just an excuse for presenting a formal appeal by one of the team captain.

As far as I remember the game was identified because an Israeli teammate looked at the game with Leela and observed almost 100% matching of CM's moves. It should be noted the game was played 1 day later than the other 3 games and its result had no bearing on the match outcome as the IL team was winning 3-0 by then. Thus, even if this one game was forfeited the match result would not change. The IL team was definitely not looking for an excuse to present a formal complaint.