Life In 19x19
http://lifein19x19.com/

“Decision: case of using computer assistance in League A”
http://lifein19x19.com/viewtopic.php?f=10&t=15538		 Page 26 of 36
Author:  Bill Spight [ Sat Jun 16, 2018 2:09 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
jlt wrote:
Bojanic wrote:
If I am not mistaken, in PGETC sgf files there is a time stamp?

This could be number in brackets, in the end, indicating remaining time:

From Metta-Ben David game
http://pandanet-igs.com/system/sgfs/637 ... 1511906173


I checked thinking time of Black for moves 51-149. He spent:

  • 2 seconds on move 51
  • 3 seconds on moves 73, 95, 137
  • 4 seconds on moves 63, 93
  • 5 seconds on moves 91, 109

About "tenuki moves" 51, 59, 65, 87, 97, 101: times spent were 2, 12, 8, 11, 56, and 8 seconds.


Botvinnik pointed out years ago that time spent on a move is an indicator of subective difficulty. (He recommended studying positions where you took a lot of time.) On that basis :b97: stands out as a difficult move for Metta. To me that makes sense. Is it kikashi or aji keshi? Not so easy to judge.

Conversely, the times taken indicate that :b91:, :b93:, and :b95: are part of a subjective unit for Metta. As is Black 109 (a different unit, OC ;)).

This suggests that time taken could be taken as an indicator of subjective significance for Metta.
Author:  Charlie [ Sat Jun 16, 2018 2:17 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Bojanic wrote:
Why do you think it can not be done quickly?
While waiting for opponent's move, you position your cursor in other program, ...


Of course, it can be done quickly. No doubt. As a StarCraft player, I'd say that the real-time latency induced by relaying the moves to Leela would be measured in milliseconds and so the only cost, as measured on the wall-clock, would be Leela's thinking time.

But what of the mental cost in concentration?

Hypothetically, if one was playing in this way -- blitz-clicking moves between two programmes -- and also being careful to sample within Leela's top-3 suggestions in order to disguise their cheating, would they really be sufficiently engrossed in the game to find their own, good moves? I think not.

I think someone could randomly deviate and chose to ignore Leela but not with any great level of skill. Not at that speed.

Of course, I am not a 4-6 dan player, myself. I also manage to misclick often enough even without cheating and needing to replicate the game in another program. Maybe rapid relaying is a skill that can be learned through practice.
Author:  Bill Spight [ Sat Jun 16, 2018 2:26 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Bojanic wrote:
Bill Spight wrote:
In this plan it seems to me that there are four significant Black choices: moves 85, 87, 97, and 101. It also seems to me that this plan, and these plays are well within the competence of a European 4 dan.

Estimating strength of the moves is very subjective, and furthermore, amateur players can be strong in one part of the game, and weak in another.
In this case, we have simpler job - there was only one AI program available and strong enough at the time, and, surprisingly, most of Metta's moves were same as program. Now, Metta could played some of the same moves - but all of them, in same sequence?


I think you underestimate the difficulty of the task. Not to belabor the issue, I have written a lot about that in this thread. Also, having identified the significant plays those are the plays to focus on, not the other plays in the sequence. Regarding all plays in the sequence equally was one failing of the statistical method used to reach the original verdict. I thought that we agreed on that. The only moves that we both consider significant are 87, 97, and 101. Those are the three plays to look at in the range being considered.

Bojanic wrote:
Bill Spight wrote:
I don't know whether you agree with me about Black 105 and 111, but this demonstrates how human judges could reach agreement about which plays are significant. :)

I partially agree - those moves are significant endgame moves. Since there is no strategic aspect in them, they carry less weight than middle game moves.
I think endgame should be analyzed separately. Also problem with endgame in some cases is that due to large difference it is not so important what to play.


Those moves may be considered to be yose, but they are not endgame plays. Check out John Fairbairn's writing about the difference between the two. Yose can occur even in the opening. :)
Author:  Bill Spight [ Sat Jun 16, 2018 2:35 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Charlie wrote:
ut what of the mental cost in concentration?

Hypothetically, if one was playing in this way -- blitz-clicking moves between two programmes -- and also being careful to sample within Leela's top-3 suggestions in order to disguise their cheating, would they really be sufficiently engrossed in the game to find their own, good moves? I think not.


An important point. That is why the mistake that Metta made that was Leela's top choice was an important piece of evidence that Bojanic discovered. :) Left to his own devices, Metta might well have found the right play.

Edit: I don't mean that I think that Metta was not left to his own devices.
Author:  Bill Spight [ Sat Jun 16, 2018 2:43 am ]
Post subject: 
Gobang wrote:
In this thread there is talk of a "we" who should come up with an anti cheating system. Who exactly are these "we"? Random people in an internet thread? What expertise do these "we" have"? What resources? How much time will "we" devote to this task? Will "we" be reimbursed by somebody? If "we" come up with a system, will anybody use it? How can it be ensured that "we" do not simply come up with a system that launders cheating, (as in pro cycling)?


Broadly speaking "we" are the community of serious go players. I expect that the Chinese pros are working on this problem without paying any attention to Western amateurs. ;)

When I was president of the New Mexico Go Association, "we" was me. I was able to enlist some help running tournaments and so on, but if I did not do the work, nobody did. I also helped with the writing of a tournament director's guide for the AGA. In that case, "we" was Terry Benson, myself, and whoever else he managed the recruit.

As a practical matter, "we" is whoever answers the call.
Author:  Jan.van.Rongen [ Sat Jun 16, 2018 2:46 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Bojanic wrote:
Why do you think it can not be done quickly?


Because I tried a year ago in many games on KGS. Look at this rank graph on KGS:

http://www.gokgs.com/graphPage.jsp?user=mrooijer

Those are all games against bots with 5-15 seconds per move. 5 seconds was very tight. I lost some of those games on time.

Your reasoning that it is possible to play faster is invalid, not because one cannot play faster, but because that assumes that Black was playing fast because he cheated.
Author:  Bill Spight [ Sat Jun 16, 2018 3:10 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Bojanic wrote:
move 97 – low suggestion, although sente move. For this move, move 101 was A suggestion from 10k - 56 seconds - it is strange in analysis and in duration, maybe some kind of disturbance, or getting up?


Was :b97: the one move of the 50 moves originally considered that did not match Leela's top three choices?
Author:  Jan.van.Rongen [ Sat Jun 16, 2018 3:11 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
AlesCieply wrote:
On the computer Carlo Metta might have used in his PGETC games. In the Italian appeal they specify what computer they performed their counter-analysis on: Intel Core i7, 2.60 GHZ, RAM 16GB, GPU NVIDIA GeForce GTX 960M. Operating system - Windows 10. They also say there that it analyses about 100k nodes in about 30s. I asked the question what computer Carlo used in his PGETC games and the answer I got (from Mirco Fanti, the Italian team captain, as he insisted any questions should not be asked Carlo directly but should go through him) that it was the one used in the analysis. I conclude from this that most (in not all) of the Italian counter-analysis was done by Carlo himself.


That conclusion is not justified. Carlos is a PHD student of AI, and it is logical that he uses a laptop with a GPU. The best buy (about 1000 euro) would be the configuration mentioned above: an 8 thread Intel i7 CPU, 16 GB and this mobile version of the GTX 960. I chose the same configuration for the same purpose, but bought it a year later when the 960 was replaced by the 1050 / 1060.

This configuration allows us to build and train medium sized neural networks, f.i. with Google's Tensorflow library. For larger networks one could use cloud computing - I even think that I saw Google giving away training time on their tensorflow GPU's that trained AlphaZero.

Anayway, that Carlos and his PhD supervisor have the same config for their laptops is quite likely.

Another point to note is that the laptop screen is not big enough to view Leela 0.11 and a Go server client at the same time. You need to toggle between three windows: Leela 0.11 board, Leela 0.11 Analysis and the (Pandanet) client. That takes time.

Thus the fact that Carlos played a lot of his answering moves almost instantly is evidence in favour of not cheating. For two reasons: (a) they are probably logical moves and the fact that Leela also recommends them is not releveant; and (b) that there is no time to switch windows and look at the Leela recommendations.
Author:  Bojanic [ Sat Jun 16, 2018 3:21 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Jan.van.Rongen wrote:
Your reasoning that it is possible to play faster is invalid, not because one cannot play faster, but because that assumes that Black was playing fast because he cheated.

Jan, I did not bring question of time, it is not even in my work.
It was brought here is argument that some of the moves could not be found in such short time. As you can see from data I posted, it is possible, but I dont think it is accurate.

Regarding motorics, it varies greatly among men. In KPMC 2006, korean playerwould wait calmly until 29, and then pick a stone, played a moveand pusha clock in less than second.

PS not all his moves were fast.
Author:  Jan.van.Rongen [ Sat Jun 16, 2018 3:29 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Bill Spight wrote:
Was :b97: the one move of the 50 moves originally considered that did not match Leela's top three choices?


I do not have that original analysis, but in my runs with Leela 0.11 it was not always recommended in the top 3. Note that the original method is not stable: there is some randomness in the behaviour of Leela 0.11

In this case Leela 0.11 thinks that K13 and N14 are the best moves, better than M6 or L7
Author:  Javaness2 [ Sat Jun 16, 2018 3:33 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Bojanic wrote:
- and now, even internet bots, who attack people who think differently.


If they are CM, I'd respect their words more if they said so directly
Author:  Javaness2 [ Sat Jun 16, 2018 3:39 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Jan.van.Rongen wrote:

Another point to note is that the laptop screen is not big enough to view Leela 0.11 and a Go server client at the same time. You need to toggle between three windows: Leela 0.11 board, Leela 0.11 Analysis and the (Pandanet) client. That takes time.


Isn't it possible just to have either extra screens, or more than 1 laptop?

Another tangent - can you easily train a network (well just make a program) to help you cheat?
All you would surely need to do would be to feed it two sets of input data. Okay, maybe it is beyond the scope of this thread.
Author:  Bojanic [ Sat Jun 16, 2018 3:40 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Jan.van.Rongen wrote:
Another point to note is that the laptop screen is not big enough to view Leela 0.11 and a Go server client at the same time. You need to toggle between three windows: Leela 0.11 board, Leela 0.11 Analysis and the (Pandanet) client. That takes time.

Jan,
There is no need for 3 windows and toggle. In Leela you can simply press f2 and activate analysis, which would show in red best moves.
Also, two boards is easily possible on any monitor.
Author:  John Fairbairn [ Sat Jun 16, 2018 3:44 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Quote:
How can it be ensured that "we" do not simply come up with a system that launders cheating, (as in pro cycling)?


I'm not sure that analogy applies.

I sympathise, because when I was young I very much enjoyed watching athletics on tv. What turned me against it (strongly) were the use of pace-makers and the use of drugs. It was a gradual process and there were other factors, such as the introduction of high-tech tracks and shoes which (then) favoured white guys from rich countries. At that time, my objections were simply about the disruption of a level playing field.

But since then it has become apparent that drugs are very dangerous in themselves, and that sporting stars who use them can be role models for youngsters - a possibly greater danger.

In contrast, in go (and chess), while there may be the occasional incident involving drugs such as beta blockers, there is no sense that contests are generally decided by who has the best pharmacist. Cheating in go and chess may be immoral (is in my view) but it is not yet endangering anyone.

If we go to the other extreme, we will no doubt see in the next few weeks many, many examples of the professional foul in the World Cup football matches. At worst the player will get a red card. But countless more times we will see deliberate minor trips and pushes and time wasting that just elicit a peep of the referee's whistle. Objectively these are surely all forms of cheating, but for many decades we have tolerated this behaviour. Indeed, there are sports such as ice hockey and pro wrestling where fans might stop going if there were no spectacular fouls and punch-ups.

Go seems to be somewhere in the middle of this spectrum. Until real and present danger presents itself, no organisation will spend big money on the problem. But cheating in go seems immoral enough to activate some ordinary fans to try to put an anti-cheating framework in place. It won't be very elaborate because there won't be any big money behind it. If pros act, it won't be because they want to help amateurs. It will be because they are terrified of losing sponsorship. Either way it will be a rickety framework, but not (intentionally, at least) a "laundry."

So in that situation, talking about the problem here is maybe the only way forward for amateurs. The discussion may seem tedious and too rumbustious to some, but it seems, in my memory, to pale into insignificance compared to the past discussions of other forms of cheating on the servers: escaping and sandbagging.
Author:  Elom [ Sat Jun 16, 2018 4:05 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
John Fairbairn wrote:
...

The discussion may seem tedious and too rumbustious to some, but it seems, in my memory, to pale into insignificance compared to the past discussions of other forms of cheating on the servers: escaping and sandbagging.


Seconded.
Author:  Charlie [ Sat Jun 16, 2018 4:44 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
John Fairbairn wrote:
... but it is not yet endangering anyone.


Hey. All this blitz-clicking and switching between windows can give you some nasty RSI.
Author:  bugsti [ Sat Jun 16, 2018 8:22 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Bojanic wrote:

This is biggest scandal in European Go I can remember of (and I play for 30 years).
It has everything:
- cheating in important game,
- referee in most important tournament involved,
- political pressure to influence referees,
- and now, even internet bots, who attack people who think differently.



Political pressure to influence referees? :shock:

Please explain this very serious sentence and prove it otherwise you only look like a paranoid conspirator.
Author:  Bojanic [ Sat Jun 16, 2018 8:40 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
bugsti wrote:
Political pressure to influence referees? :shock:

Please explain this very serious sentence and prove it otherwise you only look like a paranoid conspirator.

As I was told, one of the italian organizers told one of the EGF officials that if they don't reverse decision on Metta's suspension, they would not be able to organize EGC. After that, and dubious statistical analysis „not 98% but 93%“, decision was reversed.
Carlo Metta is one of the main organizers of EGC, and is supposed to be be main referee.

Somewhere here or on fb was published reply to Cieply's analysis from EGF official, claiming that further analysis cannot be done because we can not disturb Italians in EGC organization.
And I got same reply on my preliminary analysis, which only prompted me to do more work.

I hope that this makes things more clear.
Author:  maf [ Sat Jun 16, 2018 9:30 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
I've wondered about this. Why did Mr. Metta not step down voluntarily? Regardless of the truth about the alleged cheating, putting him in this position only calls for more trouble, doesn't it? It seems sensible to simply avoid this altogether. Was there an announcement in this regard?
Author:  Jan.van.Rongen [ Sat Jun 16, 2018 9:46 am ]
Post subject:  Re: “Decision: case of using computer assistance in League A
Bojanic wrote:
....
There is no need for 3 windows and toggle. In Leela you can simply press f2 and activate analysis, which would show in red best moves.
Also, two boards is easily possible on any monitor.


You are mistaken. F2 toggles the analysis MODE, but does not bring the analysisi window to the forefront when it is already hidden behind another window.

Bojanic wrote:
....
This is biggest scandal in European Go I can remember of (and I play for 30 years).
It has everything:
- cheating in important game,
- referee in most important tournament involved,
....


Are you serious? This is completely over the top.
Page 26 of 36 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/