jlt wrote: I am not talking about kyu players here. Bojanic (5d) as well as other strong players say that it is easy to see from a game if a player is 4d or 6d. In addition, Bojanic can use computer tools to make more accurate analyses. Other people like Uberdude (4d) and Robert Jasiek (5d) think that it is not possible to judge from a single game or from a small number of games.
Actually, I think strong players can do a fairly good job in judging the strength of similar players from their moves (and I was actually impressed how well my wife did on the test, with caveats noted there), but indeed suspect they may not be quite as good as they think, particularly in the case of a 4d playing well or a 6d playing badly (note for my test I didn't look at the games before choosing them so have no idea if any of the 4ds played well or 6ds badly), and small samples are always problematic. I like evidence, and will happily update my views on its production. Lukan (7d) said he'd give it a go, I hope he does.
Gobang wrote: (I had the nerve a critical comment about it and was slammed for my "negativity").
As the someone who "slammed" Gobang, I'd call that more of a counter-throw than a slam of my initiation as I said "I disagree, though I think your constant repetition of negativity is a waste of time." in response to "So all this is is a waste of time, just like 99% of the babble around the topic of detecting online cheats.". That was said out of frustration of his repeated calls to throw our hands in the air, give up on detecting or preventing cheating (at least attempting in an imperfect way) and just cancel the entire PGETC league (plus demeaning the participants as "kids"). This is despite admitting he is new to serious/tournament Go and the accounts from various people actually involved of how much they value the league (e.g. dsatkas in Greece, quantumf in SA, Simba and me in UK).
Gobang wrote: It is also questionable to construct this test with online games where there is no way of verifying who was in fact playing.
As for the possibility of the actual player not being the named one, yes it is non-zero and in the ~4500 games played in the history of the league I think it likely some may be so, but the chance it happened in the 14 cases I picked pretty slim, more so because they are from higher leagues so who exactly would the replacement be? Most of the top players of the countries involved participate so either it's: 1) another player on the team, who isn't playing at the same time and they all collude to keep the cheating secret, 2) some secret strong player from their country not in the league or unknown to the Go community, are there many of these? 3) some pro or strong player in Asia?. Also by being from the same event they have the same time controls, seriousness etc (and were easy for me to obtain). If someone else could collect game records from e.g. the EGC and we judge those too it would be interesting, maybe a 4d in 1 hour PGETC game online is generally weaker / plays worse / judged lower than a 4d at the 2.5 hour EGC? How about at the WAGC? Or faster KPMC? Or your average 1 hour game from a 3-a-day McMahon (last game worse from tiredness perhaps?).
Java's proposed test is also an interesting one, I hope someone conducts it (but I'm rather busy atm, and won't be able to work on my extension of Ales's mistake analysis for some time; my hope is that will give typical profiles for 4 and 6 dans, be better at identifying them than humans, and by knowing their variance we can answer questions like how unlikely is it a 4d plays as well as a 6d, or did Dragos play particularly poorly in that game vs Carlo particularly well). A lot of consideration in this thread has been on false positives, false negatives are also important to test for (but perhaps less so if we consider punishing an innocent worse than not punishing a guilty).
Edit: As Ales said, the 4 or 6 dan test isn't really relevant to Bojanic's analysis, it was prompted by the "I'm a strong player and I looked at the game and there's no way that's a 4 dan (even on a good day)" type arguments (which has a side premise of "and we don't think Carlo is really a 6 dan based on the WAGC").
Edit 2:
Gobang wrote: For this 6d or 4d test to make any sense, then it should be done in the context that it was created. A 6d player played an entire serious game with someone who is allegedly 4d, (but most probably just acting a bot for Leela). The 6d player said that it felt nothing like playing against a 4d.
Then someone decided to construct a "test", apparently for the purpose of showing that this 6d may not be a reliable judge of whether his opponent was 4d or stronger. My perception is that someone, with the intention of calling the 6d player's judgement into doubt created this "test".
My prompt wasn't just the Simba vs Carlo game, but also using the views of strong players looking at other past games of Carlo to decide whether he cheated instead of statistical Leela similarity approaches (e.g. suggested by Lukan). Gobang makes the distinction between the actual person playing the game (Simba most recently, we've not heard from Reem\Dragos\Kulkov etc) vs an observer and that they will be better at detecting the opponent's strength or if they cheated. I agree playing is different to watching, but it's not clear to me the player is a better judge: in Javaness's test we could ask the opponent of the sometimes-cheater as well as observers to identify the cheating games. But also if we are trying to distinguish "4d cheating with Leela beating 6d" and "4d not cheating getting that expected 1 in 10 win against a 6d" we'd need lots more than 5 games. Also I would like to cheekily point out the EGF rank of the 6d mentioned is 3d (though I believe he is at least 5d).
Edit 3:
Gobang wrote: Getting kyu players to decide if a 6d or 4d was playing, just by looking at the games is obviously absurd.
I'd say fun but irrelevant:
Uberdude wrote: What threshold should we take as demonstrating the truth of "It easy to tell the difference between a 4 dan and 6 dan"? (for sufficiently strong players, weaker players might like to play this game for fun/interest but them being bad at it doesn't show 6 or 7ds couldn't be good at it, I hope some strong players participate).
|
