It is currently Wed Dec 11, 2019 9:05 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 720 posts ]  Go to page Previous  1 ... 29, 30, 31, 32, 33, 34, 35, 36  Next
Author Message
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #621 Posted: Mon Jun 18, 2018 10:32 pm 
Lives with ko

Posts: 141
Liked others: 26
Was liked: 89
Rank: 5 dan
Bill Spight wrote:
Actually, focusing on the top choice is the most sensitive measure of agreement with Leela. Adding other choices makes the statistics look impressive, but also makes them less sensitive.

Edit: Yes, I know that Frejlak is not making a statistical argument. :)

This sensitivity might be a problem if we compare it to nothing. If we compare it to other game in same criteria, it is not.

Also, focusing just on A and numbers has another drawback - A is most common forced move - in one analyzed game there was more than 20% of such moves.

Edit: I also did not make statistical argument.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #622 Posted: Mon Jun 18, 2018 10:37 pm 
Dies in gote

Posts: 45
Liked others: 31
Was liked: 18
Rank: 2kyu
RobertJasiek wrote:
Gobang wrote:
the difference between 4d and 6d+ must be as clear as day and night


As I have explained in https://lifein19x19.com/forum/viewtopic ... 62#p232862 , it is not clear at all for only one game or a few games. It is only clear for many played games.


I don't agree, and I am old enough to know that something is not necessarily true, just because someone thinks he has "explained" it.


This post by Gobang was liked by: AlesCieply
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #623 Posted: Mon Jun 18, 2018 10:55 pm 
Lives in gote
User avatar

Posts: 636
Liked others: 51
Was liked: 226
Winning statistics are available on the website http://www.europeangodatabase.eu/EGD/winning_stats.php

Between Jan. 2003 and June 2018,

between a 1d and a 5d, the 1d won 6.5 % of the time.
between a 2d and a 6d, the 2d won 3.7 % of the time.
between a 3d and a 7d, the 3d won 1.5 % of the time.


This post by jlt was liked by 4 people: Bill Spight, Charlie, daal, dsatkas
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #624 Posted: Mon Jun 18, 2018 10:56 pm 
Judan

Posts: 6236
Location: Cambridge, UK
Liked others: 356
Was liked: 3378
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
There is an easy way to test the claim of people who say it is easy to distinguish a 4d and 6d. Someone picks some 4/6d games, removes identifying information and shows them to these people. They then have to say which players were 4d and which 6d. How many correct will they get. This thread could do with less talking and more testing.

maf's site https://kyudan.net is a good way to demonstrate your skill (or lack thereof) of judging player ranks from play. I look forward to seeing some familiar usernames at the top of the leaderboard with average rank guess error less than 2.

Edit: I made this test at viewtopic.php?f=10&t=15832


This post by Uberdude was liked by 2 people: Bill Spight, Charlie
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #625 Posted: Mon Jun 18, 2018 11:30 pm 
Honinbo

Posts: 9206
Liked others: 2819
Was liked: 3107
Bojanic wrote:
Bill Spight wrote:
Actually, focusing on the top choice is the most sensitive measure of agreement with Leela. Adding other choices makes the statistics look impressive, but also makes them less sensitive.

Edit: Yes, I know that Frejlak is not making a statistical argument. :)

This sensitivity might be a problem if we compare it to nothing. If we compare it to other game in same criteria, it is not.

Also, focusing just on A and numbers has another drawback - A is most common forced move - in one analyzed game there was more than 20% of such moves.


Actually, that's a plus, not a drawback. If the move is forced, the player and Leela should almost always agree. Counting B or C as a match when the move is forced is problematic.

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

"Once in a very great while his eyes light up for a moment, and he says "Whee!" very quietly."
— Lion Miller

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #626 Posted: Mon Jun 18, 2018 11:45 pm 
Lives with ko

Posts: 141
Liked others: 26
Was liked: 89
Rank: 5 dan
Bill,
I think that in Metta-Kim game is a B forced move. Player can answer on two similar places.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #627 Posted: Tue Jun 19, 2018 12:42 am 
Lives in gote
User avatar

Posts: 308
Location: Deutschland
Liked others: 268
Was liked: 126
Rank: EGF 4 kyu
jlt wrote:
Winning statistics are available on the website http://www.europeangodatabase.eu/EGD/winning_stats.php

Between Jan. 2003 and June 2018...


To save everyone, here, the effort of the long query-time over at the EGD:

Out of 4505 even games in the same time period, a weaker 4-dan player defeated their stronger 6-dan opponent 14.4% of the time.

Anyone who thinks that 14.4% is "rare" or argues that a sample of 4505 data is anything but statistically significant needs to encounter a clue at high velocity. I suppose one could argue that the data are skewed or biased or downright erroneous but doing so would constitute an own-goal: it would render the ranks meaningless and the whole question of whether a 4-dan victory against a 6-dan opponent is exceptional or not would become moot.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #628 Posted: Tue Jun 19, 2018 1:00 am 
Judan

Posts: 6236
Location: Cambridge, UK
Liked others: 356
Was liked: 3378
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Charlie, I think it is fair to point out that if a 4d plays a 6d in a McMahon tournament then (with some dependence on the level of the bar) there is a bias in that the 4d was more likely in good form and the 6d poor form for them to be drawn together. So maybe an average 4d beats an average 6d only 10% not 14% but I agree neither of those are rare.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #629 Posted: Tue Jun 19, 2018 1:24 am 
Lives in gote
User avatar

Posts: 308
Location: Deutschland
Liked others: 268
Was liked: 126
Rank: EGF 4 kyu
Uberdude wrote:
Charlie, I think it is fair to point out that if a 4d plays a 6d in a McMahon tournament then (with some dependence on the level of the bar) there is a bias in that the 4d was more likely in good form and the 6d poor form for them to be drawn together. So maybe an average 4d beats an average 6d only 10% not 14% but I agree neither of those are rare.


You're right about the bias induced by McMahon but, without digressing too far down this tangent, I'd still argue that 14.4% is correct because our definitions of "4-dan" and "6-dan" are based on the same data -- mostly from McMahon tournaments, surely. In fact, the McMahon draws would also be based on that data.

Perhaps it is true that a "true 4-dan" defeats a "true 6-dan" less than 14.4% of the time but to test that hypothesis, we would have to record decades of data from tournaments that use a mode that does not induce a bias. Within such an "ideal" system, however, both Metta and Ben-David may end up with different ranks.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #630 Posted: Tue Jun 19, 2018 2:20 am 
Judan

Posts: 6236
Location: Cambridge, UK
Liked others: 356
Was liked: 3378
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
On the "It's easy to tell a 4 dan from a 6 dan" issue, I made a little test here, can you figure out the players' ranks? viewtopic.php?f=10&t=15832.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #631 Posted: Tue Jun 19, 2018 2:40 am 
Lives in sente

Posts: 1259
Liked others: 102
Was liked: 265
Lukan wrote:

Javaness (#638):
Yes, this discussion is running in circles a lot, I have already expressed that like 400 posts ago. Especially many useless stats and numbers are being presented here again and again. Shouldn't we keep this mainly as a Go-case and listen more to strong players' Go-feelings? I feel like that is being mostly ignored here.


I don't see much new in this thread for the last few days. I guess everyone made their mind up already. So now either:
  • stage 3 of the appeals process is invoked by Israel - via EGF Appeals(?) Committee, we already had stages 1 and 2.
  • UK team, or Chris Bryant directly, launches a new appeal based on suspicions of cheating with LeelaZero.
  • talk endlessly here until a revolutionary system to detect cheating appears. I don't think it will appear quickly.

_________________
North Lecale


This post by Javaness2 was liked by: AlesCieply
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #632 Posted: Tue Jun 19, 2018 2:43 am 
Judan

Posts: 6236
Location: Cambridge, UK
Liked others: 356
Was liked: 3378
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Lukan wrote:
Especially many useless stats and numbers are being presented here again and again. Shouldn't we keep this mainly as a Go-case and listen more to strong players' Go-feelings? I feel like that is being mostly ignored here.

I don't think stats are necessarily useless, they have some advantages (objective, can be automated and fast for many games), but they are difficult to develop and use well. I agree that analysis by strong players is a useful tool, particularly given our go stats analysis is just beginning and a baby compared to chess, but I think it's very important to make the test fair and blind: i.e. don't just show a strong player like yourself 3 of Carlo's online games and 3 of his offline and asks "Is there a big difference in the play? Do you think he cheated", but show you 3 games online and 3 offline each of players A, B, C and D (where one is Carlo and the others are random other people of similar strength with games from similar conditions) and then ask if you think any of them have the suspicious big difference. If you and say 2 other strong players all picked out the same blinded player as Carlo being the suspicious one then that is much stronger evidence. A problem is a lot of top European players probably followed the case so can recognise these games of Carlo, maybe we could ask some pros in Asia to be the judges.


This post by Uberdude was liked by: AlesCieply
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #633 Posted: Tue Jun 19, 2018 2:50 am 
Lives with ko

Posts: 141
Liked others: 26
Was liked: 89
Rank: 5 dan
I have read (again, this time in details) Italian appeal.
Google found it here:
http://frmor.net/downloads/statistical_ ... _David.pdf

My comments:
- as other statistical analysis, it is focused mainly on, ehrr, statistics, and less on go game.
- question of spent time and time for analysis in Leela:
I already wrote here on about it. Spent time in the game cannot be same as used in program. First it is necessary to manually input moves into Leela, and later to client. Player could be out of place for some reason, or distracted, and some time could be wasted. Also, sometimes spent time on analysis can be even longer than recorded, because analysis could be done in opponent's time.
- high percentage similarities of other player's games is not relevant. Those player's games should be compared to their live games.
- it is completely omitted that some moves could be forced, some expected, others could be important, and to analyze longer sequences of related moves - basically player's side of story
- Only positive thing is that it showed some of the lacking of original analysis, but it suffers from same lacking.
- two more recent live games were not analyzed, but hey were not played at the time.

Edit, most important: in this paper are listed two Metta's games from EGC 2017. I would be very interested to take a look at them.


Last edited by Bojanic on Tue Jun 19, 2018 3:05 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #634 Posted: Tue Jun 19, 2018 3:05 am 
Dies in gote

Posts: 65
Liked others: 31
Was liked: 55
Bojanic wrote:
I have read (again, this time in details) Italian appeal.
Google found it here:
http://frmor.net/downloads/statistical_ ... _David.pdf

This is only a part of the Italian appeal. The document submitted to the PGETC appeals committee contained more of it. Specifically, a section about the outstanding ethical profile of Carlo Metta is missing here and there are some other parts missing as well. I do not think it would be proper if I shared the complete document on this forum but would welcome if the Italians did, so anyone can make a judgement for himself.


This post by AlesCieply was liked by 2 people: Bojanic, Hidoshito
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #635 Posted: Tue Jun 19, 2018 3:09 am 
Dies in gote

Posts: 65
Liked others: 31
Was liked: 55
Uberdude wrote:
On the "It's easy to tell a 4 dan from a 6 dan" issue, I made a little test here, can you figure out the players' ranks? https://lifein19x19.com/viewtopic.php?f=10&t=15832.

The problem with this kind of test is that it is not the same as playing a game and than judging the strength of your opponent. For me, it is also hard to tell whether the game was played by a 4d or 6d. I am a 1d player and I think I could more realistically estimate whether the player is 2k or 3d. Going away from my rank I guess my estimates become much less reliable.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #636 Posted: Tue Jun 19, 2018 3:17 am 
Dies in gote

Posts: 65
Liked others: 31
Was liked: 55
Charlie wrote:
jlt wrote:
Out of 4505 even games in the same time period, a weaker 4-dan player defeated their stronger 6-dan opponent 14.4% of the time.

Is the statistics based on grades declared by players or on ratings? Is a player with GoR=2449 counted as 4d? How is the statistics affected by games when strong 4d beats weak 6d (this relates to the point raised by Uberdude)? It would be more proper to see statistics on how often the players with ratings between 2350 and 2450 beat opponents about 200 rating points higher.

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #637 Posted: Tue Jun 19, 2018 4:12 am 
Honinbo

Posts: 9206
Liked others: 2819
Was liked: 3107
Lukan wrote:
Shouldn't we keep this mainly as a Go-case and listen more to strong players' Go-feelings? I feel like that is being mostly ignored here.


I, for one, would be glad to do so if we had any evidence that strong go players were good at detecting cheating by their feelings. We have no such evidence. Unfortunately.

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

"Once in a very great while his eyes light up for a moment, and he says "Whee!" very quietly."
— Lion Miller


This post by Bill Spight was liked by: zermelo
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #638 Posted: Tue Jun 19, 2018 4:15 am 
Judan

Posts: 6236
Location: Cambridge, UK
Liked others: 356
Was liked: 3378
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
<moderator>
As several have noted this is a rather long/popular/confusing/messy/repetitive thread. To try to make it easier to follow I will split off the anonymous accuser discussion to another thread, and possibly other sub-threads too if they can be nicely extracted. And if you want to start a new thread on a sub-topic instead of cluttering this one please feel free!
</moderator>


This post by Uberdude was liked by 2 people: Bill Spight, Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #639 Posted: Tue Jun 19, 2018 4:26 am 
Honinbo

Posts: 9206
Liked others: 2819
Was liked: 3107
Bojanic wrote:
Bill,
I think that in Metta-Kim game is a B forced move. Player can answer on two similar places.


As I said, there should almost always be a match with A.

But your point about forced moves vs. unforced moves gets us back to the point about which we agree, and which Regan also gets at. We need to distinguish the difficulty level of plays.

_________________
The Adkins Principle:

At some point, doesn't thinking have to go on?

— Winona Adkins

"Once in a very great while his eyes light up for a moment, and he says "Whee!" very quietly."
— Lion Miller

Top
 Profile  
 
Offline
 Post subject: Re: “Decision: case of using computer assistance in League A
Post #640 Posted: Tue Jun 19, 2018 5:27 am 
Dies in gote

Posts: 31
Liked others: 3
Was liked: 7
Rank: 5 kyu
Lukan wrote:
Shouldn't we keep this mainly as a Go-case and listen more to strong players' Go-feelings? I feel like that is being mostly ignored here.


Of course you were referring to Ke Jie's and Park Junghwan's Go feelings and not to those of laughably weak European "strong" players, right?

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 720 posts ]  Go to page Previous  1 ... 29, 30, 31, 32, 33, 34, 35, 36  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group