Promoting the AGA Professional program

[xed_over]

This tournament is meant to divide 1&2 from 3-16...and that's what it does.

Not only that, but it also cleanly selects 3&4 for seeds, as well as 5&6 for alternates.

you'll have to get the TD who designed this system to explain it... he'll do that better than me.

[shapenaji]

I mentioned 'inadequate' because the first winner only plays with four other players and the second winner only plays with five (or four) other players. They are not well exposed to many other players. This system equivalents to single elimination albeit the two winners have to defeat some of their opponents twice.

The problem with your argument is this: If you failed to play either of the final qualifiers in direct competition, that means there were at least two other competitors in the field you have lost to. If you have already lost to two other competitors in the field before challenging the two players who have (thus far) proved to be strongest, it is very hard to make a legitimate claim you were one of the top two competitors in the tournament. One could perhaps reasonably argue that there is not enough information to distinguish the "1st place" qualifier (winner of the winners bracket] from the "2nd place" qualifier (winner of the loser's bracket), because there is a chance the 1 loss could occur to an opponent not faced by the champion...but that's not really the goal of this tournament. This tournament is meant to divide 1&2 from 3-16...and that's what it does.

Here's a thought experiment:

You have 16 players of identical strength, On any given day, one of them will beat another one with a 50-50 shot.

The double elimination tournament MUST produce a first and second place (by design).

But that doesn't mean that it's a particularly good test of relative strength.

In this case, a proper test should find all of the players tied. And this test is not sensitive to that.

[jts]

While we would want rating information to accurately capture a 50-50 situation, sometimes it's enough to know who played better in a particular game, or match, or tournament. The problem with, say, a ko isn't that the results aren't statistically robust, but rather that it gives no information about most pairs.

[hyperpape]

Here's a thought experiment:

You have 16 players of identical strength, On any given day, one of them will beat another one with a 50-50 shot.

The double elimination tournament MUST produce a first and second place (by design).

But that doesn't mean that it's a particularly good test of relative strength.

In this case, a proper test should find all of the players tied. And this test is not sensitive to that.

I'm gonna guess that there's no tournament that you can feasibly hold in a week that gives you reliable information about relative strength in a case where players are extremely close to 50-50. That's just the nature of coin flips--someone will get lucky.

That's not to say a better tournament couldn't be selected, but your thought experiment is really the worst case scenario, and I think any tournament design will choke on it.

[tapir]

Here's a thought experiment:

You have 16 players of identical strength, On any given day, one of them will beat another one with a 50-50 shot.

The thought experiment has a major flaw: With identical strength every result is justified. (And even the most elaborate tournament system can't produce a better pick than k.o., double k.o. or even a coin toss competition.)

Or in other words, the qualifier isn't intended to produce an evaluation of relative strength, but to select two players.

[yoyoma]

I mentioned 'inadequate' because the first winner only plays with four other players and the second winner only plays with five (or four) other players. They are not well exposed to many other players. This system equivalents to single elimination albeit the two winners have to defeat some of their opponents twice.

The problem with your argument is this: If you failed to play either of the final qualifiers in direct competition, that means there were at least two other competitors in the field you have lost to. If you have already lost to two other competitors in the field before challenging the two players who have (thus far) proved to be strongest, it is very hard to make a legitimate claim you were one of the top two competitors in the tournament. One could perhaps reasonably argue that there is not enough information to distinguish the "1st place" qualifier (winner of the winners bracket] from the "2nd place" qualifier (winner of the loser's bracket), because there is a chance the 1 loss could occur to an opponent not faced by the champion...but that's not really the goal of this tournament. This tournament is meant to divide 1&2 from 3-16...and that's what it does.

Here's a thought experiment:

You have 16 players of identical strength, On any given day, one of them will beat another one with a 50-50 shot.

The double elimination tournament MUST produce a first and second place (by design).

But that doesn't mean that it's a particularly good test of relative strength.

In this case, a proper test should find all of the players tied. And this test is not sensitive to that.

shapenaji, if it was your job to select 2 players to become pros from this hypothetical field of 16 players, how would you do it?

[shapenaji]

The thought experiment has a major flaw: With identical strength every result is justified. (And even the most elaborate tournament system can't produce a better pick than k.o., double k.o. or even a coin toss competition.)

Or in other words, the qualifier isn't intended to produce an evaluation of relative strength, but to select two players.

The point of the thought experiment was to illustrate that the knock out tournament says nothing about how the winners would have fared against the players they didn't play.

The most elaborate tournament system certainly CAN produce a better pick than ko or dko.
Simply, round robin, you may have to break ties, but the round robin gives you a very strong understanding of the field. (Technically, I think a double round robin, where each player plays each other player as both white and black, is the strongest though)

I don't think the best of 3 is particularly useful. There are considerations of playing style. Beating one person in 2 games may be considerably easier than beating 2 people in 2 games.

shapenaji, if it was your job to select 2 players to become pros from this hypothetical field of 16 players, how would you do it?

We have 11 games right now, I think we can find a way to fit in 4 more. 2 games a day for a week?

Basically, I think this tournament should be a big deal, and we shouldn't jump to the format that guarantees a winner in a shorter time.

[shapenaji]

I'm gonna guess that there's no tournament that you can feasibly hold in a week that gives you reliable information about relative strength in a case where players are extremely close to 50-50. That's just the nature of coin flips--someone will get lucky.

See above post, Round Robin, 2 games a day. The tournament would be finished in 8 days.

If you're willing to cram 3 games into each day (making the test into a real endurance challenge) you could do it in 5 days.

[tapir]

The point of the thought experiment was to illustrate that the knock out tournament says nothing about how the winners would have fared against the players they didn't play.

My point wasn't that you are necessarily wrong, but that your thought experiment simply doesn't work as you need no data at all to make a good selection, if the players are of the same strength.

I often see this kind of argument about tournaments when people seem to assume that the aim is the best measurement of relative strength, but usually this isn't the aim (take a look at the rating if you need this measurement). You want to determine a winner, based on performance (in the games played) and as long as not obviously inferior players get selected everyone is happy with the result. And of course nobody takes players serious, who complain, that they lost their games because they were paired against Peter (6d) instead of Hans (6d), although they technically could be right. Multiple rounds of an elimination tournament should be enough to eliminate weaker players, but whether someone eliminated would be a tenth of a stone stronger than the actual winner on a good day with sunshine doesn't matter at all. In fact, it adds drama to the whole thing.

[jts]

I'm gonna guess that there's no tournament that you can feasibly hold in a week that gives you reliable information about relative strength in a case where players are extremely close to 50-50. That's just the nature of coin flips--someone will get lucky.

See above post, Round Robin, 2 games a day. The tournament would be finished in 8 days.

If you're willing to cram 3 games into each day (making the test into a real endurance challenge) you could do it in 5 days.

Huh? Hyperpape's point is that anyone with no prior information about the strength of the players would be forced to draw the conclusion that the player who went 10-5 was significantly stronger than the player who went 5-10, but we know ex hypothesi that these two players are equally strong; their records in this tournament are statistical noise. Thus: "There's no tournament that you can feasibly hold in a week that gives you reliable information on relative strength..."

[shapenaji]

See above post, Round Robin, 2 games a day. The tournament would be finished in 8 days.

If you're willing to cram 3 games into each day (making the test into a real endurance challenge) you could do it in 5 days.

Huh? Hyperpape's point is that anyone with no prior information about the strength of the players would be forced to draw the conclusion that the player who went 10-5 was significantly stronger than the player who went 5-10, but we know ex hypothesi that these two players are equally strong; their records in this tournament are statistical noise. Thus: "There's no tournament that you can feasibly hold in a week that gives you reliable information on relative strength..."

The point is that the Round Robin has the capacity of showing that the players are very close in strength, whereas a KO does not. The point wasn't that the Round Robin works perfectly, just better.

[shapenaji]

The point of the thought experiment was to illustrate that the knock out tournament says nothing about how the winners would have fared against the players they didn't play.

My point wasn't that you are necessarily wrong, but that your thought experiment simply doesn't work as you need no data at all to make a good selection, if the players are of the same strength.

Here's the thing though, the thought experiment still holds as we move the ranks away from 50% gradually. Sure, if they're all exactly equal, we can take whatever player we like. But when the difference between players is small, we need as many games as we possibly can to test that.

I often see this kind of argument about tournaments when people seem to assume that the aim is the best measurement of relative strength, but usually this isn't the aim (take a look at the rating if you need this measurement). You want to determine a winner, based on performance (in the games played) and as long as not obviously inferior players get selected everyone is happy with the result. And of course nobody takes players serious, who complain, that they lost their games because they were paired against Peter (6d) instead of Hans (6d), although they technically could be right. Multiple rounds of an elimination tournament should be enough to eliminate weaker players, but whether someone eliminated would be a tenth of a stone stronger than the actual winner on a good day with sunshine doesn't matter at all. In fact, it adds drama to the whole thing.

The aim here IS the best measurement of relative strength though. We're after someone to represent us in international competition. I think we can fairly say that it's not the pro qualifier that should be the source of our excitement, but the professional career. The pro qualifier should be difficult, and accurate.

[Dazz]

I think we can fairly say that it's not the pro qualifier that should be the source of our excitement, but the professional career. The pro qualifier should be difficult, and accurate.

I think that a tournament system that was more difficult and accurate, would be more exciting. I would have enjoyed seeing a round robin tournament.

[jts]

The point of the thought experiment was to illustrate that the knock out tournament says nothing about how the winners would have fared against the players they didn't play.

My point wasn't that you are necessarily wrong, but that your thought experiment simply doesn't work as you need no data at all to make a good selection, if the players are of the same strength.

Here's the thing though, the thought experiment still holds as we move the ranks away from 50% gradually. Sure, if they're all exactly equal, we can take whatever player we like. But when the difference between players is small, we need as many games as we possibly can to test that.

Well, you are the one who suggested "recognizes when all players are equally strong" as a test of a tournament set up. But in fact (I think you now agree?), this is not something that distinguishes various tournament formats. In all cases, the only way to increase our chances of getting the strongest player in a tight field is to play more games.

As to tapir's comment; I think you've dodged his main point, which is "If you are going to be an extremist about accuracy, why not use official lifetime AGA rating as the tool for selecting pros?"

[shapenaji]

Well, you are the one who suggested "recognizes when all players are equally strong" as a test of a tournament set up.

Right, the point of the test was to illustrate that when players were closely rated, a knockout tournament is a poor judge of who is the best player.

How are we arguing about the fact that Round Robin is a better test of relative strength??

But in fact (I think you now agree?), this is not something that distinguishes various tournament formats. In all cases, the only way to increase our chances of getting the strongest player in a tight field is to play more games.

But this does distinguish formats. It is pathological in both cases, but the round robin is capable of allowing ties... So obviously, if the players are closely rated, it will do a better job of establishing this.

The Knockout tournament simply lacks depth perception.

As to tapir's comment; I think you've dodged his main point, which is "If you are going to be an extremist about accuracy, why not use official lifetime AGA rating as the tool for selecting pros?"

I don't think I'm being an extremist, I'm suggesting we follow the same pro format as Japan...

Because lifetime AGA rating is a test of past performance.


EDIT:

More generally, if someone is going to become a pro, they should need to fight against all the other opponents in the qualifier. Everybody gets a chance at the pinata.