Page 1 of 3
Round Robin: SODOS or Direct comparison?
Posted: Wed Jun 30, 2010 3:45 pm
by willemien
While studying Round robin tournaments (see other tread in this forum)
I discovered something curious
Sometimes Direct Comparison and SODOS give different results, and therefore the question arises What is the better tiebreaker?
Example a 6 players round robin tournament
Code: Select all
Round 1 A wins from B : C wins from D : E wins from F
and so on see table:
Opponents
a b c d e f Win Sodos dc
Player A - W L W W W 4 -
Player B L - W W L W 3 6 W
Player C W L - W W L 3 8 L
Player D L L L - W W 2 3 W
Player E L W L L - W 2 4 L
Player F L L W L L - 1 -
A is the clear winner with 4 wins
But who is second?
B won the game between B and C
But C has a higher Sodos(Sum Of Defeated Opponents Scores) so had his wins against bigger odds
so player B is the Direct comparison 2nd place
and player C is the Sodos 2nd place
Or more concrete
Does a win against the winner/ better player (A) count for less than a win betweem tied players (B & C) ?
A similar problem occurs between D and E and also here the same question is at stake.
My provisional personal opinion is that C should get the second place cup but i am willing to chance my mind with good arguments.
Re: Round Robin: SODOS or Direct comparison?
Posted: Wed Jun 30, 2010 4:32 pm
by Bill Spight
willemien wrote:While studying Round robin tournaments (see other tread in this forum)
I discovered something curious
Sometimes Direct Comparison and SODOS give different results, and therefore the question arises What is the better tiebreaker?
Example a 6 players round robin tournament
Code: Select all
Round 1 A wins from B : C wins from D : E wins from F
and so on see table:
Opponents
a b c d e f Win Sodos dc
Player A - W L W W W 4 -
Player B L - W W L W 3 6 W
Player C W L - W W L 3 8 L
Player D L L L - W W 2 3 W
Player E L W L L - W 2 4 L
Player F L L W L L - 1 -
A is the clear winner with 4 wins
But who is second?
B won the game between B and C
But C has a higher Sodos(Sum Of Defeated Opponents Scores) so had his wins against bigger odds
so player B is the Direct comparison 2nd place
and player C is the Sodos 2nd place
Or more concrete
Does a win against the winner/ better player (A) count for less than a win betweem tied players (B & C) ?
A similar problem occurs between D and E and also here the same question is at stake.
My provisional personal opinion is that C should get the second place cup but i am willing to chance my mind with good arguments.
To me the question is, given the results, who is more likely to win a game between B and C? (Assuming equal odds a priori.) You can approach this in Bayesian terms, OC.
But off of the top of my head, here are some thoughts.
Both B and C beat D, so D is irrelevant. Both form cycles with A and E. ( C>A>B>C>... and C>E>B>C>....) So maybe they are irrelevant, too. That leaves B>F>C. The fact that F "lies between" B and C bolsters the notion that B would win a rematch.
Edit: The fact the the bottom player (F) beat C but not B is what gives C a better SODOS score, isn't it?
Re: Round Robin: SODOS or Direct comparison?
Posted: Wed Jun 30, 2010 5:01 pm
by pwaldron
willemien wrote:Sometimes Direct Comparison and SODOS give different results, and therefore the question arises What is the better tiebreaker?
This question is known as the ranking problem and has been flogged many times over in the statistical literature. A variety of tiebreaking methods have been proposed, including SODOS, sometimes iterated to infinite order. Others have suggested ordering the players so that the tournament minimizes the number of upsets; in some cases this would place a player with 3 wins behind a player with 4 wins. As Bill suggests, Bayesian methods are also possible.
About the only method that hasn't been proposed is the direct confrontation because it relies on the result of a only single game.
Re: Round Robin: SODOS or Direct comparison?
Posted: Wed Jun 30, 2010 5:14 pm
by willemien
The fact the the bottom player (F) beat C but not B is what gives C a better SODOS score, isn't it?
No not really C is 2 sodod points ahead of B below where neither player loses against F
The cycles elimination always gives the same result as direct comparison.
( a list of C > ?? > B can always be extended to C > ?? > B > C and become a cycle so C can never win)
An example where neither loses to F
Code: Select all
Opponents
a b c d e f Win Sodos dc
Player A - W L W W W 4 -
Player B L - W W L W 3 6 W
Player C W L - L W W 3 7 L
Player D L L W - W L 2 5 W
Player E L W L L - W 2 4 L
Player F L L L W L - 1 -
Re: Round Robin: SODOS or Direct comparison?
Posted: Wed Jun 30, 2010 5:38 pm
by Bill Spight
willemien wrote:The fact the the bottom player (F) beat C but not B is what gives C a better SODOS score, isn't it?
No not really C is 2 sodod points ahead of B below where neither player loses against F
I was pointing out the fact that I was arguing that B>F>C bolstered the notion that B would win a playoff, while that very fact along with the fact that F was the bottom player, was used to argue that C should win the tie break. I thought it funny that the very same fact was used to support opposite conclusions.
Re: Round Robin: SODOS or Direct comparison?
Posted: Wed Jun 30, 2010 6:01 pm
by pwaldron
For what it's worth, I ran this tournament through the old AGA Bayesian rating system. The results put A > B > C > D > E > F. The margin between B & C and D & E is awfully small though, well below the sigma.
Re: Round Robin: SODOS or Direct comparison?
Posted: Thu Jul 01, 2010 3:01 am
by Harleqin
SODOS does not have a legitimation as tiebreaker in a round robin setting, because having won against stronger opponents means having lost against weaker ones. In summa, two players with equal wins have gained them against the same field of opponents, so there is no distinction to be made there.
Direct comparison ignores cycles. The best that can be done in this direction seems to be Bill's cycle elimination method. This does not always break ties, though.
In my opinion, if you need a clear winner or even a clear total ordering, do not play round robin.
Re: Round Robin: SODOS or Direct comparison?
Posted: Thu Jul 01, 2010 3:04 am
by topazg
willemien wrote:Sometimes Direct Comparison and SODOS give different results, and therefore the question arises What is the better tiebreaker?
There is no "better" tiebreaker, merely one that achieves the objectives of your tiebreaker system.
As a general rule with these two in my practical experience, SODOS rewards more strongly the results against higher ranked players, and will often match head to head for the top of the draw, and tiebreak in the other direction at the bottom of the draw. If you argue that the worse performing player should be rewarded for beating a high performing player more than he is punished for the loss against his tied player, then use SODOS. If you think that in all tiebreaks, "who won between them" is the most logical tiebreak, use that instead.
At least with a single round robin, head to head tiebreak
always breaks ties aside from jigos.
Re: Round Robin: SODOS or Direct comparison?
Posted: Thu Jul 01, 2010 3:07 am
by topazg
Harleqin wrote:SODOS does not have a legitimation as tiebreaker in a round robin setting, because having won against stronger opponents means having lost against weaker ones. In summa, two players with equal wins have gained them against the same field of opponents, so there is no distinction to be made there.
Define legitimation. It has been used with happiness in chess for over 100 years, and still people are not complaining about it much in top level chess tournaments. That, in my mind, is practical legitimation.
There are plenty of good reasons why someone may choose to dislike the logic SODOS uses to break ties, but that's not the same as the system objectively not being a legitimate tiebreaker. As you say, in a rr setting, winning against stronger opponents means losing against weaker ones (to still end up tied). So, if you want to reward the wins against stronger players more than you punish against weaker player losses, SODOS is an excellent solution. It depends what the tournament director wishes to reward with regards to performance criteria.
Re: Round Robin: SODOS or Direct comparison?
Posted: Thu Jul 01, 2010 3:25 am
by Harleqin
topazg wrote:Define legitimation. It has been used with happiness in chess for over 100 years, and still people are not complaining about it much in top level chess tournaments. That, in my mind, is practical legitimation.
Well, I am talking about theoretical legitimation. People also happily play lottery.
So, if you want to reward the wins against stronger players more than you punish against weaker player losses, SODOS is an excellent solution.
Winning and losing are perfectly symmetric. Why should winning against the first place be valued more than losing against the last one?
Re: Round Robin: SODOS or Direct comparison?
Posted: Thu Jul 01, 2010 3:43 am
by RobertJasiek
Needless to say, not breaking ties is the best. Second best is playing an equal number of games among the tied players until the tie will be broken. Third best is playing an equal number of games among the tied players until the tie will be broken, but with short(er) thinking times. Don't anybody tell me that 5 minutes games could not be done! And if the tied players won't solve the tie after 3 extra 5 min. thinking time tournaments, then in God's sake let them remain tied! They have proven to be of equal strength so let them be equal!
However, if tiebreaking by numbers is set as a requirement, then first of all critiria of aims expressing the purpose of tiebreaking should be specified. (There is the problem, of course, then one can reverse engineer suitable aims to justify any specific tiebreaker...)
My major concern with SODOS esp. in round-robin is the question "Why not SOLOS instead?" and it would inverse the order. I do not have the same concern as to the complement of a direct comparison tiebreaker because I am strongly impressed by the reasoning that, since the tied players are to be compared, their games against each other are of greater interest than their games against third players.
There are valid reasons supporting opinions against every numeric tiebreaker. Is MutualGameScore (applied to only exactly 2 tied players, else the tie is kept) a tiebreaker that, although one can state reasons against it, at least might have one advantage not seen in any other tiebreaker? The advantage of being the one and only numeric tiebreaker that is socially acceptable for everybody because it is the minimal consensus of all opinions on numeric tiebreakers? Or is there anybody with a strong objection?
Concerning the thread's topic more strictly, in round-robin I prefer DC very much over SODOS, mainly for the stated reason. (I have other reasons, which I would need to dig out, but their importance pales in comparison to the stated reason.)
Re: Round Robin: SODOS or Direct comparison?
Posted: Thu Jul 01, 2010 3:58 am
by topazg
Harleqin wrote:Winning and losing are perfectly symmetric. Why should winning against the first place be valued more than losing against the last one?
Because some people may choose that the values to be assigned to wins and losses are to be asymmetric. There is no requirement of symmetry on the point of rewarding performance criteria. That you feel they should be symmetric is reasonable, but it is not inherently the nature of winning and losing.
Re: Round Robin: SODOS or Direct comparison?
Posted: Thu Jul 01, 2010 4:36 am
by Matti
pwaldron wrote:For what it's worth, I ran this tournament through the old AGA Bayesian rating system. The results put A > B > C > D > E > F. The margin between B & C and D & E is awfully small though, well below the sigma.
I did not get any margin between B & C or between D & E donw to seven digits.
Re: Round Robin: SODOS or Direct comparison?
Posted: Thu Jul 01, 2010 5:23 am
by zinger
In my opinion, head to head (American term for direct comparison) should always be the first tiebreaker, if tiebreakers are used at all.
This in particular seems just silly to me:
Bill Spight wrote:To me the question is, given the results, who is more likely to win a game between B and C?
Err ... they already played each other.
I would add, that I think tiebreakers should only be used for first place. Beyond that there is no need.
Re: Round Robin: SODOS or Direct comparison?
Posted: Thu Jul 01, 2010 5:37 am
by topazg
I assumed Bill's question was effectively more of a "if they played each other 10,000,000 times, who would win" question, rather than an individual match.