Proposal for a New Ranking System for Insei League

[topazg]

One of the difficulties is that the system need to be reliable not only at the top, where the prizes are won, but also at the bottom, where the demotions are decided

Indeed, but how do we define reliability?

If a player needs only to win a game to get above those who haven't won any, then playing lots of games for that one win will put you above someone who only won a single game and stopped playing. It seems that the participants want active players as well as strong players in the league, and 1-0 (win-loss) is seen as less favourable than 1-7, where 8 games have been played.

It is questionable that wins against strong players are rewarded more highly than losses against weak players are punished, but it is also questionable whether they should be valued the same, or in the other direction. I don't see any argument for one over the other two that is simply "right".

It seems to be more a case of which achieves the overall purposes of the system. I'm sure Alex wants people playing as many games as possible so that they can improve (his purpose of the leagues), so rewarding wins and not punishing losses increases activity, and thus achieves the purposes of the system. Not only that, but the three people who have posted with concerns about the current system also feel happier with rewarding the wins more greatly than the punishment of losses. In fact, anything that discourages players with wins from playing more games (such as punishment of losses) appears to be the strongest concern.

I think few participants will resent the demotion of people on a 1-0 record.

[HermanHiddema]

One of the difficulties is that the system need to be reliable not only at the top, where the prizes are won, but also at the bottom, where the demotions are decided

Indeed, but how do we define reliability?

If a player needs only to win a game to get above those who haven't won any, then playing lots of games for that one win will put you above someone who only won a single game and stopped playing. It seems that the participants want active players as well as strong players in the league, and 1-0 (win-loss) is seen as less favourable than 1-7, where 8 games have been played.

From this, "more games = higher score" would seem a strongly desirable property. But do players feel the same when it is 14-0 vs 14-6?

It is questionable that wins against strong players are rewarded more highly than losses against weak players are punished, but it is also questionable whether they should be valued the same, or in the other direction. I don't see any argument for one over the other two that is simply "right".

It seems to be more a case of which achieves the overall purposes of the system. I'm sure Alex wants people playing as many games as possible so that they can improve (his purpose of the leagues), so rewarding wins and not punishing losses increases activity, and thus achieves the purposes of the system. Not only that, but the three people who have posted with concerns about the current system also feel happier with rewarding the wins more greatly than the punishment of losses. In fact, anything that discourages players with wins from playing more games (such as punishment of losses) appears to be the strongest concern.

I think few participants will resent the demotion of people on a 1-0 record.

A simple system we used for the free pairing tournament at the EGC was:

Score = Wins + Sqrt(Games)

With this system:

• Every game always increases your score
• Losing always gains less than winning
• Playing more games is progressively rewarded less when you've already played plenty.
• For players with roughly the same number of games, higher number of wins is always better

[HermanHiddema]

A simple system we used for the free pairing tournament at the EGC was:

Score = Wins + Sqrt(Games)

With this system:

• Every game always increases your score
• Losing always gains less than winning
• Playing more games is progressively rewarded less when you've already played plenty.
• For players with roughly the same number of games, higher number of wins is always better

Disadvantage of this system is that it benefits you to play against the weaker players, again.

To avoid that, you could use the formula per opponent:

Score = Sum (Wins(X) + Sqrt(Games(X))

Where X is the opponent. In this way, your first game against a new opponent is worth at least 1 point (Sqrt(Games(X)) and at most 2. The second is worth 0.41 or 1.41, the third is 0.32 or 1.32 and the fourth is 0.27 or 1.27 (depending on if you win or lose them).

If the top players have about a 50-50 chance against each other, then their first game is worth 1.5 to them, on average, while a 99% certain win in their second game against a weaker player is worth 1.4 on average.

And on the bottom end of the league, a 50-50 chance in your 4th game against another bottom player is worth 0.82 on average, while a 99% certain loss in a second game against a top player is only worth 0.42 on average.

This system gives you incentive to play at least one game against every player, and to seek out players of your own strength more often.

[topazg]

From this, "more games = higher score" would seem a strongly desirable property. But do players feel the same when it is 14-0 vs 14-6?

I have no idea It would be consistent with the desires elsewhere, so I hope so!

A simple system we used for the free pairing tournament at the EGC was:

Score = Wins + Sqrt(Games)

With this system:

• Every game always increases your score
• Losing always gains less than winning
• Playing more games is progressively rewarded less when you've already played plenty.
• For players with roughly the same number of games, higher number of wins is always better

I like this system - it's better IMO than #wins, as it fulfils two priorities, namely the number of victories (top priority) and the number of games. However, in my mind the total number of games was a 3rd priority (though I could be wrong on the order of these last two), and the 2nd priority was a differentiation between who those wins were against. This is something that the simple system doesn't address at all, and I think it may lead to all the strong players only playing all the weak players and trying to pick up perfect records with lots of wins (purely for efficiency of maximising wins, as opposed to wanting the winning percentage itself). Part of the desire seemed to be to get the top players playing each other, so I think there has to be an incentive to play high scoring participants, and wins + sqrt(games) doesn't do this - I think it may have slightly the opposite effect.

EDIT: Ok, you've just partly answered that last paragraph.

In response, I think "If the top players have about a 50-50 chance against each other, then their first game is worth 1.5 to them, on average, while a 99% certain win in their second game against a weaker player is worth 1.4 on average." will result in playing that 1.4 every time. Not only will they almost guarantee 1.4 points, but they won't risk giving their opponent 1.5 points instead, which at 50% chance is too high when compared to the alternatives.

I think the same will be true to a lesser extent on your other example too - although 50/50 for 0.82 and 99/1 for 0.42 is pretty much same number of expected games per point, the 50/50 for 0.82 threatens to give your rivals those points if you lose, whereas playing the top player doesn't.

It is certainly an improvement over the straight wins sqrt(games), but I intuitively still prefer mine as I think it rewards beating the high scoring players more richly, and partly because the 3 parameters are so easily changeable to adapt to differing priority values. It would be good to see Division A worked out for each of these ranking systems - I'll see if I can find time this evening to do it and put them side by side.

[HermanHiddema]

From this, "more games = higher score" would seem a strongly desirable property. But do players feel the same when it is 14-0 vs 14-6?

I have no idea It would be consistent with the desires elsewhere, so I hope so!

A simple system we used for the free pairing tournament at the EGC was:

Score = Wins + Sqrt(Games)

With this system:

• Every game always increases your score
• Losing always gains less than winning
• Playing more games is progressively rewarded less when you've already played plenty.
• For players with roughly the same number of games, higher number of wins is always better

I like this system - it's better IMO than #wins, as it fulfils two priorities, namely the number of victories (top priority) and the number of games. However, in my mind the total number of games was a 3rd priority (though I could be wrong on the order of these last two), and the 2nd priority was a differentiation between who those wins were against. This is something that the simple system doesn't address at all, and I think it may lead to all the strong players only playing all the weak players and trying to pick up perfect records with lots of wins (purely for efficiency of maximising wins, as opposed to wanting the winning percentage itself). Part of the desire seemed to be to get the top players playing each other, so I think there has to be an incentive to play high scoring participants, and wins + sqrt(games) doesn't do this - I think it may have slightly the opposite effect.

EDIT: Ok, you've just partly answered that last paragraph.

In response, I think "If the top players have about a 50-50 chance against each other, then their first game is worth 1.5 to them, on average, while a 99% certain win in their second game against a weaker player is worth 1.4 on average." will result in playing that 1.4 every time. Not only will they almost guarantee 1.4 points, but they won't risk giving their opponent 1.5 points instead, which at 50% chance is too high when compared to the alternatives.

I think the same will be true to a lesser extent on your other example too - although 50/50 for 0.82 and 99/1 for 0.42 is pretty much same number of expected games per point, the 50/50 for 0.82 threatens to give your rivals those points if you lose, whereas playing the top player doesn't.

It is certainly an improvement over the straight wins sqrt(games), but I intuitively still prefer mine as I think it rewards beating the high scoring players more richly, and partly because the 3 parameters are so easily changeable to adapt to differing priority values. It would be good to see Division A worked out for each of these ranking systems - I'll see if I can find time this evening to do it and put them side by side.

I think this misunderstands the math somewhat. With 50/50 chance for 0.82 on average it takes only half the number of games compared to 99/1 for 0.42. The 0.82 is already averaged! They're playing a 50/50 game for 1.64 points total.

Similarly, you're not risking giving your opponent 1.5 points, but you're giving him 1.5 points on average. You give him 1 point guaranteed (and you're taking 1 guaranteed yourself) and you run the risk of losing and giving him one more (but if you win, you get one more). In a situation where there are others also in contention for the top spot, you're both ahead of the game afterward.

Suppose three players (A, B and C) are tied at the top, and have not played each other, but have played only weaker players. If one evening we have the match up:

A - B (50/50 chance)
C - D (99/1 chance, second game they play)

Then who is doing better, on average? A and B? Or C?

[topazg]

I think this misunderstands the math somewhat. With 50/50 chance for 0.82 on average it takes only half the number of games compared to 99/1 for 0.42. The 0.82 is already averaged! They're playing a 50/50 game for 1.64 points total.

Similarly, you're not risking giving your opponent 1.5 points, but you're giving him 1.5 points on average. You give him 1 point guaranteed (and you're taking 1 guaranteed yourself) and you run the risk of losing and giving him one more (but if you win, you get one more). In a situation where there are others also in contention for the top spot, you're both ahead of the game afterward.

Suppose three players (A, B and C) are tied at the top, and have not played each other, but have played only weaker players. If one evening we have the match up:

A - B (50/50 chance)
C - D (99/1 chance, second game they play)

Then who is doing better, on average? A and B? Or C?

Yeah, many thanks, I had misunderstood. Ok, that makes the system better than I had thought. The other issue for me is that if 1st doesn't want to play 8th again because he's already played him once and is losing returns on his games, that doesn't stop him from playing 7th, 9th, and 10th to get his games played while keeping 99% result high rewards. We have a total of 14 players in a division (including teachers, and a total allowed of 4 games per person, totalling 52 games). However, in the latest visible division A table, only 4 of the 14 have played more than 7 games, which makes me feel that there will be plenty of opportunity to get a lot of full, or nearly full value, games against weaker opponents.

The reward in your system is only by proxy in that games against the same opponent are valued less. This will encourage a wider distribution of games as opposed to games against stronger (or equal) rank, and that may be difficult / punishing for those in unusual time zones. It doesn't actually reward wins against people who have been directly successful this particular cycle.

[HermanHiddema]

Yes, my system rewards playing each opponent at least once strongly, but beyond that it does not reward playing stronger players.

It's a hard thing to solve. If you reward playing stronger players too much, then you get the whole "I got 0/16 against the top 5 and beat theis guy who got 4/1 against the bottom players" thing. If you son't reward it enough, they don't want to play each other.

Regardless, 7 games is very little, and very vulnerable to statistical noise from "fluke" wins or losses.

BTW: I'm not advocating any particular system, really. It really depends on your goals. I'm just throwing out some suggestions in the hope that they will help in formulating a good system

[topazg]

Yes, my system rewards playing each opponent at least once strongly, but beyond that it does not reward playing stronger players.

It's a hard thing to solve. If you reward playing stronger players too much, then you get the whole "I got 0/16 against the top 5 and beat theis guy who got 4/1 against the bottom players" thing. If you son't reward it enough, they don't want to play each other.

So true!

Regardless, 7 games is very little, and very vulnerable to statistical noise from "fluke" wins or losses.

I agree, it's not possible to avoid this issue. The top two players on 3-0 and 1-0 feels particularly wrong to me intuitively, having only played 4 games between them!

BTW: I'm not advocating any particular system, really. It really depends on your goals. I'm just throwing out some suggestions in the hope that they will help in formulating a good system

Ditto I think my rankings so far would be as follows:

My system
Sum (Wins(X) + Sqrt(Games(X)) per opponent
Wins + Sqrt(Games)
#Wins
Mine but with SOS instead of SODOS
Lower bound of Wilson score confidence interval for a Bernoulli parameter
TPR
Current System

My justification is because of having a high priority of valuing "wins against those with a good record" which I felt would encourage people to seek games against the top players, and for the top players to seek games against each other, and mine is the only one I think that actively encourages it. I can't see an easy way to abuse the system (including, as HaHa said, by collusion), and it rewards #wins as the primary driver, which also seems to be favourable as it encourages playing lots of games. I also like the fact it doesn't punish defeats, as again I think that encourages more games played. All of this is by trying to consider the options based on the following goals:

* lots of games is better
* lots of wins is better
* wins against the top players are valuable
* playing more games is almost always better than not playing more games

[Arlequin]

Hi all

I like topazg system, it is probably the best one for our needs.


However I would like to add one other important goal, especially for top A-class players and teachers:
- "both teachers and students should be able to compete for first place and prices"
This is important for the motivation of top players and teachers, who themselves are important for the motivation of other players .


Due to the special positions of teachers, the number of games they can play might be far higher (or far lower as currently in A group), which would eventually means impossibility for competition with some rule-set.
In theory, people should be willing to play as many simultaneous games with teachers as possible, and so the teachers should be the most active players. It used to be true, but it is recently the opposite in top group.

Therefore I think a good system should encourage people to play a lot of games with teachers, and then, because teachers would be very active, being able to adapt to the special situation of teachers if necessary.



With topasg idea, I think the current fear of A-group player to play against teacher may disappear which is good.
However, will it be the case that teacher play twice as much as top-players ? In that case, topasg's algorithm would give a enormous advantage to the teachers, or maybe I misunderstood it. Would it be easy to make a special case for teachers if necessary ?



-- I am the only Arlequ1 ! Don't let Harlequin steel my identity ^^ --

[topazg]

With topasg idea, I think the current fear of A-group player to play against teacher may disappear which is good.
However, will it be the case that teacher play twice as much as top-players ? In that case, topasg's algorithm would give a enormous advantage to the teachers, or maybe I misunderstood it. Would it be easy to make a special case for teachers if necessary ?

Yes, this is very true. Especially as the teachers would expect to win the majority of their games, I don't know how easily to make sure that they don't take advantage of this.

[haha]

If we consider than teachers are better than top players, whatever the system, only teachers will compete for the prices, so we need a handicap :

• handicap in game => teachers play against top players in simul or in 1 to 1 but with H2 handicap. I feel teachers will argue than players must have hard games to improve and top players will want to play teachers in 1 to 1, challenge more interesting.
• handicap in score => in the topazg system (let's call it "June system" ), teachers can begin with an initial score of -20 or teachers can get less points per win/lose. Here again, simulation and/or experience can adjust the parameter

still the problem come from teachers able to compete for prices. I understand it's a motivation for top players but could teachers be only motivate to have an attractive league with lot of regular members ? this is somewhat their prices.

Feedback of top players and teachers are welcome !

[topazg]

A couple of ideas are:

a) initial negative score as you say. 20 feels like it might not be so big come the end though.
b) smaller bonus, such as 0.3 * SODOS rather than 0.5

For those that want to play with figures, I've gone and put it into an Excel spreadsheet: http://www.topazg.com/insei.xls

All the figures for each division are up to date as of today, and all the key parameters are adjustable at the bottom of each table. The teachers have their own parameters there too, including an initial handicap value.

One further thing of note: All the other divisions record wins against the teachers, but exclude losses. Firstly, this may be unnecessary now as losing is likely to give rewards and won't punish their percentages (ok, not as big a reward as a win, but bigger than not playing at all). If the teachers want to carry on excluding these losses from the records, which is my initial assumption, I've adjusted the spreadsheet worksheets from Div B through to Div E to ignore losses. This means that the reward for simply playing a game in my system (only 1 point per game admittedly) isn't there for the lower division games against the teachers.

And for those interested, current standings in A (with -20 and 0.3 modifier for teachers) are now:

1. danigabi - 138 (13-1)
2. Kalmah - 96.5 (8-5)
3. ha - 69.5 (5-1)
4. Nata - 63 (5-2)
5. Syptryn - 59.5 (5-10)
6. breakfast - 43.7 (8-1)
7. RamenBoya - 29.5 (2-2)
8. OohAah - 27.5 (3-0)
9. fantastigo - 26.5 (1-9)
10. Arlequ1 - 24 (1-15)
11. DRhazar - 20.5 (1-3)
12. Teamrocket - 17 (1-5)
13. roln111 - 11.8 (4-0)
14. YraUkr - 11 (1-4)

[haha]

i think the weight of bonus is too big.
0.2 bonus modifier both for teachers and students seems right.
with this smaller bonus, the initial handicap of -20 for teachers become ok too.

[gaius]

A scoring system using the actual score + bonuses based on a few formulas sounds rather difficult if you ask me... Might I interject with a simple(r) system:

* The players are ranked by their SODOS score.
* In the case of a player playing multiple games against the same opponent, the score against that opponent is multiplied by Sqrt(1/n) where n is the number of games they played against each other.

Let's give an example. Suppose I play three games against somebody with 10 wins; out of those games, I win two and lose one. This would give me Sqrt(1/3)*(10+10+0) = 11.55 points. That's more than if I just play 1-0 against this opponent. It also gives me a much better result than if I would only play lower ranked players - so much better, in fact, that it might become almost impossible to win a division without some wins against strong players. Which is exactly what you want.

With this system:

• Almost all games either increase your score or leave it the same. OK, if you're 1-0 against someone, then losing will cost you a few points, but winning the game will earn you more than losing will cost you so there's still no reason not to play.
• Playing more games against the same opponent is progressively rewarded less, and playing against a new opponent never costs you points.
• It is vital for strong players to play other strong players.
• Most importantly, it is easy to understand and accurate both at the top and at the bottom end of the rankings!

EDIT: I just noticed this is kind of a necropost... hopefully it's still useful to someone anyway...

[topazg]

A scoring system using the actual score + bonuses based on a few formulas sounds rather difficult if you ask me... Might I interject with a simple(r) system:

* The players are ranked by their SODOS score.
* In the case of a player playing multiple games against the same opponent, the score against that opponent is multiplied by Sqrt(1/n) where n is the number of games they played against each other.

Let's give an example. Suppose I play three games against somebody with 10 wins; out of those games, I win two and lose one. This would give me Sqrt(1/3)*(10+10+0) = 11.55 points. That's more than if I just play 1-0 against this opponent. It also gives me a much better result than if I would only play lower ranked players - so much better, in fact, that it might become almost impossible to win a division without some wins against strong players. Which is exactly what you want.

With this system:

• Almost all games either increase your score or leave it the same. OK, if you're 1-0 against someone, then losing will cost you a few points, but winning the game will earn you more than losing will cost you so there's still no reason not to play.
• Playing more games against the same opponent is progressively rewarded less, and playing against a new opponent never costs you points.
• It is vital for strong players to play other strong players.
• Most importantly, it is easy to understand and accurate both at the top and at the bottom end of the rankings!

EDIT: I just noticed this is kind of a necropost... hopefully it's still useful to someone anyway...

It's not yet, I know Alex is still deciding what to do on this issue.

Can you simulate your system on Div A - it looks potentially interesting, and I'd like to see how it compares...