Proposal for a New Ranking System for Insei League

Syptryn · Post by **Syptryn** » Fri Apr 30, 2010 4:09 pm

Ha, Danagabi and I have been discussing some of the flaws in the current ranking system, and thw two biggest points are probably the following

1. The current ranking system discourages players with higher win % to play games. For

Example:
If a player gets 12-0, he has 100% win.
If he wins to get 13-0, he has 101%, a gain of 1%
If he loses, he gets 12-1, which is 91 + 1 = 92%, a loss of 8%

Thus, the player is likely to avoid playing further games in fear of losing his score.

2. The current ranking system encourages people to hunt down weaker members of the group and avoid games vs strong members.

Example:

Suppose players A, B, and C are the top 3 players in a group, with 50/50 chance of beating each other, and almost 100% chance of beating all other members of the group. They all player 12 games against other players and are now on 12-0.

Players A and B play each other four times. And end with with 2 losses each, they are now on

14-2, which is about 88 + 2 = 90%

Player C simply doesn't play. And stays on 100%

***

These two problems cause the phenomena where many players do not play games for fear of ruining their ranking. This is quite bad, given that in Group C, for example, this month, half the people didn't player enough games.

So we propose that we should perhaps try out a different system, whatever such a system may be, it should fulfill the following:

A) Encourage people to play as much as possible.

- The system should not penalize people for losing. Doing so, means that people will refrain from playing for fear of losing their rank.

B) Encourage people to not avoid opponents stronger than them.

- The system to try to create incentives to get people to play all opponents, not just ones they think they can beat.

***

PROPOSAL A:

The simplest system, and can be further expanded on.

Score: Total Number of Wins + Number of Opponents Played.

Advantage:
This is is simple, but it fulfills both criteria. People will want to play as many games as possible, because they are guaranteed that they won't lose out for it. The extra Bonus of number of opponents also ensures that players will actively hunt down different opponents.

Disadvantage:
Players with limited time might not like it, since it may overly encourage no of games played.

PROPOSAL B:

The simplest system, and can be further expanded on.

Score calculated as follows:

For Each Opponent: First game between two players gives 2 pts, 2nd games gives 1 pt, 3rd and 4th win gives 0.5 pts.
Score if the Sum of all points.

Advantage:
The advantages of A. With some constraint on how much a player can benefit by playing lots and lots of games. This way, if Player A is stronger than player B, he will need to play a lot less games to win.

***

I would like to hopefully see Proposal B implemented some time

. I think it'll help out in addressing the problem of a lot of people not playing enough games!

Numsgil · Post by **Numsgil** » Wed May 05, 2010 6:28 pm

I like the games played + games won = score idea. But let's explore it from a theoretical point of view:

Basically, if you have a score function s(g, w), where g is the games played and w is the number won, you want it so that s(g,w) < s(g+1, w). So that playing more games always increases your score, even if you lose. This encourages more games to be played. We also obviously want s(g,w) < s(g+1,w+1). (So winning games improves your score). And we want s(g+1,w) < s(g+1,w+1). (So that winning a game improves your score more than losing.)

Right away we've eliminated all existing Elo rating type systems, since it's obviously possible to lose Elo rating as you lose games.

The current system in the insei league: s(g,w) = w/g. Since w/g > w/(g+1) always, there's no incentive to play more games unless you're fairly certain you can win (I'm ignoring the 1% bonus to make the math easier). Or more precisely:

let p be the probability that you will win the game. You will only play if:

-(w/g - w/(g+1)) *(1-p) + p((w+1)/(g+1)-w/g) > 0.

Simplifying, and assuming my algebra isn't off, you will only play if:

p > g/w

If you've won 20 games, and have a 100% chance of winning, there's still no incentive to play: 1 == 1. If you've won 10 games out of 20, and expect to win with a 49% chance, you shouldn't play: .49 < .5.

The 1% bonus skews that a bit, but not significantly.

...

If we do the additive system: s(g,w) = g+w. Since g+w < g+w+1, there's always incentive to play more games.

At least to maximize your score (you still might not want to play against a stronger opponent you're tied with).

BUT! let's think about ways to game the system. What if I play 50 games and resign after the first move on all of them? I have a 0% win rate, but I could easily be ranked #1 because I have more points than someone who carefully plays 16 games and wins all of them. That's the opposite problem.

So we want to add another constraint as well: s(10,10) > s(50,0).

...

What if we let s(g,w) = w. That is, your score is the number of wins. s(g,w) == s(g+1,w), so there's no disincentive to play more games. If there's even a slight chance of winning, you're better off playing.

BUT! You're competing your score against other players. Whoever has the best score wins. So even if your score goes up, if the opponent's score goes up more, you're hurting yourself.

...

Some math again: assume there are only two players in a league:

Let d_0 be the difference in your scores before you play. Let d_w be the difference in score if you win. Let d_f be the difference if you lose. Let g_m be the games you played before this game. And let w_m be the wins before this one. Likewise g_o and w_o for the opponent.

d_0 = s(g_m, w_m) - s(g_o, w_o)
d_w = s(g_m+1, w_m+1) - s(g_o+1, w_o)
d_f = s(g_m, w_m+1) - s(g_o+1, w_o+1)

You will only play if, for the probability of winning (p):

p * (d_w + d_f - 2 * d_0) - d_f + d_0 > 0

(hrmm, I didn't check that math, but that's the basic way to approach the problem).

...

Anyway, to sum: I think #wins as the scoring mechanism is the better than either the first proposal or the current system. But it's still fundamentally flawed. I think as long as the choice exists for playing or not playing a game, there's always going to be a way to game the system.

haha · Post by **haha** » Mon May 10, 2010 6:29 am

I'm 'ha' but i have to chose a nickname with at least 3 characters

your explanations are clear and you point the main problems :

activity is different between players and no problem to encourage those with more games at long as it is not exaggerated. the current bonus system (initially proposed by me and a bit modified by Breakfast) is here for that and also to solve the situation of one with 100% score not willing to play anymore.
one can chose to play or not, chosing to not hurt himself or chosing to increase his chances to win (beat the weaker !). the pervert root cause of this problem is the fear to demote and for top players to secure prizes.

It's hard to say that but perhaps we have to remove the freedom to 'play or not' or at least to diminish it. A solution with scheduled games or random pairings must be found but difficult to imagine how ?

how about a random list (generated by the system at the beginning of each month) of 10 'must play with' game ?

topazg · Post by **topazg** » Tue May 11, 2010 5:56 am

Here's my non-uber-mathematical proposal:

Basically, we want to encourage playing, and encourage winning even more, especially winning against someone else that has won lots. So, my idea is as follows:

3 points for a win
1 point for a loss

Bonus points = {x * number of points of the opponents you beat (excluding their bonus points)}

It may be rather crude and hackish, but the more important factors are
a) does it reward games and results appropriately? And
b) is it abusable / manipulatable?

To compare, taking x = 0.5 or something for an example, the current division "A" is as follows:

Current System

With my system:

In an ordered list:

So, this way, playing more games is rewarded, winning is rewarded more, and winning against successful players is really rewarded. If the last one is rewarded too much {x} can be less than 0.5, but the principle I think (to me at least) is sound. I think this is superior to #wins, superior to the first proposal, and superior to the current system, for the above reasons. I don't think it's particularly abusable either. I'd be interested in feedback

-----

EDIT: In response to numsgil's proposal of s(g,w) < s(g+1, w) being desirable. With this system, playing any game gives s(g,w) < s(g+1, w), winning it gives s(g,w) << s(g+1, w), winning that game against a successful player gives s(g,w) <<< s(g+1, w), which feels like it achieves its end. A top player would benefit from beating his challenger enormously, especially if that challenger is close, because of the win being worth a large amount for both players. However, if he didn't want to play, the second ranked player would be best off playing the 3rd and 4th players to quickly overtake first anyway - it seems to encourage the top players matching up quite highly, and it doesn't suffer from "I played and lost 20 games but I'm better than that guy who played only 3 and won them all" problems.

-----

EDIT 2: Another advantage this system has is mathematical simplicity. It's easy to understand, easy to implement and easy to automate. A lot of rating theoreticians tend to be mathematicians, and offer as close as possible to pure and complete algorithms. The purpose here is to score appropriately, not perfectly, and simplicity I consider to be very positive attribute for whatever system gets used.

breakfast · Post by **breakfast** » Tue May 11, 2010 7:50 am

Thx a lot!
I will study these suggestions

Harleqin · Post by **Harleqin** » Tue May 11, 2010 8:23 am

One more proposal: http://www.evanmiller.org/how-not-to-so ... ating.html

haha · Post by **haha** » Tue May 11, 2010 2:45 pm

i must admit the idea of Topazg has a lot of qualities :

push to play more games
continue to try to win his games
rewards well the wins against difficult opponent
parameters configurable (points for a win, points for a lose, x bonus factor) to get a good balance with the 3 points mentionned above.

and the top players have to continue to win but they have to crush more difficult opponents if they want to get more bonus ^^ ...but still not sure they will fight against other top players. At least, it will be a more exciting 'run' between them...
this problem remains but this is the unique bad point i saw, so i'm favorable to try this system.

Syptryn · Post by **Syptryn** » Tue May 11, 2010 2:55 pm

I like Topzag's idea, especially the bonus being depend on the strength of your opponents.

Something like this would certainly gives players incentives to play highly ranked players... something that is definitely lacking in the current system!

Numsgil · Post by **Numsgil** » Tue May 11, 2010 5:59 pm

Harleqin wrote:One more proposal: http://www.evanmiller.org/how-not-to-so ... ating.html

This still suffers from the case where these's an active dis-incentive to play more games. eg:

wins = 4 games = 7: s = .845
wins = 5 games = 8: s = .866
wins = 4 games = 8: s = .789

So depending on your confidence of winning there's clearly a point where you should stop playing games.

...

topazg's system seems to be pretty good for the couple of cases I've thrown at it on paper. I don't think there's a way to game the system at all as a single person. I also don't think there's a way to game the system if two people are in collusion (so long as bonus points are awarded based on your opponent's base points).

However, I think you need to make sure that bonus points are updated based on current standings and not on historical data. That is, bonus points = sum over all opponents of (#wins against opponent * scaling factor * opponent's current base points). If you update it based on historical data there's a weird asymmetry between games at the beginning of the month (worth very little) and games at the end of the month (worth crazy a lot). The incentive would be to save all your games until after other people have built up some base points. Which means few games at the beginning of the month, and then lots at the end, which isn't good.

I think some more complex analysis needs to be done to see if there really aren't any scoring exploits. But I don't see any.

topazg · Post by **topazg** » Wed May 12, 2010 1:44 am

Numsgil wrote:However, I think you need to make sure that bonus points are updated based on current standings and not on historical data. That is, bonus points = sum over all opponents of (#wins against opponent * scaling factor * opponent's current base points). If you update it based on historical data there's a weird asymmetry between games at the beginning of the month (worth very little) and games at the end of the month (worth crazy a lot). The incentive would be to save all your games until after other people have built up some base points. Which means few games at the beginning of the month, and then lots at the end, which isn't good.

This is interesting. If I get what you are meaning, you are saying that bonus points are always calculated on the latest standings as opposed to "the opponents base points at the time they played" - if so, I agree completely, and that's what I had in mind, for exactly that reason. It does mean that you might overtake someone from 4th to 3rd (for example) without playing a game, if one of the opponents you have defeated wins an important game, as your bonus points go up - but this seems fair, as any defeated opponent of yours does really well towards the end of the month you should still be rewarded (IMO) for beating a successful opponent.

HermanHiddema · Post by **HermanHiddema** » Wed May 12, 2010 3:17 am

This bonus points system is basically just adding SODOS (multiplied by some factor) to your score. One problem with that is that the same criticism that applies to SODOS also applies to this, which is that is is questionable to reward winning against strong players when you're no punishing losing against weak players.

Example:

Suppose two players in position 5 and 6 of the 10 player league have both played 8 games. They've played all opponents except each other. Both have scored 4/8. The player in position 5 has defeated players 1, 2, 3 and 4, but lost to 7, 8, 9 and 10. The player in position 6 has defeated 7, 8, 9 and 10, but has lost to 1, 2, 3 and 4. In this scenario, player 5 gets waaaaay more bonus points that player 6. But why? After all, he lost to players 7, 8, 9 and 10, that's pretty weak!

HermanHiddema · Post by **HermanHiddema** » Wed May 12, 2010 3:22 am

Harleqin wrote:One more proposal: http://www.evanmiller.org/how-not-to-so ... ating.html

This is quite a good system, but it has a weakness when applied to this case, and that is that the player can bias the sample by only playing against the weakest players.

Any system that ranks purely on some score derived from win ratio, such as this one, will suffer from that problem.

A system that would probably work rather well is to use Tournament Performance Rating. TPR doesn't work when a player has only wins (or only losses), but that can be solved by giving every player one additional jigo result against a virtual opponent with some base rating.

topazg · Post by **topazg** » Wed May 12, 2010 3:31 am

HermanHiddema wrote:This bonus points system is basically just adding SODOS (multiplied by some factor) to your score. One problem with that is that the same criticism that applies to SODOS also applies to this, which is that is is questionable to reward winning against strong players when you're no punishing losing against weak players.

Example:

Suppose two players in position 5 and 6 of the 10 player league have both played 8 games. They've played all opponents except each other. Both have scored 4/8. The player in position 5 has defeated players 1, 2, 3 and 4, but lost to 7, 8, 9 and 10. The player in position 6 has defeated 7, 8, 9 and 10, but has lost to 1, 2, 3 and 4. In this scenario, player 5 gets waaaaay more bonus points that player 6. But why? After all, he lost to players 7, 8, 9 and 10, that's pretty weak!

I agree, this is a weakness, but part of the motivation for changing the system is simply to encourage more games. One of the risks of putting the burden on punishment rather than reward is the temptation to play less games to minimise the punishment. Unlike McMahon systems (where SODOS receives so much criticism), in the Insei league people can pick and choose their opponents and the number of games they play - in itself this is a situation that gives rise to potential abuse, and any system must be designed to cater for this. For example, in your situation, the player in 5th wouldn't even bother playing the bottom 4 - the risk of punishment would outweigh any potential reward.

I also slightly suspect that it will be hard to contrive a situation such as the one you show in your example in my system. Beating the top 4 players is unlikely to leave you in 5th due to the bonuses, depending on their results... if you can mock up a simulated league position I'd be interested to have a look at what would be required in practice to generate that scenario.

HermanHiddema wrote:A system that would probably work rather well is to use Tournament Performance Rating. TPR doesn't work when a player has only wins (or only losses), but that can be solved by giving every player one additional jigo result against a virtual opponent with some base rating

This would be interesting - however, I don't think any rank in the league should be based on "official GoR or KGS rating/rank", and TPR normally is based on the starting ratings of the players in question. What other info would you feel into TPR?

The reason I don't think it should be based on rating is each player can play something like 52 games (including the teaches, 4 games against each of the other 13 players). In reality, it is rare for players to reach 20 - which means with 32 assumed draws TPR will be naturally lower for higher rated players. This seems unreasonable as the majority of the possible games they could have played are marked as results that never happened, which feels like a poor way of ranking.

There may be other variables you can feed into a TPR, but I can't think of any off the top of my head.

topazg · Post by **topazg** » Wed May 12, 2010 3:39 am

topazg wrote:The reason I don't think it should be based on rating is each player can play something like 52 games (including the teaches, 4 games against each of the other 13 players). In reality, it is rare for players to reach 20 - which means with 32 assumed draws TPR will be naturally lower for higher rated players. This seems unreasonable as the majority of the possible games they could have played are marked as results that never happened, which feels like a poor way of ranking.

There may be other variables you can feed into a TPR, but I can't think of any off the top of my head.

Actually, this is nonsense, sorry. I misread what you were suggesting. TPR doesn't have this problem, but it does have another that numsgil has pointed out: s(g,w) < s(g+1, w) is a really important feature. If you win 3 or 4 big games, and have a 3-0 or 4-0 record, you aren't going to want to play any more - even with the "jigo against self" which is often used in TPR, your performance is sufficiently high that it will be hard for someone to knock you off top spot with a 100% record. I think this is a slightly better system than the current one being used, but the disincentive to continue playing with a good record puts it behind the other proposals (including #wins).

Harleqin · Post by **Harleqin** » Wed May 12, 2010 4:31 am

I think that "free pairing" is OK for a little fun on the side, but for "serious" competition, you need to actively balance things out. For leagues, I think that round-robin systems are standard (each player plays each other player once).

There are often "free pairing" side tournaments at weekend tournaments, which give little prizes to whoever played most games, most wins, and perhaps best win ratio.

"Free pairing" tournaments just do not have a clear winner.

Life In 19x19

Proposal for a New Ranking System for Insei League

Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League

Re: Proposal for a New Ranking System for Insei League