[Note: see below for attached original MS Word file and excel charts.] [Note: You can choose two options in the poll, and you can change your vote later on.]
ASR League 2010: The Math By Usagi
I’ll go ahead and use my KGS handle for this paper. The ASR league’s policies have been designed by hand with a best-intentions strategy. Now that the league has been active for over an entire year, there is enough data to analyze how the rules and what not of the league have affected it’s membership base. Normally this would not be a question; however at first glance it would appear there are certain serious problems with the ASR that demand immediate attention. This, of course, does depend on the people in charge to a certain extent – it is one thing to propose that something is a problem – it is quite another for others to agree on this and then to care enough to take positive action to solve the problem. So assuming that people are interested in making the ASR league a better place, I would like to make three strong statements about three key problem areas with the ASR as it now stands: 1. Activity rewards are imbalanced. 2. There are management issues or negligences which contribute to inactivity. 3. Member retention is low, what to do?
Activity Rewards The biggest problem with the ASR right now is that it is focused on rewarding activity. This was done because a strong player is not necessarily an active player. Activity rewards are a good thing; however the way they are implemented into the league has unbalanced the basic idea of a room-based league; to promote stronger players, in general, and demote weaker players thus allowing players to play people close to their level and have a relatively smooth gradient of people to play against. The idea behind this is that it will help people learn to play Go. The way the league rewards activity (currently) is to give essentially 1 point for a loss, and 2 points for a win. The danger with this system is easy to express mathematically. If there are two players X and Y in any room, where Y is X+1 (such as 2d and 3d) then we know by the ranking system that player Y’s chances of winning an even game against player X are greater than 2/3rds (~67%). We can thus generalize that over a long period of time, player X will get 2(1/3) + 1(2/3) points for every game he plays, and that player Y will get 2(2/3) + 1(1/3) points for every game they play. With this generalization we may make some probabilistic statements about what is likely to happen during a period given any particular room structure.
Assume a 14 player room with 13 players of strength X and one player of strength Y, where Y is X+1. We now know that over 10 games, player Y will get ~16.5 (16.67) points, and over ten games, each player of strength X will be expected to get ~15 points. In the long run this works as expected; stronger players promote. However, if just one of the players ranked X plays more games than the Y-strength player, they will end up with more points. One game with a win or two losses will tend to give the X-rated player 17 points, ½ point higher than what the Y-rated player is expected to get. Therefore we arrive at the general principle that playing an extra two games is sufficient to cover a deficiency of one stone of strength. This is an incredibly serious problem. This means if a 5 dan plays 15 games in a month, a 1 dan is statistically likely to beat him if he can just play 22 or 23 games. This is the reason why there are no 6 or 7 dans in the league; statistically they just can’t make it past the dozens and dozens of “active” 1 and 2 dans in their way. The Math For any player x, there are two variables; strength (winning percentage) and # of games. If we examine thousands of cases, we can chart the results as follows:
(SEE CHART #1: "POINTS BY STRENGTH" BELOW) The above chart shows the distribution of points scored by the strength of the player in the league. The variation in the noise on the Y axis is due to players playing different numbers of games. This chart clearly shows the average points a player of strength X can expect to get has little to do with his playing strength – although the expected score does rise with strength it does not significantly reflect the that difference. For example, The difference between a 40% win ratio and a 60% win ratio (approximately two stones in strength) is only 4 points; i.e. between 2 and 4 games played, depending on if those games were won or lost. This is in line with the earlier analysis.
(SEE CHART #2: "POINTS BY ACTIVITY" BELOW) This chart bears a striking difference to the one presented previously; it is a chart of expected point score by number of games played. It now becomes visually clear what has been said so many times; playing strength is not the deciding factor in who gets promoted in the ASR; it is not even close. The expected score falls in a very narrow range based on number of games played. This is solid proof that the activity reward of 1 point for a lost game is horribly flawed and must be changed. Reccomended Course of Action Of course, no one is proposing that the reward system be eliminated, but it must be reduced if the league is to move closer to fulfilling it’s role of providing a structure whereby players can play competitively serious games (not just “serious” games). Using the same software I used to produce the above two charts I determined that a reward system of 5 to 1 (versus the current 2 to 1) will increase the number of games required to game the system from 2 to 8. Even this is somewhat disturbing but any improvement in this area will have a positive effect on the league. Because of this I recommend an immediate move to a 3 points-win, 1 point loss system, phasing in 4 to 1 after two months, and 5-1 two months after that. There are other ways to cap the activity reward. The one with the most popular support appears to be limiting the number of lost games which count for points to the number of won games. Therefore a player with a 3/10 record will have 6 points for his wins, but only 3 of his losses (3 points) will count towards his score, for a total of 6+3 or 9 points (not 16!) Now that this has been covered in detail we can move on to another pressing issue; management. Management Issues No, I don’t mean that management is incompetent; far from it, Stalkor, CGBSpender, Vortex and others have done an amazing job running the league. This isn’t about a lack of abilty or a lack of dedication. It is about what could be done, and done very easily to make the league a better place. Communication Issues As it stands, a large number of people drop out of the league and are not removed for that month. Some examples; in October 2010, at the end of the month, there were over two dozen completely absent players, for a total of 55 players with 3 games or less. Alpha:1, Beta I: 3(+3 with less than 4 games), Beta II: 2 (+1 with less than 4 games), Gamma I: 2+2, Gamma II: 2+1, Gamma III: 5+2, Gamma IV: 4+3, Delta I: 4+2, Delta II: 3+3, Delta III: 0+3, Delta IV: 2+3, Delta V: 0+1, Delta VI, 0+3 The painful thing about this is that it would be extremely simple to make changes in how the league is organized to prevent this from ever happening again – in contrast to having it occur in almost every room, every single month. Suggestion: Keep in Contact with Members There are several ways this could be done. A mailing list is one good solution, but would require keeping player’s e-mail addresses when possible. Another option is sending messages on KGS. Either way, the general idea is to send a reminder sometime during the last week of the month that if a player is not planning to participate in the league, they should inform the admins. A Better Suggestion: Early Removal Rule and Discounted Game Rule Sending messages to members has it’s flaws, least of which, it’s admin intensive. A better solution is probably adding an early removal rule. This idea is both small and beautiful and can be expressed simply: “A player must play at least one game in the first two weeks of the month, or they will be removed from the league and replaced by the someone on the waiting list” (I am aware that this violates current policy). Another Interesting Suggestion: reorganize by time zone. No matter how you slice it, this makes sense. Organizing people by the time they are likely to be available (or by the time they are available) makes sense. In the past the counter to this suggestion was that some rooms would become imbalanced; it is so happened that the “European” time zone room was stronger than the “American” time zone room it may not be fair; however, this is not as relevant as it may seem at first. For one, there will be some bleed over as people join and leave the league. Secondly the I, II and so on of each class can begin with a random timezone each month, thereby mixing it up while still keeping people in the same room with others who are likely to be on at the same time. Finally, the Alpha room question. This has zero effect on Alpha room; As the activity factor is reduced in the lower rooms, the strongest players from each timezone class will be promoted to alpha. If we assume that of Beta I and Beta II, beta I contains players who are on average stronger, then players from Beta I’s timezone-class will tend to dominate Alpha anyways. There is really no drawback to not organizing by timezone. All it takes is a simple question whenever anyone joins the league; “what timezone are you in?” Please do this. Member Retention Problems This month perhaps 55 people will be removed for inactivity and several others may announce their retirement. This is over 1/3rd of our player base and is a serious, recurring problem. Turnover is simply too high and long-term retention is ridiculously low. I’d ask the administrators to seriously consider the rationale behind the league in the first place. What is it for? Without any clear focus people will leave. Please listen to me, I’ve run the league in the past and in the meantime I’ve run two separate and successful top-end raiding guilds on World of Warcraft. I know something about motivating a team. And if you want to motivate people in this league to stay, you need to make them think they are getting something out of it. Focus point: learning go Put it in the rules, the FAQ, mention it everywhere: Winner is expected to review. You don’t have to, but….. //just talk about the game, share it//. Make this a part of the ASR culture. Just do it. This is a necessity. There is zero reason to play here if you don’t learn anything. Seriously, playing serious games is not really enough… Focus point: prizes There’s been some debate about prizes and member rewards. You have to be careful about people who are only there to win the prizes and have no intention of reviewing games. I’ll say it again, mandate the reviewing of games. If you have to, do it like this, just state that you will not be eligible to win a prize if you don’t review at least half the games you win. You don’t have to check this all the time, but it is useful to have it in the rules if you need to step in and do something. Focus point: being part of something, being part of the community Involving the ASR in other things on KGS is extremely important. It is very important to tie the ASR league to KGS and to anything else which is familiar to KGS Go players. Having prizes of audiogolessons, having prizes related to atsumi school, insei league, etc. is EXTREMELY POSITIVE. Strong suggestion; add rewards such as 1-month KGS Plus memberships (a $5 value per winner), get sponsorship from a pro and advertise his service (suggest to the pro of donating one or two SHORT game reviews per month in return for trumpeting the fact he is doing this and how great his reviews are.. yes, include testimonials if at all possible, and links.)
Focus Point: Letting them know you care This sounds CHEESY but you would be surprised how WELL it works. On a regular basis, chat up members in the league. Just talk for a bit, offer to play a game, maybe a game outside the league structure. Offer to teach a weaker member. How often? Pfft, once a week. You don’t have to be their friend. You just have to show them that you are paying attention by paying attention to them, even for just 5 minutes a month. If somehow all of the 4 or 5 admins (and junior helpers, whomever) in the room manage to talk to every member for just five minutes every 2 or 3 months, believe me, everyone in the room will have a completely different picture of the ASR and the league. Make regular news releases. Talk about your plans for the future. Create some project which members can focus on. The ASRScore software is something like that. Focus on that. I’d even bring back the “usage edition” of goproblems.com, or even start a new collection for members, if you want, or perhaps a lecture series on memorizing shusaku games. But it must be institutionalized as a part of the room, for it to have the greatest effect. Meaning the information/material is somewhat separate and independently accessible without me. I can explain that in greater detail if need be.. I’m going to cut this short now because it’s late here. This is pretty much everything I’d like to say which is easily implenentatable, NOW. I really suggest that all this be taken seriously, and implemented swiftly.. the problems discussed are extremely easy to fix and don’t require any lasting investment of time…
Attachments: |
ptsbystr.PNG [ 43.65 KiB | Viewed 8150 times ]
|
ptsbyact.PNG [ 35.26 KiB | Viewed 8150 times ]
|
ASR 2010 The Math.docx [128.37 KiB]
Downloaded 379 times
|
|