Life In 19x19

Due to exams haven't had time to update the site for a while. I'll try to get some improvements done within the next few days.

Hi, I just discovered your site... and it's an immediate fall in love !
just great !

Hi,

I have a question about the ranking system. I have noticed that as I keep having correct results, my rank may get lower. From 4.3kyu to 1.6dan -> 4.3kyu to 1.9kyu for example even I have not made any mistakes. What's this about? Are the correct results worth different or something like that?
Great webpage btw.

fey wrote:[..] I have noticed that as I keep having correct results, my rank may get lower. From 4.3kyu to 1.6dan -> 4.3kyu to 1.9kyu for example even I have not made any mistakes.

they actually get higher http://senseis.xmp.net/?Rank

BUT careful, ootakamoku apparently estimates strength about 10 stones stronger than it actually is, at least it does so for me and some others I know.

And yes, it is a wonderful site

Greetings, Tom

Back to Fey's question -- among all the "correct" answers, are some more correct than others? From the observed behavior of the program, this seems likely. If so, could the relative values be indicated when you go back to review the answers?

Pretty awesome

mitsun wrote:Back to Fey's question -- among all the "correct" answers, are some more correct than others? From the observed behavior of the program, this seems likely. If so, could the relative values be indicated when you go back to review the answers?

I would also like if there would be some kind of note that you'r answer was correct but not the one with biggest points.

mitsun wrote:Back to Fey's question -- among all the "correct" answers, are some more correct than others? From the observed behavior of the program, this seems likely. If so, could the relative values be indicated when you go back to review the answers?

For now there are three levels of answers, wrong, right and uncertain. When you look at the position the mostly transparent stones shown are uncertain, where as the more solid ones are correct. Uncertain answers are not included in the rank calculations. As for rank dropping when you answer correctly, it has to do with it being a range, your average rank goes up, but the certainty of your rank increases so the maximum possible rank might actually come down faster. In anycase, there are some bugs with the ranking algorithm still, and Im working on a new version which will hopefully fix some of the problems.

Ootakamoku wrote: ... your average rank goes up, but the certainty of your rank increases so the maximum possible rank might actually come down faster...

This does not make sense mathematically. The rank probability distribution has some tail at the high end, with low probability for a high rank. When the only new information is a correct answer, those low probabilities should not decrease.

mitsun wrote:

Ootakamoku wrote: ... your average rank goes up, but the certainty of your rank increases so the maximum possible rank might actually come down faster...

This does not make sense mathematically. The rank probability distribution has some tail at the high end, with low probability for a high rank. When the only new information is a correct answer, those low probabilities should not decrease.

The 95% interval (assuming a 95% is used) shrinks. The amount of data reduces the likelihood of a large deviation.

RBerenguel wrote:

mitsun wrote:

Ootakamoku wrote: ... your average rank goes up, but the certainty of your rank increases so the maximum possible rank might actually come down faster...

This does not make sense mathematically. The rank probability distribution has some tail at the high end, with low probability for a high rank. When the only new information is a correct answer, those low probabilities should not decrease.

The 95% interval (assuming a 95% is used) shrinks. The amount of data reduces the likelihood of a large deviation.

Mitsun is correct. If a series of correct answers lowers the maximum rank level, that indicates that the rating algorithm is incorrect. (As I mentioned before, I have a bit of experience in a methodology course, having written an interactive Rasch testing algorithm, which estimated both the difficulty level of questions as well as the skill level of test takers, as Ootakamoku's site does.)

To see this, let us suppose that a 7 dan has been answering questions known to be at the 1 dan level, and furthermore, that he has answered enough questions that the tail of the player's skill level distribution below 1 dan is extremely small. Now he answers another 1 dan level question correctly. The shape of the distribution will change only slightly, but how will it change? The tail below 1 dan will shrink very slightly, while the tail above 7 dan will grow. It will grow because the probability that a 3 dan will get the answer correct is less than the probability that a 5 dan will get it correct, which is less than the probability that a 7 dan will get it correct, which is less than the probability that a high dan pro will get it correct. This means that the estimated skill level and also the maximum skill level will move up (unless the max is already at the maximum possible level). I chose this extreme case because if the lower tail below 1 dan is substantial, then it is possible to imagine that both tails move towards the mean more than the mean increases.

Bill Spight wrote:Mitsun is correct. If a series of correct answers lowers the maximum rank level, that indicates that the rating algorithm is incorrect. (As I mentioned before, I have a bit of experience in a methodology course, having written an interactive Rasch testing algorithm, which estimated both the difficulty level of questions as well as the skill level of test takers, as Ootakamoku's site does.)

To see this, let us suppose that a 7 dan has been answering questions known to be at the 1 dan level, and furthermore, that he has answered enough questions that the tail of the player's skill level distribution below 1 dan is extremely small. Now he answers another 1 dan level question correctly. The shape of the distribution will change only slightly, but how will it change? The tail below 1 dan will shrink very slightly, while the tail above 7 dan will grow. It will grow because the probability that a 3 dan will get the answer correct is less than the probability that a 5 dan will get it correct, which is less than the probability that a 7 dan will get it correct, which is less than the probability that a high dan pro will get it correct. This means that the estimated skill level and also the maximum skill level will move up (unless the max is already at the maximum possible level). I chose this extreme case because if the lower tail below 1 dan is substantial, then it is possible to imagine that both tails move towards the mean more than the mean increases.

But this assumes the answered question is of this same level, and also makes assumptions on what the ranking system is... and I guess they are different than mine I'm starting with the assumption that most solid questions are between 3k-5k, and as such, answering them correctly tends towards 3k-5kyushness. And it also depends on how Oota's algorithm is prepared, as I imagine it,it doesn't try to assign a meaningful rank (with "proper" properties), but only an estimation of an arbitrary number, as such it's almost like finding a fixed point by iteration. Each iterate (problem correctly solved) narrows down on a "correct" rank, reducing the distance to a correct rank "radius."

RBerenguel wrote:

Bill Spight wrote:Mitsun is correct. If a series of correct answers lowers the maximum rank level, that indicates that the rating algorithm is incorrect. (As I mentioned before, I have a bit of experience in a methodology course, having written an interactive Rasch testing algorithm, which estimated both the difficulty level of questions as well as the skill level of test takers, as Ootakamoku's site does.)

To see this, let us suppose that a 7 dan has been answering questions known to be at the 1 dan level, and furthermore, that he has answered enough questions that the tail of the player's skill level distribution below 1 dan is extremely small. Now he answers another 1 dan level question correctly. The shape of the distribution will change only slightly, but how will it change? The tail below 1 dan will shrink very slightly, while the tail above 7 dan will grow. It will grow because the probability that a 3 dan will get the answer correct is less than the probability that a 5 dan will get it correct, which is less than the probability that a 7 dan will get it correct, which is less than the probability that a high dan pro will get it correct. This means that the estimated skill level and also the maximum skill level will move up (unless the max is already at the maximum possible level). I chose this extreme case because if the lower tail below 1 dan is substantial, then it is possible to imagine that both tails move towards the mean more than the mean increases.

But this assumes the answered question is of this same level, and also makes assumptions on what the ranking system is... and I guess they are different than mine I'm starting with the assumption that most solid questions are between 3k-5k, and as such, answering them correctly tends towards 3k-5kyushness. And it also depends on how Oota's algorithm is prepared, as I imagine it,it doesn't try to assign a meaningful rank (with "proper" properties), but only an estimation of an arbitrary number, as such it's almost like finding a fixed point by iteration. Each iterate (problem correctly solved) narrows down on a "correct" rank, reducing the distance to a correct rank "radius."

That is only an illustration to make things clearer. It does not matter that the questions is at the same level. As Mitsun indicates, with each correct answer the lower tail shrinks and the upper tail grows.

RBerenguel wrote:I'm starting with the assumption that most solid questions are between 3k-5k, and as such, answering them correctly tends towards 3k-5kyushness.

It does not. Answering them correctly, assuming that the player is stronger than 3 - 5 kyu, tends towards the player's level, not towards the level of the question.

Edit: Unless it tends towards both at the same time, OC.

Edit2: Or away from both at the same time.

RBerenguel wrote: ... I'm starting with the assumption that most solid questions are between 3k-5k, and as such, answering them correctly tends towards 3k-5kyushness.

That is the mathematical mistake. Answering any question correctly should make your rating move upward (tend toward +infinity). Of course the gain will not be large unless the question is difficult for your current rating. But a decrease indicates a problem with the rating algorithm.

For the mathematically inclined, think of Bayes' theorem. We have a prior probability (rating distribution based on previous answers), plus a new piece of information (correct answer to a problem at a particular difficulty level), and we want to derive a new probability (rating distribution based on all answers). How does inclusion of the new information alter the probability? The new information is consistent with any rank above the difficulty of the problem, but actually becomes more likely as that rank increases, even for ranks well above the problem difficulty. So the new distribution must be pulled toward higher ratings by the inclusion of the new information.