Announcing GoReviewPartner - v0.15 with Pachi and PhoenixGo

[pnprog]

If anyone else is interested in something like this - the following Linux shell-script will read the rsgf file and plot a graph over black's win probability.

I just ran it and it works well
But the grep/cut command looks like black magic to me

Seems like the win rate graph is becoming more and more popular those days. Maybe I should consider adding one into GoReviewPartner.

I just had a look at the link to Crazy Sensei provided by Uberdude, it's quite similar to GoReviewPartner, the interface is nice.
Regarding the graph, it is described as:

Red bars indicate errors. Errors of Black are below the middle line, and errors of White are above.

I think it should read more like:

Red bars indicate where Crazy Sensie believes he could have find a better move

To build upon djhbrown comment, and if my understanding is correct, Crazy Sensei compares the win rate % of the players moves with that of its own move for each board position. Then, there are three cases:

• A better win rate is found, then a red bar is displayed
• The same move was played, or the win rate is exactly the same, no display
• A lower win rate is found, meaning that the player played a move that Crazy Sensei evaluates as better than it's own play...

The third case is apparently ignored by Crazy Sensei (yes, it makes sense from Crazy Sensei's author point of view ). But still it would be interesting (it would be fair) to display that as well.

I will consider that if I implement a graph into GoReviewPartner.

[djhbrown]

re graphs: my comment was a response to John Smith's suggestion. in it, i tried to explain that win rate graphs are misleading, in that they don't necessarily show you where you went wrong, since your opponent can be going wrong too.

my suggestion was, and is, that instead of a separate display, you colour in your stones on the board according to their "temperature" ie how hot (good) each move is as adjudged by your oracle; i chose red/green/yellow since it will work whether you are black or white (an alternative colour scheme could be red = significantly worse than oracle, green = better; otherwise normal stone colour). i imagine it might produce some interesting and possibly informative colour patterns across the board by the end of the game. grp could have a toggle button to switch colours on/off.

graphs are useful visual tools for comparing two variables, showing the distribution of one variable (such as prowess at Go, or blood pressure, or weight, or money in the bank) over another (such as space or time; eg you vs everyone else, or you vs yourself over time). graphs are handy for providing an oversight of things like trends, or variability, or distribution. they can correct basic misunderstandings, such as the notion of "middle class", a term which implies a normal distribution of income or capital across a population. it's a term which derives from intuitive notions that there are three kinds of people: rich, middle and poor. in US culture, it is tacitly assumed by economics commentators that most people are middle class - but nothing could be further from the truth! here is a graph of Australian household wealth in 2008; if you were to make one for US or indeed any country in the modern era, you would find a similar picture: a tiny fraction with most of the wealth, and all other class intervals (eg deciles) having virtually nothing by comparison, as shown by the flatline shape of the graph on the left, apart from its top end. so there are just two classes: them and us. there is no middle.

[pnprog]

Hi there!

So I have been playing around a bit with graphs in GoReviewPartner this week. Here is what it looks like so far:

As an example, I took one game from the Game Analysis section between negapesuo and wqz2017, ran the analysis at 20s/move on my old computer, and get that graph for player negapesuo (black):

chart.png (4.16 KiB) Viewed 21636 times

This is black player win rate, starting from a position higher than 50% (that was a 2 stones handicap game) with black managing to keep his advantage during game the until the final blunder at the end that cost him the game.

So you will notice red and green rectangles, here is what they mean:

• Red means that for that position, Leela believes she has find a better move than negapesuo. If that move had been played instead of actual game move, then the win rate would have been increased by the red quantity.
• Green means that Leela believes that negapesuo came up with a better move that Leela itself. If Leela's move had been played instead of the actual game move, then the win rate would have decreased by the red quantity

So red means margin for improvement, and green means better than Leela.

The very last move win rate, when added the red quantity, exceed 50%, meaning that this blunder was indeed a game reversal move.

A bar diagram may not be conveying this idea very well, I am not so sure, because it is not obvious that the red bar is higher than the grey bar, and that the green bar is lower than grey bar. If I replace the grey bars with a blue line, it would look like this:

chart1.png (4.41 KiB) Viewed 21636 times

But somehow, I don't like it that much.

You will notice this graph is only for one player, not both players at the time (it's becoming too confusing then). In fact, only half the moves are displayed (even moves for black player here, because handicap stones so white played first). And so GoReviewPartner would propose 3 graphs:

• The usual win rate graph
• The comparison between Leela/Ray and the black player (like the one I showed)
• he comparison between Leela/Ray and the white player

Then, I plan to add a few things to make it easier to use, like displaying detailed stats for each moves when the mouse is over the bar, and clicking on the bar would go directly to that move in the analysis windows.

[Dontbtme]

Wow, that's huge! Bravo!
I think one way you could change how you feel about the grey bars would be to just lighten their shade a bit so that the red would shine more by contrast...
Just an idea.
But it's looking promising already, so good job!

[pnprog]

Wow, that's huge! Bravo!
I think one way you could change how you feel about the grey bars would be to just lighten their shade a bit so that the red would shine more by contrast...
Just an idea.
But it's looking promising already, so good job!

Thank four your idea!

I think I will go for this intermediate solution (this is also a way to make the grey bars lighter in fact):

chart2.png (4.72 KiB) Viewed 21622 times

Some area have no grey bar at all, indicating no data from Leela, I don't know why yet, I will need to look into it.

[roy7]

The way Crazy Sensei handles the red/green thing is by having a graph making "made the same move Leela made" as 0, and then the bars are above or below zero by the amount better or worse than Leela. Makes it easy to instantly spot the big dips or better moves you made.

[Monadology]

I actually strongly prefer the original grey bar graph over any of the alternatives presented so far.

[djhbrown]

For comparison, https://www.crazy-sensei.com says:

Description of plots

The plot with the blue line indicates Black's probability of winning as a function of move number
Red bars indicate errors. Errors of Black are below the middle line, and errors of White are above.
The histogram shows the distribution of scores in random game playouts. White bars are White wins, black bars are Black wins. Jigo is a red bar.

An "error bar" is a conventional statistical term, defined as deviation from the mean.
https://egret.psychol.cam.ac.uk/statist ... orbars.htm

[pnprog]

Hi!

I have more or less finish implemented support for graphs, for Leela and Ray. The final result looks like this for the game I mentioned above:

Win rate graph:

Win rate graph

chart0.png (5.5 KiB) Viewed 21527 times

Comparison between black and bot:

Comparison between black and bot

chart1.png (5.14 KiB) Viewed 21527 times

Comparison between white and bot:

Comparison between white and bot

chart2.png (5 KiB) Viewed 21527 times

The missing data are when Leela did not provided a win rate, this can be:

• Leela is using a move from fuseki dictionary
• Leela would resign

For example, to evaluate the player move (as black) with Leela's move (as black) for move N, I need Leela's evaluation for N and for N+1, then I will compare those 2 win rates:

• Win rate of Leela's move as black for move N
• Win rate of Leela's answer (move N+1) as white to the actual game move played as black (move N)

If one of those two is missing, I cannot display the comparison, only the usual win rate. So if one decide to only analyse black move or white move, the comparison charts are not available.

As it is now, moving the mouse over the chart displays the detailed stats on the status bar. Click on the bar will bring the the game position in the main analysis windows. I will add an easy way to export the graph as image.

I will polish a bit more and release a v0.8, but new code has been pushed to Github if some of you want to try.

[Satorian]

This looks amazing! I'm more interested in looking at my three to five biggest mistakes per match with reasonable continuations for better moves instead of a minute and detailed look at every move, and this should make it very easy to find those in my games. Looking forward to it!

[djhbrown]

I'm more interested in looking at my three to five biggest mistakes per match

the biggest red bars are your biggest blunders, and they do stand out on the graph, but how much nicer would it be if grp could highlight them on the board itself? easy enough to compute - eg those that are more than 2 standard deviations from the mean difference between you and the oracle.

being shown where you went wrong is useful, but Monte-Carlo can't tell you why it is wrong - for that, you need either Swim or Michael Redmond.

[pnprog]

being shown where you went wrong is useful, but Monte-Carlo can't tell you why it is wrong - for that, you need either Swim or Michael Redmond.

Well, as soon as there is a GTP version of Michael Redmond available for download, I will add support for Michael Redmond in GoReviewPartner

But in the meantime, here is GoReviewPartner v0.8, the new features are:

• Win rate graph and comparison graph, as explained above
• Added a way to select what game variation to use for the analysis, in case the SGF tree has more than one branch

To have the graphs work in a elegant way, I added some "private SGF properties" to the RSGF file, that I use to store the informations needed to plot the charts. Namely:

• BWR: black win rate
• WWR: white win rate
• CBM: computer best move

I plan to use this sort of non standard SGF properties more intensively in the future, instead of passing the information through the SGF comments. This is a point where I really think the SGF standard needs an update. Clearly, we are entering a new era of Go, where computers are showing the direction, and the SGF format was not designed with that in mind 20 years ago...

Beside, I also put a lot of effort into writing some (much needed) documentation on how to install, configure and use GRP. I also separated the windows download in 3, depending of what bot you plan to use, this make the download much faster. Finally, I made a home page for this project, that looks like a real project home page

So as usual:

• Windows download from the home page
• Source code from Github

I have not real to do list for v0.9, GRP is full-filling all my needs now. What I have in head for a potential next version:

• I will probably spend a bit of time doing some code re-factoring, to make it easier to add support for other bots, a bit like add-ons.
• I am also thinking in adding support for other language, not only English
• I need to find a way to export the pictures in *.png, not *.ps
• I am opened to adding support for other strong bots

Let me know if any issue or comments!

[djhbrown]

a GTP version of Michael Redmond

i guess there is such a thing, for as a professional, i am sure Michael would tell you his move for a small fee. A freeware Michael is a different matter; you would have to ask pnprog to make one for you based on Swim.

GRP could report (YourWR - 50%) by a traffic light next to your name on the display - then, keeping your finger on the arrow (next move) button would create an animated colour change as the game progresses.

... and at the risk of repeating myself, at the risk of repeating myself, you could colour the Your stones with a colour gradient according to temperature(YourWR-OracleWR) instead of drawing a graph.

the traffic light tells you when you started to lose (or win), and the stone colours tell you how much each of your moves is wrong

as to other bots, there is one called "gomap".... it doesn't make moves but it does draw pretty pictures that could be a useful learning aide, if it were available as an optional display toggle button; eg when the button is down, the gomap is displayed on the board, and when it is up, it isn't. if the gomap toggle were the letter "c" (or caps lock), you could press/unpress it with the left hand, leaving your right hand free to press/unpress the arrow button.

[pleiade67]

Tested version 0.8 on Mac. Everything works perfectly with Gnugo and Leela (the graphs are a most useful feature). Thank you so much.

If I may be permitted to add one more suggestion:
- an option for batch processing several games (useful for processing a collection of games overnight).
Besides, If you consider introducing go board and stone artwork, there are some available. For instance by the author of sabaki sgf editor:
https://github.com/yishn/Sabaki/tree/master/img/goban
There was also a forum thread on this topic: For developers: Easy ways to render nice stone graphics:
forum/viewtopic.php?f=10&t=10226

Best,

[pnprog]

Tested version 0.8 on Mac. Everything works perfectly with Gnugo and Leela (the graphs are a most useful feature). Thank you so much.

Thanks for reporting it working on Mac, that's the one platform I cannot test myself!

an option for batch processing several games (useful for processing a collection of games overnight).

At the very beginning, that was possible to run the analysis from a terminal command, even in a no graphical environment. The possibility was lost as the application became more and more "graphical"... but I plan to bring it back. In fact, originally, I had made a script that would check every few hours for new game on my OGS account, then run an analysis automatically if new games were played. As I used to play at work during lunch time, I would have the analysis ready when I get home.

Besides, If you consider introducing go board and stone artwork, there are some available. For instance by the author of sabaki sgf editor:
https://github.com/yishn/Sabaki/tree/master/img/goban

Those are nice. I guess I really should try to make GRP more visually appealing