Announcing GoReviewPartner - v0.15 with Pachi and PhoenixGo

Tell the community about tournaments, new go sites, software updates, etc.
Post Reply
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.6

Post by pnprog »

johnsmith wrote:9x9 is really puzzling.
Agreed!

But anyway, I made the modifications to now clearly indicate the MC values, policy values, network value. And the enigmatic R% value for 9x9 or 13x13.

This is the v0.6.1 available for download there: http://ns1297.imingo.net/ (or github)

By the way, the windows download is set up to use the non-opencl version of Leela, so if your computer can run Leela with GPU, make sure to modify the command line in the Settings menu.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
roy7
Dies in gote
Posts: 41
Joined: Sat Jan 28, 2017 8:36 pm
GD Posts: 0
OGS: roy7
Universal go server handle: roy7
Been thanked: 7 times

Re: Announcing GoReviewPartner - v0.6

Post by roy7 »

pnprog wrote:And the enigmatic R% value for 9x9 or 13x13.
It is the RAVE value. See here:

viewtopic.php?f=18&t=13692&p=216390&hilit=rave#p216390

:)
johnsmith
Lives with ko
Posts: 136
Joined: Fri May 27, 2016 3:55 am
Rank: KGS 6 dan
GD Posts: 0
Has thanked: 47 times
Been thanked: 21 times

Re: Announcing GoReviewPartner - v0.6

Post by johnsmith »

Thanks pnprog!

This tool makes me wanna play more deep and serious games! It's like having a strong teacher that can pinpoint your mistakes.
johnsmith
Lives with ko
Posts: 136
Joined: Fri May 27, 2016 3:55 am
Rank: KGS 6 dan
GD Posts: 0
Has thanked: 47 times
Been thanked: 21 times

Re: Announcing GoReviewPartner - v0.6

Post by johnsmith »

Players on reddit have noticed that a completed file was 2.3mb large and as such can't be uploaded to websites for online review. I am not a programmer myself but I have awful lot of ideas :) Maybe there can be a 'save as' option to keep the comments only for the main variation. File as such probably wouldn't be that large. And keep all other comments in the original app. Or just say "W=..., MC=..., VN=..." so the file will definitely be smaller. Some people suggested a toggle for various stuff, too.

My other idea is to have a graph in the app, below the left board for example, that will show you graphically where the game went wrong. Sgf will be smaller that way. I hope pngprog will like this idea :) Just like in Leela with 3 colors. And I just hope it is not a violation of copyright, right? When you have visual representation of the game you can easily notice which move was wrong and specifically analyze that one. When I briefly see difference between two moves with evals like 55.xy and 56.xy I immediately think it's not a bad move, but it may be almost 2% difference (56.99-55.01).

The third idea is to have all those evals (Win, Monte Carlo, Value) on the left board as well. Though file will be extremely huge, but oh well :)
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.6

Post by pnprog »

Hi!
johnsmith wrote:Players on reddit have noticed that a completed file was 2.3mb large and as such can't be uploaded to websites for online review. I am not a programmer myself but I have awful lot of ideas :) Maybe there can be a 'save as' option to keep the comments only for the main variation. File as such probably wouldn't be that large. And keep all other comments in the original app. Or just say "W=..., MC=..., VN=..." so the file will definitely be smaller. Some people suggested a toggle for various stuff, too.

My other idea is to have a graph in the app, below the left board for example, that will show you graphically where the game went wrong. Sgf will be smaller that way. I hope pngprog will like this idea :) Just like in Leela with 3 colors. And I just hope it is not a violation of copyright, right? When you have visual representation of the game you can easily notice which move was wrong and specifically analyze that one. When I briefly see difference between two moves with evals like 55.xy and 56.xy I immediately think it's not a bad move, but it may be almost 2% difference (56.99-55.01).

The third idea is to have all those evals (Win, Monte Carlo, Value) on the left board as well. Though file will be extremely huge, but oh well :)
Thanks you fore your comments. Redditors are right to say that it could be improved. In fact, all the comments from Leela for all variations are duplicated for each move of the variation... this is redundant, and so useless. You can see it if you open a rsgf file using a standard sgf editor (rename the file from *.rsgf to *.sgf).

I had never noticed that, so I fixed that for the next version (already on GitHub). I think this should already make for a huge size reduction.

I never imagined somebody would go as far as renting Google cloud computers and run GoReviewPartner on it :lol: that much power allows for a lot of branches, all branches being very deep. Hahaha, it's cool seeing people doing unexpected things with GoReviewPartner :tmbup:

Then, the real constraint is the sgf format. It's a standard, and so, I cannot take too much freedom with the format standard, otherwise the compatibility with other sgf editors (or eidogo) will be lost. I encourage you to open one of your sgf files using the windows block note (drag and drop the sgf file directly into it) to get an idea of how it work.

"W=..." instead of "black/white win rate for this move is..." will definitively help, and it could be a toggle. Another toggle to deactivate the influence analysis would help also.

Otherwise, the long term solution would be, in my opinion, to issue a new sgf standard, that takes in consideration all the new possibilities brought by computers being now stronger than humans. The last version of this standard, FF4, is already 20 years old. The new version could include standard properties like MTCS score, win rate, number of simulation.

In fact, I am considering introducing a few new properties in my rsgf files to make things easier (like drawing a win rate graph). If I am careful enough, I can do that without breaking the compatibility with FF4 readers (they would just ignore the new properties).

I guess there would be not copyright issue with a graph (such graph already existed before Leela, for example in The Many Faces Of Go). It is just that sgf format is not intended for that, I need to find a workaround.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.7 (now supporting zakki/

Post by pnprog »

Hi everybody!

Here come v0.7 !

Admittedly, I am late compared to my initial planning, but here it is, this version supports the zakki version of Ray (aka "Rayon") the most promising Opensource bot.

And this is more or less the only new feature of this version. Other notable change is that it now proposes to directly open the rsgf file for review at the end of the analysis. It also improves (decreases) the size of the rsgf file by removing duplicated comments.

I prepared the usual "Windows bundle", but because it now includes Ray, its size is much bigger. For other OS, it is best to download the code on GitHub and run it as an usual python script.

Unfortunately, I was not able to compile Ray on Linus so far. If somebody succeeds to do that for version 4.2.5, then I would be happy to offer a place to store the binaries online for everybody to download.

I did not test the Ray analysis thoroughly, at least not the gpu version (because I don't have one), so i will need your help to try that. The default settings in the Windows bundle run both Leela and Ray with no gpu, so make sure to change that if you want to try with gpu.

EDIT: For Ray to run, it needs CNTK to be installed. CNTK can be downloaded there: https://github.com/Microsoft/CNTK/releases
Last edited by pnprog on Sat Jun 17, 2017 9:46 pm, edited 1 time in total.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
Tiktaalik
Beginner
Posts: 5
Joined: Thu Jun 15, 2017 1:43 pm
GD Posts: 0
Has thanked: 1 time
Been thanked: 2 times

Re: Announcing GoReviewPartner - v0.7 (now supporting zakki/

Post by Tiktaalik »

Thanks for the lovely tool. I enjoyed using it with Leela.
However, I could not run it with Rayon for some reason. I got the following error message:

-----

bot selection: Ray
white only
========= move selection
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98, 100, 102, 104, 106, 108, 110, 112, 114, 116, 118, 120, 122, 124, 126, 128, 130, 132, 134, 136, 138, 140, 142, 144, 146, 148, 150, 152, 154, 156, 158, 160, 162, 164, 166, 168, 170, 172, 174, 176, 178, 180, 182, 184, 186, 188, 190, 192, 194, 196, 198, 200, 202, 204]
Exception in Tkinter callback
Traceback (most recent call last):
File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1540, in __call__
return self.func(*args)
File "/home/nikoryu/Go/programs/goreviewpartner/toolbox.py", line 337, in start
self.popup=RunAnalysis(newtop,self.filename,move_selection,intervals)
File "/home/nikoryu/Go/programs/goreviewpartner/ray_analysis.py", line 39, in __init__
self.initialize()
File "/home/nikoryu/Go/programs/goreviewpartner/ray_analysis.py", line 291, in initialize
alert("Could not run Ray using the command from config.ini file (\""+Ray_command_line+"\")")
NameError: global name 'Ray_command_line' is not defined

-----

Do you have any idea how to fix this?
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.7 (now supporting zakki/

Post by pnprog »

Hi!
Thanks for the lovely tool. I enjoyed using it with Leela.
However, I could not run it with Rayon for some reason. I got the following error message:
This is apparently an issue running Ray. I forgot to mention that Ray needs CNTK to be used. CNTK can be downloaded there: https://github.com/Microsoft/CNTK/releases

Please have a try after installing CNTK and let me know.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
Raspingbard
Beginner
Posts: 2
Joined: Sun Apr 21, 2013 5:20 pm
Rank: 2D
GD Posts: 0

Re: Announcing GoReviewPartner - v0.7 (now supporting zakki/

Post by Raspingbard »

I keep reading where GoReviewPartner runs on the Mac, but all I see is "For Linux/Mac, it is best to get the source code from Github." What do I do once I download the source code? Could someone give me step-by-step instructions once that's done?

(I already have the Leela GTP engine running under another GUI on the Mac.)

Thanks
Tiktaalik
Beginner
Posts: 5
Joined: Thu Jun 15, 2017 1:43 pm
GD Posts: 0
Has thanked: 1 time
Been thanked: 2 times

Re: Announcing GoReviewPartner - v0.7 (now supporting zakki/

Post by Tiktaalik »

pnprog wrote: This is apparently an issue running Ray. I forgot to mention that Ray needs CNTK to be used. CNTK can be downloaded there: https://github.com/Microsoft/CNTK/releases

Please have a try after installing CNTK and let me know.
Actually, I had already installed CNTK and was running Rayon using GPU.
Anyway, I found that the main problem was that I used a wrong path to ray binary.. :oops:
I compiled a couple of different versions of Rayon and then moved some of them to my archive directory, so I somehow mixed those up and used a non-existing path..

A minor thing I would like to inform you is that there was a typo and a syntax error in one of your scripts.
Specifically, in line 291 of your "ray_analysis.py", "Ray_command_line" should be "ray_command_line".
Also in this line, it seems you cannot simply use "+" sign to join error message and the ray command line, which is a list. I just borrowed a code you used in the "leela_analysis.py" and modified it as:
alert("Could not run Ray using the command from config.ini file:\n"+" ".join(ray_command_line))
This enables the GUI pop-up as you intended.
Anyway, the fact that this part of the script is triggered means that your Ray program is not running properly, so this is really a minor thing, but just so you know..
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.7 (now supporting zakki/

Post by pnprog »

Tiktaalik wrote:
pnprog wrote: This is apparently an issue running Ray. I forgot to mention that Ray needs CNTK to be used. CNTK can be downloaded there: https://github.com/Microsoft/CNTK/releases

Please have a try after installing CNTK and let me know.
Actually, I had already installed CNTK and was running Rayon using GPU.
Anyway, I found that the main problem was that I used a wrong path to ray binary.. :oops:
I compiled a couple of different versions of Rayon and then moved some of them to my archive directory, so I somehow mixed those up and used a non-existing path..

A minor thing I would like to inform you is that there was a typo and a syntax error in one of your scripts.
Specifically, in line 291 of your "ray_analysis.py", "Ray_command_line" should be "ray_command_line".
Also in this line, it seems you cannot simply use "+" sign to join error message and the ray command line, which is a list. I just borrowed a code you used in the "leela_analysis.py" and modified it as:
alert("Could not run Ray using the command from config.ini file:\n"+" ".join(ray_command_line))
This enables the GUI pop-up as you intended.
Anyway, the fact that this part of the script is triggered means that your Ray program is not running properly, so this is really a minor thing, but just so you know..
Good job in solving your issue and finding that bug in my python code.

I am currently travelling until end of July, "away from keyboard", so I cannot work on GoReviewPartner for now. I will fix that when I am back.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
Tapani
Dies with sente
Posts: 96
Joined: Tue Mar 22, 2016 9:53 pm
Rank: SDK
GD Posts: 0
IGS: 8k
Has thanked: 38 times
Been thanked: 16 times

Re: Announcing GoReviewPartner - v0.6

Post by Tapani »

johnsmith wrote: My other idea is to have a graph in the app, below the left board for example, that will show you graphically where the game went wrong.
If anyone else is interested in something like this - the following Linux shell-script will read the rsgf file and plot a graph over black's win probability.

Code: Select all

#!/bin/sh

if [ X$1 = X -o ! -f $1 ]
then
	echo Need rsgf file as argument
	exit 1
fi

grep -e '^C\[Move' -e '^Leela' $1 | tr '\n' ' ' | tr ']' '\n' | cut -d ' ' -f 3,11 | cut -d '%' -f1 | grep -v Move > /tmp/g.dat
gnuplot -p -e "set yrange [0:100]; plot '/tmp/g.dat' with lines"
rm -f /tmp/g.dat
It is a 30 minute three-liner, so don't expect too much :)

Oh, and it only works with Leela's analysis output.
User avatar
djhbrown
Lives in gote
Posts: 392
Joined: Tue Sep 15, 2015 5:00 pm
Rank: NR
GD Posts: 0
Has thanked: 23 times
Been thanked: 43 times

Re: Announcing GoReviewPartner - v0.7 (now supporting zakki/

Post by djhbrown »

win% is as much a function of how bad opp's last move was as it is of how good your move was. a more meaningful indicator of where your play was deficient (according to your chosen oracle) is the difference between win% of your move and win% of your oracle's best move for that same position.

graphs are useful tools for seeing the relationship between two variables, so if you wanted to see how your performance behaves over time, a graph of win% over time would be a suitable instrument. on the other hand, colour coding of your moves would show you which of them were below par:

your move win% - oracle best move win%
< - 5% red
+- 5% green
> 5% yellow (you significantly bettered the machine)

5 is an arbitrary number; a more statistically meaningful one might be sd of the distribution of historical differences - then, over several games, you would discover whether or not you were improving. another way of judging that would be to glance at your rank graph, which will probably tell you that you are not because you fondly imagine that you only lost because you went wrong on one particular move, whereas it is your overall strategy and tactics and reading ability that are wanting because you are trying to reinvent the wheel, learning by experience of playing other weak players instead of reading a book and standing on the shoulders of wise (wo)men down the ages. alphago needed millions of trials to improve by trial and error and self-flagellation, what makes you think you are any different?
i shrink, therefore i swarm
Tapani
Dies with sente
Posts: 96
Joined: Tue Mar 22, 2016 9:53 pm
Rank: SDK
GD Posts: 0
IGS: 8k
Has thanked: 38 times
Been thanked: 16 times

Re: Announcing GoReviewPartner - v0.7 (now supporting zakki/

Post by Tapani »

djhbrown wrote:win% is as much a function of how bad opp's last move was as it is of how good your move was. a more meaningful indicator of where your play was deficient (according to your chosen oracle) is the difference between win% of your move and win% of your oracle's best move for that same position.

graphs are useful tools for seeing the relationship between two variables, so if you wanted to see how your performance behaves over time, a graph of win% over time would be a suitable instrument. on the other hand, colour coding of your moves would show you which of them were below par:

your move win% - oracle best move win%
< - 5% red
+- 5% green
> 5% yellow (you significantly bettered the machine)

5 is an arbitrary number; a more statistically meaningful one might be sd of the distribution of historical differences - then, over several games, you would discover whether or not you were improving. another way of judging that would be to glance at your rank graph, which will probably tell you that you are not because you fondly imagine that you only lost because you went wrong on one particular move, whereas it is your overall strategy and tactics and reading ability that are wanting because you are trying to reinvent the wheel, learning by experience of playing other weak players instead of reading a book and standing on the shoulders of wise (wo)men down the ages. alphago needed millions of trials to improve by trial and error and self-flagellation, what makes you think you are any different?
Sorry if my quick-hack script offended you somehow. How about posting your code for those move-by-move oracle comparisons - it sounds cool indeed.
Uberdude
Judan
Posts: 6727
Joined: Thu Nov 24, 2011 11:35 am
Rank: UK 4 dan
GD Posts: 0
KGS: Uberdude 4d
OGS: Uberdude 7d
Location: Cambridge, UK
Has thanked: 436 times
Been thanked: 3718 times

Re: Announcing GoReviewPartner - v0.7 (now supporting zakki/

Post by Uberdude »

Crazy-sensei, the online analysis version of CrazyStone from Remi Coulom, displays mistake bars like this, showing how much your move differs from the bot's suggestion. I think it works nicely: see the red bars on https://www.crazy-sensei.com/?lang=en&l ... =47&move=0 for example.
Post Reply