Announcing GoReviewPartner - v0.15 with Pachi and PhoenixGo

Tell the community about tournaments, new go sites, software updates, etc.
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

johnsmith wrote:Right after it played the last move, I immediately opened "start review" but there were only 7 moves analyzed. So I closed it, let the anlysis finish and then opened it again. And just then there were all the moves. So I was confused... it first calc-ed answers to my moves (up to move 12), then went back to move number 7 and calc-ed 7,8,9,10,11,12? Does it lose previous calcs that way? Cuz it may had an idea when played certain move, but had to find it again? Or it just cal-ed 7,9,11 beacuse 8,10,12 were stored somewhere? :)


What happened is that you had 2 instances of Leela running at the same time on your computer (so they where not at the same game move). One that was performing the analysis, and one that was playing as white (with the second in advance because it does not have to analyse black moves). In this case, yes they won't necessary agree on the best move.

If you want to have only one instance, you needs to select the bot that is labelled "Bot used for the analysis (XXXXX)":
live_img0.png
live_img0.png (35.98 KiB) Viewed 20975 times


The drawback in this case is that you have to wait for the bot to finish the analysis of its move AND your move before it continues playing :( So it's fine only if you take much more time thinking about your move than it does. (<- which is the case for me :lol: )

Note that if more than one bot has to be used, then you will see the option to make sure they don't think at the same time appears (see, the check box "No overlap thinking time" is not on the first image above):
live_img1.png
live_img1.png (25.72 KiB) Viewed 20975 times


At the beginning, I had considered something like you say: play move 12, then go back to move 3 for the analysis, then move again to move 14 if the opponent played move 13. But yes, it's a big big headache.

In fact, I already had a very very hard time implementing the "Undo" feature considering the analyser and the players "work" on the same SGF tree (players extend the first branch, analyser adds other branches at each node, so one need to be careful when removing nodes of the first branch).

The way I am using it now is having the slow instance performing the analysis, and the fast instance playing white or black. I don't use the thinking time limit any more, only the playouts limit from the command line. Plus, on Linux (but should be doable on windows) I set the analyser instance as a low priority process, so it won't slow down the fast instance. But because I use a playout limit without time limit, the quality of the analysis is still the same.

Edit: I should probably just use human vs human in live analysis (just like last night) and everything is just perfect :)
Yes, Human VS Human is what I imagine as "spectator mode" or "kibitz mode" :lol:

There are still plenty of work in progress before the next release, I have plenty of things to polish, and I will have to spend time on the documentation as well.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
johnsmith
Lives with ko
Posts: 136
Joined: Fri May 27, 2016 3:55 am
Rank: KGS 6 dan
GD Posts: 0
Has thanked: 47 times
Been thanked: 21 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by johnsmith »

Oh, I see now, thanks for clarification! I will use "Bot used for the analysis (XXXXX)" when necessary.
So it's fine only if you take much more time thinking about your move than it does. (<- which is the case for me :lol: )
Me too :) I also like using constant number of playouts/visits instead of time settings just because of these overlapings. Or maybe something else you're doing on your computer.
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

snippy wrote:Hi again :)

Might there still be a bug concerning handicap stones?

I played a high handicap game against leela 0.11 and lost (White wins by -9 + 0.5 Komi + 14 Handicap).

But after analyzing the game with GRP (with leela as bot) the win graph always stays around 100% win for black and for white leela even resigns close before endgame and keeps resigning every move just until the game ended. How can that be?
I spent quite a bit of time investigating this issue with the help of snippy.

As it appears, the issue was due to Leela value network unable to estimate games with too high number of handicap stones. In this specific case, the value network was giving black winner by a lot, and the Monte Carlo simulations were giving white winner by a lot. The winning rate (a function of both previous estimations) was around 50%, slightly above.

This pushed me to implement graphs and "delta graphs" (<- this is the new name I gave to the "Comparison graph for black/white") for Value Network and Monte Carlo estimations. In case of doubt with the win rate, it is better to double check the two other "metrics":
all.png
all.png (99.76 KiB) Viewed 20922 times


Ray, AQ and Leela have the three metrics available. Leela Zero has only the Value Network one (that is wrongly named Win Rate at the moment). (GnuGo has none of them :lol: )

As I was investigating this issue, I found an unexpected result with AQ and Ray. Both of them, in Chinese rule set, in handicap games:
  • Do not add the number of handicap stones to white score (this could be ok depending of the rule set used)
  • Only consider a komi of 0.5, even if the komi was set at a different value

Leela, LeelaZero and GnuGo all account for the number of handicap stones when calculating white score.

I opened issues on both projects [1][2]. I mention that so that until it is fixed, be careful when playing handicap games against Ray and AQ.

I also noticed some behaviour that may be a violation of the GTP protocol for Leela and Leela Zero:
When they choose to resign, they create a move in their move history that requires to be undo if one want to go back in the game tree.
  • GnuGo does not consider "Resign" as a valid move.
  • GoGui also agrees with GnuGo interpretation of the standard.
  • Sabaki avoid this issue by replacing "undo" by "clearboard" and replaying the whole game up to the previous move.
  • Ray does not undo.
  • AQ undo command is broken at the moment (did not check for the previous version behaviour).

I think this is an area where the GTP protocol documentation is not clear enough. I did not opened an issue for this one, seems everybody has its work around. But clearly, this poses problem when using Leela/LeelaZero with Gogui.

This was giving me headaches for the live analysis with Leela/LeelaZero. This is for such cases that I am happy I kept GnuGo around. It's implementation of the GTP protocol is somehow the reference implementation. So when it works with GnuGo but not for another bot, I can start to look into such details.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

pnprog wrote:I also noticed some behaviour that may be a violation of the GTP protocol for Leela and Leela Zero:
When they choose to resign, they create a move in their move history that requires to be undo if one want to go back in the game tree.
  • GnuGo does not consider "Resign" as a valid move.
  • GoGui also agrees with GnuGo interpretation of the standard.
  • Sabaki avoid this issue by replacing "undo" by "clearboard" and replaying the whole game up to the previous move.
  • Ray does not undo.
  • AQ undo command is broken at the moment (did not check for the previous version behaviour).

I think this is an area where the GTP protocol documentation is not clear enough. I did not opened an issue for this one, seems everybody has its work around. But clearly, this poses problem when using Leela/LeelaZero with Gogui.
Apparently, this has been fixed for Leela Zero in v0.10, v.11 or v.12 :tmbup:
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

This is me again!

A gentleman on Github has proposed a different layout for GRP, that integrate the graph inside the main view:
new_layout0.png
new_layout0.png (20.86 KiB) Viewed 20893 times


He went as far as to make a modification of the code to get a better idea. It would looks more or less like that (consider this as a draft, it would need some more polishing):
new_layout.png
new_layout.png (214.83 KiB) Viewed 20893 times


I also believe GRP GUI would require a new layout. I would like to ask all of you for suggestion, comments, and so on. Maybe I should start a different thread for that, but I don't want to spam L19 with GRP threads :lol:

For reference, the initial discussion is on github there.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
johnsmith
Lives with ko
Posts: 136
Joined: Fri May 27, 2016 3:55 am
Rank: KGS 6 dan
GD Posts: 0
Has thanked: 47 times
Been thanked: 21 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by johnsmith »

I like this new layout idea. But only wonder how would graph fit in such a small area, would you be able to see red and green deltas?
redreoicy
Beginner
Posts: 5
Joined: Thu Jan 23, 2014 8:29 pm
Rank: Eternal KGS 4d
GD Posts: 0
KGS: redreoicy
Online playing schedule: T-f
Been thanked: 1 time

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by redreoicy »

Does anyone know how to use updated versions of leelazero with GoReviewPartner? (current weights and/or .exe)
Edit: I think I've figured out the weights, but not how to use updated leela versions.
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

redreoicy wrote:Does anyone know how to use updated versions of leelazero with GoReviewPartner? (current weights and/or .exe)
Edit: I think I've figured out the weights, but not how to use updated leela versions.

It works the same way as for example, Sabaki:
  • Download and unzip Leela Zero
  • Download the weight file you plan to use
  • In GRP setting panel, Leela Zero section, indicate the path to leelaz.exe and the parameters tu run it in GTP with proper weight file.

Beware that latest version of Leela will forcefully run the tuner when starting the bot the first time. This will block GRP for a while.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
redreoicy
Beginner
Posts: 5
Joined: Thu Jan 23, 2014 8:29 pm
Rank: Eternal KGS 4d
GD Posts: 0
KGS: redreoicy
Online playing schedule: T-f
Been thanked: 1 time

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by redreoicy »

Always I get an error saying
ERROR: Leela Zero did not identified itself as expected 'Leela Zero' != ''
and nothing happens :(

But when I completely replaced the Leela Zero folder with a fresh download of the windows 64 (notcpuonly) it is working. I'm not sure if the type of leela I downloaded mattered (earlier I downloaded cpuonly as I wasn't sure if my computer had a gpu or if there was something else I did differently. Anyway it all seems well now.

By the way, thanks for the help and amazing program!
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

redreoicy wrote:Always I get an error saying
ERROR: Leela Zero did not identified itself as expected 'Leela Zero' != ''
and nothing happens :(
The error message above typically appears when Leela Zero fails to start in GTP mode. It could be that the program fails because it cannot find the weight file, or the --gtp parameter was not added.

On the next version, I have added a handful of error detection for Leela Zero, related to weight file, this will hopefully avoid more headaches.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
Sneegurd
Lives with ko
Posts: 129
Joined: Fri Mar 23, 2012 8:57 am
GD Posts: 0
Has thanked: 20 times
Been thanked: 17 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by Sneegurd »

What is this? Installed Python, use absolute path for Leela. Weightsfile is "bestnet" here. Saving the settings in UI it says below. It's on Win7x64, thanks!

Path: c:\Users\Michel\programme\Drago\Engines\LeelaZero\leelaz.exe
Settings: gtp --noponder --weights bestnet

Code: Select all

STDIN encoding: cp850
STDOUT encoding: cp850
STDERR encoding: cp850
File system encoding: mbcs
GRP path: C:\Users\Michel\Programme\GRP
Config file: C:\Users\Michel\Programme\GRP\config.ini
Reading language setting from config file
lang=de
Loading translation file: C:\Users\Michel\Programme\GRP\translations\de.po
Could not import the WX GUI library, please double check it is installed:
No module named wx
=> Falling back to tkFileDialog
Checking availability of config file
refreshing
Adding new popup
Totally 1 popups


Code: Select all

Saving LeelaZero settings
refreshing
Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Python27\lib\lib-tk\Tkinter.py", line 1541, in __call__
    return self.func(*args)
  File "C:\Users\Michel\Programme\GRP\leela_analysis.py", line 465, in <lambda>
    Button(self, text=_("Test"),command=lambda: self.parent.parent.test(self.gtp
,"fast")).grid(row=row,column=1,sticky=W)
  File "C:\Users\Michel\Programme\GRP\settings.py", line 218, in test
    from gtp_terminal import Terminal
ImportError: No module named gtp_terminal
Initializing GRP setting interface
Initializing LeelaZero setting interface
User avatar
pnprog
Lives with ko
Posts: 286
Joined: Thu Oct 20, 2016 7:21 am
Rank: OGS 7 kyu
GD Posts: 0
Has thanked: 94 times
Been thanked: 153 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by pnprog »

Hi!
Sneegurd wrote:

Code: Select all

STDIN encoding: cp850
STDOUT encoding: cp850
STDERR encoding: cp850
File system encoding: mbcs
GRP path: C:\Users\Michel\Programme\GRP
Config file: C:\Users\Michel\Programme\GRP\config.ini
Reading language setting from config file
lang=de
Loading translation file: C:\Users\Michel\Programme\GRP\translations\de.po
Could not import the WX GUI library, please double check it is installed:
No module named wx
=> Falling back to tkFileDialog
Checking availability of config file
refreshing
Adding new popup
Totally 1 popups

This is normal: originally, GRP used the tkinter (GUI tollkit) file pickers (the widgets used to select files to open or save). Then a contributor sent a patch to replace those tkinter widget by the file pickers of wxpython, another GUI toolkit, because the latter are more beautiful and easier to use. They are the same as the system, with your usual bookmark. So if you happen to have wxpython installed on your system (which is often the case for Linux) then it will be used. Otherwise, GRP uses the usual tkinter widget.

In fact, for the future, I am considering moving entirely GRP to wxpython.


Sneegurd wrote:

Code: Select all

Saving LeelaZero settings
refreshing
Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Python27\lib\lib-tk\Tkinter.py", line 1541, in __call__
    return self.func(*args)
  File "C:\Users\Michel\Programme\GRP\leela_analysis.py", line 465, in <lambda>
    Button(self, text=_("Test"),command=lambda: self.parent.parent.test(self.gtp
,"fast")).grid(row=row,column=1,sticky=W)
  File "C:\Users\Michel\Programme\GRP\settings.py", line 218, in test
    from gtp_terminal import Terminal
ImportError: No module named gtp_terminal
Initializing GRP setting interface
Initializing LeelaZero setting interface
Yep, that was a bug in the development version.

I just published the version 0.11 of GRP, you can download it for windows from here: http://ns1297.imingo.net/ (make sure to refresh the page, F5, in case you visited it recently). I haven't finished updating the documentation, so no announcement yet.
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!
Caesura
Beginner
Posts: 12
Joined: Tue Dec 19, 2017 11:31 am
Rank: Tygem 5d
GD Posts: 0
Has thanked: 3 times
Been thanked: 1 time

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by Caesura »

Hi, is it possible to provide an option to download GRP with CPU-only Leelazero? Thank you!
snippy
Beginner
Posts: 4
Joined: Sat Feb 10, 2018 4:21 am
GD Posts: 0

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by snippy »

Hi there,

would it be possible to change the table behavior in such a way, that it auto-updates when you step through the game moves? For me it will stay on the move that it has been opened with!

Take care and thanks a lot!
Sneegurd
Lives with ko
Posts: 129
Joined: Fri Mar 23, 2012 8:57 am
GD Posts: 0
Has thanked: 20 times
Been thanked: 17 times

Re: Announcing GoReviewPartner - v0.10 (supporting Leela Zer

Post by Sneegurd »

Hi, I'm using the Leela Zero version from http://ns1297.imingo.net/

The below text boxes do not scale when the window is stretched vertically. Text boxes end with "Black Value Network win probability:" - for the interesting part I have to scroll. Instead, GRP shows plenty of unused space. Can this be changed?

I used "Convert RSGF" on a rsgf file it just created, this is what happens:

Code: Select all

Convertion of C:/Users/Snee/Desktop/11468837-103-Haylee-maufred.rsgf into C:/U
sers/Snee/Desktop/11468837-103-Haylee-maufred.rsgf.sgf
Exception in Tkinter callback
Traceback (most recent call last):
  File "Tkinter.pyc", line 1532, in __call__
  File "main.py", line 74, in r2sgf
  File "r2sgf.pyc", line 16, in rsgf2sgf
  File "gomill\sgf.pyc", line 194, in set
  File "gomill\sgf_properties.pyc", line 730, in serialise
  File "gomill\sgf_properties.pyc", line 695, in serialise_as_type
  File "gomill\sgf_properties.pyc", line 264, in serialise_text
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2022' in position
 261: ordinal not in range(256)
Post Reply