It is currently Sun Dec 17, 2017 2:29 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 135 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7  Next
Author Message
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.5
Post #61 Posted: Mon May 15, 2017 1:43 pm 
Lives with ko
User avatar

Posts: 270
Liked others: 15
Was liked: 33
Rank: NR
yup - that was it. thanks.

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.5
Post #62 Posted: Mon May 22, 2017 9:16 am 
Dies with sente

Posts: 87
Liked others: 3
Was liked: 44
Rank: OGS 7 kyu
Hi everybody, that's me again!

One more update regarding v0.6 that should be released this week. Leela 0.10 has been released last week, and GoReviewPartner v0.5 works just fine with that latest Leela. The new Leela GUI now displays the moves sequences with numerated stones on the Goban just like uses to do GoReviewPartner. I would like to believe that GoReviewPartner prompted Leela's author to implement the same feature, but that's probably not the case :lol:

I was reading the comments about Leela on Reddit, and somebody mentioned an improved influence GTP function. I had not noticed that Leela had such function, so I just had a look and implemented the result into GoReviewPartner the same way GnuGo territory is implemented. Here is the result for the same position as in the previous screen-shot for GnuGo:
Attachment:
Screenshot0001.png
Screenshot0001.png [ 52.8 KiB | Viewed 1748 times ]

Two things to notice:
  • It's an influence function, not a territory function (I will rename the button) so comparing GnuGo and Leela maps is probably not a good idea
  • I noticed Leela provides the data instantly when asked for the map, it must be some sort of fast heuristic (so no simulation or whatever)

As for what I mentioned in the previous post, about completely separating GnuGo and Leela, this is now completed. So before running the analysis, the user is asked what bot to used:
Attachment:
Screenshot0002.png
Screenshot0002.png [ 21.53 KiB | Viewed 1748 times ]

This forced me to make some big code factorization, but the resulting code is (hopefully) much cleaner and it should be easier to finally implement support for additional bots.

All this new code has been pushed to GitHub already, but not heavily tested yet.

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!


This post by pnprog was liked by 3 people: djhbrown, ez4u, pleiade67
Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.5
Post #63 Posted: Mon May 22, 2017 5:58 pm 
Lives with ko
User avatar

Posts: 270
Liked others: 15
Was liked: 33
Rank: NR
such a fickle friend of the gnu - you have now made it possible for the discerning customer to ignore the old cow completely.

at last!... thank you very much! :clap:


Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner
Post #64 Posted: Thu May 25, 2017 12:39 pm 
Dies with sente

Posts: 73
Liked others: 14
Was liked: 7
Rank: KGS 6 dan
pnprog wrote:
I also added the neural network value. But I am not sure what it really means


Can you please add "Net Win %" instead of "Net Prob %" (it's the last row on the right screen)? It's very useful number to evaluate the game, much better than MC... Can you please add it on the left screen too (for the current position as well as variations)?

Thanks for contributing to the go community with this app! I find it very useful and use it very often to analyze what was the best sequence in my tournament and online games. I tried replacing Leela 0.9.0 files with 0.10.0 and it works just perfectly.

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.5
Post #65 Posted: Sun May 28, 2017 3:19 am 
Dies in gote

Posts: 35
Liked others: 0
Was liked: 2
@pnprog thx it looks great.

Can you create an ios version?
It would be great to have it on the ipad pro 12.9 screen and especially on the ipad pro 2 and iphone 8.

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner
Post #66 Posted: Thu Jun 01, 2017 8:22 am 
Dies with sente

Posts: 87
Liked others: 3
Was liked: 44
Rank: OGS 7 kyu
Hi johnsmith,

Thanks for your positive feedback, it's very motivating for me!
johnsmith wrote:
Can you please add "Net Win %" instead of "Net Prob %" (it's the last row on the right screen)?
If I am not wrong, it is what GoReviewPartner calls black/white win probability, or am I wrong?

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.5
Post #67 Posted: Thu Jun 01, 2017 8:34 am 
Dies with sente

Posts: 87
Liked others: 3
Was liked: 44
Rank: OGS 7 kyu
Hi goame,
goame wrote:
@pnprog thx it looks great.

Can you create an ios version?
It would be great to have it on the ipad pro 12.9 screen and especially on the ipad pro 2 and iphone 8.
It is unlikely that Leela will run on ios anytime soon (if ever). So the analysis part would not be available on ios or android.

It would be possible to have a application of ios/android for reviewing the *.rsgf files just like a viewer (without possibility to test alternative sequence against Leela), but it would have to be another project, because the graphical framework I use (tkinter) is not available on ios or android.

One possible option would be using Kivy, another graphical framework dedicated to touch interface. I did experiment with it and made some small go applications on my android phone last year, so it would be possible. But the "work-flow" would be awkward: running the analysis from computer, then transferring it on the tablet, then reviewing it. Plus, some "mouse gestures" (like "mouse over" or "mouse scrolling") would not be available on tablet, so the interface would have to be rethink from scratch.

So... yes, another project :sad:

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.5
Post #68 Posted: Thu Jun 01, 2017 8:55 am 
Dies with sente

Posts: 87
Liked others: 3
Was liked: 44
Rank: OGS 7 kyu
Hi everybody!

Here come v0.6 !

So v0.6 does not really bring new features. In fact, this version is more about "polishing" the software (code cleaning, code re-factorization, better error handling) and is a big step forward v0.7 that will add support for Alphago that will add support for Ray.

So what's new:
  • Separate analysis for Leela and Gnugo
  • Territory map for GnuGo and influence map for Leela
  • A few more settings to review a game
  • Possibility to export an image for the gobans (postscript for now, i will try to change it for png later)
  • Tons of bug fixes
  • v0.6 for windows ships with Leela 0.10 (but v0.5 was working fine already)

Same as before, there is a direct download for GoReviewPartner+Leela for Windows users.
For Linux/Mac, it is best to get the source code from Github.

So next step is adding support for Ray, zakki, the author of Ray, has just added a feature (a GTP command) to allow GoReviewPartner to get similar data than what it get already from Leela (best moves and sequences with %win rate). So I just started playing with it, in fact, I have the analysis/review part already quite working, I need more testing and integration to complete v0.7

I target to release v0.7 before the end of the month.

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!


This post by pnprog was liked by 4 people: ez4u, johnsmith, pleiade67, Sailer
Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #69 Posted: Fri Jun 02, 2017 5:20 am 
Dies with sente

Posts: 73
Liked others: 14
Was liked: 7
Rank: KGS 6 dan
Thanks for all the great work, I'm sure many many more people appreciate it than just those who liked your post ;-)

What I meant in one of my previous posts was explained on the very, very simplified example (maybe even stupid example :cry: even though I'm dan player) in the attached images.

Image 1: first column is some kind of average of second and third column, while fourth is the probability of pro player spotting that move.
Image 2: if I understood GoReviewPartner correctly, then these columns are represented in this particular fashion as shown.

With current version I let GoReviewPartner analyzes the game for one hour, and then I have to switch to Leela UI to check number 3 (Net Win%). Which is small effort for me but still it's an effort :) What I'd like very much to have is number 3 in both current game window and variation window.

Cheers!


Attachments:
2.png
2.png [ 46.25 KiB | Viewed 1466 times ]
1.png
1.png [ 175.64 KiB | Viewed 1466 times ]
Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #70 Posted: Fri Jun 02, 2017 7:58 am 
Dies with sente

Posts: 87
Liked others: 3
Was liked: 44
Rank: OGS 7 kyu
Hi!
johnsmith wrote:
Image 1: first column is some kind of average of second and third column, while fourth is the probability of pro player spotting that move.
Image 2: if I understood GoReviewPartner correctly, then these columns are represented in this particular fashion as shown.

Below is the "dialogue" that will happen between GoreviewPartner and Leela "in the terminal" (this is the GTP protocol), for the same board position (I assumed a komi of 6.5) (blue are messages sent by GoReviewPartner, and red replies by Leela):
Quote:
komi 6.5
=

play black t1
=

play black t19
=

play white d4
=

genmove white
MC winrate=0.609375, NN eval=0.732318, score=W+18.0
Nodes: 2869, Win: 79.88% (MC:64.70%/VN:85.92%), PV: D16 Q17 R4 P4 Q3
Nodes: 6744, Win: 80.93% (MC:64.06%/VN:87.64%), PV: D16 Q17 R4 P4 P3 O3 Q3 O4
Nodes: 9966, Win: 81.53% (MC:63.83%/VN:88.56%), PV: D16 Q17 Q4 R6 O3 P3
Nodes: 12727, Win: 81.49% (MC:63.84%/VN:88.50%), PV: D16 Q17 Q4 R6 O3 S4
Nodes: 15975, Win: 81.68% (MC:64.04%/VN:88.69%), PV: D16 F17 C14 Q17 R4 P4 Q5
Nodes: 18602, Win: 82.02% (MC:64.22%/VN:89.10%), PV: D16 Q17 Q4 R6 O3 R11 R10
Nodes: 23200, Win: 82.41% (MC:64.36%/VN:89.58%), PV: D16 Q17 Q4 R6 O3 R11 N16 O16
Nodes: 26451, Win: 82.47% (MC:64.31%/VN:89.70%), PV: D16 F17 C14 Q17 R4 P4 Q5 P5
Nodes: 30230, Win: 82.62% (MC:64.24%/VN:89.93%), PV: D16 F17 C14 Q17 R4 K16 Q14

D16 -> 29533 (W: 83.23%) (U: 64.20%) (V: 90.80%: 668) (N: 21.9%) PV: D16 F17 C14 Q17 R4 K16 Q14
D17 -> 1331 (W: 79.37%) (U: 64.12%) (V: 85.43%: 30) (N: 26.4%) PV: D17 Q16 R4 D15
Q16 -> 1170 (W: 81.01%) (U: 63.94%) (V: 87.80%: 28) (N: 13.3%) PV: Q16 Q4 D16 R14 O17 S16 R17 R11 S17
Q4 -> 672 (W: 79.06%) (U: 62.34%) (V: 85.71%: 14) (N: 14.2%) PV: Q4 D16 Q16 O3 O4
R4 -> 302 (W: 77.13%) (U: 65.55%) (V: 81.74%: 8) (N: 9.4%) PV: R4 D16 E16
C16 -> 250 (W: 77.23%) (U: 65.10%) (V: 82.06%: 8) (N: 7.6%) PV: C16 Q17 O17
====================================
29533 visits, score 83.23% (from 82.71%) PV: D16 F17 C14 Q17 R4 K16 Q14

33417 visits, 11858 nodes, 33417 playouts, 1336 p/s

= D16


The interesting part is:
Quote:
D16 -> 29533 (W: 83.23%) (U: 64.20%) (V: 90.80%: 668) (N: 21.9%) PV: D16 F17 C14 Q17 R4 K16 Q14

So the information currently displayed are:
  • Black/white win probabilities: W: 83.23%
  • Number of play-outs: 29533
  • Neural network value: N: 21.9%
So we are left with those 2 informations: (U: 64.20%) and (V: 90.80%: 668) and I don't know what they mean. Is (V: 90.80%: 668) the one you want? (to be understood as 100%-90.80% for white maybe? so 9.20%)

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #71 Posted: Fri Jun 02, 2017 9:22 am 
Dies with sente

Posts: 73
Liked others: 14
Was liked: 7
Rank: KGS 6 dan
Yes, (V: 90.80%: 668) is what I'd like :D it is the same like those parameters above:
W => Win
U => Monte Carlo (MC)
V => Value Network (VN)

The thing is, similar to this example, that MC can sometimes show evaluation of 50% while value net can be as high as 75%. So MC might trick you the game is even, while value net tells you how things really are. The same thing goes for Win since it takes MC into account...
So you're right - since it is white's turn it means white is leading by 90.8%, and black is losing by 100-90.8=9.2%. That number 668 I have no idea what it means :D

edit:
pnprog wrote:
to be understood as 100%-90.80% for white maybe? so 9.20%

for black I think...

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #72 Posted: Fri Jun 02, 2017 11:36 am 
Dies in gote

Posts: 36
Liked others: 0
Was liked: 7
OGS: roy7
Universal go server handle: roy7
V is the value network win rate estimate. It moves much quicker, and generally is more accurate, than the U (Monte Carlo) win rate. However it is subject to blind spots where it makes errors (like the policy network does) so the author uses a weighted average W of the U and V values (bit heavier weight on the Monte Carlo value) for the decision making process when Leela does tree search.

N is the policy network based odds a pro player would choose that move. The tree search is seeded and weighted based on the N values. If a move has an N value of 0, Leela won't consider it, which can lead to the rare "blunder" moves where Leela just can't see a move.

The number after the V % is (I think) the number of nodes evaluated by the value network in that variation. The NN evaluation, even with GPU, is much slower than the tree search itself can explore.


This post by roy7 was liked by: johnsmith
Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #73 Posted: Sat Jun 03, 2017 4:42 am 
Dies with sente

Posts: 87
Liked others: 3
Was liked: 44
Rank: OGS 7 kyu
Ok, so it seems I was quite wrong regarding all those values :roll:
I will fix that quickly and add all those values in a v0.6.1 that I will try to release tomorrow.

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #74 Posted: Sun Jun 04, 2017 6:22 am 
Dies with sente

Posts: 87
Liked others: 3
Was liked: 44
Rank: OGS 7 kyu
Hi!

When trying on a 9x9 games, I got this type of data from Leela instead:
Quote:
C3 -> 17924 (U: 89.10%) (R: 87.78%: 33386) (N: 144.9%) PV: C3 C4 C5 B3 B4 B5 D4 B6
So:
  • 17924: Number or play-outs
  • (U: 89.10%): I guess this is still the MonteCarlo search result
  • (R: 87.78%: 33386): Do you know what that one would be?
  • (N: 144.9%): Apparently, this is: Eval: displayed in non-19x19 games instead of Net Prob%. A dimensionless number expressing the likelihood of the move being good, based on a pattern database and the moves' features.
Best regards,

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #75 Posted: Sun Jun 04, 2017 7:53 am 
Dies with sente

Posts: 73
Liked others: 14
Was liked: 7
Rank: KGS 6 dan
9x9 is really puzzling.

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #76 Posted: Sun Jun 04, 2017 9:27 am 
Dies with sente

Posts: 87
Liked others: 3
Was liked: 44
Rank: OGS 7 kyu
johnsmith wrote:
9x9 is really puzzling.
Agreed!

But anyway, I made the modifications to now clearly indicate the MC values, policy values, network value. And the enigmatic R% value for 9x9 or 13x13.

This is the v0.6.1 available for download there: http://ns1297.imingo.net/ (or github)

By the way, the windows download is set up to use the non-opencl version of Leela, so if your computer can run Leela with GPU, make sure to modify the command line in the Settings menu.

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!


This post by pnprog was liked by: pleiade67
Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #77 Posted: Sun Jun 04, 2017 9:44 am 
Dies in gote

Posts: 36
Liked others: 0
Was liked: 7
OGS: roy7
Universal go server handle: roy7
pnprog wrote:
And the enigmatic R% value for 9x9 or 13x13.


It is the RAVE value. See here:

viewtopic.php?f=18&t=13692&p=216390&hilit=rave#p216390

:)

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #78 Posted: Sun Jun 04, 2017 10:06 am 
Dies with sente

Posts: 73
Liked others: 14
Was liked: 7
Rank: KGS 6 dan
Thanks pnprog!

This tool makes me wanna play more deep and serious games! It's like having a strong teacher that can pinpoint your mistakes.

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #79 Posted: Tue Jun 06, 2017 1:30 am 
Dies with sente

Posts: 73
Liked others: 14
Was liked: 7
Rank: KGS 6 dan
Players on reddit have noticed that a completed file was 2.3mb large and as such can't be uploaded to websites for online review. I am not a programmer myself but I have awful lot of ideas :) Maybe there can be a 'save as' option to keep the comments only for the main variation. File as such probably wouldn't be that large. And keep all other comments in the original app. Or just say "W=..., MC=..., VN=..." so the file will definitely be smaller. Some people suggested a toggle for various stuff, too.

My other idea is to have a graph in the app, below the left board for example, that will show you graphically where the game went wrong. Sgf will be smaller that way. I hope pngprog will like this idea :) Just like in Leela with 3 colors. And I just hope it is not a violation of copyright, right? When you have visual representation of the game you can easily notice which move was wrong and specifically analyze that one. When I briefly see difference between two moves with evals like 55.xy and 56.xy I immediately think it's not a bad move, but it may be almost 2% difference (56.99-55.01).

The third idea is to have all those evals (Win, Monte Carlo, Value) on the left board as well. Though file will be extremely huge, but oh well :)

Top
 Profile  
 
Offline
 Post subject: Re: Announcing GoReviewPartner - v0.6
Post #80 Posted: Wed Jun 07, 2017 1:14 am 
Dies with sente

Posts: 87
Liked others: 3
Was liked: 44
Rank: OGS 7 kyu
Hi!
johnsmith wrote:
Players on reddit have noticed that a completed file was 2.3mb large and as such can't be uploaded to websites for online review. I am not a programmer myself but I have awful lot of ideas :) Maybe there can be a 'save as' option to keep the comments only for the main variation. File as such probably wouldn't be that large. And keep all other comments in the original app. Or just say "W=..., MC=..., VN=..." so the file will definitely be smaller. Some people suggested a toggle for various stuff, too.

My other idea is to have a graph in the app, below the left board for example, that will show you graphically where the game went wrong. Sgf will be smaller that way. I hope pngprog will like this idea :) Just like in Leela with 3 colors. And I just hope it is not a violation of copyright, right? When you have visual representation of the game you can easily notice which move was wrong and specifically analyze that one. When I briefly see difference between two moves with evals like 55.xy and 56.xy I immediately think it's not a bad move, but it may be almost 2% difference (56.99-55.01).

The third idea is to have all those evals (Win, Monte Carlo, Value) on the left board as well. Though file will be extremely huge, but oh well :)

Thanks you fore your comments. Redditors are right to say that it could be improved. In fact, all the comments from Leela for all variations are duplicated for each move of the variation... this is redundant, and so useless. You can see it if you open a rsgf file using a standard sgf editor (rename the file from *.rsgf to *.sgf).

I had never noticed that, so I fixed that for the next version (already on GitHub). I think this should already make for a huge size reduction.

I never imagined somebody would go as far as renting Google cloud computers and run GoReviewPartner on it :lol: that much power allows for a lot of branches, all branches being very deep. Hahaha, it's cool seeing people doing unexpected things with GoReviewPartner :tmbup:

Then, the real constraint is the sgf format. It's a standard, and so, I cannot take too much freedom with the format standard, otherwise the compatibility with other sgf editors (or eidogo) will be lost. I encourage you to open one of your sgf files using the windows block note (drag and drop the sgf file directly into it) to get an idea of how it work.

"W=..." instead of "black/white win rate for this move is..." will definitively help, and it could be a toggle. Another toggle to deactivate the influence analysis would help also.

Otherwise, the long term solution would be, in my opinion, to issue a new sgf standard, that takes in consideration all the new possibilities brought by computers being now stronger than humans. The last version of this standard, FF4, is already 20 years old. The new version could include standard properties like MTCS score, win rate, number of simulation.

In fact, I am considering introducing a few new properties in my rsgf files to make things easier (like drawing a win rate graph). If I am careful enough, I can do that without breaking the compatibility with FF4 readers (they would just ignore the new properties).

I guess there would be not copyright issue with a graph (such graph already existed before Leela, for example in The Many Faces Of Go). It is just that sgf format is not intended for that, I need to find a workaround.

_________________
I am the author of GoReviewPartner, a small software aimed at assisting reviewing a game of Go. Give it a try!

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 135 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group