Life In 19x19 http://lifein19x19.com/ |
|
Announcing GoReviewPartner - v0.15 with Pachi and PhoenixGo http://lifein19x19.com/viewtopic.php?f=9&t=14050 |
Page 12 of 16 |
Author: | mb76 [ Sat Apr 21, 2018 7:25 am ] | |||
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis | |||
AQ 2.1.1 (GPU with Cuda) does not handle handicap games properly. For example suggestion moves 8/233(G), 15/233(C), 28/233(C) and 64/233(E) overlaps the already placed handicap stone (RGSF). Attachments: -SGF file -RSGF file
|
Author: | pnprog [ Mon Apr 30, 2018 12:01 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
sleepyEDB wrote: In the meantime, would altering the komi value to account for the false handicap stone (I'm assuming decreasing it?) be a sort of workaround? Though since this behavior seems to be exclusive to OGS, maybe this is a good excuse to start playing on other servers... Yes, this is the way to go, especially if you play 9x9, as I imagine you are using Leela for that, and Leela does not use Value Network for 9x9.Yes, you can decrease the komi by one point. For other sizes of goban, things would get a bit more complicated: The komi becomes 6.5, but Leela (and all other bots) have value network trained for 7.5pt, so in very closed games (0.5pt wins) they get confused. Leela Zero is the most confused, other bot do better because their monte carlo rolls mitigate that issue a bit. |
Author: | pnprog [ Mon Apr 30, 2018 12:21 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
Dontbtme wrote: So, anyway, these were my thoughts (even though I know some things were on the work already). What do you think? I am not sure I understood everything 100% (I realize we may switch to French and continue through PM).Somehow there is some sort of contradiction in the analysis settings, because the options to analyse only black or only white moves does remove a lot of interesting data (the win rate deltas), so most of the time, it best to keep analyse for both colors. But in the same time, some players don't want to be bothered with the opponent best lines of play*. One possibility could be to offer the possibility to perform the analysis for both colors, but discard the variations for one of the two colors. But I am afraid this would be confusing to many users. Another possibility would be to propose to discard variations for one or both colors prior to converting from RSGF to SGF. This match your use case, but probably not the use cases for most of other users. One third possibility would be to somehow replace the RSGF to SGF conversion by a more complete tool box, that allows quick editing of a RSGF file (keep only variations based on several criteria). And converting to SGF would be one of that toolbox features. This could even came separately with GRP. I need to think about that *Something I quite don't agree with, because I think it is interesting to see what were the opportunities to crush me that my opponent missed, but anyway. |
Author: | pnprog [ Mon Apr 30, 2018 12:44 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
mb76 wrote: AQ 2.1.1 (GPU with Cuda) does not handle handicap games properly. Woh, yes, AQ seems to be definitively confused!For example suggestion moves 8/233(G), 15/233(C), 28/233(C) and 64/233(E) overlaps the already placed handicap stone (RGSF). Attachments: -SGF file -RSGF file Nice find, this is definitively a problem in my code:
I will fix that! |
Author: | sleepyEDB [ Tue May 01, 2018 12:37 pm ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
Hello, I switched to Pandanet to try and avoid the 'phantom handicap stone' bug on OGS but I seem to have found another situation in which GRP and Leela do not like 9x9 games...maybe I just need to start playing 19x19, eh? For whatever reason, Pandanet assigns a negative komi to black rather than a positive komi to white which throws Leela for a loop toward the end of the game. In this example, Leela considers the game to be over around move 31 and would resign as black even though I actually won by 4.5 points during the actual match. This makes reviewing the entire match rather difficult as the last 1/3 of my moves aren't evaluated. I don't think I have the option of editing the komi when creating the game on Pandanet, but I've tried changing the komi when selecting the file to review in GRP (removing the negative, doubling the given value, selecting a positive value like 3.5 used on OGS) but none of my mods have worked; Leela still thinks black loses and resigns. What's strange is if I convert the RSGF file into an SGF, the outcome is correct and black wins by 4.5 points. Again, I'm just reporting this for posterity and if fixing them isn't deemed worthwhile I fully understand. I can appreciate that this rabbit hole seems to get pretty deep with different servers, komi values, and AI sources and may not be worth fixing. I've uploaded some screenshots (hidden), the original SGF, and the RSGF file (renamed to an SGF to allow the upload) for reference. Attachment: Attachment: Thanks, sleepy |
Author: | pnprog [ Tue May 01, 2018 9:25 pm ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
sleepyEDB wrote: For whatever reason, Pandanet assigns a negative komi to black rather than a positive komi to white which throws Leela for a loop toward the end of the game. In this example, Leela considers the game to be over around move 31 and would resign as black even though I actually won by 4.5 points during the actual match. This makes reviewing the entire match rather difficult as the last 1/3 of my moves aren't evaluated. I don't think I have the option of editing the komi when creating the game on Pandanet, but I've tried changing the komi when selecting the file to review in GRP (removing the negative, doubling the given value, selecting a positive value like 3.5 used on OGS) but none of my mods have worked; Leela still thinks black loses and resigns. I just had a look at it, and it is definitively a GRP bug, not a issue with Leela handling negative komi. I opened the game with GoGui and asked Leela to play some end game moves, and Leela sees black winning by 4.5pt, with 100% confidence. So it is a GRP bug, I don't know yet what causes this bug (seems like the black and white probabilities are switched). Anyway, thanks for reporting, this is helping me a lot! |
Author: | sleepyEDB [ Wed May 02, 2018 5:35 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
Awesome! I'm glad this was well-received and that I could be of help. sleepy |
Author: | pnprog [ Fri May 04, 2018 12:49 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
mb76 wrote: AQ 2.1.1 (GPU with Cuda) does not handle handicap games properly. For example suggestion moves 8/233(G), 15/233(C), 28/233(C) and 64/233(E) overlaps the already placed handicap stone (RGSF). For information, I fixed this bug in the development version. The bug was also affecting Ray. |
Author: | pnprog [ Fri May 04, 2018 12:54 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
pnprog wrote: I will see what's doable, ideally, the best would be to recognize those three possibilities:
I implemented that in the development version, I am quite satisfied with the solution:
|
Author: | pnprog [ Fri May 04, 2018 1:13 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
sleepyEDB wrote: Awesome! I'm glad this was well-received and that I could be of help. In fact, those seemingly unimportant bugs quite often hide deeper and more vicious bugs that are hard to detect and recreate. So chasing bugs that appear in exotic situations is in fact useful. And this may be the case for that bug. I have been scratching my head all the morning with it:
Could it be that previous win rate calculation affected the following win rate calculation? To check that, I modified Leela's dedicated code in GRP so that, instead of undoing moves the "normal way", she undoes moves by complete board reset and replay to previous position. That's the way AQ and Ray work already because they don't accept the standard undo command. And then, finally, GRP agrees with Gogui that black indeed is winning this game: Win rate before modification: Attachment: before.png [ 8.96 KiB | Viewed 14240 times ] Win rate after modification: Attachment: after.png [ 10.65 KiB | Viewed 14240 times ] So at this point, I believe this is an issue with Leela, I will check a bit further then probably contact Gian-Carlo Pascutto for this one. |
Author: | ez4u [ Fri May 04, 2018 4:08 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
Thanks very much for working on this so hard! |
Author: | gdgcb [ Fri May 04, 2018 5:35 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
Hi Very appreciate your sharing and it works quite well for analysing games. Do you have any plan to release an updated version that can support ELF OpenGo hash? |
Author: | sleepyEDB [ Fri May 04, 2018 7:31 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
pnprog wrote: So at this point, I believe this is an issue with Leela, I will check a bit further then probably contact Gian-Carlo Pascutto for this one. Wow, awesome stuff! Thank you for your hard work on a great program...where's the donate button?! sleepy |
Author: | pnprog [ Fri May 04, 2018 10:41 pm ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
sleepyEDB wrote: pnprog wrote: So at this point, I believe this is an issue with Leela, I will check a bit further then probably contact Gian-Carlo Pascutto for this one. Wow, awesome stuff! Thank you for your hard work on a great program...where's the donate button?! I was able to reproduce the issue on another 9x9 game, as well as an 19x19 game. From what I understand, this bug appears under the following conditions:
I sent my finding to Leela's author: GCP wrote: Thanks, this should be enough to investigate the problem. I am not actively working on Leela 0.x right now, as Leela Zero is still taking up all of my time, but if I ever get back to it, I will try to fix this bug. I will patch GRP in the next version so that Leela use the "clear board then replay up to previous move" procedure instead of conventional undo. I will do the same for all bots, as it will make the code cleaner. In the meantime sleepyEDB, for further analysis, tell GRP to start the analysis at move 2, it avoids that bug. |
Author: | pnprog [ Sat May 05, 2018 7:35 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
Hi! gdgcb wrote: Hi Hum... let me see my check list:Very appreciate your sharing and it works quite well for analysing games. Do you have any plan to release an updated version that can support ELF OpenGo hash?
The only aspect that could delay it is potential difficulties to run it on my (pretty deprecated) computer. So it will just be a matter or time. |
Author: | dfan [ Sat May 05, 2018 7:55 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
The latest version of Leela Zero (as of yesterday) can run with the ELF-format weights, so I suspect that will be the easiest way for anyone to review games using the brains of ELF Open Go. |
Author: | gdgcb [ Sun May 06, 2018 12:44 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
pnprog wrote: Hi! gdgcb wrote: Hi Hum... let me see my check list:Very appreciate your sharing and it works quite well for analysing games. Do you have any plan to release an updated version that can support ELF OpenGo hash?
The only aspect that could delay it is potential difficulties to run it on my (pretty deprecated) computer. So it will just be a matter or time. As the GO program is getting far more stronger than human, I find no meaning to play with computer But can help me improve go skills by reviewing games. Your program is quite helpful. Thanks very much |
Author: | Dontbtme [ Sun May 06, 2018 8:49 am ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
Hi You said my explanations weren't clear last time, plus I have a better idea of what would be (in my opinion) the best/easiest way to propose the settings, so let me try again: _ First, when you analyze a game, it records EVERYTHING: every variation, every data into the rsgf. Next, if you wanna review the rsgf, a window pops up asking you what you wanna see: how many variations per move, from one or two color's point of view, every mistake or only those with more than a given win rate drop, etc. (the window would have the settings used last time, so you would either confirm or change them) What that means is that you can review your game one way the first time, then another the next time, USING THE SAME RSGF: the settings didn't change what was recorded, only what was showed during a given review session. _ Second, when you convert the rsgf, since everything was recorded on it, a window would also pop up asking you what you want in the sgf (it would have the settings used last time, so you would either confirm or change them). Like for the review, you can ask to create an sgf from one or two color's point of view, what variations to keep and so on... plus what kind of data you want to show up as comments. In my case, I only keep the win rate shift between my move and the variation. For example: "White Value Network win probability: • before c6: 54.48% • after c6: 52.43% (-2.05pp)" Like before, you could convert your rsgf one way at first, and then another (for whatever reason) the next time, since the rsgf got EVERYTHING recorded anyway. For example, you could make a separate sgf for each color, or you could have an sgf with only one variation and another one with many, or one with big mistakes only and another one with every mistakes, etc. using THE SAME RSGF. Here's the kind of sgf I edit myself nowadays based on the rsgf (I couldn't put it here directly, apparently, so here's the Eidogo link): http://eidogo.com/#tNq4zS60 That's the way I like my sgf (clear and simple) but obviously that's only one example: someone else could chose different settings. What do you think? |
Author: | sleepyEDB [ Mon May 07, 2018 12:55 pm ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
pnprog wrote: Haha, no donation needed, working on GRP is already quite rewarding Fair enough, but the offer stands should you change your mind! pnprog wrote: In the meantime sleepyEDB, for further analysis, tell GRP to start the analysis at move 2, it avoids that bug. The next time I run into negative komi I will definitely do that. Thanks! sleepy |
Author: | MikeKyle [ Mon May 07, 2018 4:25 pm ] |
Post subject: | Re: Announcing GoReviewPartner - v0.11.2 (with Live Analysis |
Thank you so much for all your work on this brilliant program. I'd like to second dontbtme's last post: it would be really brilliant to do all the analysis, but then be able to choose the depth of review when you actually start the reviewing. Sometimes I want to see all the interesting sequences the AI would expect in my games but at other times I just want to concentrate on understanding my big mistakes. To have everything stored but then be able to filter it to a set number of variations or a certain mistake threshold would be really ideal. To volunteer a more tame idea - would it be possible to allow batch analysis of a few sgf files? I guess the file picker could just allow you to pick more than one file and then it would queue up the files for processing? |
Page 12 of 16 | All times are UTC - 8 hours [ DST ] |
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |