It is currently Tue Apr 23, 2024 2:08 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 11 posts ] 
Author Message
Offline
 Post subject: Realistic Go problems
Post #1 Posted: Tue Mar 26, 2024 9:22 am 
Dies in gote

Posts: 24
Liked others: 0
Was liked: 4
Rank: OGS 3k
Hi,

I quite like Go but never liked tsumego, finding them tedious and too small in scope, so I've made a website with what I think to be a more rewarding and fun approach to Go problems. They are whole board positions from pro games turned into problems by AI (curated by me). It's still a work in progress but should be able to handle some traffic. I'd love to know what you all think of the concept and the quality of the problems.

I assigned the initial difficulty ratings myself, and especially in the 2200+ (4d+) range they are not likely to be accurate. Some of them could be simply too difficult, but it's hard for me to judge what's too difficult for high dan players.

https://realgoproblems.com/

Would appreciate any and all feedback!

Top
 Profile  
 
Offline
 Post subject: Re: Realistic Go problems
Post #2 Posted: Tue Mar 26, 2024 2:25 pm 
Gosei
User avatar

Posts: 1754
Liked others: 177
Was liked: 492
Nice idea, however I don't always understand why my wrong answers are wrong and why Katago's answers are correct. Perhaps show a few followup moves?

Top
 Profile  
 
Offline
 Post subject: Re: Realistic Go problems
Post #3 Posted: Tue Mar 26, 2024 2:54 pm 
Oza
User avatar

Posts: 2411
Location: Ghent, Belgium
Liked others: 359
Was liked: 1019
Rank: KGS 2d OGS 1d Fox 4d
KGS: Artevelde
OGS: Knotwilg
Online playing schedule: UTC 18:00 - 22:00
Is the concept similar to
https://neuralnetgoproblems.com/ ?

Top
 Profile  
 
Offline
 Post subject: Re: Realistic Go problems
Post #4 Posted: Tue Mar 26, 2024 4:13 pm 
Lives in gote

Posts: 586
Location: Adelaide, South Australia
Liked others: 208
Was liked: 265
Rank: Australian 2 dan
GD Posts: 200
mart900 wrote:
Would appreciate any and all feedback!

The link takes me to a login page. I can't see anything else without registering. Literally nothing, it's just a black background, not even a bit of basic info about the site.

I've already got more accounts on more sites than I want to keep track of. I'm not going to create another one before I've had a look around and decided if I'm interested in the content. Compare goproblems.com, tsumego-hero.com, 101weiqi.com...


This post by xela was liked by: ez4u
Top
 Profile  
 
Offline
 Post subject: Re: Realistic Go problems
Post #5 Posted: Wed Mar 27, 2024 4:06 am 
Dies in gote

Posts: 24
Liked others: 0
Was liked: 4
Rank: OGS 3k
jlt wrote:
Nice idea, however I don't always understand why my wrong answers are wrong and why Katago's answers are correct. Perhaps show a few followup moves?
I considered adding the AI variation but given the (low) amount of playouts per position it's likely to confuse the user more than help them. I will probably add a few extra moves as played in the game though, in case AI and pro played the same move (usually the case).

One feature I want to add is allowing users to comment on problems and add variations, which would become visible with the solution and could help clarify things.

Knotwilg wrote:
Is the concept similar to
https://neuralnetgoproblems.com/ ?
Yes, though I don't know how they generated the problems. I suppose the biggest difference is the rating system, which I believe to be important because it gives users something to play for, a way to measure progress, and it allows the ratings of the problems to grow organically, which will eventually make them perfectly accurate.

xela wrote:
The link takes me to a login page. I can't see anything else without registering. Literally nothing, it's just a black background, not even a bit of basic info about the site.

I've already got more accounts on more sites than I want to keep track of. I'm not going to create another one before I've had a look around and decided if I'm interested in the content. Compare goproblems.com, tsumego-hero.com, 101weiqi.com...
Fair point, I'll add a link to the about page there.

Top
 Profile  
 
Offline
 Post subject: Re: Realistic Go problems
Post #6 Posted: Fri Mar 29, 2024 1:58 am 
Dies in gote

Posts: 24
Liked others: 0
Was liked: 4
Rank: OGS 3k
xela wrote:
mart900 wrote:
Would appreciate any and all feedback!

The link takes me to a login page. I can't see anything else without registering. Literally nothing, it's just a black background, not even a bit of basic info about the site.

I've already got more accounts on more sites than I want to keep track of. I'm not going to create another one before I've had a look around and decided if I'm interested in the content. Compare goproblems.com, tsumego-hero.com, 101weiqi.com...
You can now play without registering

Top
 Profile  
 
Offline
 Post subject: Re: Realistic Go problems
Post #7 Posted: Fri Mar 29, 2024 9:14 pm 
Lives in gote

Posts: 586
Location: Adelaide, South Australia
Liked others: 208
Was liked: 265
Rank: Australian 2 dan
GD Posts: 200
mart900 wrote:
You can now play without registering

Thanks, this is what I was looking for :-) I think the concept is promising but difficult to execute well. Please keep working on it!

Some low hanging fruit (*) for the UI:
  • The thing I most want is a "download SGF" button. This is for playing out some variations on the board, and also asking my local KataGo installation (a) why my wrong answers are actually wrong; (b) what a good followup looks like after the right move.
  • In similar vein, the ability to scroll through the next few moves of the game continuation would be nice.
  • Browse problems by difficulty level. Some days I might be looking more for "easy target practice", other days for a serious challenge. I'm happy with the default being that the site chooses problems appropriate to your level, but it would be good to have other options too.
  • The bottom of the board is right at the bottom of the page. Some space below would look better.
  • Personally I'm not a fan of "fake realism". In particular, wood grain on a computer screen just looks wrong to me. A settings menu with the option of a plainer board style (something more similar to the diagrams on this site) would be nice.
  • The artwork on the login page isn't exactly inviting :-) I guess it's still early days.
Also, I know this is much less easy, but I'd strongly prefer discussion forums integrated with the site rather than a discord channel. On-site discussion means you don't need a separate login, and it means that things get indexed by search engines so it's easier to find old conversations.

* I say that, but I know these things are often harder than they look!

Top
 Profile  
 
Offline
 Post subject: Re: Realistic Go problems
Post #8 Posted: Fri Mar 29, 2024 9:24 pm 
Lives in gote

Posts: 586
Location: Adelaide, South Australia
Liked others: 208
Was liked: 265
Rank: Australian 2 dan
GD Posts: 200
mart900 wrote:
Knotwilg wrote:
Is the concept similar to
https://neuralnetgoproblems.com/ ?
Yes, though I don't know how they generated the problems.

The info is all there on the site. Just click the "about" button at top right.

For me the big difference is that most of the positions on neuralnetgoproblems are what I think of as basic instinct problems: things you can solve based on a sense of shape or direction of play, without reading or deep positional judgement. I think realgoproblems is already covering a wider variety of problem types. This has both pros and cons.

The "basic instinct" character comes from the fact that the right answer is the highest policy move in a particular neural net. This somehow captures the idea that the move is "right" based on general considerations, rather than needing to read out variations. It also means that once the right answer is on the board, you often have a sense that it feels right, rather than thinking "why would anyone want to play that move???"

Two other key differences:
  • I believe neuralnetgoproblems is 100% automatically generated, whereas realgoproblems is manually curated (some automation for making a list of candidate positions, but a human has judged which ones make suitable problems, right?)
  • neuralnetgoproblems does a couple of things to soften the feeling of "it's right or it's wrong, no shades of grey, no explanation". Most of the problems have not just a single right answer, but also a number of "near miss" choices. And you have the option of seeing the next 9 moves of the game, which sometimes (not always!) give an insight into the logic of the right answer.

Top
 Profile  
 
Offline
 Post subject: Re: Realistic Go problems
Post #9 Posted: Fri Mar 29, 2024 9:37 pm 
Lives in gote

Posts: 586
Location: Adelaide, South Australia
Liked others: 208
Was liked: 265
Rank: Australian 2 dan
GD Posts: 200
xela wrote:
I think the concept is promising but difficult to execute well.

I tried to do something similar early this year, but gave up!

The biggest issue I see with https://realgoproblems.com/ in its current state is that it's a bit unsatisfying to put just a single move on the board and be told "right" or "wrong" with absolutely no explanation. I believe you plan to address this in the long term by allowing users to submit comments and variations. If you get enough users and build up the right culture, this could work well. But it's a long road...

For a few weeks, when I reviewed my games with KataGo I was noting the positions where it surprised me. Then I went through the list looking for things I could post on goproblems.com. My criteria were:
  • A unique best move.
  • A continuation making it clear why the best move leads to a good result.
  • Refutations of plausible but wrong attempts. If a solver plays the wrong answer, they should get some feedback as to why it's wrong.
The main issue I had was the continuation. Finding a problem position with a clear-cut best move is one thing. Having a unique and clear best move at each step of the variation is much harder to achieve! I kept finding positions with several almost-equally good alternatives, and thinking that in order for the problem to be "fair" (you shouldn't mark it wrong if the user picks a move that's 0.1 point worse than the "best") I needed to add variations for all of those options. So the number of variations kind of exploded.

Listing the other plausible attempts and refutations also caused a bit of an explosion of variations.

Ultimately I discarded three quarters of my original candidate positions, and spent 30-60 minutes per problem working out the variations for what was left. Then I uploaded three of them, waited for some feedback before posting the rest, and found that goproblems.com is no longer the lively discussion forum it used to be :-/

I've just uploaded the rest of the set in their current state. Chances are there are few errors, and a few important things missing. At this stage I'm unlikely to do more. If you want to see how far I got, check out https://goproblems.com/problems/groups? ... ection=247. Hopefully https://realgoproblems.com/ can get over or around these sorts of roadbumps and go further!


This post by xela was liked by: saxmaam
Top
 Profile  
 
Offline
 Post subject: Re: Realistic Go problems
Post #10 Posted: Sat Mar 30, 2024 2:21 am 
Dies in gote

Posts: 24
Liked others: 0
Was liked: 4
Rank: OGS 3k
Thanks so much for the thorough feedback! I added most of it to my rapidly growing todo-list.

Initially there were more problems per game, so I decided against a "download SGF" feature early on. After all, users would be able to analyze the game, then go to the next problem and immediately know the answer if it's from the same game. But over time I've landed on a way to generate the problems that leads to much less problems per game (typically one or two), so it's safe to add now.

I like your problem set! I considered eventually adding a way for people to submit their own problems but wasn't sure if anyone would. This has given me renewed interest in the feature.

xela wrote:
The biggest issue I see with https://realgoproblems.com/ in its current state is that it's a bit unsatisfying to put just a single move on the board and be told "right" or "wrong" with absolutely no explanation.
I think that's right, and it's a hard problem to solve. As you mentioned, eventually I want users to be able to add comments and variations but that is a long road indeed. Something else I recently thought of and you mentioned as well is, as more users attempt the problems, "common mistakes" might emerge. And I could have KataGo have an automated look at those common mistakes and give refutations.

I'm afraid that adding the recommended variations other than what was played in the game, or multiple-move refutations to these common mistakes, and doing this with any kind of confidence that the recommendation/refutation is actually correct all the way through, is simply too costly when it comes to GPU resources. But this might change in the future.

Top
 Profile  
 
Offline
 Post subject: Re: Realistic Go problems
Post #11 Posted: Sat Mar 30, 2024 4:52 am 
Lives in gote

Posts: 586
Location: Adelaide, South Australia
Liked others: 208
Was liked: 265
Rank: Australian 2 dan
GD Posts: 200
mart900 wrote:
I'm afraid that adding the recommended variations other than what was played in the game, or multiple-move refutations to these common mistakes, and doing this with any kind of confidence that the recommendation/refutation is actually correct all the way through, is simply too costly when it comes to GPU resources. But this might change in the future.

Yes, efficiency is an issue here. That's why I only made 11 problems before giving up, while you're at several hundred and counting :-)

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group