Life In 19x19

Hello. I'm writing a tsumego app/website, focused on speed and configurability. I've made good progress on the app, but I'm having a hard time finding go problems I'll be able to include with it.

Does anyone know of set(s) of go problems, ideally with solutions, that are either in the public domain (expired copyrights) or with permissive licenses?

Ideally I'd like problem sets that are already digitized, but if I get desperate enough, books with solutions that are old enough for the copyright to be expired might be okay too.

I've done a good bit of looking around, and lists of sets exist (eg https://senseis.xmp.net/?GoProblemsOnTheInternet ), but I generally can reject everything for:

- no license mentioned
- no solutions provided
- license only mentioned in the form of eg "thanks to X for letting me host these"
- obviously copied from books (new enough to be copyrighted)
- obviously copied from ancient books, but unclear who did it or if they added enough to themselves have copyright
- in difficult formats (eg the problems on Sensei's Library itself)

kadoban wrote:Does anyone know of set(s) of go problems, ideally with solutions, that are either in the public domain (expired copyrights) or with permissive licenses?

Ideally I'd like problem sets that are already digitized, but if I get desperate enough, books with solutions that are old enough for the copyright to be expired might be okay too.

It is not so clear to me what it is that you want, the question seems open ended. There are many problem collections online that have uncertain provenance. One thing that you don't mention at all is the quality of the problems or the solutions.

If you are interested then it is possible to download PDFs of various editions of Gokyo Shumyo from the National Diet Library. I think some have solution variations written in the margins. I haven't checked if it says anything about copyright but I imagine that since these are available as PDFs from the National Diet Library it is possible that these are out of copyright.

This is going to be tricky. In go, the problem sites that I'm aware of seem to rely on user contributions, and aren't careful about checking the copyright status or licensing of contributions. In contrast, for chess it's the norm that problems are computer-generated from game records. But the nature of the game is very different: it's much easier in chess to create a reasonable quality of automatically generated problems with clear-cut solutions. (There's https://neuralnetgoproblems.com/, but they're not exactly tsumego, a bit different in character.)

The classical problem sets are of course public domain by now, but the traditional solutions won't satisfy the average user's expectations on a modern problem site (choosing my words carefully as some people on this forum don't agree with those expectations...) And the majority of those problems are dan level, which is unlikely to be your core audience. So you'd need to put a fair bit of effort into checking the correctness of solutions, adding explanatory variations for wrong attempts, and estimating the difficulty level of each problem. And perhaps make easier problems out of followup positions. A lot of this effort should be automatable, but I don't think there's existing software which will just do all of it out of the box.

For kyu level problem books, you'll often see the same problem appearing in multiple books, which suggests that the position is actually public domain. If you could somehow get a large collection of books in machine-readable form and then search for repeated positions...

Honestly, I suspect your best option is to write your own web scraping code to put Sensei's Library problems into a suitable format. Remember that SL is Open Content licenced, so you need to acknowledge your sources. And remember that you can download a snapshot, so you don't need to hit the server with thousands of requests.

If you go ahead with this, or find a better approach, please update here and let us know what you came up with!