On the Collection of Game Records

[nagano]

I originally posted this on StarBaduk several months ago, and due to a lack of response am reposting it here:

Database compilers often have to scour many newspapers, magazines, yearbooks, and internet sites to find the records they seek. Even then they never find them all, some are never published in print or on the internet, and if there are differences in the records from different sources, it is often impossible to be certain which version of the record is correct.
This gives rise to the question, how can these problems be solved? What is the best way to reform the system? It has improved somewhat with the increase in availability of records on the internet, especially since many games are now broadcast live. My idea is for a central database of all professional games to be created that would serve as the ultimate authority and eliminate these issues. Since recorders input many games to the servers live, they could simply input them to the database instead, which would then automatically relay the top boards to the servers.
Obviously I am aware that this would be rather difficult to achieve, due to copyright issues and difficulty in getting the professional organizations to agree on a plan. I would like to know your thoughts, and whether or not you think the idea is feasible, especially if you have any knowledge of the internal politics of the professional organizations or the copyright issues involved.

[TMark]

As someone who has been producing sgf files since about 1993 and has seen the expansion of internet records, there would still be a minor problem. GoGoD does harvest a number of game records from Far Eastern sites, translates the data and checks for faults in the files, before including the games. However, we also purchase books from China, Japan and Korea which have the printed record of the game. I then go through each book to check that all the games are there, that the players' names are the same and that the number of moves and result match. Where the games haven't been provided on the internet, I then do the work, but I also have to clean up game records where the printed copy does not match the internet copy. It may be that the copier got distracted by some fast interplay or just mis-clicked, but there are a significant number of games that make no sense on internet sites. The worst examples are when player A, taking White, wins the game and advances to the next round. However, the internet game record shows an additional Black move and it says that Black won, while the book shows the correct result. The job of checking the Oriental tournament tables is also an onerous one.

Like many suggestions, this one depends on who will be doing the base work. If you are suggesting an English-language site, you will need people able to translate each of the Oriental languages and able to produce standard versions of the record from those used by the original recorder. It is a laudable concept but too often such ideas run into the sand when the originator finds that real life interferes.

Best wishes.

[nagano]

Well, I never said I thought it would be easy. And yes I understand that there would still be occasional errors. I just think it would be easier, and prevent games from slipping through the cracks. The biggest issue, I'm sure, would be the conflicting copyrights on the records.

[hyperpape]

The easiest solution, by far, is to let John and TMark continue doing their work, and keep paying them on a regular base.

[nagano]

Yes, I know, but I like to make things complicated. Plus, they only release a CD twice a year so unless you do all the work yourself, you cannot study new patterns as easily until the update. For stronger players this could be an issue. The model that I'm looking at is Chessbase, which has an online database version that is constantly updated. But Chess game records are not copyrighted, and FIDE has far more power than the IGF, which makes things a lot easier. Also, the strange lack of interest in databases from professionals assures that the issue is not even brought up. I think that is why there is nothing comparable for Go yet. It's a pity.

[hailthorn011]

Yes, I know, but I like to make things complicated. Plus, they only release a CD twice a year so unless you do all the work yourself, you cannot study new patterns as easily until the update. For stronger players this could be an issue. The model that I'm looking at is Chessbase, which has an online database version that is constantly updated. But Chess game records are not copyrighted, and FIDE has far more power than the IGF, which makes things a lot easier. Also, the strange lack of interest in databases from professionals assures that the issue is not even brought up. I think that is why there is nothing comparable for Go yet. It's a pity.

I actually agree with you. I think a data base that's set up like this would be immensely popular:

1. By year played.
2. What type of game it is (title match, ect.)
3. Country of origin.
4. Professional Rank (possible)

And then imagine being able to access this from any computer? If money is an issue, you could still charge for it. I can, of course, see flaws with this. Mainly the whole copyright thing. Just my thoughts.

[kirkmc]

Ideally, you'd want a sort of wiki, where anyone can make changes and comments, as long as their sourced, but where you would keep all versions so anyone can see if there are multiple versions.

Sounds like a lot of work though. I vote to let the GoGoD Guys keep doing their thing. There's also GoBase.

[hailthorn011]

There's also GoBase.

And GoKifu

Oh, wow, I really need to get out more. That's awesome.

[masek]

As someone who has been producing sgf files since about 1993 and has seen the expansion of internet records, there would still be a minor problem. GoGoD does harvest a number of game records from Far Eastern sites, translates the data and checks for faults in the files, before including the games. However, we also purchase books from China, Japan and Korea which have the printed record of the game. I then go through each book to check that all the games are there, that the players' names are the same and that the number of moves and result match. Where the games haven't been provided on the internet, I then do the work, but I also have to clean up game records where the printed copy does not match the internet copy. It may be that the copier got distracted by some fast interplay or just mis-clicked, but there are a significant number of games that make no sense on internet sites. The worst examples are when player A, taking White, wins the game and advances to the next round. However, the internet game record shows an additional Black move and it says that Black won, while the book shows the correct result. The job of checking the Oriental tournament tables is also an onerous one.

Like many suggestions, this one depends on who will be doing the base work. If you are suggesting an English-language site, you will need people able to translate each of the Oriental languages and able to produce standard versions of the record from those used by the original recorder. It is a laudable concept but too often such ideas run into the sand when the originator finds that real life interferes.

Best wishes.

The idea of game hashing come into mind trust or not trust.

Completely agree translation of game data and making transparent language making problems .
http://gokifu.com/zh/ - i have made transparent Chinese. Took a lot time for proper translation.

Database compilation very hard job, sometimes very routine .