Life In 19x19
http://lifein19x19.com/

An SGF Collection (2)
http://lifein19x19.com/viewtopic.php?f=10&t=12428
Page 1 of 2

Author:  aeb [ Thu Nov 05, 2015 3:18 pm ]
Post subject:  An SGF Collection (2)

Let me try a new thread, now that the previous one ended a bit sadly.
The past few days I added another 3000 games to the collection at
http://homepages.cwi.nl/~aeb/go/games/index.html

I wondered what fraction this is of the Japanese tournament games that
are available in SGF form.
Look at the Japanese tournaments where go4go has at least 100 games.
Code:
tournament           go4go       aeb
Kisei                1294        1846
Meijin               2182        2841
Honinbo              2048        3094
Tengen               1211        2054
Oza                  1095        1616
Gosei                1310        1679
Judan                1745        2270
Shinjin O            564         678
NHK                  808         1705
Agon                 285         579
Ryusei               2330        2343
FHoninbo             303         440
FMeijin              320         883
FKisei               317         318
Kansai Kiin 1st      153         373

tot                  15965       22719

Gobase is asleep, I count 11268 games there (for these tournaments).
Remains the question about statistics for GoGoD.

Part of the required infrastructure was a list of pro names
with their rank development over time, see
http://homepages.cwi.nl/~aeb/go/misc/progression.html
Comments are welcome.

Maybe this more or less completes the Japanese tournament part.
It will take a nontrivial amount of time to do China/Korea in a similar way.

Author:  macelee [ Mon Jan 11, 2016 4:25 am ]
Post subject:  Re: An SGF Collection (2)

You might want to have a look at Gosei/Gos32/14.sgf. This game is not on Nihonkiin's Gosei page. On the other hand, there is a game in Kisei tournament having the same date and the same players. Please verify your source. If you rely on certain Asian servers' record, they mix up tournaments quite often.

Author:  aeb [ Tue Jan 12, 2016 9:46 am ]
Post subject:  Re: An SGF Collection (2)

macelee wrote:
You might want to have a look at Gosei/Gos32/14.sgf.
Thanks! Fixed.
Good that someone checks the details carefully.
(In fact this was on my todo list: I had 11 sources for this game, 6 called it Gosei and 5 called it Kisei.)

Now that I write anyway, I can update the sentence "Gobase is asleep". It woke up for a short while, and today it has 88642 games.

Author:  macelee [ Thu Jan 14, 2016 3:04 am ]
Post subject:  Re: An SGF Collection (2)

Shinjin-O/ShO36/P12.sgf and Shinjin-O/ShO36/P13.sgf are the same game. The former contains incorrect player name - should be removed.

In the same tournament Shinjin-O/ShO36/P6.sgf's kifu is broken. In fact it is the same game as P7.sgf. P6 should be removed.

Author:  aeb [ Thu Jan 14, 2016 7:57 am ]
Post subject:  Re: An SGF Collection (2)

macelee wrote:
P6.sgf and P12.sgf should be removed from ShO36.

Thanks again! Fixed.

Author:  aeb [ Sun Jul 17, 2016 10:19 am ]
Post subject:  Re: An SGF Collection (2)

aeb wrote:
I wondered what fraction this is of the Japanese tournament games that
are available in SGF form.

On http://homepages.cwi.nl/~aeb/go/games/index.html one can find
a freely available collection of Japanese professional go games
in SGF format.

Last time I published stats was when it had about 30000 games.
Today this collection has some 35000 games, mostly from Japanese
pro tournaments. There are a few older games as well.

I think this is the most extensive resource on Japanese games
but I may be unaware of recent or Asiatic sources.

An attempt to compare with gogod and go4go:

The Winter 2015 GoGoD collection has 86093 SGF files. There is no easy
way to sort them into Japanese / Chinese / Korean / other.
My collection has a bit over 10000 games from before 2016 not in GoGoD.
(JF, or anybody else, is of course free to take them.)

Go4Go reports 55831 games today. It is very uptodate, and always has
the latest games. It does not often add old games. Statistics of the
difference between aeb and go4go are similar to last time: go4go
has almost 18000 games in Japanese tournaments, and for these same
tournaments my collection has somewhat more than 24000 games.
The games missing in go4go are mostly older games.

Author:  Bonobo [ Sun Jul 17, 2016 11:14 am ]
Post subject:  Re: An SGF Collection (2)

Are the individual games which you posted there separately also included in the big archive?

Author:  aeb [ Sun Jul 17, 2016 12:46 pm ]
Post subject:  Re: An SGF Collection (2)

Bonobo wrote:
Are the individual games which you posted there separately also included in the big archive?

Yes. The many smaller per-tournament archives are updated more frequently, but eventually all ends up in the single big archive.

Author:  aeb [ Fri Jul 22, 2016 8:29 am ]
Post subject:  Re: An SGF Collection (2)

macelee wrote:
..mix up tournaments..

A moment ago I sorted some Meijin games and noticed that some sources give
2016-06-15 Nishi Takenobu vs Yukawa Mitsuhisa
as 42nd Meijin, and other sources call it 43rd Tengen.
I think it should be Tengen, but go4go says Meijin. Am I mistaken?

Author:  hyperpape [ Fri Jul 22, 2016 10:07 am ]
Post subject:  Re: An SGF Collection (2)

aeb wrote:
Go4Go reports 55831 games today. It is very uptodate, and always has
the latest games. It does not often add old games. Statistics of the
difference between aeb and go4go are similar to last time: go4go
has almost 18000 games in Japanese tournaments, and for these same
tournaments my collection has somewhat more than 24000 games.
The games missing in go4go are mostly older games.
I understand everyone wishes to be very careful with their own database, but I look at this and see opportunities for collaboration.

Author:  macelee [ Fri Jul 22, 2016 12:06 pm ]
Post subject:  Re: An SGF Collection (2)

aeb wrote:
macelee wrote:
..mix up tournaments..

A moment ago I sorted some Meijin games and noticed that some sources give
2016-06-15 Nishi Takenobu vs Yukawa Mitsuhisa
as 42nd Meijin, and other sources call it 43rd Tengen.
I think it should be Tengen, but go4go says Meijin. Am I mistaken?


You are right this is a Tengen game. My original source 新浪围棋 was wrong. This is a Kansai Kiin game - they do not keep tournament records as well as Nihon Kiin. They only get three static web pages covering tournament games from previous 3 weeks. These pages then get overwritten when new data is available. Luckily from about a year ago I started using a script to collect and save these pages. So I have an official record to check against.

Author:  aeb [ Sat Jul 23, 2016 1:48 am ]
Post subject:  Re: An SGF Collection (2)

hyperpape wrote:
I see opportunities for collaboration.

On the one hand, it is unclear how cooperation between an open source project and a commercial project should be organized.
On the other hand, to each the existence of the other is somewhat useful.
Let me take the opportunity to mention that more Ibero-Japan Cup games are available.

Author:  hyperpape [ Sat Jul 23, 2016 8:06 am ]
Post subject:  Re: An SGF Collection (2)

Your collection is public domain, so that by itself opens up some potential--I don't know whether you thought about the possibility of some other database incorporating yours, and whether you'd welcome it or regard it as a bit rude (the law isn't everything).

Anyway, I'm not trying to tell anyone what to do. That would be completely unreasonable, since you two are doing the legwork of creating those collections, not me.

Author:  aeb [ Sat Jul 23, 2016 2:57 pm ]
Post subject:  Re: An SGF Collection (2)

hyperpape wrote:
Your collection is public domain--I don't know whether you thought about the possibility of some other database incorporating yours, and whether you'd welcome it or regard it as a bit rude

I would welcome it, of course. It happened several times, sometimes because I suggested it.

Author:  aeb [ Wed Dec 28, 2016 7:53 am ]
Post subject:  Re: An SGF Collection (2)

aeb wrote:
Today this collection has some 35000 games, mostly from Japanese pro tournaments.

That was half a year ago. The first published collection was on 2000-01-01 and had 1111 games, probably the only largish public domain collection. Yesterday the count passed 40000.

Author:  Gomoto [ Wed Jan 04, 2017 4:42 pm ]
Post subject:  Re: An SGF Collection (2)

Thank you!

Author:  Uberdude [ Thu Mar 09, 2017 6:33 am ]
Post subject:  Re: An SGF Collection (2)

aeb: Michael Redmond recently did a video and sgf review of a Dosaku game, and his sgf differs from yours at http://homepages.cwi.nl/~aeb/go/games/g ... ku/052.sgf with info
1670-05-06 Kanbun 10-3-17 Kikugawa Yuseki Honinbo Dosaku 2 W+R 143

His record at http://www.usgo.org/news/wp-content/upl ... edited.sgf has black 66 at d2 as a tesuji instead of the more natural e3 block, to prevent c4 atari being sente which is where black wants to link up. When someone linked your collection (thanks!) before the Redmond sgf was available I did wonder when replaying it why white didn't atari there (maybe black ignores and pushes in the centre?).

Author:  aeb [ Fri Mar 10, 2017 5:12 am ]
Post subject:  Re: An SGF Collection (2)

Uberdude wrote:
aeb: Michael Redmond recently did a video and sgf review of a Dosaku game, and his sgf differs from your Dosaku/052.sgf. His record has black 66 at d2 as a tesuji instead of the more natural e3 block, to prevent c4 atari being sente.

Yes, thanks. I saw that sgf (but not the video). Concerning this game, I have five versions, and
Code:
$ for i in [1234].sgf; do sgfcmp -1 $i 5.sgf; done
files have 99 and 143 moves; after truncation to 99 moves, the differences are: move 68 : bo vs bm
move 68 : bo vs bm
moves 66,120 : dr,bo vs eq,bp
move 66 : dr vs eq
(where the Redmond version is 4.sgf and my published version is 5.sgf).
The reason that I picked 5.sgf was purely formal: it is the "middle" one: two versions differ from it in move 66, two other versions in move 68.

Author:  John Fairbairn [ Fri Mar 10, 2017 10:36 am ]
Post subject:  Re: An SGF Collection (2)

Andrew

The traditional version of this game (i.e. the version known the longest) has just 99 moves and 66 is at E3 and 68 at B5. When Go Seigen gave his commentary on it, he raised no query about 66 (but did discuss the alternative of 67 at 69 - he concluded 67 was "inevitable"). But he did say 68 (at B5) was a mistake, though in the sense that it was a bad play rather than a typo (or brusho).

When Sakai Takeshi published his more recent book on Dosaku he gave a version with 143 moves and with 66 at D2 and 68 at B7. This version was also used by the Nihon Ki-in in a complete collected of Dosaku's games but they added a note to say that this record was full of misprints.

For that reason, the GoGoD version retains the Go Seigen moves up to 99 and adds the other moves, with a suitable note, but that was an arbitrary choice, swayed no doubt by Go's eminence! However, Go came in for some criticism for his Dosaku book. I can't remember if it was Sakai who led the chorus

I have seen no discussion of reasons for favouring one version over the other, though I imagine it would always be normal to plump for the longer one.

The game also comes with two different dates. All the versions simply give the result as W+ (not W+R), and some specifically say that the rest of the moves are unknown (this is very common with games of this era, both in Japan and China).

The AGA/Redmond version gives Black as Kikukawa. This is a possible reading but Hayashi Yutaro researched these things by going to family registration records and he gives Kikugawa, which is also the commoner reading anyway.

At least a dozen Dosaku game records have major discrepancies and there are a few that may not even be genuine.

Author:  aeb [ Fri Mar 10, 2017 11:47 am ]
Post subject:  Re: An SGF Collection (2)

John Fairbairn wrote:
The traditional version of this game (i.e. the version known the longest) has just 99 moves and 66 is at E3 and 68 at B5. When Sakai Takeshi published his more recent book on Dosaku he gave a version with 143 moves and with 66 at D2 and 68 at B7.
The GoGoD version retains the Go Seigen moves up to 99 and adds the other moves,
All the versions simply give the result as W+ (not W+R), and some specifically say that the rest of the moves are unknown.
Good! Your 99-move version is my 1.sgf, the GoGoD version is my 2.sgf, and you explain the status of this latter version: it is a GoGoD construct combining the start of 1.sgf with the tail of 4.sgf. In that case I should probably follow Redmond with 4.sgf.
Redmond writes LB[of:A][qi:B]C[White is alive with A or B. Black can only resign at this point.], suggesting that there are no further moves.

Page 1 of 2 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/