It is currently Mon Sep 23, 2019 12:37 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 29 posts ]  Go to page 1, 2  Next
Author Message
Offline
 Post subject: An SGF Collection (2)
Post #1 Posted: Thu Nov 05, 2015 3:18 pm 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
Let me try a new thread, now that the previous one ended a bit sadly.
The past few days I added another 3000 games to the collection at
http://homepages.cwi.nl/~aeb/go/games/index.html

I wondered what fraction this is of the Japanese tournament games that
are available in SGF form.
Look at the Japanese tournaments where go4go has at least 100 games.
Code:
tournament           go4go       aeb
Kisei                1294        1846
Meijin               2182        2841
Honinbo              2048        3094
Tengen               1211        2054
Oza                  1095        1616
Gosei                1310        1679
Judan                1745        2270
Shinjin O            564         678
NHK                  808         1705
Agon                 285         579
Ryusei               2330        2343
FHoninbo             303         440
FMeijin              320         883
FKisei               317         318
Kansai Kiin 1st      153         373

tot                  15965       22719

Gobase is asleep, I count 11268 games there (for these tournaments).
Remains the question about statistics for GoGoD.

Part of the required infrastructure was a list of pro names
with their rank development over time, see
http://homepages.cwi.nl/~aeb/go/misc/progression.html
Comments are welcome.

Maybe this more or less completes the Japanese tournament part.
It will take a nontrivial amount of time to do China/Korea in a similar way.


This post by aeb was liked by: ZeroKun
Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #2 Posted: Mon Jan 11, 2016 4:25 am 
Lives in sente

Posts: 905
Location: UK
Liked others: 68
Was liked: 469
Rank: 5 dan
KGS: macelee
You might want to have a look at Gosei/Gos32/14.sgf. This game is not on Nihonkiin's Gosei page. On the other hand, there is a game in Kisei tournament having the same date and the same players. Please verify your source. If you rely on certain Asian servers' record, they mix up tournaments quite often.


This post by macelee was liked by: aeb
Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #3 Posted: Tue Jan 12, 2016 9:46 am 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
macelee wrote:
You might want to have a look at Gosei/Gos32/14.sgf.
Thanks! Fixed.
Good that someone checks the details carefully.
(In fact this was on my todo list: I had 11 sources for this game, 6 called it Gosei and 5 called it Kisei.)

Now that I write anyway, I can update the sentence "Gobase is asleep". It woke up for a short while, and today it has 88642 games.

Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #4 Posted: Thu Jan 14, 2016 3:04 am 
Lives in sente

Posts: 905
Location: UK
Liked others: 68
Was liked: 469
Rank: 5 dan
KGS: macelee
Shinjin-O/ShO36/P12.sgf and Shinjin-O/ShO36/P13.sgf are the same game. The former contains incorrect player name - should be removed.

In the same tournament Shinjin-O/ShO36/P6.sgf's kifu is broken. In fact it is the same game as P7.sgf. P6 should be removed.


This post by macelee was liked by: aeb
Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #5 Posted: Thu Jan 14, 2016 7:57 am 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
macelee wrote:
P6.sgf and P12.sgf should be removed from ShO36.

Thanks again! Fixed.

Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #6 Posted: Sun Jul 17, 2016 10:19 am 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
aeb wrote:
I wondered what fraction this is of the Japanese tournament games that
are available in SGF form.

On http://homepages.cwi.nl/~aeb/go/games/index.html one can find
a freely available collection of Japanese professional go games
in SGF format.

Last time I published stats was when it had about 30000 games.
Today this collection has some 35000 games, mostly from Japanese
pro tournaments. There are a few older games as well.

I think this is the most extensive resource on Japanese games
but I may be unaware of recent or Asiatic sources.

An attempt to compare with gogod and go4go:

The Winter 2015 GoGoD collection has 86093 SGF files. There is no easy
way to sort them into Japanese / Chinese / Korean / other.
My collection has a bit over 10000 games from before 2016 not in GoGoD.
(JF, or anybody else, is of course free to take them.)

Go4Go reports 55831 games today. It is very uptodate, and always has
the latest games. It does not often add old games. Statistics of the
difference between aeb and go4go are similar to last time: go4go
has almost 18000 games in Japanese tournaments, and for these same
tournaments my collection has somewhat more than 24000 games.
The games missing in go4go are mostly older games.


This post by aeb was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #7 Posted: Sun Jul 17, 2016 11:14 am 
Oza
User avatar

Posts: 2160
Location: Germany
Liked others: 7899
Was liked: 896
Rank: OGS SDK
Universal go server handle: trohde
Are the individual games which you posted there separately also included in the big archive?

_________________
“The only difference between me and a madman is that I’m not mad.” — Salvador Dali ★ Play a slooooow correspondence game with me on OGS? :)

Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #8 Posted: Sun Jul 17, 2016 12:46 pm 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
Bonobo wrote:
Are the individual games which you posted there separately also included in the big archive?

Yes. The many smaller per-tournament archives are updated more frequently, but eventually all ends up in the single big archive.


This post by aeb was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #9 Posted: Fri Jul 22, 2016 8:29 am 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
macelee wrote:
..mix up tournaments..

A moment ago I sorted some Meijin games and noticed that some sources give
2016-06-15 Nishi Takenobu vs Yukawa Mitsuhisa
as 42nd Meijin, and other sources call it 43rd Tengen.
I think it should be Tengen, but go4go says Meijin. Am I mistaken?

Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #10 Posted: Fri Jul 22, 2016 10:07 am 
Tengen

Posts: 4268
Location: North Carolina
Liked others: 447
Was liked: 702
Rank: AGA 3k
GD Posts: 65
OGS: Hyperpape 5k
aeb wrote:
Go4Go reports 55831 games today. It is very uptodate, and always has
the latest games. It does not often add old games. Statistics of the
difference between aeb and go4go are similar to last time: go4go
has almost 18000 games in Japanese tournaments, and for these same
tournaments my collection has somewhat more than 24000 games.
The games missing in go4go are mostly older games.
I understand everyone wishes to be very careful with their own database, but I look at this and see opportunities for collaboration.

_________________
Occupy Babel!


This post by hyperpape was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #11 Posted: Fri Jul 22, 2016 12:06 pm 
Lives in sente

Posts: 905
Location: UK
Liked others: 68
Was liked: 469
Rank: 5 dan
KGS: macelee
aeb wrote:
macelee wrote:
..mix up tournaments..

A moment ago I sorted some Meijin games and noticed that some sources give
2016-06-15 Nishi Takenobu vs Yukawa Mitsuhisa
as 42nd Meijin, and other sources call it 43rd Tengen.
I think it should be Tengen, but go4go says Meijin. Am I mistaken?


You are right this is a Tengen game. My original source 新浪围棋 was wrong. This is a Kansai Kiin game - they do not keep tournament records as well as Nihon Kiin. They only get three static web pages covering tournament games from previous 3 weeks. These pages then get overwritten when new data is available. Luckily from about a year ago I started using a script to collect and save these pages. So I have an official record to check against.


This post by macelee was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #12 Posted: Sat Jul 23, 2016 1:48 am 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
hyperpape wrote:
I see opportunities for collaboration.

On the one hand, it is unclear how cooperation between an open source project and a commercial project should be organized.
On the other hand, to each the existence of the other is somewhat useful.
Let me take the opportunity to mention that more Ibero-Japan Cup games are available.

Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #13 Posted: Sat Jul 23, 2016 8:06 am 
Tengen

Posts: 4268
Location: North Carolina
Liked others: 447
Was liked: 702
Rank: AGA 3k
GD Posts: 65
OGS: Hyperpape 5k
Your collection is public domain, so that by itself opens up some potential--I don't know whether you thought about the possibility of some other database incorporating yours, and whether you'd welcome it or regard it as a bit rude (the law isn't everything).

Anyway, I'm not trying to tell anyone what to do. That would be completely unreasonable, since you two are doing the legwork of creating those collections, not me.

_________________
Occupy Babel!


This post by hyperpape was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #14 Posted: Sat Jul 23, 2016 2:57 pm 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
hyperpape wrote:
Your collection is public domain--I don't know whether you thought about the possibility of some other database incorporating yours, and whether you'd welcome it or regard it as a bit rude

I would welcome it, of course. It happened several times, sometimes because I suggested it.


This post by aeb was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #15 Posted: Wed Dec 28, 2016 7:53 am 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
aeb wrote:
Today this collection has some 35000 games, mostly from Japanese pro tournaments.

That was half a year ago. The first published collection was on 2000-01-01 and had 1111 games, probably the only largish public domain collection. Yesterday the count passed 40000.

Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #16 Posted: Wed Jan 04, 2017 4:42 pm 
Lives in sente

Posts: 1146
Location: Earth
Liked others: 377
Was liked: 187
Thank you!

Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #17 Posted: Thu Mar 09, 2017 6:33 am 
Judan

Posts: 6072
Location: Cambridge, UK
Liked others: 345
Was liked: 3267
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
aeb: Michael Redmond recently did a video and sgf review of a Dosaku game, and his sgf differs from yours at http://homepages.cwi.nl/~aeb/go/games/g ... ku/052.sgf with info
1670-05-06 Kanbun 10-3-17 Kikugawa Yuseki Honinbo Dosaku 2 W+R 143

His record at http://www.usgo.org/news/wp-content/upl ... edited.sgf has black 66 at d2 as a tesuji instead of the more natural e3 block, to prevent c4 atari being sente which is where black wants to link up. When someone linked your collection (thanks!) before the Redmond sgf was available I did wonder when replaying it why white didn't atari there (maybe black ignores and pushes in the centre?).

Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #18 Posted: Fri Mar 10, 2017 5:12 am 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
Uberdude wrote:
aeb: Michael Redmond recently did a video and sgf review of a Dosaku game, and his sgf differs from your Dosaku/052.sgf. His record has black 66 at d2 as a tesuji instead of the more natural e3 block, to prevent c4 atari being sente.

Yes, thanks. I saw that sgf (but not the video). Concerning this game, I have five versions, and
Code:
$ for i in [1234].sgf; do sgfcmp -1 $i 5.sgf; done
files have 99 and 143 moves; after truncation to 99 moves, the differences are: move 68 : bo vs bm
move 68 : bo vs bm
moves 66,120 : dr,bo vs eq,bp
move 66 : dr vs eq
(where the Redmond version is 4.sgf and my published version is 5.sgf).
The reason that I picked 5.sgf was purely formal: it is the "middle" one: two versions differ from it in move 66, two other versions in move 68.

Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #19 Posted: Fri Mar 10, 2017 10:36 am 
Oza

Posts: 2352
Liked others: 15
Was liked: 3417
Andrew

The traditional version of this game (i.e. the version known the longest) has just 99 moves and 66 is at E3 and 68 at B5. When Go Seigen gave his commentary on it, he raised no query about 66 (but did discuss the alternative of 67 at 69 - he concluded 67 was "inevitable"). But he did say 68 (at B5) was a mistake, though in the sense that it was a bad play rather than a typo (or brusho).

When Sakai Takeshi published his more recent book on Dosaku he gave a version with 143 moves and with 66 at D2 and 68 at B7. This version was also used by the Nihon Ki-in in a complete collected of Dosaku's games but they added a note to say that this record was full of misprints.

For that reason, the GoGoD version retains the Go Seigen moves up to 99 and adds the other moves, with a suitable note, but that was an arbitrary choice, swayed no doubt by Go's eminence! However, Go came in for some criticism for his Dosaku book. I can't remember if it was Sakai who led the chorus

I have seen no discussion of reasons for favouring one version over the other, though I imagine it would always be normal to plump for the longer one.

The game also comes with two different dates. All the versions simply give the result as W+ (not W+R), and some specifically say that the rest of the moves are unknown (this is very common with games of this era, both in Japan and China).

The AGA/Redmond version gives Black as Kikukawa. This is a possible reading but Hayashi Yutaro researched these things by going to family registration records and he gives Kikugawa, which is also the commoner reading anyway.

At least a dozen Dosaku game records have major discrepancies and there are a few that may not even be genuine.

Top
 Profile  
 
Offline
 Post subject: Re: An SGF Collection (2)
Post #20 Posted: Fri Mar 10, 2017 11:47 am 
Dies with sente

Posts: 75
Liked others: 5
Was liked: 26
John Fairbairn wrote:
The traditional version of this game (i.e. the version known the longest) has just 99 moves and 66 is at E3 and 68 at B5. When Sakai Takeshi published his more recent book on Dosaku he gave a version with 143 moves and with 66 at D2 and 68 at B7.
The GoGoD version retains the Go Seigen moves up to 99 and adds the other moves,
All the versions simply give the result as W+ (not W+R), and some specifically say that the rest of the moves are unknown.
Good! Your 99-move version is my 1.sgf, the GoGoD version is my 2.sgf, and you explain the status of this latter version: it is a GoGoD construct combining the start of 1.sgf with the tail of 4.sgf. In that case I should probably follow Redmond with 4.sgf.
Redmond writes LB[of:A][qi:B]C[White is alive with A or B. Black can only resign at this point.], suggesting that there are no further moves.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 29 posts ]  Go to page 1, 2  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: mipli and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group