It is currently Wed May 27, 2020 5:09 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 20 posts ] 

How long until a bot reaches consistent 7d on KGS?
Poll ended at Fri Apr 13, 2012 10:45 pm
<6 months 4%  4%  [ 1 ]
<1 year 37%  37%  [ 10 ]
<2 years 41%  41%  [ 11 ]
<3 years 7%  7%  [ 2 ]
<5 years 0%  0%  [ 0 ]
>5 years 7%  7%  [ 2 ]
never 0%  0%  [ 0 ]
I'll quit Go first 4%  4%  [ 1 ]
Total votes : 27
Author Message
Offline
 Post subject: Computers reach 6d on KGS
Post #1 Posted: Wed Mar 14, 2012 10:45 pm 
Lives in gote

Posts: 637
Location: Austin, Texas, USA
Liked others: 48
Was liked: 210
Zen19D is 6d on KGS, and is doing decently at that level, winning 7 of 14 rated games so far. It's been less than a year since I made the "Computers reach 5d on KGS" thread, will the steady progress of computer Go slow down or stop anytime soon? Let's see your vote for when Computers reach 7d(!) on KGS.

Even Zen's weakest account on KGS, Zen19N, is 5d now. Zen19N plays on slower hardware and slower time settings than Zen19D, but it is still quite formidable!

Here is the last thread: http://www.lifein19x19.com/forum/viewto ... =18&t=3951


Last edited by yoyoma on Thu Mar 15, 2012 8:35 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #2 Posted: Thu Mar 15, 2012 5:42 am 
Lives in sente

Posts: 800
Liked others: 141
Was liked: 123
Rank: AGA 2kyu
Universal go server handle: speedchase
I don't think this is really "reaching 6d" after all, it is playing 9X15, which is the ideal timing for computer go. That being said, the further they go, the harder it will be for them to improve.

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #3 Posted: Thu Mar 15, 2012 6:03 am 
Lives in sente

Posts: 953
Liked others: 0
Was liked: 162
Maybe.

I think this all depends on the discovery of another "breakthrough" that might be able to address one or more of the outstanding weakenesses. It is entirely possible that the MCTS programs might be strengthened by some change to how the tree "grows".

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #4 Posted: Thu Mar 15, 2012 7:47 am 
Tengen

Posts: 4316
Location: North Carolina
Liked others: 468
Was liked: 709
Rank: AGA 3k
GD Posts: 65
OGS: Hyperpape 4k
Yeah, I think the most accurate measure is the performance of medium-slow playing bots, but fortunately, it seems that they move in line with each other, with a stone's difference or so in favor of the blitz bots.

And re: Mike, there does seem to be some possibility of a surprise upside, but that's not the only way that there can be advances. Right now the best bots have been steadily and (not-so) slowly improving through incremental changes. That could easily stop, but I think the best prediction is continued short-term improvement. Whether that can lead to sustained progress seems more dubious.

It's personally hard for me to wrap my head around a MCTS engine playing at strong professional level. Take Sakata's wrong-side peep as discussed in modern master games. If I recall correctly, it required a 21 move sequence to see the value of the peep. There are variants of Monte Carlo that are almost hybrids of alpha-beta (http://www.ru.is/faculty/yngvi/pdf/WinandsB10a.pdf), and I wonder if something more analogous to that is where we'll eventually have to go. The approach taken in that paper is obviously not applicable to Go, but perhaps the general idea of hybridization is applicable.

_________________
Occupy Babel!

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #5 Posted: Fri Mar 16, 2012 7:18 am 
Lives in gote

Posts: 637
Location: Austin, Texas, USA
Liked others: 48
Was liked: 210
I'm pretty surprised there are so few people voting the longer periods, almost all the votes so far are for <1 or <2 years. Where did all the doubters go? I usually fall on the side expecting steady progress, but my own doubts are growing as we reach these higher and higher levels.

Zen19D has now won 62% of 58 rated games at the 6d level, quite a solid performance! It's offline now, probably getting equipment setup for the Takemiya vs Zen19 match. Thread about that match is here:
http://www.lifein19x19.com/forum/viewto ... =18&t=5572

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #6 Posted: Fri Mar 16, 2012 7:42 am 
Tengen

Posts: 4316
Location: North Carolina
Liked others: 468
Was liked: 709
Rank: AGA 3k
GD Posts: 65
OGS: Hyperpape 4k
Even taking 2 years would be a major slowdown in progress. I think a few years ago, a lot of us thought that programs would jump up into the dan ranks and then stall, but it just hasn't happened.

_________________
Occupy Babel!

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #7 Posted: Fri Mar 16, 2012 8:13 am 
Beginner
User avatar

Posts: 7
Liked others: 2
Was liked: 1
GD Posts: 5
The time lag from a 5 dan fast bot 0+9x:15 to a 5 dan slow bot 20+5x:30 was 6 months. Zen19D reached 5 dan in June 2011. Zen19S reached 5 dan in January 2012. If the improvement stays steady then a slow bot 20+5x:30 will reach 7 dan in September 2012. I think the steady progress of 1 dan in 1 year is great. Even 1 dan in 2 years is good. In 2 years, a bot may be in the Top 100 KGS Players!

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #8 Posted: Fri Mar 16, 2012 9:10 am 
Lives in gote

Posts: 637
Location: Austin, Texas, USA
Liked others: 48
Was liked: 210
hyperpape wrote:
Even taking 2 years would be a major slowdown in progress. I think a few years ago, a lot of us thought that programs would jump up into the dan ranks and then stall, but it just hasn't happened.


Ah the poll option intervals I made make it hard to know how if people think it will take slightly longer (e.g. 1 year, 2 months) or almost 2x longer (1 year, 11 monnths). I voted for <2 years, but that's only because I expect it to take slightly longer than 1 year.

Still I expected more people voting for the <3, <5, or >5 options.

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #9 Posted: Wed Jan 27, 2016 10:09 pm 
Lives in gote

Posts: 637
Location: Austin, Texas, USA
Liked others: 48
Was liked: 210
Well it's been almost 4 years, and while Google's AlphaGo hasn't played publicly on KGS, I think it's safe to say that it's level is at least 7d KGS.

viewtopic.php?f=18&t=12644

5-0 against Fan Hui
494-1 against Zen/CrazyStone/Pachi in even games (I wonder which one beat it once)
99% against Pachi giving 4 stones
86% against Zen giving 4 stones
77% against CrazyStone giving 4 stones

Giving 4 stones to Zen and CrazyStone... The version of the software they have and the hardware they're using are probably KGS 5d... So you could even argue AlphaGo is KGS 9d...

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #10 Posted: Thu Jan 28, 2016 6:03 am 
Lives in sente

Posts: 953
Liked others: 0
Was liked: 162
But we really do need to put some kind of qualification on that in terms of "with how much computing power"

A program like Zen might be playing at 6d on kgs given a machine with about an order of magnitude more crunch power than the average home laptop. On our home machines it would play a few stones weaker*.

This latest report is of a program playing on a machine with an order of magnitude more than what Zen on KGS is using. While the MCTS algorithm scales poorly, not out of the question that it would play a stone or two better given an order of magnitude more crunch.


* The problem is the MCTS usually need modification to be optimized for the crunch power vs time control change. They typically have their pruning coded for particular values. So not really just a matter of changing the number of playouts used. Perhaps with the right changes to the pruning used, only a stone or two weaker. But without, there is danger of falling below the threshold where MCTS works well at all, and could be much worse.

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #11 Posted: Thu Jan 28, 2016 8:04 pm 
Lives in gote

Posts: 637
Location: Austin, Texas, USA
Liked others: 48
Was liked: 210
Zen19X has just broken through to 7d KGS! If only Zen had achieved this 2 days earlier it would have been the biggest news in computer go in 4 years. Instead their accomplishment is somewhat overshadowed by Google's AlphaGo. Nevertheless they are the first program to do this publicly on KGS.

http://www.gokgs.com/gameArchives.jsp?u ... dAcounts=y

I update the senseis page on the history of KGS bot ratings (and also compressed it a bit since it's spanning almost 10 years now).
http://senseis.xmp.net/?KGSBotRatings

Also Zen19X is now the #45 player on the KGS top 100 list (Zen19 is #95, a few days ago it was the first bot to breakthrough to the top 100).
http://www.gokgs.com/top100.jsp

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #12 Posted: Fri Jan 29, 2016 12:53 am 
Judan

Posts: 6413
Location: Cambridge, UK
Liked others: 375
Was liked: 3510
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
The vast majority of its games are moderate handicap against low dans, with only one even game against a 7d which it won (unsurprisingly by a big centre kill against a human who underestimated its centre killing ability). I think we need more even games to say it is 7d with confidence (problem is the scarcity of KGS 7ds, and if they are online probably the larger number of weaker players will snipe the game offers).

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #13 Posted: Sun Jan 31, 2016 10:00 am 
Judan

Posts: 6413
Location: Cambridge, UK
Liked others: 375
Was liked: 3510
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Zen19X just beat 2 6ds, 2 7ds, and 2 8ds (inc. cheater = Pavol Lisy Euro 7d/1p) so that 7d is looking more believable now. Its play is impressive. The game vs Pavol:

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #14 Posted: Sun Jan 31, 2016 3:33 pm 
Lives in gote

Posts: 436
Liked others: 1
Was liked: 38
Rank: KGS 5 kyu
Uberdude wrote:
Zen19X just beat 2 6ds, 2 7ds, and 2 8ds (inc. cheater = Pavol Lisy Euro 7d/1p) so that 7d is looking more believable now. Its play is impressive. The game vs Pavol:


There is no komi however so Zen had a handicap of some sort.

Winning against european pro with komi handicap is a great improvement however AlphaGo can do so much more :)

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #15 Posted: Sun Jan 31, 2016 4:59 pm 
Gosei
User avatar

Posts: 1435
Location: California
Liked others: 53
Was liked: 169
Rank: Out of practice
GD Posts: 1104
KGS: fwiffo
Props to the Zen folks though; it's running on *way* less hardware (2x12 core Xeon server).

_________________
KGS 4 kyu - Game Archive - Keyboard Otaku

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #16 Posted: Wed Feb 10, 2016 12:28 am 
Judan

Posts: 6413
Location: Cambridge, UK
Liked others: 375
Was liked: 3510
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
On KGS Hideki (distributed Zen programmer) said Zen was due to play Ichiriki Ryo on 6th Feb. Does anyone have any news on that?

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #17 Posted: Wed Feb 10, 2016 1:48 am 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
Do you mean this?
2016-02-07 Atsushi Ida 8p Zen 19×19 4 stones 0-1
SGF Event sponsored by the Nihon Ki-in Zen ran on a dual Xeon E5-2687W v3 server, 2×12 3.1 GHz cores, as used by Zen19X on KGS.

Time limits were 30 minutes + 30 seconds byoyom

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #18 Posted: Wed Feb 10, 2016 2:28 am 
Judan

Posts: 6413
Location: Cambridge, UK
Liked others: 375
Was liked: 3510
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
pookpooi wrote:
Do you mean this?
2016-02-07 Atsushi Ida 8p Zen 19×19 4 stones 0-1
SGF Event sponsored by the Nihon Ki-in Zen ran on a dual Xeon E5-2687W v3 server, 2×12 3.1 GHz cores, as used by Zen19X on KGS.

Time limits were 30 minutes + 30 seconds byoyom


Yes, that's it, thanks! I got my young Japanese pros beginning with 'I' mixed up. Given Zen19X's performance on KGS recently I thought 4 stones would be too much for it. Where did you get that information, is the "SGF" text a link to the game record?

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #19 Posted: Wed Feb 10, 2016 2:30 am 
Judan

Posts: 6413
Location: Cambridge, UK
Liked others: 375
Was liked: 3510
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Uberdude wrote:
Where did you get that information, is the "SGF" text a link to the game record?


Google is my friend:

http://www.computer-go.info/h-c/ (but the SGF link there is broken :sad: Edit: emailed Nick the webmaster and he fixed it)

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 6d on KGS
Post #20 Posted: Wed Feb 10, 2016 4:33 am 
Judan

Posts: 6413
Location: Cambridge, UK
Liked others: 375
Was liked: 3510
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
I had a quick look through the game and it seemed pretty dull and unimpressive, none of the sharp attacking play Zen19X showed on KGS. Part of that might be because Ida played simply without overplays that could be punished, but it seems more like the handicap was too much and Zen just played making enough concessions to still win.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 20 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group