It is currently Tue Mar 19, 2024 3:27 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 440 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7, 8, 9 ... 22  Next
Author Message
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #101 Posted: Wed Mar 17, 2021 5:23 am 
Beginner

Posts: 8
Liked others: 0
Was liked: 0
Where do u take those 60b nets? I searched all over https://katagotraining.org/networks/ and didn't find any. And how much stronger this 60b in comparison with 40b 697?

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #102 Posted: Wed Mar 17, 2021 8:04 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
while there are only two networks 60 blocks:
https://media.katagotraining.org/upload ... 221.bin.gz
https://media.katagotraining.org/upload ... 314.bin.gz

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #103 Posted: Wed Mar 17, 2021 12:01 pm 
Beginner

Posts: 8
Liked others: 0
Was liked: 0
Yeah, I know, but those links are not to the site, but download links. And where do you find them? I couldn't find anything about 60b nets on the katago site (https://katagotraining.org/networks/). Does someone upload them on github or what? Is there any other source where I can look for new ones when they eventually come, other than you and go4thewin who uploaded previous 60b net?

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #104 Posted: Wed Mar 17, 2021 4:04 pm 
Lives in sente

Posts: 757
Liked others: 114
Was liked: 916
Rank: maybe 2d
Join the discord server:
https://discord.gg/bqkZAz3

60b is not ready to officially take over 40b because it's not strong enough yet given the extra compute cost, these are nets that are posted to see how it's progressing. Join discord if you care about news or discussions of these things, or you want to know when cool stuff is posted (nets, experimental branches of code, stats, test results, games)

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #105 Posted: Thu Mar 18, 2021 9:33 am 
Beginner

Posts: 8
Liked others: 0
Was liked: 0
That's awesome, thanks!

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #106 Posted: Sat Apr 03, 2021 4:32 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
New b60 net https://media.katagotraining.org/upload ... 898.bin.gz

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #107 Posted: Mon Apr 19, 2021 6:06 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
KataGo v1.8.2
"This is a minor release mainly of interest to contributors to KataGo's distributed run, or to users who run KataGo self-play training on their own GPUs. This release doesn't make any changes to the KataGo engine itself, but it does fix an issue that was believed to be limiting the diversity in KataGo's self-play games. Switching to this version should, over the long term of training, improve KataGo's learning, particularly on small boards, as well as enable a few further parameter changes in the future once most people have upgraded, which should also further improve opening diversity."
https://github.com/lightvector/KataGo/releases

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #108 Posted: Sun Apr 25, 2021 4:58 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
New b60 nets
http://media.katagotraining.org/uploade ... 932.bin.gz
https://media.katagotraining.org/upload ... 424.bin.gz
https://discord.com/channels/4170221623 ... 8804732928

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #109 Posted: Mon Apr 26, 2021 8:09 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
KataGo b60c320-20210424 playouts 1, komi 0 - CS Zero 9d:


Attachments:
CS Zero - KataGo.sgf [1.7 KiB]
Downloaded 2001 times

This post by And was liked by: go4thewin
Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #110 Posted: Mon Apr 26, 2021 2:45 pm 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
KataGo b60c320-s275 playouts 1 - CS Zero 9d H2:


Attachments:
CS Zero - KataGo.sgf [1.78 KiB]
Downloaded 2002 times

This post by And was liked by: go4thewin
Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #111 Posted: Wed Apr 28, 2021 5:34 am 
Oza
User avatar

Posts: 2401
Location: Tokyo, Japan
Liked others: 2337
Was liked: 1332
Rank: Jp 6 dan
KGS: ez4u
And wrote:
KataGo v1.8.2
"This is a minor release mainly of interest to contributors to KataGo's distributed run, or to users who run KataGo self-play training on their own GPUs. This release doesn't make any changes to the KataGo engine itself, but it does fix an issue that was believed to be limiting the diversity in KataGo's self-play games. Switching to this version should, over the long term of training, improve KataGo's learning, particularly on small boards, as well as enable a few further parameter changes in the future once most people have upgraded, which should also further improve opening diversity."
https://github.com/lightvector/KataGo/releases


The new v1.8.2 seems to run much slower than 1.8.1 on my machine. I am running contribute on windows 10 with an Nvidia GTX1650 super (no tensor cores, no FP16) using the opencl version. Moving from 1.8.1 to 1.8.2, I am seeing: -33% on plays per second, -30% on nn evals per second, and +61% on seconds per game. This is from the information in the log files.

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #112 Posted: Wed Apr 28, 2021 6:27 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
I compared versions of eigenavx2 - 1.8.2 10% slower, several moves 20% slower, watched over 100 moves. The default_gtp.cfg files are the same, network 20b, 10 playouts :sad:

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #113 Posted: Wed Apr 28, 2021 6:50 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
OpenCL version 1.8.2 is 2.5 times slower! but there are differences in opencltuning files (GT 610)
EDIT I replaced the opencltuning file, nothing changed. 1.8.2 is also 2.5x slower! The default_gtp.cfg files are the same, network 20b, 10 playouts

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #114 Posted: Wed Apr 28, 2021 6:58 am 
Oza
User avatar

Posts: 2401
Location: Tokyo, Japan
Liked others: 2337
Was liked: 1332
Rank: Jp 6 dan
KGS: ez4u
When I run the cuda version on google colab, I do not see any difference. If anything, 1.8.2 might be a little faster. However, that was with quite short runs versus over 24 hours when I looked at the opencl versions on my machine.

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #115 Posted: Wed Apr 28, 2021 7:01 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
obviously something is wrong :)

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #116 Posted: Thu Apr 29, 2021 4:06 pm 
Lives in sente

Posts: 757
Liked others: 114
Was liked: 916
Rank: maybe 2d
We've figured out what's going on - it's not a matter of the version, but apparently on some GPUs KataGo's tuning is noisy enough that it it has an occasional chance of picking to not use FP16 storage when FP16 storage is beneficial, so on those GPUs, every time you manually re-tune you have a chance of getting different performance.

I'm not entirely sure how to fix it without making the OpenCL tuning process heavier and slower. But definitely something I'll think about for a future release - @ez4u thanks for the report and being so careful to check and notice things. :tmbup:

Also, is there a place in the docs where I can write or fix to avoid giving the impression that people have to manually retune each version? Although, I guess it was really good that you did anyways, otherwise we wouldn't have known about this quirk about tuning. :)

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #117 Posted: Mon May 03, 2021 6:58 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
KataGo s790 playouts 1 - CS Zero 9d H2:


Attachments:
CS Zero - KataGo.sgf [1.48 KiB]
Downloaded 1798 times
Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #118 Posted: Tue May 11, 2021 12:10 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
KataGo s800 playouts 1 - CS Zero 9d H3:


Attachments:
CS Zero - KataGo.sgf [2.46 KiB]
Downloaded 1731 times
Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #119 Posted: Thu May 20, 2021 3:49 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
New b60 net https://media.katagotraining.org/upload ... 170.bin.gz

Top
 Profile  
 
Offline
 Post subject: Re: KataGo Distributed Training and new networks
Post #120 Posted: Sat Jun 19, 2021 5:57 am 
Gosei
User avatar

Posts: 1348
Liked others: 202
Was liked: 203
New b60 net https://media.katagotraining.org/upload ... 705.bin.gz

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 440 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7, 8, 9 ... 22  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group