It is currently Thu Mar 28, 2024 1:30 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 216 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6 ... 11  Next
Author Message
Offline
 Post subject: Re: KataGo V1.3
Post #41 Posted: Mon Jan 27, 2020 9:31 am 
Dies in gote

Posts: 52
Liked others: 2
Was liked: 12
Ah I totally missed the 'p' :) Thanks for clarifying

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #42 Posted: Mon Jan 27, 2020 3:05 pm 
Oza
User avatar

Posts: 2401
Location: Tokyo, Japan
Liked others: 2338
Was liked: 1332
Rank: Jp 6 dan
KGS: ez4u
@lightvector
Do you have any plans to release any further updates to the 10b net? On my old hardware it is a joy to fool around with. :)

I understand you have to prioritize things though.

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21


This post by ez4u was liked by: Gomoto
Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #43 Posted: Mon Jan 27, 2020 3:21 pm 
Lives with ko

Posts: 150
Liked others: 200
Was liked: 30
Rank: 25 kyu
Yakago wrote:
@lightvector

Do you have some additional details on your internal rating tests btw?

On CGOS, some users have put up 800 visits katago bots.

This one I would guess is the first 1.3 release, and it ranks slightly higher than LZ ELFv2 v800, but is worse in the head-2-head matchup.

Another one should be the 2nd release. It has not played that much, and has been taken offline, but had even poorer results against LZ ELF v2

We of course don't know the settings of these, but what do you make of it?

Cgos rankings have katago 1.3.1 s103 at 800 visits at a comparable strength to lz elfv1 at 1600 playouts, so 9d
http://www.yss-aya.com/cgos/19x19/cross ... 3v800.html

Estimates of different bots human rankings (overestimation)
https://github.com/breakwa11/GoAIRatings

So basically you will never in your life beat katabot on ogs in an even game. Which is such an amazing thing for such a young bot

_________________
set katago to play at your level https://docdro.id/sHZU1ti or experiment with gtp4zen ( https://rb.gy/kx2ilb )

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #44 Posted: Mon Jan 27, 2020 4:04 pm 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
I also love the 10b network, even on a 1080ti. Responsivness is everything ;-)


This post by Gomoto was liked by: lightvector
Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #45 Posted: Mon Jan 27, 2020 8:01 pm 
Lives in sente

Posts: 757
Liked others: 114
Was liked: 916
Rank: maybe 2d
ez4u wrote:
@lightvector
Do you have any plans to release any further updates to the 10b net? On my old hardware it is a joy to fool around with. :)

I understand you have to prioritize things though.


I already stopped training that one - it seemed mostly to be capped out and not getting noticeably stronger at the time I released it. Perhaps you could gain still another 100-200 Elo with really really really long training times on big net data with low learning rates, but I think overall it's close to as strong as possible at that size given KataGo's current methods.

Glad you like it! :D

And if you do want to go one step up the ladder of slower but stronger, the 15-block net here https://github.com/lightvector/KataGo/r ... 1.3.1-nets is just shy of the old KataGo (g104) 20 block net in strength. I'm continuing to train it a bit more and it's now just a little past g104 20 block in strength. - I'll release an updated version when it's done - it's slowing down a lot and probably going to cap out soon.


This post by lightvector was liked by: ez4u
Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #46 Posted: Fri Jan 31, 2020 6:05 am 
Lives in gote

Posts: 502
Liked others: 1
Was liked: 153
Rank: KGS 2k
GD Posts: 100
KGS: Tryss
At how many playing games (approximate numbers) did the 10b capped out?

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #47 Posted: Sat Feb 01, 2020 10:55 pm 
Lives in sente

Posts: 757
Liked others: 114
Was liked: 916
Rank: maybe 2d
I'd guess somewhere around 4-5 million games is when I stopped training it. It's a little annoying to check more precisely, since all my data is recorded in terms of number data rows, not games. Only about half of these games were 19x19, mind you. The rest are on smaller board sizes. :)

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #48 Posted: Sat Feb 01, 2020 11:11 pm 
Lives in sente

Posts: 757
Liked others: 114
Was liked: 916
Rank: maybe 2d
v1.3.2 is now released. If you're using the OpenCL version, you should get a noticeable performance boost. And along with stronger nets - new net gains another 100ish Elo over the old one from a week ago, at least in self-play testing, and there is a very slightly better 15-block net too.

If you were finding KataGo to be roughly evenly matched with LZ before, then with the speed boost and the better nets here hopefully this puts KataGo more clearly ahead. :rambo:

As usual, find things here:
https://github.com/lightvector/KataGo/releases

If you find a bug or something silly that I messed up in this release that isn't working, poke me and I will fix it. Enjoy!


This post by lightvector was liked by 5 people: ez4u, go4thewin, Vargo, Waylon, wineandgolover
Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #49 Posted: Sun Feb 02, 2020 2:34 am 
Dies with sente

Posts: 108
Location: France
Liked others: 14
Was liked: 18
Rank: FFG 1d
lightvector wrote:
v1.3.2 is now released. If you're using the OpenCL version, you should get a noticeable performance boost.

I have a GTX 1660 Ti GPU but I have issues when trying to install the CUDA version. I didn't report it because the OpenCL version is working fine and I have no experience with compiling so helping me would have been a hassle. That being said, I wondered if I was missing out on performance.
I can confirm the boost in performance. With the 1.3.1 katago and 15 blocks, Lizzie reports about 500 visits per second. With the 1.3.2 katago and new 15 blocks, Lizzie shows around 800 visits per second.

Edit: Spent some more time to explore the CUDA version. Here is what I've done to set up katago so far:
- I downloaded the lastest version of Lizzie not long ago.
- I donwloaded the new version(s) of katago and put them in new folders
- I added new engines in Lizzie, pointing to the correct katago folder and net
This has been working fine with the OpenCL versions so far.
I had done the same with the CUDA versions but in Lizzie the engine was loading forever. Trying to execute the katago file would prompt a missing dll file. I finally (today) found that dll file and copied it in the katago folder. The engine loaded :clap: But it was super slow :sad: (about 100 visits per second). I guess I need to tweak the parameters but I don't know how. It seems it was done automatically with OpenCL.

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #50 Posted: Sun Feb 02, 2020 6:37 am 
Lives in sente

Posts: 757
Liked others: 114
Was liked: 916
Rank: maybe 2d
I'm not exactly sure why it would be slow. There aren't actually that many parameters you can tune that would have that big of an effect, just one mainly: have you run the benchmark command in a console/command window to select a number of numSearchThreads to use? (See the readme, and/or the main page https://github.com/lightvector/KataGo#how-to-use for the command).

Also, is it automatically using FP16?

Edit: Oh, interesting, looks like according to this: https://www.tomshardware.com/reviews/nv ... ,6002.html GTX 1660 Ti doesn't have any tensor cores, despite being on a high compute capability past where tensor cores were introduced. If that's indeed true and I'm not misreading things, then that would explain it. So you can also try going into your config and specifying:

cudaUseFP16 = false
cudaUseNHWC = false

to explicitly disable FP16 and the attempt to use tensor cores, but I suspect the OpenCL will be better for you, as the big reason for the CUDA on top-end cards is the tensor cores.

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #51 Posted: Sun Feb 02, 2020 7:41 am 
Dies with sente

Posts: 108
Location: France
Liked others: 14
Was liked: 18
Rank: FFG 1d
lightvector wrote:
I'm not exactly sure why it would be slow. There aren't actually that many parameters you can tune that would have that big of an effect, just one mainly: have you run the benchmark command in a console/command window to select a number of numSearchThreads to use? (See the readme, and/or the main page https://github.com/lightvector/KataGo#how-to-use for the command).

Also, is it automatically using FP16?

Edit: Oh, interesting, looks like according to this: https://www.tomshardware.com/reviews/nv ... ,6002.html GTX 1660 Ti doesn't have any tensor cores, despite being on a high compute capability past where tensor cores were introduced. If that's indeed true and I'm not misreading things, then that would explain it. So you can also try going into your config and specifying:

cudaUseFP16 = false
cudaUseNHWC = false

to explicitly disable FP16 and the attempt to use tensor cores, but I suspect the OpenCL will be better for you, as the big reason for the CUDA on top-end cards is the tensor cores.

I had not run the benchmark command. It's done now (correctly I hope). It suggested numSearchThreads=16 so I changed accordingly.
The katago-gtp10.cfg file has cudaUseFP16 = true and cudaUseNHWC = true. I changed them to false, but saw no difference in Lizzie.

I can confirm my card doesn't have tensor cores. The GTX 1660 cards are based on the same architecture as the RTX cards so the fp16 thing should also work, but they don't support Ray Tracing and DLSS which are the gaming features using the tensor cores.

I'll stick to the OpenCL version for the moment. If someone has a RTX 2060 I'm curious how fast it is compared my my graphic card.

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #52 Posted: Sun Feb 02, 2020 12:42 pm 
Lives in sente

Posts: 757
Liked others: 114
Was liked: 916
Rank: maybe 2d
A bug has been now fixed in the v1.3.2 release where KataGo could crash if you have pondering enabled and the GUI you are using then tries to ask it to score the game while it's pondering.

The release and all the downloadable executables have been updated at the same place as before:
https://github.com/lightvector/KataGo/releases/

(thanks to the LZ discord user who reported it).

Additionally, KataGo will now tolerate interleaving white passes with black moves on an otherwise empty board as a way of "setting up" a handicap game - which should work around some issues with Lizzie and free handicap placement (although hopefully Lizzie gets fixed eventually too).

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #53 Posted: Sun Feb 02, 2020 12:50 pm 
Lives in sente

Posts: 757
Liked others: 114
Was liked: 916
Rank: maybe 2d
A bit more detail about the Lizzie handicap thing:

Due to KataGo not detecting such a handicap placement method before, if you tried to manually place stones on the board to play a handicap game within Lizzie, then KataGo would treat it as an even game, and therefore quickly attempt to resign, because if you are behind by that much in an even game at high levels of play, there is no hope.

Lizzie has a bug where if the bot attempts to resign in such a situation (or maybe attempts to resign ever in any situation?), Lizzie treats it as a pass - which might be very very confusing if you're trying to play the game. Additionally, Lizzie then sends a nonsensical command to the bot after that but ignores the response, which causes Lizzie GUI and the bot to be desync'ed in their idea of what the board position.

This bug still exists in Lizzie as far as I can tell, but you're less likely to hit it now because KataGo will not resign so early any more if you use this method of setting up an initial handicap.

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #54 Posted: Sun Feb 02, 2020 8:39 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
40 game test at time parity, 1600 visits for LZ :

Katago v1.3.2 (20b_s19_d43) v LZ017 (#261)

Katago wins 30-10 (75%)

Gogui-twogtp 1.5.1, all games by resignation, no error, no duplicate game.
LZ : --gtp -w 261_0a054a.gz -v 1601 --noponder -r 10
Katago : gtp -model g170-b20c256x2-s1913382912-d435450331.txt.gz -config gtp.cfg
LZ takes slightly more time per game (LZ average : 239s, Katago : 232s)

lightvector wrote:
If you were finding KataGo to be roughly evenly matched with LZ before, then with the speed boost and the better nets here hopefully this puts KataGo more clearly ahead.
Yes indeed ! :clap:

Games details : (LZ always apear as B, because of the command -alternate)
Attachment:
k1.jpg
k1.jpg [ 66.67 KiB | Viewed 8300 times ]
Details :
Attachment:
k2.jpg
k2.jpg [ 188.05 KiB | Viewed 8300 times ]
If someone wants the games, I'll upload them


This post by Vargo was liked by: Maharani
Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #55 Posted: Mon Feb 03, 2020 4:19 am 
Lives with ko

Posts: 150
Liked others: 200
Was liked: 30
Rank: 25 kyu
Thanks! Can you do a similar test between the new 15b net and Lz-Elfv2?

_________________
set katago to play at your level https://docdro.id/sHZU1ti or experiment with gtp4zen ( https://rb.gy/kx2ilb )


This post by go4thewin was liked by: Maharani
Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #56 Posted: Mon Feb 03, 2020 8:39 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
go4thewin wrote:
Thanks! Can you do a similar test between the new 15b net and Lz-Elfv2?
40 game test at visits parity (not far from time parity on this PC : average 245s/game for ELFv2, 275s for KataGo)

KataGo v1.3.2 : g170e-b15c192-s1672170752-d466197061, maxVisits = 3200
v
LZ017 : leelaz.exe --gtp -w ELFv2 -v 3201 --noponder -r 10

Gogui-twogtp 1.5.1, all games by resignation, no error, but 3 duplicate games (=37 games for the results)

Katago wins 24:13 (~65%)

Details (KataGo always appears as W, because of the command -alternate)
Attachment:
k1b.jpg
k1b.jpg [ 75.58 KiB | Viewed 8163 times ]
Attachment:
k2b.jpg
k2b.jpg [ 210.51 KiB | Viewed 8163 times ]

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #57 Posted: Mon Feb 03, 2020 10:20 pm 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
Another 40 game test at near time parity (ELF takes ~57s/game KataGo 51s)

KataGo v1.3.2 : g170e-b15c192-s1672170752-d466197061, maxVisits = 700
v
LZ017 : leelaz.exe --gtp -w ELFv2 -v 801 --noponder -r 10

Gogui-twogtp 1.5.1, all games by resignation, no error, no duplicate games

Katago wins 32:8 (80%)


Details : (KataGo always appears as W, because of the command -alternate)
Attachment:
k1c.jpg
k1c.jpg [ 82.19 KiB | Viewed 8153 times ]
Attachment:
k2c.jpg
k2c.jpg [ 206.59 KiB | Viewed 8153 times ]

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #58 Posted: Tue Feb 04, 2020 4:17 am 
Oza
User avatar

Posts: 2401
Location: Tokyo, Japan
Liked others: 2338
Was liked: 1332
Rank: Jp 6 dan
KGS: ez4u
Vargo wrote:
40 game test at time parity, 1600 visits for LZ :

Katago v1.3.2 (20b_s19_d43) v LZ017 (#261)

Katago wins 30-10 (75%)

Gogui-twogtp 1.5.1, all games by resignation, no error, no duplicate game.
LZ : --gtp -w 261_0a054a.gz -v 1601 --noponder -r 10
Katago : gtp -model g170-b20c256x2-s1913382912-d435450331.txt.gz -config gtp.cfg
LZ takes slightly more time per game (LZ average : 239s, Katago : 232s)

...

How many visits did this mean for Katago?

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #59 Posted: Tue Feb 04, 2020 7:22 am 
Lives in gote

Posts: 337
Liked others: 22
Was liked: 97
ez4u wrote:
How many visits did this mean for Katago?
maxVisits=1600 (or maybe 1500, not 100% sure, because I changed the config since)

Top
 Profile  
 
Offline
 Post subject: Re: KataGo V1.3
Post #60 Posted: Tue Feb 04, 2020 2:07 pm 
Lives with ko

Posts: 150
Liked others: 200
Was liked: 30
Rank: 25 kyu
Vargo wrote:
go4thewin wrote:
Thanks! Can you do a similar test between the new 15b net and Lz-Elfv2?
40 game test at visits parity (not far from time parity on this PC : average 245s/game for ELFv2, 275s for KataGo)

KataGo v1.3.2 : g170e-b15c192-s1672170752-d466197061, maxVisits = 3200
v
LZ017 : leelaz.exe --gtp -w ELFv2 -v 3201 --noponder -r 10

Gogui-twogtp 1.5.1, all games by resignation, no error, but 3 duplicate games (=37 games for the results)



Details (KataGo always appears as W, because of the command -alternate)
Attachment:
k1b.jpg
Attachment:
k2b.jpg

wow thank you!
for anyone else interested, cgos now has a kata bot 191 playing with 800 playouts http://www.yss-aya.com/cgos/19x19/cross ... _p800.html
with 1 playout http://www.yss-aya.com/cgos/19x19/cross ... t1_p1.html

_________________
set katago to play at your level https://docdro.id/sHZU1ti or experiment with gtp4zen ( https://rb.gy/kx2ilb )

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 216 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6 ... 11  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group