Nvidia RTX 30xx

[pwaldron]

Robert, perhaps you can clear something up for me. I've been reading a little recently, and I haven't been clear whether many cores is essential for *using* a neural network (i.e., using Katago) compared to training the network.

[RobertJasiek]

Training AI:

I think that multiple GPUs would be best, which then require quite a few of CPU cores to feed the GPUs.

Using AI:

It is just a matter of thinking time. More CPU cores are better. Here I mainly rely on the journal c't, which studied numbers of CPU cores for using deep learning. From 4 to ca. 8 cores, the acceleration was roughly proportional, and 6 was said to be the reasonable minimum. I would choose at least 8 (real cores). Above 8, the curve becomes flatter and flatter (when only using 1 GPU), although not too flat. It is then mostly a matter of cost per core. If you can pay them, get 10 or 12 cores. OTOH, instead of 16 cores, you might first consider getting 2 GPUs (subject to the problems discussed before).

For 1 GPU, I'd say that the sweet range is 6 - 16 real CPU cores, of which 6 - 8 is the most cost-efficient but more cores help more (if you can handle their heat).

So many cores are not "essential" but at least 6 cores are "essential" because with fewer cores you risk under-employing your GPU.

[wineandgolover]

I'm planning on buying a better card solely for Katago. My budget is around 300-400, so a 5600XT or a RTX 2070 is what I'm looking at right now... maybe a 3060 or 3070 if they come out soon.

Can anyone (@lightvector, @RobertJasiek) tell me if KataGo will be running better on a GPU with tensor cores than on one without? I presume it will, but I am not sure whether the tensor cores are only helpful at training time? And if they are helpful at runtime, is there a order-of-magnitude estimate how much better such a card will be (compared to a similar card without TC)? There is a very in-depth blog about GPUs for deep learning that says something like twice as fast...?

https://timdettmers.com/2020/09/07/whic ... -learning/

My question basically asks if a midrange NVIDIA card will still be a lot better for Katago than a similarily priced AMD card (that have no TC as far as I know?). The reason I'm even thinking about that is that NVidia cards used to have (and I think they still do) a lag problem in computation mode ... when the computation of the go bot runs, the GUI will become very unresponsive, even on a fast computer (correct me if that is no longer an issue). I asked the programmer of Leela (gcp) about that and he said that he knows about it, and that it has something to do with "schedulers" of which Nvidia cards have only one and AMD cards have more so they don't lag...

And is there a difference in regard to the usefulness of tensor cores between the OpenCL and the CUDA versions?

And is it possible to solve the lag issue by simply using two cards? One AMD for all display purposes, and an NVIDIA only for the computing part? Has anyone done that (on Linux)?

Any hints appreciated!!

Edit: Sorry if that is slightly OT, but I couldn't find a better place to ask this question, since lightvector is probably the one person who knows most about that.

Some NVIDIA cards run CUDA a bit faster, but it is fiddly, I’m told.

Version 1.61 gives NVIDIA cards a big speed bump over AMD equivalents because it supports FP16, which AMD doesn’t support.

If you have the choice of either, and your purpose is katago, I’d definitely go NVIDIA, and I say this, owning an AMD card.

[honinbo-shusaku]

Thanks @wineandgolover. From what I gathered about CUDA under Linux, yes, it seems a bit tricky.

I asked on the github page of kata about the tensor core support, and someone there said that Kata now supports tensor cores also with the OpenCL version, and that performance is very similar. So no need to tinker with CUDA anymore.

I guess I'll just wait for the AMD show end of October to see if they have anything interesting, and surely prices won't go up either after that... but it looks like it's going to be NVidia.

[RobertJasiek]

As to Ryzen, I consider 3700X, 3900X, some XT, 5800X, 5900X or await a still unannounced 5700X. XT may require a suitable mainboard or UEFI update presuming an earlier CPU. Ryzen 5000, too, a new mainboard launched later or a rare expensive mainboard with UEFI update without CPU.

Having inquired two manufacturers for a few mainboards, I can reconfirm most of this. Typically, currently available mainboards originally issued for 3700X, 3900X would now be offered with a newer UEFI also suitable for XT variants of the CPUs. 5800X, 5900X do indeed require an UEFI update; some mid priced mainboards allow this without CPU, but other mainboards might require an earlier CPU for that. In case of doubt, ask the manufacturers and retailers, identify UEFI versions and study compatibility lists for them.

[RobertJasiek]

Let me discuss RTX 3080 (if available) system prices in EUR.

Code: Select all

Item      Cheap  Low   Fair  Silent  Upgrade  High
---------------------------------------------------------------------------------------------
GPU         700   700   700   720     720      800
CPU          45   170   260   260     400      800
RAM          45    50   100   100     200      200
Storage      15    50    90    90     100      300
Mainboard    50   110   140   140     160      300
PSU          45    80   100   240     240      240
Case         40    70    80   130     130      200
Cooling       0     0     0   160     160      700
Misc          0     0     0    30      30       30
---------------------------------------------------------------------------------------------
Sum         940  1230  1470  1870    2140     3570
Difference -530  -240     0  +400    +670    +2100

In theory, a Cheap build is below €1000 but falls apart from watching it. A Low Build for some €1200 requires compromises with CPU (might bottleneck the GPU), RAM (many sample positions are impossible) and mainboard (VRAM runs hot). A Fair build (also typical Youtube build) approaches €1500 but makes intermediate or somewhat loud noise.

Silence (ca. 37 dB or less) adds €400 resulting in about €1900. The PSU is the major factor when doubling the watt to run it at ca. 50% for lowest noise. (The alternative is a passive 700W PSU saving €40 but risking occasional power failures.) A silent CPU cooler costs €60 - 100 (or more, if you prefer). Silent case fans add €30 - 80. Miscellaneous includes a GPU stand and extra PWM cables to some fans.

I have not listed RGB, but you might spend, say, +€200 on that.

An Upgrade build for +€270 (compared to the Silent build) yielding some €2100 includes a faster (but not the fastest gaming) CPU, doubles the RAM and chooses a mainboard with even better VRM temperatures. If you don't need silence, you spend ca. €1750.

The High End build for almost €3600 is not excessive but includes the fastest gaming CPU, a large PCIe 4.0 SSD, a medium to high end priced mainboard (but not one for two GPUs 4 PCIe slots apart, then the mainboard is €400 - 600) and a rough estimate of water cooling.

The wide range of expense is really interesting. One can chicken out but it makes little sense if one actually wants to use the GPU well. €1500 is the expected price of a system but a strict desire for silence adds another ca. €400. It is always tempting to get the best of everything but I think one should compromise on the CPU unless one also wants to use the PC for CPU intensive non-Go tasks. You can save €100 if you consider 32GB RAM sufficient. As explained earlier, building a 2 GPU system is expensive and difficult.

I guess I will go for something between Silent and Upgrade. What are your plans?

[honinbo-shusaku]

Interesting comparison, and sounds pretty correct to me. I'll wait for the AMD announcements, which will at least lower the GPU prices overall, and maybe they have something interesting with tensor cores in the pipeline (which I doubt).

At the moment I'm running a 120 EUR AMD RX 570 card, which already gives me enough power to analyze my games. If I take away all the "wanna have" feeling, that is already more than enough. I use Lizzie with kata and usually 100 playouts, which gives very reliably results at my SDK level of strength. Sometimes I double check with more playouts, but it basically never makes any difference. There was one game once, not my own but friend's, there was one complex position where the correct move (that was rather easy to spot for a human) took a few thousand playouts to find. Other than that...

My feeling is, as long as one is below 2-4 dan, a GPU like that is fine.

The only real profit I can get from a better card is more speed, so I could use maybe 200-500 playouts and still be faster than now. So I guess a used 2070-2080 will be quite sufficient for me. It depends on the prices. If I have the money to spare I might go for a 3060 - 3070, we'll see. Right now my old system (i7-3770) is still plenty for everything I do, so I'm not in the mood for a full upgrade, neither technically nor financially

@Robert: I wonder, what are you planning to do with such a machine, I mean in Go specific applications? As long as you're not training a neural net, what good will such a system do you? I'm just wondering because I'm asking myself the same question. Are you looking at pro games on a regular basis? I guess then it would make sense.

[RobertJasiek]

As somebody who has never been in Asia for insei-like study, with little chance to get a variety of regular real word opponents, affected by the KGS rating system giving me not strong enough opponents on average and currently having too little time to attend many top amateur tournaments, I expect that AI gives me good training games.

Besides, I hope that I can use some of the AI programs for backtracking errors to better find the sources of my mistakes when I cannot do it well enough on my own.

I need to study the opening again because, nowadays, opening books from the 70s are not good enough any more for my level. Since around 1995, openings have advanced a lot with many middle game aspects in them. Although I have deciphered some of that, it is very easy to make mistakes and overlook other things.

On average, my middle game fighting is not that of a 6d so I expect AI to help me training it. I have a few certain other middle game weaknesses of that I know that AI don't have them, so I should learn there, too.

Furthermore, it won't hurt to check one book example or another, AFA they can be feeded using a 19x19 embedding.

(See my earlier description for what I expect AI to not be able to do.)

[honinbo-shusaku]

So from a purely pragmatic point of view, I'd say what you want to do you can do with almost any card, a 1080TI will be plenty, a 2080 will be way ahead of any 9 dan, from what I gather. Pros streaming katago games take like 3 stones handicap, so in an even game, I doubt they have a chance even against a 1080TI on several hundred playouts.

I doubt it will add any real advantage over a 2080 or even 2060, when it comes to analyzing amateur (even 6D) games, and like others said, I'd bet my money on such a card vs. any 9p, on any time setting / no handicap. But if you have the money and don't mind spending a few hundred more, yes, why not get the 3080. It will be faster and be useful a few years longer. I'd do the same. Future bots will have bigger nets and demand more power, and it will be fun to play around with those! I just don't have that much to spare at the moment, so will look for a more down-to-earth-option. Until I'm ready to spend several hundred EUR. Also, I had a more or less high-end card (at the time, VEGA56) once, and it was so fu***g loud I sent it back within 5 minutes But that may have been a VEGA-specific problem. I just fear that the 3080s are just as annoying once they run under full load.

Regarding KGS: what about the asain servers? Not finding strong opponents there as well? KGS is sort of withering away it seems...

[RobertJasiek]

IIRC, Asus RTX 3080 TUF Gaming 10G or O10G in quiet mode is ca. 36 dB and MSI RTX 3080 Gaming X Trio is ca. 35.5 dB under non-overclocking load around 98%. Values vary a bit depending on who measures. You can listen to noise samples on Youtube, keyword "noise". IMO, not dead silent but not much more than constant whispering wind.

The (used as) case fan Arctic P14 (not F14), CPU cooler Scythe Fuma 2 (note the 2; alternative Arctic Liquid Freezer II 280, but beware the thickness of 65mm or with screws 67.6 mm) and power supply Corsair HX1200i (note the 'i') are at same noise level.

Hence, nowadays, fast PC and reasonable silence are not mutually exclusive any more. It is just a matter of expense, careful choice and compatibility checking.

[RobertJasiek]

After the paper launches, RTX 3000 (with Nvidia chips), RX 6000 (with AMD chips) and Ryzen 5000 CPUs are extremely short on stock. Apparently demand is 20x the supply. Rather than higher than usual demand, the common major reason seems to be too low capacities for producing 7nm and 8nm chips at TSMC and Samsung. Shortages are said be prevail until January, some sources say March. Apple's switch to 5nm TSMC must contribute to the problem. Intel can't deliver so everybody wishing up-to-date dies for top hardware wants to let TSMC or Samsung manufacture. They don't have enough production streets and output per wafer. The computer industry has not seen such a situation of overall extreme supply shortage before.

There are also side effects. At Ryzen 5000 launch, one might have expected the usual behaviour of older CPUs dropping in price. However, Ryzen 3000 has risen in price or remained constant, depending the model. Now that people realise they can't buy Ryzen 5000, some need to buy 3000. As this has become clear, also some other computer parts have become out of supply at reasonable prices, such as high end PSUs or new case models. Their restocking is stated as mid-December to mid-January.

Meanwhile, more and more paper launches of further RTX 3000 occur or are awaited. A Ryzen 5700X might also appear in the following months - on paper.

Ugh. Building a new RTX 3000 computer has become a pipe dream! (Unless you buy the worst model, of which a few are available per country and day, or an RTX 3090 at an unreasonable price around €2000.)

Although I was in no hurry to build such a PC and expected a calm situation in January, chances increase that I (and millions others) need to wait 1/2 or 1 year after the paper launch on September 17. More than daily checking of availability does not help.

[honinbo-shusaku]

Although I was in no hurry to build such a PC and expected a calm situation in January, chances increase that I (and millions others) need to wait 1/2 or 1 year after the paper launch on September 17. More than daily checking of availability does not help.

If you need to buy everything new, it really may take half a year before everything is back to normal. The corona season surely didn't help in production, and also it increased demand as so many people are stuck at home, gaming or needing computers for home office work. And of course it's christmas season, so demand is 2-3x the usual anyway.

I guess starting late Jan the situation will get better. I fear the 3000 GPUs and 5000 mid-range Ryzens will be hard to get even longer. They both are pretty neat improvements that many people have waited for. I think if one only needs a new GPU or a new CPU, it will be possible to pay a little extra and get it pretty soon. But a new system... tough. And not worth the premium in my opinion. It's just a crazy situation right now.

I seriously think I'll just get a 2000 series card or maybe the 5700, if I can get that for a good price, and wait with the CPU/MB. My system is still quite ok, no need to rush.

[RobertJasiek]

Finally, the first closed and very well cooling case, the new Fractal Meshify 2 XL, is out (Europeans might need to wait 2 or 3 months for import by retailers) that can fit 2x RTX 3090 FE or 2x RTX 3080/3090 each up to 3 slots thick installed directly on one of the few, expensive motherboards (€380 ~ €580) with two PCIe 4.0 x16 running at x8 speed slots being 4 slots apart! (If you can find and spend the necessary change on your preferred 2 GPUs.) So a low noise solution is possible and 2x RTX 3080 FE is not needed any more for a PC with 2x RTX 3080.

[iopq]

FWIW, my processor's not really utilizing 12 threads in KataGo. It does have some peaks, but when I disabled boost it didn't decrease the performance. In other words, 3600 is more than enough for KataGo with a 2060.

[RobertJasiek]

The AMD RX 6800XT equals the Nvidia RTX 3080 in 1080p / 1440p gaming without streaming, ray tracing, DLSS and "only" has a few driver bugs. The RTX 3080 is slightly better in 4K gaming and much faster in Blender benchmarks, which are more relevant for deep learning. Therefore, as expected, go players should prefer Nvidia.

In Germany, AMD's new GPUs are essentially unavailable. Several, but not all, models of Nvidia's RTX 3070, and a few models of RTX 3090, are available for at least ca. €100 ~ 150 shortage surcharge, but especially the 3090 often only for a very short period per retailer.

RTX 3080 is still rare: the worst models (Zotac, Palit) are occasionally very shortly available in tiny numbers for that surcharge, a few other models from retailers behaving as scalpers (€200 ~ 500 surcharge), the best and most sought models (all Asus; MSI GX3) are still unavailable in the €100 ~ 150 surcharge range. Availability is very slightly improving and surcharges are slowly dropping but it will last at least one or two more months before prices might approach MSRPs. Even then, I presume that the best 3080 models will still be unavailable.