How to Run Your Threadripper 3000 Cool & Quiet, for a Change.
On a truly workstation-oriented, fanless motherboard.
In this brief article, we are going to discuss what follows:
- Pros and Cons of a 3rd-Gen Threadripper HEDT configuration, and why all the Cons are quite manageable.
- My TR3000 build: how and why I choose my components.
- How to run your TR3000 on air well below its termal-throttling threshold, and spare money upon your electricity bill, while at the same time achieving solid acoustic comfort as a bonus
Should I buy one?
To say the least, Zen2-based Threadripper 3000s had astonished the world of computing. While their Zen and Zen+ predecessors managed to live up to their name by exhibiting strong multithreaded performance and a very good price-performance ratio, they remained inferior to their direct (much more expensive) Intel counterparts in terms of raw horsepower.
Unfortunately for Intel, things changed drastically with the latest AMD HEDT-oriented creation: not only these processors stand out from the competition in multithreaded applications, but they managed to deliver jaw-dropping performance even in single-threaded benchmarks, which had always been sort of an Achilles’ heel for AMD.
Just to capture visually how good these CPUs actually are, let me show you two traditional benchmarks. Both on them come from the in-depth review made by Servethehome about the cheapest member of the family, the 3960X.
Cinebench R20 (rendering):
Linux kernel compiles per hour (software development):
Note that the 28-core W-3275 ($4449) is the absolute best that Intel can offer for workstations at the moment, but even the 24-core 3960X ($1399) crushes it quite effortlessly.
But the story doesn’t end here. Both the number of PCIe lanes and their bandwidth are important factors for determining how good a workstation processor actually is, since many professionals want to adopt multi-GPU configurations and fast NVMe storage solutions.
While TR3000s do support PCIe 4.0, Xeons and i9–109xx-XE limit themselves at PCIe 3.0, that is, half the bandwidth; PCIe 4.0 will come to Intel platforms only with Ice Lake.
More to the point, the monstrously expensive W-32xx do actually match the lanes that a TR3000 gives to the user (64), but i9 and Xeon W-22xx will deliver just 48 lanes.
The Downsides
Alas, such performances and feature-richness do have a price. In my opinion, there are two serious drawbacks with the Threadrippers:
1 — It’s difficult to cool them effectively.
These processors draw a lot of power, and as a consequence they have a tremendous heat output. In turn, this makes them difficult to be properly cooled.
Leaving apart custom loop liquid coolers, that are expensive, difficult to manage, and need maintenance, one has to choose between massive air coolers and All-In-One liquid solutions (AIOs).
Both will struggle to keep a 3rd-Gen Threadripper below its 85C throttling temperature, and both will annoy the user by being awfully loud as a consequence of operating above their limits.
The official AMD subreddit is full of such complaints.
2 — No professional motherboards.
These are definitely not gaming processors. Would you spend well over a thousand of dollars on a Threadripper when you could be much better served by a 5950X, in gaming?
TR3000s are clearly made to do actual work rather than gaming, and just happen to be good at gaming as well.
Still, TRX40 motherboards are annoyingly game-ish, full of RGB stuff, and lacking important features for professional scenarios, like IPMI (Integrated Platform Management Interface).
Perhaps more importantly, almost all of them do have a slot layout not suited for multi-GPU usage scenarios (four dual-slot GPUs or at least two triple-slot GPUs like the new 3090 FE).
To make things even worse, all of them employ tiny chipset fans. Other than producing a typically high-pitched whine under stress, these fans do have very weak motors. They would be easily blocked by an insect wandering into your case or other particles or untidy cables, and that could have nasty consequences.
Both of these drawbacks can be taken out of the table.
The Build
Not surprisingly, I decided to go for the cheapest member of the family, that is the 3960X. As you may see in the benchmarks above, it offers the best performance vs. cost ratio.
For example, the 3970 is 22.5% faster than its smaller sibling in CBR20, but it costs 43% more (retail prices: $1399 and $1999 respectively).
Apart from the CPU, I bought the only professional TRX40 motherboard there is: the Asrock Rack TRX40D8–2N2T.
Most people are unaware of its existence since it’s totally neglected by the reviewers. As you may see, there are no fans upon the chipset or the VRMs (it wasn’t so difficult to manufacture one in such a way, was it?) and still it’s a server part certified for 24/7 operation.
As previously said, it’s equipped with an IPMI interface with integrated video output and a dedicated ethernet port.
The number of USB ports is a bit scanty (two 3.1 and two 3.2, one of which is USB-C, plus four by the front panel), but is has two 2.5G NICs and two 10G NICs (Intel X710), as well as two M.2 slots for NVMe SSDs.
The slot layout is ideal for multi-GPU usage, and while the last slot is 8X, it is also open-ended, so you can install a GPU into it and still enjoy the bandwidth you would have had with a full-length PCIe 3.0 16X slot.
That wide heatsink you may see in front of the PCIe slots is there to cool the two Intel X710 10G ethernet chips, and no, it doesn’t interfere with the GPUs.
Another generous heatsink takes care of the VRMs.
As discussed above, one can choose to cool the Threadripper by using a 280/360mm AIO or a big air cooler.
I’m not a big fan of AIOs, due to reliability concerns. On top of that, the machine will be used remotely via IPMI and/or VNC when I’m not at my place, so it can’t be constantly monitored.
As for air coolers, there were many options by Noctua, BeQuiet!, Cooler Master, and Thermalright. I decided to choose the Noctua because it does well in the reviews, but more importantly for me it’s narrow enough to allow easy access to the memory modules once you remove the fans.
The cooler is damn big. You need a case clearance of 165mm at least, in order to use it. It’s size becomes apparent as it’s mounted upon the socket, but note that it doesn’t block access to anything important.
It can also be configured with two fans in a push-pull fashion.
I adopted ECC memory as I always do with any professional machine I build. People often underestimate the importance of Error Correction (and reporting) RAM.
If a bit flip occurs in memory, and some files are open in memory because you are editing them, they will end up corrupted as they are written back on the hard disk. If you do important work with you computer, you can’t really afford that.
ECC also reports and logs defective memory modules. A perfectly functional module can become faulty in time, as a consequence of current spikes or silicon degradation.
High frequency ECC memory is now relatively cheap. I went for four 32Gb Micron modules, 3200Mhz.
I had a spare case, a Jonsbo U4 bought a few months ago for an HTPC project I never actually realized. Maybe such a build deserved a better case, but I was somewhat excited at the perspective of making a Threadripper work on air in such a tight environment.
The U4 is one of the most compact cases capable of supporting ATX boards: it’s indeed smaller than most micro-ATX gaming cases, while still allowing a CPU heatsink clearance of 170mm.
Furthermore, being a case with HTPCs in mind, it’s equipped with just two 120mm intake fans and one 120mm exhaust fan. That would make the job of our Noctua cooler even harder.
As you may see, the heatsink is now installed with only one fan. Two fans would fit, but there wouldn’t be enough space for the second one to exhaust properly. Another challenge added to our seemingly impossible cooling mission.
How to run it cool & silent
First of all, I attached the Noctua to the motherboard fan header for automatic control, and left the BIOS parameters on their factory setting, that is, everything on [Auto].
Then I did a quick benchmark, CBR20. The same used above by the folks at Servethehome.
Leaving aside that I got the same score as in the review by Servethehome, you can observe that the maximum observed temperature (highlighted in yellow) went over the throttling threshold, despite the fact that this benchmark has finished rather quickly. On the other hand, the Noctua cooler became annoyingly loud in such circumstance at more than ~80C.
As a result, I cursed and tried to do a trick I always do to make my HTPCs as silent as possible, that is, reducing the CPU’s maximum performance level using Windows’ advanced power settings:
I left the stress test provided by CPU-Z running for 30 minutes, and the maximum observed temperature was 78C, well below the throttling wall at 85C, with the power draw oscillating between ~160W and 190W (the frequency multiplier oscillated accordingly between 37.25 and 38.50). Here I restarted briefly the stress test provided by CPU-Z just to show you the score: ~15300, against the usual ~15700 (all-core boost frequency is usually 4.1 Ghz).
The loss in performance has been negligible, while the savings in power and heat output were in fact substantial.
The Noctua cooler was way quieter.
Nonetheless, no matter how effective, this is quite an inconvenient way of obtaining such result. Custom power settings are easily overridden by apps and even OS upgrades. And I still don’t like 78C, since I want complete silence. And what if one wants to use this processor with Linux?
Thus, I started googling for more permanent solutions. It turned out that people at Computerbase.de already did some experiments, and I find it to be quite surprising no other reviewer thought about doing them. Here are some interesting results:
Note that the 3960X lost just 7% of its usual performance by being configured at 180W.
How did they do that? Well, the article show us two methods:
- By using the AMD proprietary software Ryzen Master (only for Windows!)
- By the BIOS.
It turns out that Ryzen Master has an ECO mode that limits to 180W the maximum current that the socket can deliver to the processor. Once again, I don’t like it, since it’s limited to Windows and forces you to have Ryzen Master always running.
The article mentions that you can do the same by tinkering with the BIOS, and even better you can set the BIOS at any power level you desire (while Ryzen Master has only that ECO mode at 180W). But it says nothing about how do it actually. And AMD Bioses do have hundreds of settings related to the processor.
Actually, in the forum post accompanying the article, people asked precisely about that, but no answers were given.
I will spare you some headaches. AMI Aptio BIOSes for Zen2 platforms are all very similar, no matter board manufacturer. The following screenshots are to be intended as sequential.
1 — Go to Advanced, and then open AMD Overclocking:
2 — Accept whatever they want you to accept to cover their arse.
3 — Then select Precision Boost Overdrive:
4 — Set it to [Advanced], and set PBO Limits to [Manual]
5 — Now you just have to change the PPT Limit from 0 (no limit) to whatever wattage you want to achieve. We can safely assume that one can go as low as 90W, probably even less than that. But don’t exaggerate or you’ll compromise stability.
I tried just 180W and 140W since I had the benchmark by Computerbase to compare against.
Thermals, Power, Performance.
I measured the power draws at wall using a Fluke power meter. All these reading were obtained while running the systems configured as follows:
- 3960X processor
- Asrock Rack TRX40D8–2N2T
- One XPG SX8200 Pro 1Tb NVMe ssd.
- EVGA 1000T2 Power supply
- One Noctua 14cm CPU fan
- Three Arctic P12 Case fans.
Note that there is no discrete video card.
Here are the results:
In the end, I did set my 3960X at 180W, attached all the fans to a $7 fan controlling bracket purchased on amazon, set them at inaudible levels, and forgot about noise and heat. Here we are, a 3960X in a small, silent box. No liquid, no hassle.
Hope this helps, and that you liked my config! Cheers!
A.
