Building a TrueNAS Scale Machine

Introduction

Since TrueNAS Scale has become generally available (GA), I thought this would be a good time to build a DIY NAS machine. I wanted to build a NAS that was optimized for sequential transfer performance, using mainstream PC parts. This post will be about how to go about building a TrueNAS Scale machine.

I already have an eight-bay Synology DiskStation DS1817+ that has eight 6TB Seagate Iron Wolf Pro drives in RAID 5 with Btrfs. It has an Intel Atom C2538 4C/4T processor running at 2.4GHz. That Atom processor was released in Q3 2013, so it is very weak by modern standards.

I’ve done what I could to expand the old Synology by maxing out the RAM at 16GB and adding a PCIe 3.0 10GbE network adapter card. Even so, I’m not satisfied with the sequential transfer performance.

I realize that this Synology unit is two generations old, but it is frustrating that Synology has a two-year release cycle. They are also slow to add integrated 10GbE support compared to other vendors such as QNAP. Another problem is that Synology is slowly forcing you to buy their own branded storage as new NAS models are released.

Building a TrueNAS Scale Machine

So, the question is whether I can build something better without spending a huge amount of money. I didn’t want to use an actual rackmount server, mainly because of noise and power usage. Instead, I wanted to build a PC tower machine that I could easily modify and expand over time.

Required Components

Just like with any other desktop type PC, there are a number of required components that you must have. These include:

• CPU (and CPU cooler)
• Motherboard
• RAM
• GPU (which can be integrated or discrete)
• Boot drive(s)
• Storage drives
• Power supply
• Case

If you are building a dedicated NAS machine, you may also need additional drives for L2ARC read caching or SLOG usage. It is also likely that you will need a PCIe AIC network card (unless your motherboard has an integrated 10GbE NIC).

You need to think about your main use case for the NAS. Is it mainly for shared storage, or do you also want to run applications on it? Perhaps you want a Plex server, or you want to run VMs or containers. Your use cases should drive your component choices

Because my main use case is shared storage, I needed to try to choose my components with that in mind.

CPU

I wanted a modern, high-performance CPU with PCIe 4.0 or better support. It was important that I could use DDR4 RAM, since I have a lot of it on hand. If possible, I wanted a CPU with integrated graphics, so I could forego the expense and power usage of a discrete GPU.

These factors pointed me at an Intel Alder Lake CPU. I was leaning towards an Intel Core i7-12700, until my friend Wendell Wilson let me know that Linux still struggles some with mixed P and E cores. This made me ultimately choose an Intel Core i5-12400 CPU. It has 6C/12T with a base clock of 2.5 GHz and a Turbo clock of 4.4GHz. There are six Performance cores and zero Efficient cores, so it works great on Linux.

• Micro Center – $169.99
• Amazon – $174.24

This CPU also comes with a decent Intel RM1 Laminar CPU cooler in the box. This CPU cooler is MUCH better looking than the old, boxed CPU coolers that Intel used for so many years. More importantly, it has better cooling performance.

Motherboard

Since I am optimizing for storage performance, I wanted a motherboard with as much storage bandwidth as possible. Looking at the number of SATA ports, PCIe slots and M.2 slots is very important here. You also need to look at the total number of PCIe lanes, what generation they are, and how they are allocated between the CPU and chipset.

Finally, you really need to take a look at the motherboard manual, so you understand what happens as you populate all of the ports and slots. It is very common for SATA ports to be disabled when you populate all of the M.2 or PCIe slots.

The chipset that the motherboard uses is also critically important for total storage bandwidth. Right now, the Intel Z690 chipset has the most total bandwidth and flexibility of any mainstream chipset for Intel or AMD. The x8 DMI 4.0 link between the CPU and the chipset gives you nearly 16GB/sec of bandwidth across that link.

After quite a bit of research, I decided to use an ASRock Z690 Steel Legend DDR4 motherboard. This particular Z690 motherboard has eight SATA 3 ports, three PCIe x16 slots, two PCIe x1 slots and three M.2 slots.

Two of the M.2 slots are PCIe 4.0 and one is PCIe 3.0. I am going to put an M.2 to five port SATA3 adapter in the PCIe 3.0 slot. The adapter I am using is PCIe 3.0 x2 so it will have about 1,700MB/sec of total bandwidth. That should be fine for five SATA 3 hard drives.

This motherboard is relatively affordable for a Z690 model. The fact that it can use DDR4 RAM is also a plus. I’ll talk more about what components are going in what ports and slots later in the post.

• Micro Center – $249.99
• Amazon – $273.99

RAM

I decided to use 64GB of DDR4-3600 CL16 RAM in this machine, mainly because I already had it on hand. To be specific, it is two G.SKILL Ripjaws V DDR4-3600 CL16 (F4-3600C16D-32GVKC) kits, for a total of 64GB of RAM. It runs at XMP speed with no problems with this CPU and motherboard.

TrueNAS Scale uses Adaptive Replacement Cache (ARC), which uses the physical memory (RAM) as a read cache. By default, it will use up to 50% of your RAM for ARC.

• Micro Center – $164.99 for each kit
• Amazon – $168.99 for each kit

GPU

Since this will mainly be a storage NAS, I am just going to use the integrated Intel UHD Graphics 730 that come with the Intel Core i5-12400 processor. This reduces my cost and power usage and also frees up a PCIe slot that I can use for storage.

TrueNAS Scale supports GPU passthrough, so I could add a higher performance discrete GPU in the future if my workload changes.

Boot Drive

TrueNAS Scale uses Debian Linux, which will be installed on your boot drive. It does not hit that drive very hard, so it does not need to be very large or have high-end performance. A 16GB drive is large enough. You could use a USB thumb drive, but that is not recommended.

I decided to use a very modest 256GB WD SN530 PCIe 3.0 M.2 NMVe drive that I had removed from a Lenovo laptop. Another choice would have been to use a 120GB SATA drive, but I didn’t want to use a SATA port. My primary storage will be using all of my SATA ports.

Storage Drives

I am going to use a number of 8TB Toshiba N300 7200rpm SATA3 HDDs for my primary storage. The 8TB size seems to be the sweet spot for price vs. size. So far, I have nine of these drives, with thirteen SATA ports available. I want to do some experiments to figure out how many drives (and what RAIDZ level) to use for each storage pool. My objective is to be able to saturate a 10GbE network link for reads and writes.

So far, I am pretty impressed with the sequential performance of the Toshiba N300 drives. With five drives in RAIDZ1 in a storage pool, I should be able to max out a 10GbE link. I could also use six drives in RAIDZ2 to have better redundancy.

• Micro Center – $194.99
• Amazon – $194.99

Cache Drive

You can also have a second level adaptive replacement cache (L2ARC), which is one or more cache drives added to a ZFS storage pool. This is a read-cache for data that doesn’t fit in your ARC cache in RAM. You should use high-performance NVMe drives for this, if possible.

I decided to use a 1TB Samsung 980 PRO M.2 PCIe 4.0 NVMe drive for this. This may be overkill, since I only have 64GB of RAM. The guidance is that your L2ARC should be 5X your ARC size.

The Samsung 980 PRO is a great PCIe 4.0 drive, and the price has come down quite a bit in recent months.

• Micro Center – $149.99
• Amazon – $149.99

Power Supply

Since this DIY NAS will be running 24×7, I wanted a power supply with excellent energy efficiency. Power supplies with 80 Plus Titanium ratings are the highest efficiency that you can get.

If you want something like that, a Seasonic power supply is where you should start looking. I decided to use a Seasonic Prime TX-750 that I already had on hand. This is an extremely high- quality power supply with a twelve-year warranty.

In an ideal world, I would have used a Prime TX-650. This is because your highest efficiency is at 50% load. Lower capacity power supplies are more efficient with small loads, since they are more likely to be close to 50% of their total capacity.

I don’t know yet what my idle and peak power usage will be for the complete system, but a Seasonic Prime Fanless PX500 or PX450 would probably be the absolute best choice.

• Micro Center – $199.99
• Amazon – $235.70

Case

Since this is a storage NAS, where I was planning on using many 3.5″ hard drives, I had some specific case requirements. Most modern tower PC cases simply don’t have very many 3.5″ drive bays. This is understandable, since most new PCs typically use mainly M.2 or 2.5″ drives for storage.

I looked around some, and finally decided to get a Fractal Design Meshify 2 XL case. This is a very large case that can hold up to eighteen 3.5″ drives (in storage mode). You will have to buy six Fractal Design Type B hard drive tray kits to do this, which will cost about $150 extra.

This case is very well ventilated, and it can hold multiple 140mm fans and even two 420mm radiators. It is an extremely modular and flexible case that is relatively pricey.

• Micro Center –
• Amazon – $273.16

Building a TrueNAS Scale Machine

At this point, I have done a bench build with the CPU, motherboard, RAM, boot drive, cache drive, and a couple of the storage drives in a mirror. I have installed TrueNAS Scale and done some initial experiments while I was waiting for additional components to arrive.

The boot drive is in the M2_3 slot and the Samsung 980 Pro drive is in the M2_1 slot (which is connected to the CPU). The M.2 to SATA adapter will go in the M2_2 slot (which is only PCIe 3.0).

I plan on putting a 10GbE NIC in the PCIE5 slot, which leave the PCIE1, PCIE2, PCIE3, and PCIE4 slots available. This should let me add additional SATA drives and some additional M.2 NVMe drives (in a PCIe to M.2 adaptor card) in the future.

Final Words

I will have more content about this system as I actually build it into the case.

If you have any questions about this post, please ask me here in the comments or on Twitter. I am pretty active on Twitter as GlennAlanBerry. Thanks for reading!