Building a TrueNAS Scale Machine

Introduction

Since TrueNAS Scale has become generally available (GA), I thought this would be a good time to build a DIY NAS machine. I wanted to build a NAS that was optimized for sequential transfer performance, using mainstream PC parts. This post will be about how to go about building a TrueNAS Scale machine.

I already have an eight-bay Synology DiskStation DS1817+ that has eight 6TB Seagate Iron Wolf Pro drives in RAID 5 with Btrfs. It has an Intel Atom C2538 4C/4T processor running at 2.4GHz. That Atom processor was released in Q3 2013, so it is very weak by modern standards.

Building a TrueNAS Scale Machine
Synology DS1817+

I’ve done what I could to expand the old Synology by maxing out the RAM at 16GB and adding a PCIe 3.0 10GbE network adapter card. Even so, I’m not satisfied with the sequential transfer performance.

Building a TrueNAS Scale Machine
Synology DS1817+ Read Performance

I realize that this Synology unit is two generations old, but it is frustrating that Synology has a two-year release cycle. They are also slow to add integrated 10GbE support compared to other vendors such as QNAP. Another problem is that Synology is slowly forcing you to buy their own branded storage as new NAS models are released.

Building a TrueNAS Scale Machine

So, the question is whether I can build something better without spending a huge amount of money. I didn’t want to use an actual rackmount server, mainly because of noise and power usage. Instead, I wanted to build a PC tower machine that I could easily modify and expand over time.

Required Components

Just like with any other desktop type PC, there are a number of required components that you must have. These include:

  • CPU (and CPU cooler)
  • Motherboard
  • RAM
  • GPU (which can be integrated or discrete)
  • Boot drive(s)
  • Storage drives
  • Power supply
  • Case

If you are building a dedicated NAS machine, you may also need additional drives for L2ARC read caching or SLOG usage. It is also likely that you will need a PCIe AIC network card (unless your motherboard has an integrated 10GbE NIC).

You need to think about your main use case for the NAS. Is it mainly for shared storage, or do you also want to run applications on it? Perhaps you want a Plex server, or you want to run VMs or containers. Your use cases should drive your component choices

Because my main use case is shared storage, I needed to try to choose my components with that in mind.

CPU

I wanted a modern, high-performance CPU with PCIe 4.0 or better support. It was important that I could use DDR4 RAM, since I have a lot of it on hand. If possible, I wanted a CPU with integrated graphics, so I could forego the expense and power usage of a discrete GPU.

These factors pointed me at an Intel Alder Lake CPU. I was leaning towards an Intel Core i7-12700, until my friend Wendell Wilson let me know that Linux still struggles some with mixed P and E cores. This made me ultimately choose an Intel Core i5-12400 CPU. It has 6C/12T with a base clock of 2.5 GHz and a Turbo clock of 4.4GHz. There are six Performance cores and zero Efficient cores, so it works great on Linux.

Building a TrueNAS Scale Machine
Intel Core i5-12400

This CPU also comes with a decent Intel RM1 Laminar CPU cooler in the box. This CPU cooler is MUCH better looking than the old, boxed CPU coolers that Intel used for so many years. More importantly, it has better cooling performance.

Motherboard

Since I am optimizing for storage performance, I wanted a motherboard with as much storage bandwidth as possible. Looking at the number of SATA ports, PCIe slots and M.2 slots is very important here. You also need to look at the total number of PCIe lanes, what generation they are, and how they are allocated between the CPU and chipset.

Finally, you really need to take a look at the motherboard manual, so you understand what happens as you populate all of the ports and slots. It is very common for SATA ports to be disabled when you populate all of the M.2 or PCIe slots.

The chipset that the motherboard uses is also critically important for total storage bandwidth. Right now, the Intel Z690 chipset has the most total bandwidth and flexibility of any mainstream chipset for Intel or AMD. The x8 DMI 4.0 link between the CPU and the chipset gives you nearly 16GB/sec of bandwidth across that link.

Intel Z690 Chipset Block Diagram

After quite a bit of research, I decided to use an ASRock Z690 Steel Legend DDR4 motherboard. This particular Z690 motherboard has eight SATA 3 ports, three PCIe x16 slots, two PCIe x1 slots and three M.2 slots.

Two of the M.2 slots are PCIe 4.0 and one is PCIe 3.0. I am going to put an M.2 to five port SATA3 adapter in the PCIe 3.0 slot. The adapter I am using is PCIe 3.0 x2 so it will have about 1,700MB/sec of total bandwidth. That should be fine for five SATA 3 hard drives.

ASRock Z690 Steel Legend

This motherboard is relatively affordable for a Z690 model. The fact that it can use DDR4 RAM is also a plus. I’ll talk more about what components are going in what ports and slots later in the post.

RAM

I decided to use 64GB of DDR4-3600 CL16 RAM in this machine, mainly because I already had it on hand. To be specific, it is two G.SKILL Ripjaws V DDR4-3600 CL16 (F4-3600C16D-32GVKC) kits, for a total of 64GB of RAM. It runs at XMP speed with no problems with this CPU and motherboard.

TrueNAS Scale uses Adaptive Replacement Cache (ARC), which uses the physical memory (RAM) as a read cache. By default, it will use up to 50% of your RAM for ARC.

GPU

Since this will mainly be a storage NAS, I am just going to use the integrated Intel UHD Graphics 730 that come with the Intel Core i5-12400 processor. This reduces my cost and power usage and also frees up a PCIe slot that I can use for storage.

TrueNAS Scale supports GPU passthrough, so I could add a higher performance discrete GPU in the future if my workload changes.

Boot Drive

TrueNAS Scale uses Debian Linux, which will be installed on your boot drive. It does not hit that drive very hard, so it does not need to be very large or have high-end performance. A 16GB drive is large enough. You could use a USB thumb drive, but that is not recommended.

I decided to use a very modest 256GB WD SN530 PCIe 3.0 M.2 NMVe drive that I had removed from a Lenovo laptop. Another choice would have been to use a 120GB SATA drive, but I didn’t want to use a SATA port. My primary storage will be using all of my SATA ports.

Storage Drives

I am going to use a number of 8TB Toshiba N300 7200rpm SATA3 HDDs for my primary storage. The 8TB size seems to be the sweet spot for price vs. size. So far, I have nine of these drives, with thirteen SATA ports available. I want to do some experiments to figure out how many drives (and what RAIDZ level) to use for each storage pool. My objective is to be able to saturate a 10GbE network link for reads and writes.

So far, I am pretty impressed with the sequential performance of the Toshiba N300 drives. With five drives in RAIDZ1 in a storage pool, I should be able to max out a 10GbE link. I could also use six drives in RAIDZ2 to have better redundancy.

Toshiba N300 CrystalDiskMark Results

Cache Drive

You can also have a second level adaptive replacement cache (L2ARC), which is one or more cache drives added to a ZFS storage pool. This is a read-cache for data that doesn’t fit in your ARC cache in RAM. You should use high-performance NVMe drives for this, if possible.

I decided to use a 1TB Samsung 980 PRO M.2 PCIe 4.0 NVMe drive for this. This may be overkill, since I only have 64GB of RAM. The guidance is that your L2ARC should be 5X your ARC size.

The Samsung 980 PRO is a great PCIe 4.0 drive, and the price has come down quite a bit in recent months.

Power Supply

Since this DIY NAS will be running 24×7, I wanted a power supply with excellent energy efficiency. Power supplies with 80 Plus Titanium ratings are the highest efficiency that you can get.

Power Supply Efficiency Standards

If you want something like that, a Seasonic power supply is where you should start looking. I decided to use a Seasonic Prime TX-750 that I already had on hand. This is an extremely high- quality power supply with a twelve-year warranty.

In an ideal world, I would have used a Prime TX-650. This is because your highest efficiency is at 50% load. Lower capacity power supplies are more efficient with small loads, since they are more likely to be close to 50% of their total capacity.

I don’t know yet what my idle and peak power usage will be for the complete system, but a Seasonic Prime Fanless PX500 or PX450 would probably be the absolute best choice.

Case

Since this is a storage NAS, where I was planning on using many 3.5″ hard drives, I had some specific case requirements. Most modern tower PC cases simply don’t have very many 3.5″ drive bays. This is understandable, since most new PCs typically use mainly M.2 or 2.5″ drives for storage.

I looked around some, and finally decided to get a Fractal Design Meshify 2 XL case. This is a very large case that can hold up to eighteen 3.5″ drives (in storage mode). You will have to buy six Fractal Design Type B hard drive tray kits to do this, which will cost about $150 extra.

Fractal Meshify 2 XL

This case is very well ventilated, and it can hold multiple 140mm fans and even two 420mm radiators. It is an extremely modular and flexible case that is relatively pricey.

  • Micro Center –
  • Amazon – $273.16

Building a TrueNAS Scale Machine

At this point, I have done a bench build with the CPU, motherboard, RAM, boot drive, cache drive, and a couple of the storage drives in a mirror. I have installed TrueNAS Scale and done some initial experiments while I was waiting for additional components to arrive.

The boot drive is in the M2_3 slot and the Samsung 980 Pro drive is in the M2_1 slot (which is connected to the CPU). The M.2 to SATA adapter will go in the M2_2 slot (which is only PCIe 3.0).

I plan on putting a 10GbE NIC in the PCIE5 slot, which leave the PCIE1, PCIE2, PCIE3, and PCIE4 slots available. This should let me add additional SATA drives and some additional M.2 NVMe drives (in a PCIe to M.2 adaptor card) in the future.

Final Words

I will have more content about this system as I actually build it into the case.

If you have any questions about this post, please ask me here in the comments or on Twitter. I am pretty active on Twitter as GlennAlanBerryThanks for reading!

Categories Build Lists, Intel, PC Hardware, StorageTags

8 thoughts on “Building a TrueNAS Scale Machine

  1. Just curious, wouldn’t it been better to go with an AMD chip and board, since most boards support ecc capable ram on their consumer grade boards, granted Alder Lake supports ecc ram but not without using the more expensive xeon chipsets. I built a Truenas core system using a x470 board and a 3700x 8 core AMD cpu, 64gb of ecc ram and 8 4TB enterprise class drives, my read speeds cannot saturate the 10G network but that’s because my drives sequential read speeds only max out around 140. Also wouldn’t you have seen a speed boost on the synology nas just by upgrading the drives in it?

    1. I don’t think ECC RAM is that important for a lightly used home NAS.

      I thought about using an AMD client CPU and a B550 or X570 motherboard. The problem with that is that only AMD APUs (SKUs with a “G” at the end, like a Ryzen 5 5600G) have integrated graphics. That means that I would have to use one of the PCIe slots for a discrete GPU. This gives me fewer PCIe slots and fewer PCIe lanes to use for storage or network needs. It also means I need more power at idle and have to spend more money.

      The Intel Z690 chipset has more PCIe lanes (including a PCIe 4.0 x8 DMI link between the CPU and chipset) than any other mainstream consumer chipset. That is why I chose it for this.

      You are correct that I could have upgraded my drives in the Synology and perhaps removed a storage drive and replaced it with a SATA SSD for caching to get better throughput for some workloads. But I was getting significantly less throughput than I should have been with eight 6TB Iron Wolf Pro NAS drives. I think the old and very slow Intel Atom CPU was part of the problem, especially for writes.

  2. Can’t wait to see how this works for you, ready to build this exact setup and ready to bid farewell to my old Synology NAS. Any ETA on a follow-up post about the build?

    1. Well, this build is done, and I am very happy with it. I am long overdue for a follow-up post. It is very fast and very quiet, and it smokes my old Synology in all respects.

  3. Really nice build. I stumbled onto the post because I was looking at the same system board for TrueNAS Scale. How well do the 8 on-board SATA ports work with TrueNAS? How many drives did you end up with?

  4. This is excellent! Thank you for the great write-up Glenn! I will be using this to build my own TrueNAS setup in the near future.

  5. Hi Glenn,

    I hope you are doing well. I recently read your article on building a TrueNAS SCALE machine, which I found very informative. However, I wanted to let you know about a small mistake. The article can be found at the following link:

    https://glennsqlperformance.com/2022/03/13/building-a-truenas-scale-machine/

    In the section discussing the CPU, the second link directing to Amazon appears incorrect. While the first link to the Intel Core i5 at Micro Center is accurate, the Amazon link leads to a CPU with socket LGA 1200 instead of the intended socket 1700.

    Unfortunately, I discovered this discrepancy after my Amazon order arrived today. Could you please update the link on your website to avoid any confusion for future readers?

    Thank you for your attention to this matter, and I look forward to reading more of your informative articles.

    Best regards from California,

    Marcus

  6. I recently came across your informative article on building a TrueNAS server, and I found it quite intriguing.

    Last week, I built a TrueNAS Core system with 48 TB using an old PC from my garage. However, the motherboard is seven years old, and I mistakenly connected a SATA adapter to the PCIe 16x slot. I also connected a 10 Gb network card to another PCIe slot. As a result, the overall access speed to my hard drives has been significantly reduced. I should have done more thorough research before proceeding.

    On a brighter note, I’ve ordered all the components you used in your build. They have become more affordable since you originally purchased them so I could upgrade to 128 GB of RAM.

    Is there anything you would do differently with your system today? I plan to connect the NAS (I also have two Synology DS916 units) to my 10 Gb network and directly to my video editing computer. As a journalist, the system’s primary function will be to store my audio and video files.

    I’m looking forward to hearing from you and your recommendations. Thank you for your time and consideration.

Please tell me what you think

%d bloggers like this:
search previous next tag category expand menu location phone mail time cart zoom edit close