50GB/Sec with an X670E Motherboard!

Introduction

I recently bought an ASUS ROG Crosshair X670E Extreme motherboard along with an AMD Ryzen 9 7950X processor. This will be the basis of a new main desktop system for me, but I am going to play with it for a bit first. One thing I am going to try is to see if I can break 50GB/Sec with an X670E Motherboard!

This may not be possible with PCIe 4.0 NVMe SSDs, but I want to see how close I can get… What I mean here is 50GB/sec of simultaneous sequential reads as measured by CrystalDiskMark. There will be one copy of CDM running against each M.2 NVMe SSD in the system. This will let me determine whether the system can sustain seven PCIe 4.0 NVMe SSD running full bore.

ASUS ROG Crosshair X670E Extreme

This motherboard has two M.2 PCIe 5.0 slots and two PCIe 5.0 x16 expansion slots that are all connected to the CPU (not the X670 chipset). It also has a special Gen Z.2 slot (to the right of the memory slots) that lets you use two M.2 NVMe PCIe 5.0 SSDs in an included daughter card. This Gen Z.2 slot is also connected to the CPU.

Finally, there is an included PCIe 5.0 AIC that lets you run one M.2 NVMe PCIe 5.0 SSD that is connected to the CPU. This card is meant to go in the PCIEX16_2 slot.

On a normal system, you would put a discrete GPU in the primary PCIE slot (PCIEX16_1). But since AMD Ryzen 7000 series CPUs now have integrated graphics, you can use PCIEX16_1 for storage. I happen to have a dual M.2 PCIe 4.0 AIC that came with an MSI MEG X570S ACE MAX motherboard that I am going to try to use in that slot.

The diagram below shows the motherboard layout and what happens as you populate all of these slots that share bandwidth.

Detailed Slot Breakdown

Here is how I plan to use all of the available slots and PCIe lanes in the system:

• PCIEX16_1 slot
  • 8 PCIe 5.0 lanes
  • MSI PCIe 4.0 AIC with two 1TB SK hynix Platinum P41 PCIe 4.0 SSDs
• PCIEX16_2 slot
  • 4 PCIe 5.0 lanes
  • ASUS PCIe 5.0 AIC with one 1TB SK hynix Platinum P41 PCIe 4.0 SSD
• M.2_1 slot
  • 4 PCIe 5.0 lanes
  • One 2TB SK hynix Platinum P41 PCIe 4.0 SSD (boot drive)
• M.2_2 slot
  • 4 PCIe 5.0 lanes
  • One 1TB SK hynix Platinum P41 PCIe 4.0 SSD
• Gen Z.2 slot
  • 8 PCIe 5.0 lanes
  • ASUS GenZ.2 AIC with two 1TB SK hynix Platinum P41 PCIe 4.0 SSDs

This is a total of 28 PCIe 5.0 lanes with seven SK hynix Platinum P41 PCIe 4.0 SSDs. Each of these SK hynix Platinum P41 SSDs can do about 7,200 MB/sec of sequential read bandwidth. With seven of those all going at once, I should be able to break 49GB/sec (assuming I don’t hit another bottleneck).

50GB/Sec with an X670E Motherboard!

What bottleneck(s) might stop me? The most likely issue is how many total useable PCIe lanes I will be able to use. I will be using the iGPU for video, with everything else dedicated to storage. All of my storage will be connected to the processor, not either of the X670 chipsets.

There are supposed to be 24 PCIe 5.0 lanes available from the processor, but I need 28…

Four PCIe 4.0 lanes are used to connect the first X670 chipset, which daisy chains to the second X670 chipset. Having two X670 chipsets is what makes it an X670E system. All of the USB ports and NICs are connected to the chipsets.

It is possible that the PCIEX16_1 slot will not bifurcate down to two times x4 that would be needed to recognize both M.2 SSDs that will be installed. If that happens, I will only be able to use one of the M.2 drives on the MSI card.

Another question is whether my two 16GB sticks of G.Skill Trident Z5 Neo RGB DDR5-6000 CL30 RAM can absorb 50GB/sec of data being thrown at them. Finally, is the question of whether the Ryzen 9 7950X CPU can consume 50GB/sec of data being thrown at it.

Final Words

As I write this, I have not tried this with seven M.2 PCIe 4.0 SSDs. With only five M.2 PCIe 4.0 SSDs, I was able to hit 34.3GB/sec last night. I will try it with seven tonight. Let’s see if I can break 50GB/Sec with an X670E Motherboard!

I will report back with the results, good or bad.

Sept 30 Update: Going to six or seven M.2 SSDs (using the primary PCIe slot with a dual M.2 card) was a big failure last night. Here are the results of the six drive CrystalDiskMark test. This was only 11.36 GB/sec for sequential reads. Why did this happen?

The AMD Radeon integrated graphics use 16 PCIe 4.0 lanes from the CPU (even though they only have two GPU cores). I can’t find anywhere in the ASUS BIOS to change that (it is probably outside of the motherboard’s control).

With all of the M.2 slots and the second PCIe slot populated, the primary PCIe slot is limited to eight PCIe 5.0 lanes, which is fine. That lets me run five M.2 PCIe 4.0/5.0 x4 SSDs at full bandwidth.

One thing that might work would be to have a dual M.2 PCIe card that is able to combine the drives or bifurcate the primary PCIe slot, and then use a discrete GPU in the secondary slot. This would get me to six M.2 drives at full bandwidth.

If you have any questions about this post, please ask me here in the comments or on Twitter. I am pretty active on Twitter as GlennAlanBerry. Thanks for reading!