Building an Unreasonable Home Server

May 2024

A home server is way more than a server at home. It is a place to experiment, a low-stakes playground for ideas and a great way to make the life of you and your roommates better and worse at the same time. Since my budget envelope for a new home server has been spilling over for some time, I set out to replace current one.

Previous setup

I’ve had an Intel Celeron J4105-based server for the last five or six years. Its motherboard provides four SATA ports which were hooked up to a few older spinning rust hard disks. Since energy is expensive here in Germany (currently around €0.30 per kWh), those were configured to spin down after 30 minutes of idle. This also reduced the noise level of the server to practically zero when it was idling. However, spinning the disks up takes 10–15 seconds and makes the first use of basically any service feel sluggish. Additionally, although the motherboard consumes almost no power (around 5 Watts), it also feels pretty slow when you’re used to, for example, a modern laptop. It runs FreeNAS, now TrueNAS, which has excellent ZFS support and also supports plugins, similar to other commercial NAS providers.

Hardware

I’ve always had the idea of using SSDs for a tiny combined NAS/home server build but until a few years ago the prices were just too high. This changed in recent years and with okay-ish prices, it became a viable option if you’re not a data hoarder. So this year I set out to build a more powerful, yet kind of efficient server using SSDs. I noticed that 2.5" SSDs are priced almost the same as NVME SSDs and had an idea: what if I use one of those 4x PCIe cards for the storage and put a smaller one directly on the motherboard for the OS?

Finding a motherboard that can hold one m.2 SSD and supports 4x bifurcation is not hard, but there is one option that stood out to me: the Minisforum BD770i is small (Mini-ITX), features an AMD Ryzen 7 7745HX with 8 cores/16 threads, supports at least 32 GB DDR5 RAM and PCIe 5.0. Although it is not documented well, after updating the BIOS, it supports 4x bifurcation on the PCIe slot. What this means is that you can have a PCIe card in the (single) slot that holds up to 4 NVMe SSDs and each one of these gets its own PCIe 5 lane, potentially supplying around 14 GB/sec of read throughput. At the same time they consume less than 1 Watt at idle. The Ryzen 7745HX is actually a high-end mobile APU, which made me confident that the energy usage wouldn’t be so bad, at least at idle.

Since the server now basically consists of this small motherboard and one fanless PCIe card, I wanted to cram it into a small form factor (SFF) case. There are lots of them, but many either don’t support PCIe cards (which makes sense for SFF cases) or reserve a few liters for a beefy GPU. One of the few cases that could at least fit the card without the bracket is the “Goodisory A09”, a 3.8L case. It is not fancy, but its tiny form factor sparks some joy in me. Normally this case supports a TFX power supply, but why take the established thing when you can get a new shiny one?

GaN power supplies are becoming more popular for USB-C, since they allow very small dimensions while being very energy efficient. HDPlex is offering one of the first, if not the first, GaN PSU for SFF cases. It is not exactly cheap, but it is so efficient that no cooler is needed and tiny. Also, the €50 extra cost are amortized in only 36000 hours or 4 years!

Fitting everything in this case, especially the 4x PCIe card, was a bit of work. In the end it all fits well. There is even a few centimeters space between the card and the PSU, since it is so small. I’m not looking forward to upgrading the RAM or the SSDs in the future, so I hope they’ll last for some time.

The setup works as expected, although the Minisforum BIOS is subpar. Configuring the bifurcation to x4x4x4x4 on the PCIe slot was possible only after booting some weird Windows live CD to update the BIOS to the most recent version. Once this is configured, all SSDs are recognized. One major drawback of this approach is that the board does not seem to save the settings on power loss. Normally that is not a big deal, but when all your data is relying on this bifurcation setting, you’ll think twice before disconnecting the power to check on something. Effectively, I now have to boot into the BIOS and reset this setting every time the power gets cut. This might be an effect of the Ryzen being a mobile processor/platform, but it is still annoying. Everything else works as intended.

• Minisforum BD770i (€590)
• 2x RAM 16GB DDR5 4800MHz (€100)
• 4x 4 TB NVME SSD with PCIe 4.0(!) (€1120)
• 1 TB NVME SSD with PCIe 4.0 (€80)
• 250 Watt HDPlex GaN power supply (€150)

This makes a total of roughly €1000, right? Right.

Software

For the operating system I went with NixOS, since it runs all my other servers and laptops. There is a lot to tell, love and hate about NixOS, but that has been done countless times on the internet before. It basically uses a functional programming language that – simply put – only has the side effect of emitting build instructions for software. If this tickles your inner monad, I recommend taking a look.

The SSDs are in a RAID-Z configuration, meaning that one of the four SSDs could fail without any data loss (I hope this never happens, or I’ll have to weld them out of the case). With 4 SSDs with 4 TB each, this gives 12 TB net capacity for the server. One additional 1 TB SSD is mounted directly on the board to hold the OS. I’m not afraid about the SSDs reaching the end of their lifetime more than was worried about the HDDs mechanic breaking down after a reboot.

The rest of the setup was a breeze, thanks to Nixpkgs and the sheer endless existing services I can choose from. If something is really finicky, I’ll just break out of the Nixpkgs walled garden and provide a docker-compose.yml, plug Watchtower in there and be done with it. The machine provides a bunch of services:

• Paperless-ngx organizes documents: scan all documents to a share on the server and paperless will nicely organize them. It also pulls invoices from my mails.
• PhotoPrism or Immich backup and organize photos, have everything you need including local face detection. The projects are pretty similar, so I just run both and pick a favorite some time later.
• Blocky is what Pi-Hole could’ve been if it was simple: a DNS server that supports block lists. It works very well without taking away one of those SSDs for a node_modules directory.
• Home assistant is the all-in-one solution for automating your home. It interfaces with basically everything and, given enough time and energy, you can achieve almost any automation with it. The web frontend doesn’t work super well with Nix, so this is something that could just be hosted in the dockers.
• Restic is a robust tool to perform backups over the network. This runs as a server to receive the data from clients over the network but also uploads backups to a remote server for offsite backup. It can both just “forward” the received backups as well as perform backups of the data stored on the server. Highly recommended.
• Sanoid does automatic snapshotting of the ZFS volume.
• Syncthing syncs files seamlessly with all configured clients. I’ve used it for years and even contributed to the project some time ago – it just works.
• Headscale is an open implementation of the Tailscale control server and provides a VPN across all my devices. I really like how smoothly Tailscale (and Headscale) works.
• Jellyfin streams media from the server to various clients.
• distcc speeds up compilations for others in the network.

There are a few more niche things on the server and there’s a lot more potential. You can see the configuration in the repository.

Conclusion

This build works really well and will hopefully satisfy my home server needs for the next 6–8 years. Yes, it is unreasonable, since similar performance can be purchased for less money, and you could potentially get more server-like features (e.g. IPMI). But it feels so good that everything is contained in this tiny, silent box, works in an instant, and that I can sleep at night knowing there’s no more spinning rust in the house.

The power usage with a BeQuiet! 300 Watt power supply is around 20 Watts idle, while with the GaN PSU it is now down to 13 Watts. This amounts to around €40 per year or €3.50 per month in energy cost.