My home labbing journey started a couple of years ago with my introduction to TrueNAS, a simple installation on my rather meek old PC, packing a Ryzen 7 3700X, an Nvidia GTX 1070 Ti, and 24GB of RAM. Despite the underwhelming specs, it got the job done as a gateway to the world of home labbing, and could host a litany of software like my reverse proxy, Home Assistant, and Nextcloud. When I eventually moved to a new, Proxmox-based server, I moved TrueNAS over too... to a virtual instance.

I know what some of you who are familiar with the process are thinking: "What about your storage?" Others may even question what the point of TrueNAS is when Proxmox has native ZFS support, making it possible to set up an SMB or NFS share on the host. As it turns out, there are some pretty neat benefits to virtualizing TrueNAS this way.

Why I first virtualized TrueNAS

And how

I built a new, much more powerful server just over a month ago, with the intent to migrate most of my self-hosted services to it. While my TrueNAS server was still chugging along, I already had two other nodes running Proxmox: one hosts OPNsense, and the other was where I deployed my new services and slowly migrated my older ones from TrueNAS. With that said, not everything had been migrated, and I didn't want my old TrueNAS server to be drawing power separately from my new primary server, especially when my new server draws less power at idle and has significantly more powerful components.

As a result, I initially planned to virtualize TrueNAS as a stopgap solution. I was still hosting important services on it, so I couldn't just permanently shut it down at that point. So, I exported my TrueNAS storage pools, backed up my TrueNAS configuration, and powered down the server. The configuration file is a simple .tar file (or a .db file if you don't export your secret key), which you can upload to a newly installed TrueNAS instance. This then restores all of your old settings, including storage settings and network shares.

Next up, I moved my hard drives to the new Proxmox machine and installed TrueNAS as a VM. I went through the normal setup procedure before shutting it down. After that, I passed through my SATA controller to the VM. This way, TrueNAS gets full control of the SATA hardware on my motherboard, which works fine in my case, as I had no other SATA devices connected. To my virtual TrueNAS instance, it looks like a normal set of drives with a normal SATA controller, and it can control the drives fully as if it weren't virtualized at all.

Finally, I uploaded my configuration file, restored my data, and imported the storage pools I had previously exported. Everything worked: it looked the same, felt the same, and all of my self-hosted services started up and kept chugging along as if nothing had changed. TrueNAS can still run SMART tests on the disks, and ZFS can properly self-heal and correct data on the drive if it needs to.

Why I'm still virtualizing TrueNAS

It feels like the best of both worlds

Given that I did that over a month ago, it seems like a bit of an elongated stopgap solution, right? Well, that stopgap has now become a permanent fixture of my Proxmox server. For example, I have an Immich instance in TrueNAS that I still back up my photos to, and as time went on, I realized I liked having the separation between Proxmox and my storage, and virtualizing TrueNAS felt like I was getting the best of both worlds.

See, no matter what tinkering I do with Proxmox, because of the SATA controller passthrough, Proxmox can't even see the drives if it wanted to. That's thanks to Virtual Function I/O, which allows the passing through of physical hardware to a user-space process so that it can be controlled. It requires you to enable IOMMU, but essentially, it wraps a layer around the hardware, dictating that it can only be controlled by the process granted access, which, in this case, is the TrueNAS QEMU virtual machine.

This layer protects my storage from tinkering, no matter what I do. If you look at the driver that's loaded to deal with hardware that's been passed through to a VM, the driver listed will be vfio-pci. If I run "lspci -nnk", this is what the output looks like for my SATA controller:

00:17.0 SATA controller [0106]: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode] [8086:7ae2] (rev 11)
 DeviceName: Onboard - SATA
 Subsystem: ASUSTeK Computer Inc. Device [1043:8694]
 Kernel driver in use: vfio-pci
 Kernel modules: ahci

While I obviously keep off-site backups, that wouldn't make it any less frustrating to restore the data if I were to experience catastrophic data loss as a result of messing about. TrueNAS is also built for storage, and Proxmox isn't. While you could certainly install software on the host to mimic TrueNAS in the same way (and even have more control overall), a generally good rule to follow is to avoid installing software on the host if you can help it.

Of course, there are several exceptions to that, and given it's your instance, you can install what you want if you'd like. I've definitely installed some software on the host that I'm sure those who truly subscribe to that rough guideline would balk at. But when there's a clear, easy path to do it, and the benefits are numerous, why wouldn't I just continue virtualizing TrueNAS? I don't need to worry about the drives as I've passed them through correctly in accordance with iXsystems' guidance, and I get the benefits of complete storage separation from the host while also not needing to install additional software on the host. It's a win-win.

It doesn't matter, so long as it works

Guidelines are just that

Truth be told, no matter what way you want to share files on your network, both methods are valid. You may encounter issues when upgrading the Proxmox host between major versions, but even then, it might all work fine. I personally like the separation, but that doesn't mean most people will particularly care about it, either.

If it works, who really cares? The conventional "right" way may not be the right way for you, but I wrote this as I frequently see people questioning others who have virtualized TrueNAS as if it were a big mistake to do that. So long as your drives are passed through correctly, it's completely fine, and it has quite a few benefits depending on your workflows.

And for those who want to access their storage on their Proxmox host, you can still do that. I've mounted my storage on my Proxmox host using the SMB share in TrueNAS, and I've passed that mount point through to an LXC without any issues. It admittedly sounds pretty convoluted, but it works perfectly and was pretty intuitive to get up and running. Migrating to a virtualized instance took me less than half an hour, and I'm really glad that I did it.