When running multiple services, platforms, and operating systems, one wouldn't be blamed for immediately considering virtual machines (VMs). They're easy to set up and forget, but there are some problems that arise from using multiple VMs within a home lab setting. With the increased availability of Linux containers (LXCs), largely with Proxmox and the release of TrueNAS 26, there's almost a better way for running self-hosted services. Linux containers won't be the greatest choice for everything, but I moved most of my home server to LXCs, and I'm not looking back.

Whether it's Immich, Jellyfin, or some other service, there's a very good chance it's running within a Linux container and not a VM.

Why use a LXC over a VM?

And what on earth are the differences?

A container is used to describe an isolated Linux user space running on a host kernel. Docker is a fine example of this, running services from images, designed to be rebuilt, replaced, and discarded. LXCs are slightly different in that they don't run within software like Docker, but rather on the system itself. It's about as barebones as you can get for running each service on its own instance, sharing the same host. And it's because of this that LXCs are generally considered better than VMs for the home lab.

A Linux container runs at near-native performance, whereas a virtual machine will have the overhead of creating and running a resource-intensive instance. A VM actually virtualizes the entire system. The guest system boots its own kernel on top of virtualized hardware, assigned to the instance. If you configure Home Assistant to run as a VM, it's similar to creating a physical system to do the same thing. An LXC uses the underlying kernel to provide a private view of the system while running almost natively.

Because we're not emulating the chipset, guest kernel, and everything else required to make the instance work as a whole, we're able to reduce the overhead, resulting in better performance and more resources for running other services. KVM virtual machines are extremely good these days, but they still require a considerable amount of system resources to exist. Linux containers on a Linux host will almost always best a VM in performance, but again, it's not all about numbers, but rather how much can be run on the system.

When it finally made sense for me

I needed more on the same host

For me, it really clicked when I noticed that some of my running VMs would still be taking up a fair chunk of the host system, be it RAM or CPU cycles. Within the VMs are init systems, background services, kernels, and more, all running and competing for resources. An LXC can sit almost idle, drawing almost nothing from the host. Because I'm not booting an entirely new OS within the virtualized platform, it no longer feels wasteful when creating and running multiple service instances.

One VM could be (and has been) successfully split into multiple LXCs without sacrificing overall performance. Then there's the speed in which these systems actually fire up from shutdown. A VM, even a lightweight one, can take several seconds to boot. I'm talking almost half a minute at times, and depending on your underlying hardware. An LXC usually starts within a few seconds. A Docker container within an LXC will be even faster, usually starting almost immediately.

Rebooting a VM can feel like restarting an entire system, hardware and all. That's because it sort of is, but an LXC can be rebooted like you would re-open an app on your desktop PC or phone. It feels instant because it's more like a restart than a reboot. Finally, backups are much easier with an LXC because snapshots can be taken of the container and nothing else. A VM needs to have the entire virtual disk image to save and then restore, making experimentation require more time.

LXCs largely replaced VMs in my home lab

But I still rely on both for everything to run

Having used both for years, I finally started to clean up shop and migrate as much as I could from virtual instances to containers. LXCs can be configured much like a separate system can, but without all the additional cost of running a second kernel and other bits and pieces to make virtual machines function. It's simply a walled-in garden on the same system as the host, which makes it both fascinating from a technological view and impressive for being lighter on load.

Having more resources to work with enabled my home lab to expand further without needing to waste money on buying more used gear as additional nodes within the Proxmox cluster. Before, I would provide LXCs with what I believed they'd require at most. VMs would have an arbitrary number entered into the respective field, sometimes offering multiples of what they'd actually use. It was refreshing to move to an approach that doesn't require over-provisioning resources to ensure everything will run well enough.

But that's not to say I don't use VMs or find them useful, as that couldn't be further from the truth. Virtual machines are fantastic for ensuring that there's absolutely no chance anything can escape from within that instance. Containers are secure enough, but there are rare occasions when this has occurred. It's notably more difficult for this to happen with VMs. Some services make more sense on VMs, too, such as a full-blown OS like Windows, macOS, or some random Linux distro.

VMs are still useful for the home lab

Some things I run will also require higher levels of privilege than can be offered through an LXC. NFS shares from within an LXC are almost an old wives' tale at this point. It's much easier within a VM, but so is GPU passthrough, as well as general hardware passthrough. It is possible with LXCs, but it can be made significantly easier with a VM. I've passed GPUs through for Jellyfin, OpenWeb UI, and other services and found VMs to be much easier, which can be useful for beginners.

I, like many others, find it incredibly useful to use both. But be sure to try out containers, if you haven't yet done so. They offer a new way to expand the home lab.