For most of the workloads I actually run at home, like Caddy, Affine, Immich, a Jellyfin frontend, and all the little Linux daemons that need to live somewhere, the default assumption is still that you spin up a VM. It's the safe answer, many tutorials suggest it, and on a Proxmox box it's the option sitting right at the top of the create menu. But almost every time I've started off with a VM, I've switched to a Linux container not too long after. The reason is simple: the math just doesn't favor the VM for the kind of stuff a homelabber runs.

First, let's sort some nomenclature. A "container" can refer to two different things, and they're easy to mix up. System containers like LXC, LXD, and Incus behave like tiny Linux servers. You SSH in, run systemd, install packages, and treat them like long-lived machines. Application containers like Docker usually run one service from an image and are designed to be rebuilt, replaced, and thrown away. They aren't the same workflow, but they refer to the same basic idea: isolated Linux user space running on the host's kernel.

For someone coming from a VM mindset, LXC is the easier on-ramp, because it feels like a tiny VM until you notice it boots incredibly quickly. Containers run at near-native performance, they idle on a fraction of the RAM, and on copy-on-write storage like ZFS or btrfs, they snapshot for almost nothing. There are workloads where a VM still wins, but not as many as you might think.

It comes down to a single architectural choice

One kernel does all the work

A VM virtualizes a machine, and while that sounds obvious on the surface, think about what a "machine" in this context actually is. Every guest boots its own kernel on top of virtualized CPU, virtualized NICs, virtualized block devices, and all the other pieces needed to convince an operating system that it owns the machine that it's running on. A container does not. It uses Linux kernel features like namespaces and cgroups to give a workload its own private view of the system while it still runs on the host's actual kernel. You're not emulating a chipset or guest kernel, and that goes a long way to reducing the typical VM overheads.

The benefit of containers is that they usually run close enough to bare metal that the difference is negligible for your typical homelab workload. KVM VMs are also very good these days, but they still need a lot more to exist as a VM. There's a decade-old IBM container-versus-KVM paper that still gets referenced occasionally, and for good reason. For Linux workloads on a Linux host, containers delivered equal or better performance than VMs in almost every test. The exact numbers vary by workload and hardware, but the actual result hasn't changed.

Personally, the idle cost is where I notice it most. An idle VM still has to run a full guest kernel, init system, and background services even when there's no ongoing workload. An idle LXC can sit comfortably under that because it's not booting a whole second OS. Instead of grouping five unrelated services into one VM because creating another VM feels wasteful, I can split them cleanly between multiple LXCs and barely think about it.

Startup times follow the same kind of pattern. A VM cold-booting through firmware, kernel startup, and services can take tens of seconds before it's usable, but an LXC is often up in a couple of seconds, and a Docker container can be running in less than that. Restarting an LXC for a single service feels closer to a systemctl restart than a reboot, and that changes a lot when it comes to tearing down, rebuilding, and testing new software.

Snapshots are the other big win here, as on ZFS or btrfs, a container snapshot is usually just a cheap copy-on-write snapshot of a subvolume. There's no giant virtual disk image to duplicate first, which makes taking a snapshot and restoring it if something goes wrong a lot more attractive of an option.

The mental model that finally clicked for me

It's a process with a fence, not a whole machine

The reason I came around on LXCs specifically is that these containers match most of the perceptions I have of a virtual machine. I've used VMs for years, largely because they felt permanent in a way a Docker container doesn't. LXCs keep the permanence, and you can give them a hostname, a static IP, persistent storage, and treat it similarly to a virtual machine. However, the advantage is that underneath, you're not paying for a second kernel or a virtual machine. It's just a cgroup, some namespaces, and a smaller slice of the host's resources than you'd have been asking for with a virtual machine.

For me, while they have the same feel as a VM, it's important to look at them more like Linux processes with a fence around them rather than a virtual machine. That's both from a security point of view, but also from a resources point of view. You don't need to over-provision RAM on services that you know are tiny, nor do you need to treat them as all-encompassing boxes that manage a bunch of services. You can start and shut down individual services as you please so long as they're split up correctly across containers.

From a security point of view, it changes how you approach your services. If you need your isolation boundary to be thicker than something that shares the host's kernel, a virtual machine is still the right answer. Container escapes are rare, and VM escapes can happen too, but there's a lot more work involved in escaping a virtual machine than there is in escaping an LXC.

Not every isolated process needs its own guest operating system, and unprivileged containers go a long way to bridging that gap and ensuring that a container only has the access it needs. Remember: it's basically just a fenced off Linux process, and the Linux permission model is genuinely pretty good.

Where a VM still earns its keep

There are many times a VM makes more sense

There are workloads where virtual machines still make sense, and I still make use of them. The first is fairly obvious: anything that isn't Linux. If a container shares the host kernel, you can't run Windows, BSD, or macOS, so that clearly needs a virtual machine. As well, some things need a higher level of privilege than you may be able to give to an LXC. For example, mounting an NFS share from inside of an LXC is nigh-on impossible. You can do it, but it's significantly easier from a virtual machine, as "root" in a virtual machine is truly root, compared to an unprivileged container where "root" actually carries a UID of 100000.

On that point, security isolation is the second big reason to use a virtual machine. As I've mentioned, containers share the host kernel, so the boundary breaking there means that every other service on that machine is potentially caught in the crossfire. A VM isn't invincible, but it puts a hypervisor and a hardware virtualization boundary between the guest and the host, which is a much stronger separation model. There's a middle ground found in microVMs like Firecracker and Kata, but they realistically solve a different problem than the ones a typical homelabber wants to solve.

GPU passthrough (and hardware passthrough in general) is also something made significantly easier in a virtual machine. I passthrough my AMD Radeon RX 7900 XTX to an LXC for llama.cpp, and it works, but it's nowhere near as simple as a virtual machine would have made it. Passthrough to a VM is the well-paved road, mature and well-documented, but it has the downside that the VM is the only one that can utilize it. With an LXC, giving hardware access to an unprivileged container requires understanding permissions and being aware of AppArmor and cgroups.

Aside from the obvious instances where a virtual machine makes sense, like running a different operating system, wanting true root access in an isolated system, or if you're running potentially unsafe software, almost every workload that I run lands in an LXC on my Proxmox server. The reason is simple: for Linux user-space workloads on a Linux host, the VM is often doing a lot of work that the service itself does not need, and it's the more expensive choice for the same outcome.

With that said, of course I'll keep a couple of VMs around for the workloads that genuinely need them. But my go-to hasn't been a VM for a long time, and LXCs get the job done perfectly.