Why Docker, Containers and systemd Drive a Wedge Through the Concept of Linux Distributions

The announcement of Rocket by CoreOS was perceived by many to be a direct challenge to Docker, particularly as it came on the eve of DockerCon Europe and threatened to overshadow news coming out at the event. Docker, Inc. CEO Ben Golub was quick to fire back with his ‘initial thoughts on the Rocket announcement’. This piece isn’t about the politics of ecosystems and VC funded startups, which I’ll leave to Colin Humphreys (and note an excellent response from Docker Founder and CTO Solomon Hykes). It also isn’t about managing open source community, which I’ll leave to Matt Asay. Here I want to look at systemd, which lies at the heart of the technical arguments.

There’s been an unholy war raging through the Linux world over systemd for some time. Pretty much everything on a system gets touched by what is selected as the first process on a system and how that impacts everything getting started up. People care a lot about this stuff, and the arguments have been passionate. Nevertheless, Mark Shuttleworth conceding defeat on behalf of Ubuntu marked the last major distribution going all in on systemd. Unless forks like Devuan become successful it’s going to be pretty hard to get Linux in a couple of years time without getting systemd as part of it.

Since CoreOS is a shiny new distribution, they went for systemd from the beginning. It was an obvious move, and frankly anything else would seem pretty ridiculous. Docker however, is a child of a different generation. Docker couldn’t align itself with systemd and then wait around for it to become popular. Docker had to work with what was already out there, which right now is mostly not systemd based installations. That’s why Docker gets installed as a daemon, which then sits uncomfortably within the process management reach of CoreOS and other systemd based distributions. Darren Shepherd does a great job of explaining the gory details in his post ‘Is Docker Fundamentally Flawed?’, where he concludes that there really is no fundamental flaw:

“It’s not that Rocket doesn’t have a stand alone daemon it is just that that daemon happens to be systemd, PID 1, which runs as root. So is Docker fundamentally flawed? I can’t imagine how you could say that because Rocket follow a very similar paradigm, it just happens to be built into systemd. The fundamental issue is that both Docker and systemd want to be the daemon to manage containers.”

The irony here is that the move to containers in general, and Docker in particular, is driving a wedge through the very concept of Linux distributions. CoreOS, RedHat’s Project Atomic, and Ubuntu’s ‘Snappy’ Core are all positioned as distributions to run Docker on. Meanwhile, the field is wide open for new distributions to run in Docker (and for the time being Docker has been hugely successful by letting developers stick with what they know and carry on using their distro of choice). What’s very clear is that systemd isn’t needed at all inside a container; an init system isn’t required for a single process (or even a handful of processes to support a given service). It also seems like systemd is both overkill for starting Docker, and a mismatch of process control approaches. So perhaps the systemd refuseniks were right — it doesn’t seem to have any proper place in a world of containerized services.

The choice of the Docker approach or the Rocket approach may come down to user experience, or the lack of it. Docker has been an instant hit with developers because the user experience is simple and familiar. Shepherd makes the case that this is where Rocket is at a disadvantage in What’s Missing From Rocket? The User Experience. But perhaps that’s the whole point — operations isn’t supposed to have a user experience; it just becomes automated stuff going on behind the scenes. The dev and ops dichotomy is a false one, brought about by over simplistic organizational structure, and addressed by the DevOps movement. Docker versus Rocket might also be a false choice, as it’s entirely possible that Docker’s developer friendly CLI, API and Dockerfile will be brought together with a portable container specification that helps with operations at scale. Whether systemd has any place in that picture remains to be seen.