Docker at 10 — 3 Things We Got Right, 3 Things We Got Wrong

Docker recently celebrated its 10-year anniversary. I am super proud of all we accomplished at Docker and the amazing things the team continues to achieve today. So much of what we see around us wouldn’t exist — microservices-based architectures, Kubernetes and so much more — if containers did not become the new unit of compute consumption. Yet, I’m sure when you look back at major transformational journeys in your life, you see both the things you got right and those you didn’t. This is certainly true for me. So, without further ado, let’s look at a few.

3 Things We Got Right

Containers Would Transform the World

When I joined Solomon Hykes to build DotCloud (later renamed Docker) in 2010, it quickly became clear that we would never be able to realize our vision if we just used the tools that existed at the time. DotCloud was the first Platform as a Service (PaaS) to support any language while Heroku and others were still limited to running a single language stack. One limitation we immediately faced while building DotCloud was a lack of alternatives to virtual machines (VMs) as the key infrastructure building block. While virtual machines were a huge step forward vis-a-vis bare-metal servers for the world of infrastructure, they failed to deliver the agility we needed to move into the cloud native era. We needed something lightweight enough that we could isolate each customer on their own namespace (compute, network, storage) while packing hundreds of developer apps on a single machine. It was the beginning of the microservices pattern. VMs were still the state of the art for infrastructure repeatability, and containers were still an obscure technology restricted to few tinkerers (remember when LXC required a kernel patch to attach to running containers?). Others thought the solution was to put the VM on a zero-carb diet (remember JeOS?). It was clear to us that despite all the challenges, it was worth the effort to build everything around containers. Eventually, we were proven right. A few years later, we extracted a core component of the DotCloud platform: the container runtime. We rewrote it and open sourced it. It was the first version of Docker. The initial goal was for Docker to be the first of many open components extracted from DotCloud. The container orchestrator, the networking layer, would come soon after. But given the immediate attention Docker had in its early days, the schedule changed quite a bit.

Developers, Developers, Developers

Steve Ballmer was right. Where VMware primarily focused on solving problems for IT, we realized early on that the way to change the world was to focus on the world’s software developers. You had to change the way software was built, not just operated, and that meant starting with developers’ needs first. As someone who has managed thousands of developers, I am keenly aware of the challenges software developers face every day. It can be one of the most incredible jobs in the world, full of challenging problems and the satisfaction of creating something wonderful, but it can also be tedious, frustrating and, at times, infuriating. Infrastructure and tools have progressed enormously, but the bar has also risen. Our North Star at Docker was to reduce distractions, overhead and allow developers to be productive and collaborate effectively. One of the first acquisitions (and successful product integrations) was a product called “Fig,” which later became Docker Compose, initially built by Ben Firshman, who is now the founder of Replicate, and Anand Prasad. Interestingly, the YAML model implemented by Fig (compose.yml) was directly inspired by the first DotCloud service composition (dotcloud.yml) that we built years prior. While we made great progress, more remains to be done here, in particular going beyond containers as the sole unit and orchestrating pipelines of containers. That’s one of the reasons why we started Dagger, a programmable CI/CD engine that runs your pipelines in containers, in 2018.

Investing in Building a Vibrant Community

We focused on building a great community first. It was our belief from Day One that we couldn’t achieve what we were looking to achieve alone. It required winning the hearts and minds of a huge set of people, and the key to achieving that was letting go of ownership of so many things. DockerCon became the gathering place for many of the best and brightest of our industry, a place where people came together who shared a common vision of how things could evolve and were willing to roll up their sleeves and pitch in to build it. In the early days of Docker when we considered building a developer conference ourselves, it initially sounded like an unreachable dream. Something that is either for big corporations or for much more mature developer community, such as PyCon. But when we organized the first DockerCon in San Francisco in June 2014 and were able to gather some talented developers in the same place, it became obvious that it was the beginning of something much bigger that would transform the whole company, and the industry. That legacy remains strong today in the dozens (or is it hundreds now?) of open source projects and communities we see in our industry. The Cloud Native Computing Foundation serves today as the host of many of these, and many more continue to pop up each day.

3 Things We Got Wrong

Adoption vs. Monetization 

The flip side of being “community first” is that it took too long for us to build a sustainable business. Our bias was to do everything in the open, listen intently to the needs of the community and try as hard as we could to deliver to them. The initial foundation of this strategy was that open source projects and commercial, proprietary solutions could co-habit nicely and be part of the same customer journey. I still believe in this model today, but it’s a tricky balance. First, you have to accept that some open source contributors and users will never be customers. And it’s OK, given that they participate in building a strong community, a strong brand, which then contributes to growing the commercial funnel. Second, the product architecture must allow building enterprise-grade features on a core open source foundation. This comes often with a complex support and release process. We definitely could have been a little more strategic in the way we created the path to building a solid business. Eventually, we got there, but it took too long and often felt scary.

Team Culture

We didn’t define the team culture and core values early on. It got defined later, either by the community or by the people who joined the company later on. This caused our team culture to drastically shift from the early days, without it being obvious at first. Our culture ended up reflecting the styles and values of those in the community rather than vice versa. A specific example of what we got wrong was in having two separate groups of people in our company — one focused on open source and the community, and the other on the business. This is one of my biggest regrets. It turned into a split brain in internal tooling, product and project management and, the most important, the team culture itself. It’s hard to balance those competing interests for anyone, but when you split the roles, you end up with internal battles, inconsistencies and open debates that can never be settled (everybody is right from their perspective). Many of the best and brightest wanted to work on the community side, and there was often a subtle (or not so subtle) judgment of the other side that came through too many of our collaborations. At times it felt like we had the “OSS religious people” pitted against the “enterprise money makers.” It just wasn’t productive. Having both a vibrant community and a sustainable business requires integrated teams where everyone struggles with the inherent tensions that our model naturally creates. It also creates a better team culture. No matter what part of the company you are working on, there is only one set of objectives to care about.

Containers as the Center of the Universe

As I took a step back, I realized we were overly attached to the container. We started to see containers as the core solution for most problems. It made us blind to all the other needs of the dev supply chain. Docker came to life because we saw containers would unlock a whole set of necessary changes within our industry, but as things progressed, we did not focus on the follow-on needs. By leaving so many needs unaddressed, we created lots of space for others to jump in and build out those areas. On one hand, that left great opportunity, but it also meant a fragmentation of the community. An example of one of the challenges we didn’t tackle at Docker was the overall automation of the software supply chain. We unlocked so much value toward the end of that supply chain but didn’t adequately address the needs of developers as they coded and collaborated, and today CI/CD remains a mess. But it is a solvable mess. Like many others from that era, as Solomon Hykes, Andrea Luzzardi and I reflected on our time at Docker, we realized our revolution remains incomplete, and so we found our mission for the next decade.

Join us at KubeCon + CloudNativeCon North America this Nov. 6-9 in Chicago for more on Kubernetes and the cloud native ecosystem.