Solomon Hykes: Dagger Brings the Promise of Docker to CI/CD

For the team behind Dagger, the “portable dev kit for CI/CD pipelines” that was launched into public beta last month, the open source startup is a bit of a second act. The trio of co-founders — Solomon Hykes, Andrea Luzzardi, and Samuel Alba — all spent nearly a decade working together at Docker, the company Hykes founded before leaving in 2018. Now, the team has rejoined to build Dagger, which according to its introductory blog post “allows DevOps engineers to build powerful CI/CD pipelines quickly, then run them anywhere.” That summary immediately calls to mind the team’s prior endeavor.

Dagger, explained Hykes in an interview with The New Stack, solves the problems that DevOps engineers face from having to keep up with the growth of software teams, the scale of the software now being shipped, and the need to automate it all with increasingly complicated pipelines.

“We identified the main pain point, the main bottleneck, which is the glue,” he said. “DevOps engineers spend most of their time gluing together a bunch of specialized tools and the glue itself is the problem. It’s basically scripts that they throw together; they’re hard to test, they’re hard to adapt to new requirements, they’re hard to debug, and usually, you need to jump between tools and languages to kind of tie it all together. We let them get rid of that glue. Instead, they get to play Legos to get a standard set of components, the standard interface between them, and a unified model for how to combine them.”

With Dagger, DevOps engineers build actions using the language of their choice — Hykes offered Python, Go, TypeScript, Ruby, Java, and shell scripts as examples — and then compose them together declaratively using CUE definitions, which describe the inputs, outputs, sub-actions, and the wiring between them. Dagger’s use of CUE over traditional options like YAML or JSON delivers the ability to do more than simply describe data — it also provides logic capabilities. For example, CUE not only describes data types but can enforce those types; as well as constraints on the data, such as maximum or minimum values.

Part of the reason that Dagger evokes memories of Docker is that containerization is core to its methodology. Dagger works to make CI/CD pipelines run in both developer and CI environments, by containerizing the actions. Those actions are then pieced together using CUE to create the directed acyclic graph (DAG), explained Hykes. This separates the entire process into the imperative actions and the declarative definitions, which Hykes suggested is the “ideal pattern”.

“Every pipeline system that lives long enough and has enough users starts converging towards the same ideal pattern,” said Hykes. “We’re just starting there, instead of getting there later and trying to bolt it on.”

Had a great first day at @dagger_io! Really great team full of talented and passionate people working on exactly what I’m interested in, basically all you could ask for.

Looking forward to proselytizing the “Everything is a DAG” philosophy of the new DevOps operating system 😊🚀

— Erik Sipsma (@ESipsma) April 12, 2022

The combination of all these parts is a CI/CD pipeline platform that not only enables the sharing of individual actions but also the ability to run the pipeline wherever you can run containers.

When I suggested that Dagger sounded a bit like “Docker for CI/CD”, Hykes couldn’t help but agree, with the caveat of “a few major differences.” One place that Docker had fallen short, in his estimation, was with truly creating an ecosystem of sharing. With Docker Hub, users need to trust that the things being shared are safe since they are binaries. With Dagger’s soon-to-be-available hub called Dagger Universe, the pieces being shared are in the form of CUE files, so they are open to being inspected. Beyond that, Hykes also explained that Dagger is being built to be intentionally more open than Docker, with all of the project development communications taking place out in the open, such as on their Discord channel. Hykes also said that Dagger provided more of a platform than his former company.

“We give you a language, we give you an API, we give you a development kit, packages to import. It’s a real developer experience, and that changes the nature of the product because it means that that community, as it grows, they’re actually able to share some of the code they’re writing with and reuse each other’s code,” said Hykes. “The whole point of Dagger is that you can reuse someone else’s actions safely. You can split up your pipelines [into] actions and share those with the world if you want, and so it’s just like a regular code ecosystem, like Python or Go.”

For now, the Dagger team and community are looking to solidify the project, which is available on GitHub under the Apache 2.0 license, and work toward an eventual 1.0 release. Beyond that, the team is also looking at providing a cloud-based Dagger that would offer additional features, such as role-based access control (RBAC) or visibility.

“One obvious low-hanging fruit is visibility into your supply chain,” said Hykes. “You know, who’s deploying what, where, and why. Or, the other way around: what’s running in production right now, and where did it come from, and who touched it?”