Krustlet Brings WebAssembly to Kubernetes with a Rust-Based Kubelet

In the world of Kubernetes, most programs extending Kubernetes are written in Go, and most applications orchestrated by Kubernetes are packaged as containers. These two things have become so common as to have become default, but neither is expressly necessary, as proven by an up and coming project out of Microsoft’s Deis Labs called Krustlet, which is currently in the process of joining the Cloud Native Computing Foundation (CNCF) at the Sandbox level.

Krustlet upends both of these Kubernetes customs, first in that it is written in Rust, the notably memory-safe language originally developed by Mozilla, and second in that its purpose is to enable Kubernetes to schedule WebAssembly modules in lieu of Open Container Initiative (OCI) containers. While the Krustlet team was careful to note during its launch a little over a year ago that WebAssembly and containers can act as “a complementary pair,” each with “their own set of unique advantages and disadvantages,” it is precisely those advantages they wanted to bring to Kubernetes.

“WebAssembly can do many of the same tasks that a container does, but it has a different operational profile. It can be run with far fewer privileges, it is smaller, and it is cross-architecture/cross-OS,” explained Matt Butcher, a principal software engineer with Azure Core in an email. “Being familiar with Kubernetes, we wanted to see if we could build a tool that executes Wasm inside of Kubernetes to illustrate the similarities and highlight the differences between Wasm and containers. That turned out to be a watershed moment. WebAssembly is showing even more promise than we originally imagined.”

According to Ralph Squillace, a principal program manager for Azure Core, there was another reason to bring WebAssembly to Kubernetes, as well.

“The driving power of the Kubernetes project was not so much in the container runtimes — even though they kickstarted the revolution — but in the control plane, the API that unifies the skillset and the tooling,” wrote Squillace in an email. “We wanted to see what could be done using a different sandboxed process approach (WASM) with that operational model — where could that go? Turns out, it goes well.”

The “Krustlet” name itself gives some indication as to what is going on here: Krustlet is essentially a kubelet written in Rust for the purpose of running WebAssembly. When the Kubernetes API receives a request, pointing to a container image, to create a new pod, kubelet normally responds and creates that pod to run the container image. In this case, when the Kubernetes API gets a request to create a new pod, but that request points to a WebAssembly module stored in an OCI registry, Krustlet responds instead. All of this happens through a Kubernetes system called “taints and tolerations,” and Butcher said that this was “what’s so fun about Krustlet: We could use Kubernetes unaltered — no patches, no special tweaks to the control plane — and Krustlet is just an alternative runtime that can be dropped into any Kubernetes cluster, instantly offering Wasm support.”

And while Squillace points out that containers are “far more useful for code reuse than WASM” — although he says the WebAssembly System Interface (WASI) may change that — WebAssembly, again, has some distinct advantages, especially when it comes to microservices and running in environments that benefit from small, fast binaries.

“We don’t see WebAssembly being used to repackage existing applications. But when it comes to the microservice architecture, we think WebAssembly might just be the sweet spot,” writes Butcher. “Startup speed, memory footprint, security, and even ease-of-development are all highly meaningful metrics when it comes to microservices. And these happen to be the strengths of WebAssembly. Our vision is to make it simple and enjoyable to build microservices in WebAssembly, and then deploy them using Kubernetes.”

By enabling WebAssembly, Krustlet offers increased density, faster startup and shutdown times, smaller network and storage footprints, and all of these are features that not only support microservices but also operation on the edge and in IoT environments. In addition, WebAssembly also offers the ability to run on multiple architectures without being recompiled, has a security model that distrusts the guest by default, and can be executed by an interpreter and streamed in, meaning it can be run on the smallest of devices.

“Krustlet, potentially combined with things like SUSE/Rancher’s k3s, can make inroads into IoT by providing a small-footprint extension to a Kubernetes cluster. This points to a sea change occurring in Kubernetes. When some folks at Google first wrote Kubernetes, they were thinking about clusters in the data center. But why think only in terms of the data center?” asks Butcher. “Imagine a world where the pod could be dynamically moved as close to the user as possible — down to a thermostat or a gaming console or a home router. And then, as the person left home, that app could ‘leave’ with them, hopping to the next closest place — yet still within the same cluster. Certainly, that’s tomorrow’s world, but Krustlet is a step toward realizing it.”

This next world of Kubernetes is certainly part of the reason the project is working to join the CNCF, explained Squillace, who also noted that the team behind the project is the same as that behind Helm, Brigade, and Porter — all CNCF projects.

“We really want to see Kubernetes succeed over the next decades. Krustlet expands what Kubernetes can do: Krustlet applications can be packaged by Helm and Porter, for example. These things indicate to us that Krustlet belongs in CNCF, where it can be one more useful piece in an expanding portfolio of cloud native tools,” said Squillace.

Currently, the Krustlet repository still displays a prominent banner declaring it to be “highly experimental,” but the project is expected to reach v1.0 in the coming months. Moving forward, Squillace said that the project “will focus on networking and adding more support for wasm runtimes, focusing on collaboration with approaches like Akri (for devices) and also smaller distributions like SUSE’s k3s.”

“Assuming that Krustlet finds its place in the Kubernetes ecosystem, it’s our hope that it becomes ‘boring’, that it is used, and it works,” Squillace said in the email. “It did, however, really solidify what could be done with WASM and in particular, the coming WASI specification work and module linking.”

Moreover, Squillace noted that there is a direct path from Krustlet, with its multi-runtime provider model, to the Bytecode Alliance — which included founding members Fastly, Intel, Mozilla and Red Hat — to support further work on WASM and the other features of WebAssembly components like linking and so on. Microsoft is also a member.

“Some of the leading lights of the cloud and WebAssembly world, like Google, Arm, and Shopify, have already joined the original founding members and we are excited for what the community can do together over the next year,” wrote Squillace.