Beyond Kube-Proxy: Project Calico Harnesses eBPF for a Faster Data Plane

Thanks to the power of the newly-introduced eBPF in the Linux kernel, Tigera has been able to outfit its Kubernetes-focused Calico network management software with a new data plane mode, one that can speed pod-to-pod data communication and eliminate the dependency on Kubernetes’ kube-proxy for traffic management.

Tigera had started releasing work with eBPF almost a year ago, but this is the first release of Calico that fully harnesses the power of the new Linux kernel technology, Alex Pollitt, Tigera co-founder and chief technology officer, said.

“We wanted to derive what we were doing from fundamentals, to be confident we were building the right thing for users,” said Pollitt. “We looked at every networking building block in the Linux kernel, understood how they worked, how they interacted.”

The new data plane will be offered as an option alongside the traditional Calico data plane. Those organizations using a new version of Linux kernel (preferably 4.8 or newer) will be able to take advantage of the new solution, Pollitt said. In particular, organizations, where performance is a top priority, should test this release, he said. The new eBPF data-plane mode, available as a tech review, can both scale to a higher throughput while using less CPU per GBit.

Currently, over 150,000 Kubernetes clusters run Calico, including not only bare-metal implementations but those running across Amazon Web Services, Azure and Google Cloud. Originally created for OpenStack, Calico was designed to make it easy to get data packets from one part of the network to another, using internet technologies like IP routing, rather than switching, virtual networks, overlay networks or other complex approaches.

Anticipating containers, Calico was designed for very dynamic environments and can manage hundreds of thousands of end-points that can change the location at any time. Calico meshes very well with Google’s Zero Trust Security model, which assumes networks and hosts will be breached, and so limits the amount of damage that can be done.

Enter eBPF

The eBPF (extended Berkeley Packet Filter), a recent introduction to the Linux kernel, is a virtual machine inside the kernel that provides advanced, high-speed extensible network packet filtering. Work that used to be done by an external module can be executed, much more quickly, by the kernel itself.

For Calico, eBPF provided the tools to hasten data traffic while reducing the networking complexity of high-end Kubernetes deployments. Use of this data plane eliminates the need for kube-proxy, Kubernetes’ built-in network proxy that handles load balancing through iptables. Although fine for everyday use, kube-proxy doesn’t scale well as the number of services that are mapped in iptables grow. It requires IPVS (IP Virtual Server) to effectively scale for thousands of services.

👁 Image
The Calico data plane offers a similar scalable performance as IPVS, by way of a more efficient lookup map, but without the need of kube-proxy at all. It reduces first packet latency to servers, which would lead to a noticeable improvement in environments with lots of provisioned services are constantly spun up. The in-kernel functionality reduces the CPU overhead. The data plane preserves external client source IP addresses all the way to the pod (great for troubleshooting), and also supports DSR (Direct Server Return), in which the return traffic doesn’t need to loop back through the original ingress.

The newly released Calico version 3.13 also includes an auto-detection capability to detect the IP address range used for pods when running on kubeadm. Calico v3.13 will be available in Calico Enterprise v2.8.

The folks at Calico will explain this technology in more detail today, March 17, on a Cloud Native Computing Foundation webinar at 10:30 a.m. PDT.

The Cloud Native Computing Foundation is a sponsor of  The New Stack.