Implementing a Secure Service Mesh

I’m currently working toward integrating all of our workloads under what is known as a service mesh.

A service mesh is a network layer that sits between every pod in all clusters. We can use the mesh and its associated tools to enroll a series of pods into discretely defined and secure network data planes.

For the purpose of this blog post, I’ll be talking about Kuma. Kuma is an open source solution built on top of Envoy that acts as a control plane for microservices and service mesh. It works with Kubernetes and virtual machines (VMs), and it can support multiple meshes in one cluster.

There are other open source and managed service mesh options out there, like Istio, Linkerd and Kong Mesh.

👁 Image

Why Use a Service Mesh

One of our main goals with using a service mesh was to get Mutual Transport Layer Security (mTLS) between internal pod services for security. However, using a service mesh provides many other benefits because it allows workloads to talk between multiple Kubernetes clusters or run 100% bare-metal apps connected to Kubernetes. It offers tracing, logging around connections between pods, and it can output connection endpoint health metrics to Prometheus.

This diagram shows what a workload might look like before implementing a service mesh. In the example on the left, teams are spending time building pipes instead of building products or services, common functionality is duplicated across services, there are inconsistent security and observability practices, and there are black-box implementations with no visibility.

On the right, after implementing a service mesh, the same team can focus on building products and services. They’re able to build efficient distributed architectures that are ready to scale, observability is consistent across multiple platforms, and it’s easier to enforce security and compliance best practices.

👁 Image

How the Kuma Service Mesh Architecture Works

The magic of taking an application pod’s socket communications from plain text over to mTLS lies within the Kuma control plane, the sidecar and Kuma Container Network Interface (CNI). When a developer merges some change, adding new services to an application, Kuma transparently detects and injects the required bits to proxy traffic automatically across its own network data plane.

Kuma service mesh has three major components:

• Kuma CNI: A CNI plugin that identifies user application pods with sidecars, based on annotations, to set up traffic redirection. It sets this up in the pod lifecycle’s network setup phase when every pod is scheduled in Kubernetes through a process called mutating webhooks.
• Kuma-sidecar: This runs on every instance exposing a service. The services delegate all the connectivity and observability concerns to an out-of-process runtime that will be on the execution path of every request. It will proxy all the outgoing connections and accept all the incoming ones. And, of course, it will execute traffic policies at runtime, like routing or logging. By using this approach, developers don’t have to worry about encrypted connectivity and can focus entirely on their services and applications. It’s called sidecar proxy because it’s another container running alongside a service process on the same pod. There is going to be one instance of a sidecar proxy for each running instance of services, and because all the incoming and outgoing requests, and their data, always go through the sidecar proxy, this is also called the Kuma data plane (DP), since it sits on the network data path.
• Kuma control plane (kuma-cp): This is a distributed executable written in GoLang that can run on Kubernetes, issue data plane certificates and coordinate data plane (DP) state within a Kubernetes API. You can use Kuma Custom Resource Definitions (CRDs) to configure Kuma settings and policy, and the sidecars pick up changes automatically from the control plane.

Conclusion

The service mesh topology of today is much like what the enterprise service bus (ESB) architecture of the 1990s and 2000s was going for. Instead of directing broker traffic along a route based on business policies as ESB architectures did, with the mesh, you now freely connect your application and the mesh manages the routes and policies from the top.

In my opinion, the biggest reason ESB architecture wasn’t more prevalent in the industry was because of the monolithic codebase requirements it had to make it possible and the ultimate dependency management problems you often encountered. You would have dozens of projects sharing dependencies for managing objects on the ESB, and it became a software management headache.

Service mesh technologies ease the pain by staying decoupled from your code. It allows developers to shift the complexity of security, reliability and observability away from their application stacks and keep it solely as part of the infrastructure equation.