Sidecars are Changing the Kubernetes Load-Testing Landscape

As your infrastructure is scaling and you start to get more traffic, it’s important to make sure everything works as expected. This is most commonly done through testing, with load testing being the optimal way of verifying the resilience of your services.

Traditionally, load testing has been accomplished via standalone clients, like GoReplay and JMeter. However, as the world of infrastructure has gotten more modern, and organizations are using tools like Kubernetes, it’s important to have a modern toolset as well.

With traditional load testing, you’ll commonly run into one of three major issues:

• Scripting load tests takes a lot of time
• Load tests typically run in large, complex, end-to-end environments, that are difficult to provision, as well as being expensive for production-scale infrastructure
• Data and realistic use cases are impossible to mirror one-to-one, unless you have production data

A more modern approach is to integrate your load-testing tools directly into your infrastructure. If you’re using Kubernetes, that can be accomplished via something like an Operator. This will give you the benefit of having less infrastructure to manage, and being able to use infrastructure-specific features like autoscaling Pods.

However, the biggest advantage you get from integrating your tool into Kubernetes directly is that you can use sidecars.

You Need to Focus on Traffic Capture

There are two main components of any proper load-testing setup: traffic capture and traffic replay. This is how most load-testing tools have worked for decades, and still holds true. However, there are many ways to capture traffic.

The next section will dive deeper into how sidecars specifically help with this. But first, it’s important to understand why there’s even an issue with traditional load testing.

First, how do you record the traffic? The most common approach is to record your own traffic. You open up a browser that’s configured to capture traffic, and you start emulating how your users would be using your service.

While this can certainly be helpful in many areas, it does mean you are relying on your own perception of how your application is being used. For a simple application, this may not be an issue but if your product is a Saas platform, there are likely many ways your application can be used.

Second, you’re missing out on important metadata. Again, perhaps not an issue for a simple application, but something that gets increasingly important as your application scales.

If you’re the one creating the traffic, you will only be getting the headers that your specific environment produces, meaning your load tests will only be testing one specific scenario. A better approach is to capture production traffic and use that in your replays, as you will be getting a range of different requests.

Third, you need to think about your dependencies. If you are load testing your entire infrastructure it likely won’t be an issue. But, if you want to do a load test on a single service, i.e. when testing is a part of your CI/CD pipeline, you want to make sure that any outgoing requests go to a mock service.

With traditional load-testing tools, you’re not going to get this. Either you’ll have to set up a new service to act as a mock, or you’ll have to change the code of your application.

Fourth, you are adding more infrastructure to manage. In a modern DevOps world, you don’t want load tests to be run from a developer’s laptop. It should be executed from a service running inside your infrastructure. So, with most tools you will have to add additional services that you need to manage.

Using Sidecars to Capture Traffic

Now that you know why traditional load testing doesn’t fit into a modern infrastructure, let’s touch on the previous points, exploring how sidecars are going to solve the problems.

In Kubernetes, it’s very easy to add a sidecar and make it act as a proxy. Already you’ve solved the issue of how to capture traffic. This sidecar will be capturing all inbound requests, and sending it to another system that can store the recorded requests.

With this approach it’s easy to implement shadowing, which is the practice of capturing production traffic and replaying it in your development environment.

Because you’re recording production data directly, you will also know for sure that you’ll be getting all the metadata from every request. You may be seeing issues with this already though. What about authentication, tokens, or sensitive information?

Well, sidecars won’t just be capturing traffic. It will take care of replaying it as well. And, because the sidecar is an application in itself, it’s easy to transform any metadata, like timestamps, before it sends it to your application.

As for sensitive data, the sidecar can easily filter that out from the metadata before it gets sent to storage.

At this point, you’ve already solved the issue of managing extra infrastructure, as sidecars can be added via a few simple lines to a manifest file. You’ve also solved the issue of missing metadata, as the proxy will capture everything, unless specifically configured to do otherwise.

Then there’s still the issue of mocking dependencies. The sidecar is still acting as a proxy, both for ingoing and outgoing requests. Because of this, you can tell the sidecar that you’re executing a load test, and it will recognize the outgoing requests. Instead of forwarding the request, it will simply respond with the data it got when the request was initially recorded.

If you want to see a specific example of how sidecars can be used to perform load tests, take a look at this example from Speedscale.

Sidecars Aren’t the Only Way to Capture Traffic

At this point, it should be clear how sidecars can be used to change the way you approach load testing. However, it’s also important to mention that sidecars are just one of many ways to improve traffic capture.

With how the industry is changing, it’s very likely that traffic capture will become commoditized, and the best tool will be the one that does the best job of replaying traffic.

Postman is used by many developers to test their APIs, and have created extensive collections with various different request definitions. While this does go somewhat against the points of not creating traffic manually, there is still value to be gained from being able to test specific use cases.

Service meshes are a popular way to manage communication between services, especially microservices. Most service meshes will enable you to have some kind of observability. Through this, you’ll gain a collection of production traffic, that you can then import into the load-testing tool of your choice.

Given the topic of this post, it’s important to note that some companies are working on removing sidecars from the implementation of service meshes. However, right now it’s just an option, and it’ll be exciting to see whether the future of services meshes are sidecar-less or not.

eBPF is becoming more and more used in modern infrastructure, like Kubernetes. Essentially, eBPF lets you execute applications at the kernel level, and, as such, it’s a very effective way of capturing traffic.

High-fidelity logs from a monitoring system is also a possible way to capture traffic. However, it’s important to note here that it has to be high-fidelity. To save costs and storage space, it’s very common to strip request logs of some headers that won’t be helpful in troubleshooting. If you are intending to use logs for traffic replay and reap the most benefits, you need to keep all the information.

API Gateways/Ingress controllers can possibly be a good way of capturing traffic, as it’s very common for gateways and controllers to log the traffic going through. If traffic capture truly becomes commoditized, it’s then simply a matter of importing the traffic into your load-testing tool.

Traffic Replay Is a Powerful Tool in Load Testing

Sidecars are going to change the way that load testing is done, and some companies are already embracing it. Whether it’s because you’ll be using them for capturing traffic, replaying it, or mocking the outgoing requests, there’s no doubt that you’ll be seeing an increase in the use of sidecars.

If you’re interested in testing what it’s like to load test Kubernetes using sidecars, go ahead and sign up for a free trial for Speedscale. Speedscale is a new pioneering product in Kubernetes load testing, focusing heavily on features like traffic replay, shadowing, API mocks, etc.