NSA on How to Harden Kubernetes

The NSA, yes, the National Security Agency, has two jobs. One you know about from my spy movies and the like is to eavesdrop on communications outside the United States. The other half of their job, though, is less well known. Their other job is to protect communications from other would-be snoopers. So, it is that the NSA made the original secure Linux (SELinux); has written guidelines on how to secure video conferencing, text chatting, and collaboration tools; and now explains how to harden Kubernetes against attackers.

This isn’t the first time the NSA has helped us to secure Kubernetes. Their new Kubernetes Hardening Guide has been updated and is more useful today. For instance, the NCC Group saw that the first version’s information about Kubernetes authentication was “largely incorrect” because it claimed Kubernetes doesn’t provide an authentication method by default.” NCC Group noted, though, that Kubernetes supports both token and certification authentication natively.

These improvements are important. We need all the help we can get to secure Kubernetes. According to the Cloud Native Computing Foundation (CNCF)’s 2021 Cloud Native Survey, 96% of organizations now use or evaluate Kubernetes. Indeed, 5.6 million developers are already using Kubernetes worldwide. That’s a resounding 31% of all backend developers.

Securing Kubernetes Properly

Now of that huge number how many do you think are securing Kubernetes properly? My guess, based on talking and watching Kubernetes developers at work, is far, far too few. As Red Hat recently pointed out, human error is a leading cause of Kubernetes security mishaps. Indeed, 94% of those surveyed admitted they have experienced a Kubernetes and container environments security incident in the last 12 months.

This is a real problem. Hackers know as well as we do that we’re now living in a world moving to containers and Kubernetes as quickly as possible for all our IT work. That means, as the NSA points out, Kubernetes clusters are a prime target for data theft, computational power theft, and denial of service attacks. And let’s not forget, in times of cyberwar, Kubernetes is a promising target.

Currently, data theft is the number one target. But increasingly cyber actors are trying to hijack Kubernetes clusters for cryptocurrency mining. In short, there are many people out there after your Kubernetes installations and it behooves you to defend as best as you can.

Specifically, the NSA recommends:

• Scan containers and Pods for vulnerabilities or misconfigurations.
• Run containers and Pods with the least privileges possible.
• Use network separation to control the amount of damage a compromise can cause.
• Use firewalls to limit unneeded network connectivity and use encryption to protect confidentiality.
• Use strong authentication and authorization to limit user and administrator access as well as to limit the attack surface.
• Capture and monitor audit logs so that administrators can be alerted to potential malicious activity.
• Periodically review all Kubernetes settings and use vulnerability scans to ensure risks are appropriately accounted for and security patches are applied.

That’s all good, but it’s also all rather generic. I hope no matter whether you’re running a single simple Linux, Apache, MySQL, PHP/Perl/Python (LAMP) server or a multi-thousand node, complex Kubernetes cluster, you’re already doing all that.

Nothing Simple about Kubernetes

Of course, patching in Kubernetes environments is hard. Besides Kubernetes itself, numerous other programs run with it and within it to do real work. There’s nothing simple about running Kubernetes so it only makes sense sadly that it’s also hard to secure.

For example, we all know we shouldn’t run applications as root, but by default, many Kubernetes container services run as the root user, and applications execute within them as root even though they don’t need privileged execution. Nonetheless, all too often, the NSA warns us, developers build container applications that execute as root. Why? Because it’s so easy. But it’s also so dangerous.

And, of course, Kubernetes has its fair share of its own security problems. For instance, the Cybersecurity and Infrastructure Security Agency (CISA), NSA’s partner in this guide, recently warned of a critical, with a terribly high CVSS severity score of 8.8, Kubernetes Capsule Operator reverse proxy privilege escalation flaw, CVE-2022-23652.

Securing Kubernetes can be a full-time job. The NSA mentions that there are third-party security programs that can help. Of course, these also come with their own security concerns. On the other hand, given Kubernetes’ complexity, any help you can get from such programs as Calico Cloud, JetStack Secure, and Falco and newer security approaches such as zero trust, is to be welcomed.