Rootkits Come to Containers and Bring Trouble with Them

Rootkits have long been a pain in the neck to servers. These collections of malicious programs would hide on servers while allowing their controllers’ administrator-level access to a computer or network. While they don’t get the security headlines they used to, they’re still alive, well, and causing damage. So, it comes as no surprise that Aqua’s Team Nautilus has found many attackers using advanced persistent threat (APT) techniques to plant rootkits in cloud native containers.

It’s never a dull moment in the security business.

APTs and rootkits go hand-in-hand. An APT is a determined effort by an enemy to use continuous, clandestine, and sophisticated hacking techniques to break into your system. Once in, they remain hidden, and your uninvited visitor grabs data or sets up malware for a bigger, better attack. A rootkit, the names say it all–root access is done by a hidden kit of programs — works well with APTs.

Usually, because of their sheer scale, APTs are done by nation-states. Rootkits, sadly, can be planted by just about any script kiddie.

Aqua has seen the following steps in this latest series of attacks:

1. Running a vanilla container image: The adversaries run an Alpine Linux container image.
2. Escaping to the host: The attackers then mount the host file system to escape the container and gain access to the host.
3. Downloading a malicious script: After they have escaped to the host, they write a command in the cron scheduler system that downloads and executes a malicious shell script (cronb.sh) from a remote source.
4. Loading and executing the malware: The script cronb.sh is the payload, which executes the attack.

Simple and ugly.

Team Nautilus has found that in “cloud native environments, attackers usually execute rootkits on the host to hide malicious processes and to reduce the chances of detection after they have escaped the containerized environment.” And, of course, “rootkits can be used to override functions or binaries in userspace (for example, using LD_PRELOAD).” This last is a really old C library trick but it’s still successfully used and abused today.

Specifically, whoever these attackers are they’re using the Linux kernel space rootkit Diamorphine. If they can’t get Diamorphine in they’ll try using a userspace rootkit as a fallback. In either case, the point is to hide the real malicious payload from your system.

No matter the rootkit, the name of the game in the current crop of rootkit attacks is to conceal malicious programs such as Bitcoin miners with their high CPU usage from process monitoring programs such as ps or top that track Linux processes. They do this by manipulating how you see the data in Linux’s /proc/ filesystem. You see nearly all userspace tools that report on processes rely on the /proc/ filesystem contents. So, when you call top you’re using the getdents or getdents64 function to pull the data from proc. This rootkit intercepts these functions and hides the hidden processes’ data.

So, what can you do about this? Well, as everyone’s been telling you all along. Keep your Linux system and open source software up to date. None of those rootkits can get in unless you’ve opened the door. Keep it closed and locked with up-to-date secure software without known and fixed holes and you’re good. Sure, once in a blue moon there’s a zero-day attack on Linux, but it’s rare. Known, fixed security holes are the real ways most attackers get in.

And, as the National Security Agency (NSA) reminds us, you should configure systems to load only modules with a valid digital signature. This makes it much harder for an attacker to push a malicious kernel module into your containers.

Aqua also recommends you use Tracee to protect yourself. This is an open source Linux runtime security and forensics tool. It’s built on top of extended BPF (eBPF). Tracee 0.5.0 and above enable you to write Open Policy Agent (OPA) Rego signatures. The point of this is it enables you to detect suspicious activity. For example, Aqua has added some basic signatures such as “Attempt to load a kernel module detection.” As the name says, this lets you detect suspicious kernel driver loading. For a more in-depth analysis on how you can detect privilege escalation with Tracee, check out the talk by Team Nautilus at BlackHat 2021.

You can also use Aqua Enforcers’ file integrity monitoring (FIM). This validates your operating system application files by comparing the current file state with a known, good baseline. If the rootkit changes the names of some binaries and inserts new files with their names it will spot these changes and alert you that there are programs up to no good in your systems.