Why Kubernetes Has Emerged as the ‘OS’ of the Cloud

Since the dawn of the computer age, IT has been defined by the operating systems (MS-DOS, Unix, Linux, Windows, macOS, Android, iOS) that quarterback them. New research has revealed that the main players and market shares in 2023 remain the same but that there’s also a rising star on this horizon as we move into the mysterious new Web3 era.

That star, to the surprise of few, is Kubernetes. The Cloud Native Computing Foundation (CNCF) reports that this “operating system” — well, really a base layer of ops management inside a modern IT stack — has made a successful mainstream leap to the cloud and seen its instances multiply greatly in just a few years’ time.

Seventy-one percent of all organizations run databases and caches in Kubernetes, representing a 48% year-on-year increase. Together with messaging systems (36% growth), organizations were increasingly using databases and caches to persist application workload states.

Kubernetes Origin

Kubernetes, which has been in general use for less than a decade, orchestrates and deploys containers with microservices to run workloads. Forty-four percent of respondents to the CNCF survey (which encompassed 2,063 qualified professionals) are already using containers for nearly all applications and business segments, and another 35% say containers are used for at least a few production applications. This is a significant threat to competing technologies such as virtual machines, which have been well-embedded in data systems for more than 20 years but are fast being replaced in production use cases.

Kubernetes, originally developed and designed by Google engineers Craig McLuckie, Joe Beda, and Brendan Burns in 2013, has quickly grown to become one of the most popular open-source projects in the world. However, the increased usage of the system orchestrator isn’t apparent in all corners of IT infrastructure.

“So, what’s happening with Kubernetes is we’re using it more and more and with more and more complex (cloud-native) applications,” said Lawrence Hecht, an analyst/researcher with The New Stack. “But if you use Kubernetes for enterprises, that (market segment) is not growing at all. Zero. Existing users, yes, and then there might be some new instances from smaller companies. But the growth is all in cloud-native apps.”

Enterprises are experimenting with different ways to deploy Kubernetes in the cloud and finding success doing it. Dynatrace, in its Kubernetes in the Wild 2023 report, noted in 2021 that in a typical Kubernetes cluster, application workloads accounted for most of the pods (59%). At that time, all non-application workloads (system and auxiliary workloads) played a relatively small part.

A year later, however, this picture was quickly reversed. In 2022, auxiliary workloads outnumbered application workloads (63% vs. 37%) as organizations increasingly adopted advanced Kubernetes platform technologies such as security controls, service meshes, messaging systems, and observability tools. At the same time, organizations tried Kubernetes for a broader range of use cases, such as building data pipelines, scheduling utility workloads, and other tasks. Kubernetes became the platform for running almost anything — thus emerging as a virtual “operating system” for cloud native apps. Versatility is its crowning glory.

Containers Are the New Normal, and WebAssembly Is the Future

The CNCF also found that with containers going mainstream in 2022, the uptake of serverless architecture is setting the stage for WebAssembly, which was asked about for the first time in this survey. Overall, 37% of organizations reported having some experience deploying applications with WebAssembly. Although many are still personally testing them, WasmEdge and WAMR are the top runtimes being used.

“WebAssembly, or Wasm, was shown to be a very practical way to run code on a  web browser, serving as a compiler of sorts,” wrote B. Cameron Gain here in The New Stack. “And it has worked so well as a language that the World Wide Web Consortium (W3C) named it a web standard in 2019, thus becoming the fourth web standard with HTML, CSS and JavaScript.”

WebAssembly is seen as a payload for containers, a way of programming sidecar services such as service meshes, and an alternative way to deliver and orchestrate workloads to edge devices.

Adoption of CNCF Projects Increased in 2022

Other highlights of the report included:

• The biggest challenges responders reported in using and deploying containers are lack of training and security. In fact, lack of training is the most significant barrier inhibiting adoption; it is the top challenge cited by 44% that have yet to deploy containers in production, and 41% of those that use containers on a limited basis. Once containers are used for nearly all applications, then security becomes the top challenge.
• The adoption of CNCF-incubated and graduated projects increased in 2022, with OpenTelemetry and Argo scoring the largest jumps in usage. The former rose from 4% in 2020 to 20% in 2022 and the latter from 10% to 28%. Meanwhile, Containerd (36% to 56%) and CoreDNS (48% to 56%) are the graduated projects with the greatest increase in use and evaluation.
• Continuous integration and delivery (CI/CD) technologies grew by 43% year-on-year, indicating that organizations are dedicating significantly more Kubernetes clusters to running software build, test, and deployment pipelines.

The worldwide survey was fielded from June 30 through Sept. 27, 2022, by the CNCF and Linux Foundation Research. It was promoted via social media, the CNCF and Linux Foundations and their respective websites, the CNCF full subscriber email, the KubeWeekly newsletter, and the Linux Foundation newsletter. The survey was available in English, Chinese, and Japanese. A third-party panel provider was also utilized to get 54% of respondents, which were provided nominal compensation for participating.