Cloud Native Ecosystem Containers Databases Edge Computing Infrastructure as Code Linux Microservices Open Source Networking Storage

AI AI Engineering API Management Backend development Data Frontend Development Large Language Models Security Software Development WebAssembly

AI Operations CI/CD Cloud Services DevOps Kubernetes Observability Operations Platform Engineering

C++ Developer tools Go Java JavaScript Programming Languages Python Rust TypeScript

One of the main reasons that Kubernetes became so popular is because it’s so flexible. When we say Kubernetes, we typically think about deploying and managing containers. And while this is, in fact, Kubernetes’s main job, it can actually do much more than that. This is possible thanks to something called custom resource definitions, or CRDs for short. In this post, you’ll learn what CRDs are and what you can use them for. We’ll also take a look at how to create them.

But by looking under the Kubernetes hood, you could see that the main component of Kubernetes is an API server and etcd data store.

Kubernetes API

Before we dive into custom resource definitions, let’s first talk about Kubernetes in general. If I asked you, “What is Kubernetes?” then you’d probably answer, “Kubernetes is a container orchestrator.” This would, of course, be one correct answer. But by looking under the Kubernetes hood, you could see that the main component of Kubernetes is an API server and etcd data store. And there are other, more important components like kube-scheduler, kube-controller-manager and cloud-controller-manager, but pretty much any operation on your cluster needs to go through an API server. That API has a few built-in objects that it understands. Things you may be familiar with like pods, namespaces, ConfigMaps, services or nodes are all API objects. That’s why whenever you execute kubectl get pods< or kubectl get nodes, you get a list of pods or nodes. But if you try to get a list of objects that don’t exist in the Kubernetes API — like, for example, kubectl get biscuits — you’d get a response similar to this: error: the server doesn't have a resource type "biscuits" And that is because there is no such thing as “biscuits” defined in the Kubernetes API. Quite logical, right? Well, what if I told you that you could add a biscuits definition to your Kubernetes cluster? In fact, you can extend your Kubernetes API with any custom object you like. That’s exactly what custom resource definitions are for.

Why CRDs?

So what’s the point of adding a biscuits definition to your Kubernetes cluster? Remember when I mentioned earlier that the success of Kubernetes comes from its flexibility? The ability to extend the Kubernetes API with custom resource definitions is a really great feature that lets you do something magical. It allows you to instruct Kubernetes to manage more than just containers. Why is that such a great thing? Because CRDs together with Kubernetes operators give you almost unlimited possibilities. You can adapt Kubernetes in a way that it will take care of older parts of your infrastructure. If you do it right, you’ll be able to avoid bottlenecks and easily modernize things that normally would require long and costly redesigns.

CRDs on Your Cluster

Before we dive into creating our own CRD, you need to know two things. First, creating a custom resource definition is an advanced topic. Many companies don’t even need to create any CRDs. The Kubernetes community finds interesting solutions for common problems all the time, and it’s likely that any use case you encounter probably already has a CRD you can use. And if you’re still new to Kubernetes, you definitely shouldn’t jump into CRDs before you understand the basics well. Second, as already mentioned, you don’t need to create any CRDs yourself if you don’t feel the need to. However, many Kubernetes tools will install their own CRDs, so even if you don’t create any yourself, you’ll probably still end up having some on your cluster. One example is cert-manager, a very popular Kubernetes tool for managing certificates. It installs a few CRDs on your cluster in order to do its job. If you execute kubectl get clusterissuers before installation of cert-manager, your cluster won’t know what ClusterIssuers are: error: the server doesn't have a resource type "clusterissuers" But if you execute the same command after cert-manager installation, you’ll get the list of ClusterIssuers on your cluster. In fact, you can list all custom resource definitions installed on your cluster by executing kubectl get crd: The above output comes in the form of <object>.<group> and tells me that I can execute commands like kubectl get addons, kubectl get helmcharts or kubectl get middlewares. None of them are defined in vanilla Kubernetes and are only there because they were defined as custom resource definitions.

How to Create CRDs

OK, forget about biscuits. Let’s take a look at some more realistic examples. Imagine that you want Kubernetes to somehow manage your custom routers in your data center. For that, you could create a custom resource definition similar to this one: You can apply the above CRD to the cluster by executing kubectl apply -f router-CRD.yaml. Once you do that, your Kubernetes cluster will already know what “router” is. Therefore, you’ll be able to execute kubectl get routers. Of course, we just applied the resource definition, not the resource itself. So kubectl get routers will return the following: No resources found. But as you can see, it doesn’t return this: error: the server doesn't have a resource type "routers" Which means we successfully added a new object to the Kubernetes API. To add an actual router resource, you can construct a YAML definition file like with any other object: Now, you can create a new router on your cluster by executing kubectl apply - f example-router.yaml, and if you try to get the list of routers again with kubectl get routers, you should see one now:

In its current form, our CRD doesn’t do anything besides being processed and stored by the Kubernetes API.

What to Do With CRDs

You may be thinking, “OK, great, but that router doesn’t do anything!” And yes, that’s right. In its current form, our CRD doesn’t do anything besides being processed and stored by the Kubernetes API. And while there are use cases where this is enough, usually CRDs are combined with custom controllers. Custom controllers are another concept in Kubernetes that lets you actually do something with your custom resources. In our case, we would like to actually create or configure the routers in our data center. Therefore, we’d have to write a custom controller and instruct it to listen to the Kubernetes API and wait for any changes to our custom router objects. Custom controllers under the hood are just applications or scripts written in your programming language of choice. They’re deployed on the cluster as pods, and their job is to listen to the Kubernetes API and perform some actions based on defined logic.

CRD vs. ConfigMap

Last but not least, by looking at CRDs, you may see some similarities with a Kubernetes built-in object, ConfigMap. And if you use CRDs without a custom controller, they may, in fact, serve a similar purpose. They both can be used to store custom configurations. However, there are noticeable differences between them. First of all, ConfigMaps by design are meant to provide configuration for your pods. They can be mounted as files or environment variables into the pod. They work well if you have well-defined config files like, for example, Apache or MySQL config. CRDs can also be consumed by pods but only by contacting the Kubernetes API. They simply have a different purpose than ConfigMaps. They’re not meant to be used to provide configuration to your pods but to extend the Kubernetes API in order to build custom automation.

Summary

Kubernetes’s flexibility is what made it so successful, among other things, of course. Now, you can use that flexibility by creating your own Kubernetes objects. The possibilities are almost limitless, and it’s only up to you how you’ll make use of CRDs. Come back to us for more Kubernetes articles. Here’s the Release article explaining another Kubernetes object, DaemonSets. Also, feel free to take a look at our offerings. We simplify the development process by providing Environments as a Service.

Release is a sponsor of The New Stack.

The New Stack does not sell your information or share it with unaffiliated third parties. By continuing, you agree to our Terms of Use and Privacy Policy.