Tutorial: Deploy the Nvidia GPU Operator on Kubernetes Based on containerd Runtime

This tutorial will explore the steps to install Nvidia GPU Operator on a Kubernetes cluster with GPU hosts based on the containerd runtime instead of Docker Engine.

In a typical GPU-based Kubernetes installation, each node needs to be configured with the correct version of Nvidia graphics driver, CUDA runtime, and cuDNN libraries followed by a container runtime such as Docker Engine, containerd, podman, or CRI-O. Then, the Nvidia Container Toolkit is deployed to provide GPU access to the containerized applications. Finally, Kubernetes is installed, which will interact with the chosen container runtime to manage the lifecycle of workloads.

Nvidia GPU Operator dramatically simplifies the process without installing the drivers, CUDA runtime, cuDNN libraries, or the Container Toolkit. It can be installed on any Kubernetes cluster that meets specific hardware and software requirements.

Below are the steps to install containerd, Kubernetes, and Nvidia GPU Operator. Towards the end of the installation, we will test the GPU access by running the popular nvidia-smi command within the pod.

Environment

Operating system: Ubuntu 18.04 LTS Server
GPU: Nvidia GeForce RTX 3090
CPU: AMD Ryzen ThreadRipper 3990X
RAM: 128GB
HDD: 4TB NVMe SSD

Step 1: Install Containerd Runtime

Load the required modules and ensure they are persisted during reboots.

sudo modprobe overlay
sudo modprobe br_netfilter
cat <

cat <

Load the sysctl parameters without rebooting the system.

sudo sysctl --system

Finally, install the containerd runtime.

sudo apt-get update
sudo apt-get install -y containerd

Let’s create the default containerd configuration file.

sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml

Set the cgroup driver for runc to systemd, which is required for the kubelet.

Within the [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] section, add the following lines:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
 SystemdCgroup = true

Your config.toml should look like this:

👁 Image

Restart containerd with the new configuration.

sudo systemctl restart containerd

Check the status of containerd runtime.

systemctl status containerd

👁 Image

Step 2: Install Kubernetes 1.21

Start by disabling swap memory.

sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

Install the required tools.

sudo apt-get update
sudo apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update sudo apt install -y kubeadm kubelet kubernetes-cni

Let’s initialize the control plane.

sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=10.0.0.54

Make sure you replace the IP address of 10.0.0.54 with the appropriate address of your host.

It’s time to configure kubectl CLI.

mkdir $HOME/.kube
sudo cp /etc/kubernetes/admin.conf $HOME/.kube/
sudo chown $(id -u):$(id -g) $HOME/.kube/admin.conf
export KUBECONFIG=$HOME/.kube/admin.conf
echo "export KUBECONFIG=$HOME/.kube/admin.conf" | tee -a ~/.bashrc

Before we can access the cluster, we need to install the CNI addon. For this tutorial, we are using Weave Net from Weave Works.

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

Since we only have one node, let’s remove the taint to enable scheduling.

kubectl taint nodes --all node-role.kubernetes.io/master-

Finally, check the status of the cluster.

kubectl get nodes

👁 Image

Step 3 – Install Nvidia GPU Operator

Start by installing the binary of Helm3.

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 
chmod 700 get_helm.sh
./get_helm.sh

Add the Nvidia Helm Repository.

helm repo add nvidia https://nvidia.github.io/gpu-operator
helm repo update

Since we are using the containerd runtime, let’s set that as the default.

helm install --wait --generate-name \
 nvidia/gpu-operator \
 --set operator.defaultRuntime=containerd

Within a few minutes, you should see the pods in the gpu-operator-resources namespace running.

kubectl get pods -n gpu-operator-resources

👁 Image

It’s time to test the GPU access from a pod. Run the below command to launch a test pod.

kubectl run gpu-test \
 --rm -t -i \
 --restart=Never \
 --image=nvcr.io/nvidia/cuda:10.1-base-ubuntu18.04 nvidia-smi

👁 Image

Congratulations! In less than 10 minutes, we configured a Kubernetes cluster based on containerd powered by a GPU.