VOOZH about

URL: https://kind.sigs.k8s.io/docs/user/private-registries/

โ‡ฑ kind โ€“ Private Registries


๐Ÿ‘ Image

Private Registries

This guide discusses how to use kind with image registries that require authentication.

There are multiple ways to do this, which we try to cover here.

Contents ๐Ÿ”—๏ธŽ

Use ImagePullSecrets ๐Ÿ”—๏ธŽ

Kubernetes supports configuring pods to use imagePullSecrets for pulling images. If possible, this is the preferable and most portable route.

See the upstream kubernetes docs for this, kind does not require any special handling to use this.

If you already have the config file locally but would still like to use secrets, read through kubernetesโ€™ docs for creating a secret from a file.

Pull to the Host and Side-Load ๐Ÿ”—๏ธŽ

kind can load an image from the host with the kind load ... commands. If you configure your host with credentials to pull the desired image(s) and then load them to the nodes you can avoid needing to authenticate on the nodes.

Add Credentials to the Nodes ๐Ÿ”—๏ธŽ

Generally the upstream docs for using a private registry apply, with kind there are two options for this.

Mount a Config File to Each Node ๐Ÿ”—๏ธŽ

If you pre-create a docker config.json containing credential(s) on the host you can mount it to each kind node.

Assuming your file is at /path/to/my/secret.json, the kind config would be:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
 extraMounts:
 - containerPath: /var/lib/kubelet/config.json
 hostPath: /path/to/my/secret.json

Important: The mounted file must contain the actual base64-encoded credentials in the auths field. It cannot reference an external credential store or helper.

Many systems store docker credentials outside of config.json using a credential helper (e.g. "credsStore": "desktop" on Docker Desktop, "credsStore": "osxkeychain" on macOS, or "credsStore": "secretservice" on Linux). If your ~/.docker/config.json contains a credsStore or credHelpers key, the credentials themselves are not in the file and mounting it into a kind node will not work because the credential helper binary is not present inside the node.

To check whether your config contains plain credentials or a credential store reference:

cat ~/.docker/config.json

A config with plain credentials looks like:

{
 "auths": {
 "myregistry.example.com": {
 "auth": "dXNlcjpwYXNzd29yZA=="
 }
 }
}

A config that uses an external credential store (which will not work when mounted) looks like:

{
 "auths": {},
 "credsStore": "desktop"
}

To generate a config file with plain credentials, create a temporary config that bypasses the credential store and log in with it:

# Create a temp dir to hold the plain-credential config
DOCKER_CONFIG=$(mktemp -d)
export DOCKER_CONFIG
# Seed an empty auths entry to disable the credential store
cat <<EOF >"${DOCKER_CONFIG}/config.json"
{
 "auths": { "myregistry.example.com": {} }
}
EOF
# Log in โ€” credentials will be written as plain base64 auth in the file
docker login myregistry.example.com
# Use ${DOCKER_CONFIG}/config.json as your hostPath

For registries that use short-lived or token-based authentication (such as Azure Container Registry with OAuth tokens), the static mount approach may not be reliable once the token expires. For ACR, prefer logging in with a service principal whose credentials are long-lived:

docker login myregistry.azurecr.io \
 --username <service-principal-id> \
 --password <service-principal-password>

Use an Access Token ๐Ÿ”—๏ธŽ

A credential can be programmatically added to the nodes at runtime.

If you do this then kubelet must be restarted on each node to pick up the new credentials.

An example shell snippet for generating a gcr.io cred file on your host machine using Access Tokens:

examples/kind-gcr.sh
#!/bin/sh
# Copyright The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -o errexit
# desired cluster name; default is "kind"
KIND_CLUSTER_NAME="${KIND_CLUSTER_NAME:-kind}"
# create a temp file for the docker config
echo "Creating temporary docker client config directory ..."
DOCKER_CONFIG=$(mktemp -d)
export DOCKER_CONFIG
trap 'echo "Removing ${DOCKER_CONFIG}/*" && rm -rf ${DOCKER_CONFIG:?}' EXIT
echo "Creating a temporary config.json"
# This is to force the omission of credsStore, which is automatically
# created on supported system. With credsStore missing, "docker login"
# will store the password in the config.json file.
# https://docs.docker.com/engine/reference/commandline/login/#credentials-store
cat <<EOF >"${DOCKER_CONFIG}/config.json"
{
 "auths": { "gcr.io": {} }
}
EOF
# login to gcr in DOCKER_CONFIG using an access token
# https://cloud.google.com/container-registry/docs/advanced-authentication#access_token
echo "Logging in to GCR in temporary docker client config directory ..."
gcloud auth print-access-token | \
 docker login -u oauth2accesstoken --password-stdin https://gcr.io
# setup credentials on each node
echo "Moving credentials to kind cluster name='${KIND_CLUSTER_NAME}' nodes ..."
for node in $(kind get nodes --name "${KIND_CLUSTER_NAME}"); do
 # the -oname format is kind/name (so node/name) we just want name
 node_name=${node#node/}
 # copy the config to where kubelet will look
 docker cp "${DOCKER_CONFIG}/config.json" "${node_name}:/var/lib/kubelet/config.json"
 # restart kubelet to pick up the config
 docker exec "${node_name}" systemctl restart kubelet.service
done
echo "Done!"

Use a Service Account ๐Ÿ”—๏ธŽ

Access tokens are short lived, so you may prefer to use a Service Account and keyfile instead. First, either download the key from the console or generate one with gcloud:

gcloud iam service-accounts keys create <output.json> --iam-account <account email>

Then, replace the gcloud auth print-access-token | ... line from the access token snippet with:

cat <output.json> | docker login -u _json_key --password-stdin https://gcr.io

See Googleโ€™s upstream docs on key file authentication for more details.

Use a Certificate ๐Ÿ”—๏ธŽ

If you have a registry authenticated with certificates, and both certificates and keys reside on your host folder, it is possible to mount and use them into the containerd plugin patching the default configuration, like in the example:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
 - role: control-plane
 # This option mounts the host docker registry folder into
 # the control-plane node, allowing containerd to access them. 
 extraMounts:
 - containerPath: /etc/docker/certs.d/registry.dev.example.com
 hostPath: /etc/docker/certs.d/registry.dev.example.com
# NOTE: the following patch is not necessary with images from kind v0.27.0+
# It may enable some older images to work similarly
containerdConfigPatches:
- |- [plugins."io.containerd.grpc.v1.cri".registry]
 config_path = "/etc/containerd/certs.d"