VOOZH about

URL: https://docs.datadoghq.com/containers/kubernetes/tag/

⇱ Kubernetes Tag Extraction

For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/containers/kubernetes/tag.md. A documentation index is available at /llms.txt.

Kubernetes Tag Extraction

The Datadog Agent can automatically assign tags to metrics, traces, and logs emitted by a pod (or an individual container within a pod) based on labels or annotations.

Out-of-the-box tags

The list of automatically-assigned tags depends on the Agent’s cardinality configuration. Tag Cardinality is added before ingestion and can impact billing, as different cardinality settings impact the number of emitted metrics.

container_id
Cardinality: High
Source: Pod status
Requirement: N/A
display_container_name
Cardinality: High
Source: Pod status
Requirement: N/A
pod_name
Cardinality: Orchestrator
Source: Pod metadata
Requirement: N/A
oshift_deployment
Cardinality: Orchestrator
Source: Pod annotation openshift.io/deployment.name
Requirement: OpenShift environment and pod annotation must exist
kube_ownerref_name
Cardinality: Orchestrator
Source: Pod ownerref
Requirement: Pod must have an owner
kube_job (Orchestrator)
Cardinality: Orchestrator
Source: Pod ownerref
Requirement: Pod must be attached to a cronjob
kube_job (Low)
Cardinality: Low
Source: Pod ownerref
Requirement: Pod must be attached to a job
kube_replica_set
Cardinality: Low
Source: Pod ownerref
Requirement: Pod must be attached to a replica set
kube_service
Cardinality: Low
Source: Kubernetes service discovery
Requirement: Pod is behind a Kubernetes service
kube_daemon_set
Cardinality: Low
Source: Pod ownerref
Requirement: Pod must be attached to a DaemonSet
kube_container_name
Cardinality: Low
Source: Pod status
Requirement: N/A
kube_namespace
Cardinality: Low
Source: Pod metadata
Requirement: N/A
kube_app_name
Cardinality: Low
Source: Pod label app.kubernetes.io/name
Requirement: Pod label must exist
kube_app_instance
Cardinality: Low
Source: Pod label app.kubernetes.io/instance
Requirement: Pod label must exist
kube_app_version
Cardinality: Low
Source: Pod label app.kubernetes.io/version
Requirement: Pod label must exist
kube_app_component
Cardinality: Low
Source: Pod label app.kubernetes.io/component
Requirement: Pod label must exist
kube_app_part_of
Cardinality: Low
Source: Pod label app.kubernetes.io/part-of
Requirement: Pod label must exist
kube_app_managed_by
Cardinality: Low
Source: Pod label app.kubernetes.io/managed-by
Requirement: Pod label must exist
env
Cardinality: Low
Source: Pod label tags.datadoghq.com/env or container envvar (DD_ENV or OTEL_RESOURCE_ATTRIBUTES)
Requirement: Unified service tagging enabled
version
Cardinality: Low
Source: Pod label tags.datadoghq.com/version or container envvar (DD_VERSION or OTEL_RESOURCE_ATTRIBUTES)
Requirement: Unified service tagging enabled
service
Cardinality: Low
Source: Pod label tags.datadoghq.com/service or container envvar (DD_SERVICE, OTEL_RESOURCE_ATTRIBUTES, or OTEL_SERVICE_NAME)
Requirement: Unified service tagging enabled
pod_phase
Cardinality: Low
Source: Pod status
Requirement: N/A
oshift_deployment_config
Cardinality: Low
Source: Pod annotation openshift.io/deployment-config.name
Requirement: OpenShift environment and pod annotation must exist
kube_ownerref_kind
Cardinality: Low
Source: Pod ownerref
Requirement: Pod must have an owner
kube_deployment
Cardinality: Low
Source: Pod ownerref
Requirement: Pod must be attached to a deployment
kube_argo_rollout
Cardinality: Low
Source: Pod ownerref
Requirement: Pod must be attached to an argo rollout
kube_replication_controller
Cardinality: Low
Source: Pod ownerref
Requirement: Pod must be attached to a replication controller
kube_stateful_set
Cardinality: Low
Source: Pod ownerref
Requirement: Pod must be attached to a statefulset
persistentvolumeclaim
Cardinality: Low
Source: Pod spec
Requirement: A PVC must be attached to the pod
kube_cronjob
Cardinality: Low
Source: Pod ownerref
Requirement: Pod must be attached to a cronjob
image_name
Cardinality: Low
Source: Pod spec
Requirement: N/A
short_image
Cardinality: Low
Source: Pod spec
Requirement: N/A
image_tag
Cardinality: Low
Source: Pod spec
Requirement: N/A
eks_fargate_node
Cardinality: Low
Source: Pod spec
Requirement: EKS Fargate environment
kube_runtime_class
Cardinality: Low
Source: Pod spec
Requirement: Pod must be attached to a runtime class
gpu_vendor
Cardinality: Low
Source: Pod spec
Requirement: Container must be attached to a GPU resource
image_id
Cardinality: Low
Source: Container image ID
Requirement: N/A
kube_autoscaler_kind
Cardinality: Low
Source: Kubernetes autoscaler type
Requirement: Kubernetes autoscaler must be used
kube_priority_class
Cardinality: Low
Source: Pod priority class
Requirement: Pod must have priority class set
kube_qos
Cardinality: Low
Source: Pod Quality of Service class
Requirement: N/A

Host tag

The Agent can attach Kubernetes environment information as “host tags”.

kube_cluster_name
Cardinality: Low
Source: DD_CLUSTER_NAME envvar or cloud provider integration
Requirement: DD_CLUSTER_NAME envvar or cloud provider integration enabled
kube_node_role
Cardinality: Low
Source: Node label node-role.kubernetes.io/<role>
Requirement: Node label must exist
kube_node
Cardinality: Low
Source: NodeName field in a pod’s specifications
Requirement: N/A
orch_cluster_id
Cardinality: Low
Source: Orchestrator cluster metadata
Requirement: Orchestrator environment
kube_distribution
Cardinality: Low
Source: Node labels and NodeSystemInfo
Requirement: N/A

Tag Autodiscovery

Starting with Agent v6.10+, the Agent can autodiscover tags from Pod annotations. It allows the Agent to associate tags to all data emitted by the entire pods or an individual container within this pod.

As a best practice in containerized environments, Datadog recommends using unified service tagging to help unify tags. Unified service tagging ties Datadog telemetry together through the use of three standard tags: env, service, and version. To learn how to configure your environment with unified tagging, refer to the dedicated unified service tagging documentation.

To apply a <TAG_KEY>:<TAG_VALUE> tag to all data emitted by a given pod and collected by the Agent use the following annotation on your pod:

annotations:ad.datadoghq.com/tags:'{"<TAG_KEY>": "<TAG_VALUE>","<TAG_KEY_1>": "<TAG_VALUE_1>"}'

If you want to apply a <TAG_KEY>:<TAG_VALUE> tag to an individual container <CONTAINER_NAME> within a pod, use the following annotation on your pod:

annotations:ad.datadoghq.com/<CONTAINER_NAME>.tags:'{"<TAG_KEY>": "<TAG_VALUE>","<TAG_KEY_1>": "<TAG_VALUE_1>"}'

Starting with Agent v7.17+, the Agent can Autodiscover tags from Docker labels. This process allows the Agent to associate custom tags to all data emitted by a container, without modifying the Agent configuration.

com.datadoghq.ad.tags:'["<TAG_KEY>:TAG_VALUE", "<TAG_KEY_1>:<TAG_VALUE_1>"]'

Starting with Agent v7.77+, tag annotations support Autodiscovery template variables for dynamic tagging based on runtime metadata. With the exception of %%env_<VAR>%%, all template variables are supported.

annotations:ad.datadoghq.com/tags:'{"pod_ip":"%%host%%","pod_name":"%%kube_pod_name%%","namespace":"%%kube_namespace%%"}'ad.datadoghq.com/nginx.tags:'{"container_port":"%%port_80%%"}'

Tag extraction

Starting in version 7.64+, the Agent and Cluster Agent can be configured to collect labels and annotations from Kubernetes resources and use them as tags from a common configuration. Datadog recommends using the following options to ensure consistent reporting across the Agent’s core tagging, the Cluster Agent’s KSM reporting, and both Agents’ Orchestrator Explorer reporting:

  • kubernetesResourcesLabelsAsTags
  • kubernetesResourcesAnnotationsAsTags

These options should be used instead of the legacy Agent options podLabelsAsTags, nodeLabelsAsTags, namespaceLabelsAsTags, and any KSM configuration overrides.

These configurations reference the resource type of the object from which to extract metadata. Each resource type must be specified in the format resourceType.apiGroup, where resourceType is the plural name of the resource. Resources in the empty API group (for example, pods and nodes) can be specified using only the resourceType name.

For example, run kubectl api-resources to retrieve this information:

NameAPI VersionDatadog Resource Configuration
podsv1pods
nodesv1nodes
namespacesv1namespaces
deploymentsapps/v1deployments.apps
rolesrbac.authorization.k8s.io/v1roles.rbac.authorization.k8s.io

Notes:

  • Tags do not cascade between the workload and child resources. For example, labels on a Deployment are not automatically applied to logs from its child Pods. To tag Pod data, configure label extraction directly on the Pods.
  • Tags do cascade from the namespace to the pods and containers inside of them.
  • Use Datadog Agent 7.73+ to use wildcards in the tag extraction rules for your KSM Metrics.

Kubernetes resources labels as tags

This option is used to extract a given label on your Kubernetes resources and send that up as a Datadog tag.

To extract a given resource label <LABEL> and transform them as tag keys <TAG_KEY> within Datadog, add the following configuration to your Operator’s DatadogAgent configuration in datadog-agent.yaml:

apiVersion:datadoghq.com/v2alpha1kind:DatadogAgentmetadata:name:datadogspec:global:kubernetesResourcesLabelsAsTags:<RESOURCE>:<LABEL>:<TAG_KEY>

For example, to extract resource labels from nodes, pods, and deployments:

apiVersion:datadoghq.com/v2alpha1kind:DatadogAgentmetadata:name:datadogspec:global:kubernetesResourcesLabelsAsTags:nodes:kubernetes.io/arch:archpods:role:pod_roledeployments.apps:team:kube_team

To extract a given resource label <LABEL> and transform them as tag keys <TAG_KEY> within Datadog, add the following configuration to your Helm datadog-values.yaml file:

datadog:kubernetesResourcesLabelsAsTags:<RESOURCE>:<LABEL>:<TAG_KEY>

For example, to extract resource labels from nodes, pods, and deployments:

datadog:kubernetesResourcesLabelsAsTags:nodes:kubernetes.io/arch:archpods:role:pod_roledeployments.apps:team:kube_team

To extract a given resource label <LABEL> and transform them as tag keys <TAG_KEY> within Datadog, add the following environment variable to both your Agent and Cluster Agent containers.

- name:DD_KUBERNETES_RESOURCES_LABELS_AS_TAGSvalue:'{"<RESOURCE>":{"<LABEL>":"<TAG_KEY>"}}'

For example, to extract resource labels from nodes, pods, and deployments:

- name:DD_KUBERNETES_RESOURCES_LABELS_AS_TAGSvalue:'{"deployments.apps":{"team":"kube_team"},"nodes":{"kubernetes.io/arch":"arch"},"pods":{"role":"pod_role"}}'

For Agent 7.73.0+, use the following configuration to add all resource labels as tags to your metrics. In this example, the tag names are prefixed with <PREFIX>_:

#(...)kubernetesResourcesLabelsAsTags:pods:"*": <PREFIX>_%%label%%

Notes: Custom metrics may impact billing. See the custom metrics billing page for more information.

Merging with legacy configurations

This configuration option is merged with other configurations set in podLabelsAsTags, namespaceLabelsAsTags and nodeLabelsAsTags. In case of conflict, kubernetesResourcesLabelsAsTags take precedence while merging the configurations.

For example, if you have the following configurations:

datadog:kubernetesResourcesLabelsAsTags:pods:label-1:tag-alabel-2:tag-bpodLabelsAsTags:label-2:legacy-tag-clabel-3:legacy-tag-d

The following mapping is used to extract tags from pod labels:

label-1:tag-alabel-2:tag-blabel-3:legacy-tag-d

Kubernetes resources annotations as tags

This option extracts a specified annotation from your Kubernetes resources and sends it as a Datadog tag.

To extract a given resource annotation <ANNOTATION> and transform them as tag keys <TAG_KEY> within Datadog, add the following configuration to your Operator’s DatadogAgent configuration in datadog-agent.yaml:

apiVersion:datadoghq.com/v2alpha1kind:DatadogAgentmetadata:name:datadogspec:global:kubernetesResourcesAnnotationsAsTags:<RESOURCE>:<ANNOTATION>:<TAG_KEY>

For example, to extract resource annotations from nodes, pods, and deployments:

apiVersion:datadoghq.com/v2alpha1kind:DatadogAgentmetadata:name:datadogspec:global:kubernetesResourcesAnnotationsAsTags:nodes:kubernetes.io/arch:archpods:role:pod_roledeployments.apps:team:kube_team

To extract a given resource annotation <ANNOTATION> and transform them as tag keys <TAG_KEY> within Datadog, add the following configuration to your Helm datadog-values.yaml file:

datadog:kubernetesResourcesAnnotationsAsTags:<RESOURCE>:<ANNOTATION>:<TAG_KEY>

For example, to extract resource annotations from nodes, pods, and deployments:

datadog:kubernetesResourcesAnnotationsAsTags:nodes:kubernetes.io/arch:archpods:role:pod_roledeployments.apps:team:kube_team

To extract a given resource annotation <ANNOTATION> and transform them as tag keys <TAG_KEY> within Datadog, add the following environment variable to both your Agent and Cluster Agent containers.

- name:DD_KUBERNETES_RESOURCES_ANNOTATIONS_AS_TAGSvalue:'{"<RESOURCE>":{"<ANNOTATION>":"<TAG_KEY>"}}'

For example, to extract resource annotations from nodes, pods, and deployments:

- name:DD_KUBERNETES_RESOURCES_ANNOTATIONS_AS_TAGSvalue:'{"deployments.apps":{"team":"kube_team"},"nodes":{"kubernetes.io/arch":"arch"},"pods":{"role":"pod_role"}}'

For Agent 7.73.0+, use the following configuration to add all resource annotations as tags to your metrics. In this example, the tag names are prefixed with <PREFIX>_:

#(...)kubernetesResourcesAnnotationsAsTags:pods:"*": <PREFIX>_%%annotation%%

Notes: Custom metrics may impact billing. See the custom metrics billing page for more information.

This configuration option is merged with other configurations set in podAnnotationsAsTags. In case of conflict, kubernetesResourcesAnnotationsAsTags take precedence while merging the configurations.

For example, if you have the following configurations:

datadog:kubernetesResourcesAnnotationsAsTags:pods:annotation-1:tag-aannotation-2:tag-bpodAnnotationsAsTags:annotation-2:legacy-tag-cannotation-3:legacy-tag-d

The following mapping is used to extract tags from pod annotations:

annotation-1:tag-aannotation-2:tag-bannotation-3:legacy-tag-d

Container environment variables as tags

Starting with Agent v7.32+, the Agent can collect container environment variables and use them as tags to attach to all metrics, traces, and logs corresponding to the container. Both docker and containerd containers are supported:

To extract a given environment variable <ENV_VAR> and transform it as a tag key <TAG_KEY> within Datadog, add the following configuration to your Operator’s DatadogAgent configuration in datadog-agent.yaml:

apiVersion:datadoghq.com/v2alpha1kind:DatadogAgentmetadata:name:datadogspec:#(...)override:nodeAgent:env:- name:DD_CONTAINER_ENV_AS_TAGSvalue:'{"<ENV_VAR>": "<TAG_KEY>"}'

For example, you could set up:

apiVersion:datadoghq.com/v2alpha1kind:DatadogAgentmetadata:name:datadogspec:#(...)override:nodeAgent:env:- name:DD_CONTAINER_ENV_AS_TAGSvalue:'{"app":"kube_app"}'

To extract a given environment variable <ENV_VAR> and transform it as a tag key <TAG_KEY> within Datadog, add the following configuration to your Helm datadog-values.yaml file:

datadog:env:- name:DD_CONTAINER_ENV_AS_TAGSvalue:'{"<ENV_VAR>": "<TAG_KEY>"}'

For example, you could set up:

datadog:env:- name:DD_CONTAINER_ENV_AS_TAGSvalue:'{"app":"kube_app"}'

To extract a given environment variable <ENV_VAR> and transform it as a tag key <TAG_KEY> within Datadog, add the following environment variable to the Datadog Agent:

DD_CONTAINER_ENV_AS_TAGS='{"<ENV_VAR>": "<TAG_KEY>"}'

For example:

DD_CONTAINER_ENV_AS_TAGS='{"app":"kube_app"}'

Note: Custom metrics may impact billing. See Custom Metrics Billing for more details.

Container labels as tags

Starting with Agent v7.33+, the Agent can collect container labels and use them as tags. The agent attaches the tags to all metrics, traces, and logs associated with the container.

The Agent can generate tags from container labels for both docker and containerd containers. In the case of containerd, the minimum supported version is v1.5.6, because previous releases do not propagate labels correctly.

To extract a given container label <CONTAINER_LABEL> and transform it as a tag key <TAG_KEY> within Datadog, add the following configuration to your Operator’s DatadogAgent configuration in datadog-agent.yaml:

apiVersion:datadoghq.com/v2alpha1kind:DatadogAgentmetadata:name:datadogspec:#(...)override:nodeAgent:env:- name:DD_CONTAINER_LABELS_AS_TAGSvalue:'{"<CONTAINER_LABEL>": "<TAG_KEY>"}'

For example, you could set up:

apiVersion:datadoghq.com/v2alpha1kind:DatadogAgentmetadata:name:datadogspec:#(...)override:nodeAgent:env:- name:DD_CONTAINER_LABELS_AS_TAGSvalue:'{"app":"kube_app"}'

To extract a given container label <CONTAINER_LABEL> and transform it as a tag key <TAG_KEY> within Datadog, add the following configuration to your Helm datadog-values.yaml file:

datadog:env:- name:DD_CONTAINER_LABELS_AS_TAGSvalue:'{"<CONTAINER_LABEL>": "<TAG_KEY>"}'

For example, you could set up:

datadog:env:- name:DD_CONTAINER_LABELS_AS_TAGSvalue:'{"app":"kube_app"}'

To extract a given container label <CONTAINER_LABEL> and transform it to a tag key <TAG_KEY>, add the following environment variable to the Datadog Agent:

DD_CONTAINER_LABELS_AS_TAGS='{"<CONTAINER_LABEL>":"<TAG_KEY>"}'

For example:

DD_CONTAINER_LABELS_AS_TAGS='{"app":"kube_app"}'

Note: Custom metrics may impact billing. See Custom Metrics Billing for more details.

Further Reading