![]() |
VOOZH | about |
Jenkins is an automation server with continuous integration and delivery features. With its plugin architecture, Jenkins can be customized to fit any CI/CD need and automates all aspects of project development, testing, and deployment.
Set up CI Visibility for Jenkins to collect data across various stages of your pipeline executions, identify performance bottlenecks, resolve operational challenges, and refine your deployment processes.
| Pipeline Visibility | Platform | Definition |
|---|---|---|
| Manual steps | Manual steps | View manually triggered pipelines. |
| Queue time | Queue time | View the amount of time pipeline jobs sit in the queue before processing. |
| Logs correlation | Logs correlation | Correlate pipeline spans to logs and enable job logs collection. |
| Infrastructure metric correlation | Infrastructure metric correlation | Correlate jobs to infrastructure host metrics for Jenkins workers. |
| Custom spans | Custom spans | Configure custom spans for your pipelines. |
| Custom pre-defined tags | Custom pre-defined tags | Set custom tags to all generated pipeline, stages, and job spans. |
| Custom tags and measures at runtime | Custom tags and measures at runtime | Configure custom tags and measures at runtime. |
| Parameters | Parameters | Set custom parameters (such as the default branch name or Git information) when a pipeline is triggered. |
| Pipeline failure reasons | Pipeline failure reasons | Identify pipeline failure reasons from error messages. |
| Running pipelines | Running pipelines | View pipeline executions that are running. Requires Jenkins plugin version >= 8.0.0 |
| Filter CI Jobs on the critical path | Filter CI Jobs on the critical path | Filter by jobs on the critical path. |
| Execution time | Execution time | View the amount of time pipelines have been running jobs. |
The following Jenkins versions are supported:
This integration supports both Agentless and Agent-based installation. Installing the Agent is required for infrastructure metrics correlation.
This table shows the mapping of concepts between Datadog CI Visibility and Jenkins:
| Datadog | Jenkins |
|---|---|
| Pipeline | Pipeline |
| Stage | Stage |
| Job | Step |
Skip this step if you do not need infrastructure metrics correlation.
Install Datadog Agent on your Jenkins controller node and on your worker nodes by following the Agent installation instructions.
If the Jenkins controller and the Datadog Agent have been deployed to a Kubernetes cluster, Datadog recommends using the Admission Controller, which automatically sets the DD_AGENT_HOST environment variable in the Jenkins controller pod to communicate with the local Datadog Agent.
If you want to report the logs of your Jenkins jobs to Datadog, make sure that custom logs collection over TCP is enabled and configured in the Agent.
If your Agent runs in a container, add the DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true environment variable to it and make sure the following ports are accessible by the Jenkins controller:
8125/udp8126/tcp10518/tcpInstall and enable the Datadog Jenkins plugin v3.1.0 or later:
There are several ways you can configure the Datadog Jenkins plugin.
8126.app.datadoghq.com) in the Datadog App hostname field.Use this option to make the Jenkins plugin report directly to Datadog without using the Datadog Agent. It requires an API Key.
app.datadoghq.com) in the Datadog App hostname field.If your Jenkins instance uses the Jenkins configuration-as-code plugin:
datadogGlobalConfiguration:unclassified:datadogGlobalConfiguration:datadogClientConfiguration:# Select the `Datadog Agent` modedatadogAgentConfiguration:# Configure Datadog Agent hostagentHost:'<your-agent-host>'# Configure Datadog Agent portagentPort:8125# (Optional) Configure logs collection port as configured in your Datadog AgentagentLogCollectionPort:10518# Configure traces collection portagentTraceCollectionPort:8126# Enable CI Visibility flagenableCiVisibility:true# (Optional) Configure your CI Instance nameciInstanceName:'jenkins'# (Optional) Configure the name of the host that you use to access Datadog UIdatadogAppHostname:'app.datadoghq.com'# (Optional) Enable logs collectioncollectBuildLogs:trueIf your Jenkins instance uses the Jenkins configuration-as-code plugin:
datadogGlobalConfiguration:unclassified:datadogGlobalConfiguration:datadogClientConfiguration:# Select the `Agentless` mode (using API key).datadogApiConfiguration:intake:datadogIntakeSite:# Configure your Datadog sitesite:''apiKey:datadogCredentialsApiKey:# Configure ID of Jenkins credentials that store your API keycredentialsId:'my-api-key-credentials-id'# Enable CI Visibility flagenableCiVisibility:true# (Optional) Configure your CI Instance nameciInstanceName:'jenkins'# (Optional) Configure the name of the host that you use to access Datadog UIdatadogAppHostname:''# (Optional) Enable logs collectioncollectBuildLogs:trueimport jenkins.model.Jenkins
import org.datadog.jenkins.plugins.datadog.DatadogGlobalConfiguration
import org.datadog.jenkins.plugins.datadog.configuration.DatadogAgentConfiguration
def jenkins = Jenkins.getInstance()
def datadog = jenkins.getDescriptorByType(DatadogGlobalConfiguration)
def agentHost = 'localhost' // Configure your Datadog Agent host
def agentPort = 8125
def agentLogCollectionPort = 10518 // (Optional) Configure logs collection port as configured in your Datadog Agent
def agentTraceCollectionPort = 8126 // Configure traces collection port
datadog.datadogClientConfiguration = new DatadogAgentConfiguration(agentHost, agentPort, agentLogCollectionPort, agentTraceCollectionPort)
datadog.datadogAppHostname = 'app.datadoghq.com' // the name of the host that you use to access Datadog UI
datadog.enableCiVisibility = true
datadog.collectBuildLogs = true // (Optional) Enable logs collection
datadog.ciInstanceName = 'jenkins' // (Optional) Set your CI Instance name
// Save config
datadog.save()
import hudson.util.Secret
import jenkins.model.Jenkins
import org.datadog.jenkins.plugins.datadog.DatadogGlobalConfiguration
import org.datadog.jenkins.plugins.datadog.configuration.DatadogApiConfiguration
import org.datadog.jenkins.plugins.datadog.configuration.api.intake.DatadogIntakeSite
import org.datadog.jenkins.plugins.datadog.configuration.api.intake.DatadogSite
import org.datadog.jenkins.plugins.datadog.configuration.api.key.DatadogTextApiKey
def jenkins = Jenkins.getInstance()
def datadog = jenkins.getDescriptorByType(DatadogGlobalConfiguration)
def site = new DatadogIntakeSite(DatadogSite.) // Pick your Datadog site
def apiKey = new DatadogTextApiKey(Secret.fromString('<YOUR_API_KEY>')) // or `new DatadogCredentialsApiKey('<YOUR_CREDENTIALS_ID>')`
datadog.datadogClientConfiguration = new DatadogApiConfiguration(site, apiKey)
datadog.datadogAppHostname = '' // the name of the host that you use to access Datadog UI
datadog.enableCiVisibility = true
datadog.collectBuildLogs = true // (Optional) Enable logs collection
datadog.ciInstanceName = 'jenkins' // (Optional) Set your CI Instance name
// Save config
datadog.save()
# Select the Datadog Agent mode
DATADOG_JENKINS_PLUGIN_REPORT_WITH=DSD
# Configure the Agent host
DATADOG_JENKINS_PLUGIN_TARGET_HOST=your-agent-host
# Configure the Traces Collection port (default 8126)
DATADOG_JENKINS_PLUGIN_TARGET_TRACE_COLLECTION_PORT=8126
# Enable CI Visibility
DATADOG_JENKINS_PLUGIN_ENABLE_CI_VISIBILITY=true
# (Optional) Configure your CI Instance name
DATADOG_JENKINS_PLUGIN_CI_VISIBILITY_CI_INSTANCE_NAME=jenkins
# (Optional) Configure Log Collection port as configured in your Datadog Agent
DATADOG_JENKINS_PLUGIN_TARGET_LOG_COLLECTION_PORT=10518
# (Optional) Enable logs collection
DATADOG_JENKINS_PLUGIN_COLLECT_BUILD_LOGS=true
# (Optional) Configure the name of the host that you use to access Datadog UI
DATADOG_JENKINS_PLUGIN_DATADOG_APP_HOSTNAME=app.datadoghq.com
# Select the Datadog Agent mode
DATADOG_JENKINS_PLUGIN_REPORT_WITH=HTTP
# Configure your Datadog site
DATADOG_JENKINS_PLUGIN_DATADOG_SITE=
# Configure your API key
DATADOG_JENKINS_PLUGIN_TARGET_API_KEY=your-api-key
# Enable CI Visibility
DATADOG_JENKINS_PLUGIN_ENABLE_CI_VISIBILITY=true
# (Optional) Configure your CI Instance name
DATADOG_JENKINS_PLUGIN_CI_VISIBILITY_CI_INSTANCE_NAME=jenkins
# (Optional) Enable logs collection
DATADOG_JENKINS_PLUGIN_COLLECT_BUILD_LOGS=true
# (Optional) Configure the name of the host that you use to access Datadog UI
DATADOG_JENKINS_PLUGIN_DATADOG_APP_HOSTNAME=
Job logs collection can be enabled optionally when configuring the Jenkins plugin (see previous section). Both Agentless and Agent-based options are supported.
Logs are billed separately from CI Visibility.
Log retention, exclusion, and indexes are configured in Log Management. Logs for Jenkins jobs can be identified by the source:jenkins tag.
If you are using Jenkins workers, you can correlate pipelines with the infrastructure that is running them. For this feature to work:
DD_CI_HOSTNAME in every Jenkins worker with the worker hostname.export DD_CI_HOSTNAME=my-hostname
If you are using Kubernetes to manage your Jenkins instances, add the DD_CI_HOSTNAME environment variable to the pod that executes the Jenkins job. The value of this environment variable depends on what you are using in your Datadog Agent daemonset when reporting the infrastructure metrics.
This is only required for Jenkins workers. For the Jenkins controller, the infrastructure metric correlation does not require additional actions.
Note: Infrastructure metric correlation is supported since Jenkins Plugin v5.0.0 or later.
This is an optional step that enables the collection of tests data using Test Optimization.
See the Test Optimization documentation for your language to make sure that the testing framework that you use is supported.
There are different ways to enable Test Optimization inside a Jenkins job or pipeline:
datadog step inside the pipeline script.For pipelines that spin up a Docker container to execute tests, you can only configure the SDK manually.
UI-based Test Optimization configuration is available in Datadog Jenkins plugin v5.6.0 or later.
This option is not suitable for pipelines that are configured entirely in Jenkinsfile (for example, Multibranch pipelines or pipelines from Organization Folder).
For these pipelines use declarative configuration with the datadog step (described in the next section).
To enable Test Optimization through the UI do the following:
datadog pipeline stepThis configuration option is available in Datadog Jenkins plugin v5.6.2 or later.
In declarative pipelines, add the step to a top-level options block like so:
pipeline {
agent any
options {
datadog(testOptimization: [
enabled: true,
serviceName: "my-service", // the name of service or library being tested
languages: ["JAVA"], // languages that should be instrumented (available options are "JAVA", "JAVASCRIPT", "PYTHON", "DOTNET", "RUBY")
additionalVariables: ["my-var": "value"] // additional tracer configuration settings (optional)
])
}
stages {
stage('Example') {
steps {
echo "Hello world."
}
}
}
}
In a scripted pipeline, wrap the relevant section with the datadog step like so:
datadog(testOptimization: [ enabled: true, serviceName: "my-service", languages: ["JAVASCRIPT"], additionalVariables: [:] ]) {
node {
stage('Example') {
echo "Hello world."
}
}
}
The other datadog settings, such as collectLogs or tags can be added alongside the testOptimization block.
Follow the manual Test Optimization configuration instructions that are specific to your language.
Some features of the Datadog Jenkins plugin need Git information associated with the Jenkins builds to work correctly.
The minimum required Git info for a build is repository URL, branch, commit SHA, and commit author email. This information can be determined by the plugin automatically, propagated from SCM, provided manually with environment variables, or obtained by combining these approaches.
Note: If a pipeline checks out multiple repositories, Git info of the repositories that are checked out later in the pipeline has higher priority.
The Jenkins plugin is capable of automatically detecting Git information associated with a build or a pipeline. However, depending on the Jenkins version and the pipeline details, there may be cases when automatic Git data detection is not possible.
In this case you can make the Git information available to the plugin by using the .each {k,v -> env.setProperty(k, v)} function after executing the checkout or git steps. For example:
If you’re using a declarative pipeline to configure your pipeline, propagate Git information using a script block as follows:
Using the checkout step:
pipeline {
stages {
stage('Checkout') {
script {
checkout(...).each {k,v -> env.setProperty(k,v)}
}
}
...
}
}
Using the git step:
pipeline {
stages {
stage('Checkout') {
script {
git(...).each {k,v -> env.setProperty(k,v)}
}
}
...
}
}
If you’re using a scripted pipeline to configure your pipeline, you can propagate the git information to environment variables directly.
Using the checkout step:
node {
stage('Checkout') {
checkout(...).each {k,v -> env.setProperty(k,v)}
}
...
}
Using the git step:
node {
stage('Checkout') {
git(...).each {k,v -> env.setProperty(k,v)}
}
...
}
In case the plugin cannot detect Git information automatically and propagating Git data via SCM is not an option, the necessary Git information can be set manually.
To do so, set the following environment variables.
Note: These variables are optional, but if they are set, they take precedence over the Git information set by other Jenkins plugins.
DD_GIT_REPOSITORY_URL (Optional)https://github.com/my-org/my-repo.gitDD_GIT_BRANCH (Optional)mainDD_GIT_TAG (Optional)0.1.0DD_GIT_COMMIT_SHA (Optional)faaca5c59512cdfba9402c6e67d81b4f5701d43cDD_GIT_COMMIT_MESSAGE (Optional)Initial commit messageDD_GIT_COMMIT_AUTHOR_NAME (Optional)John SmithDD_GIT_COMMIT_AUTHOR_EMAIL (Optional)john@example.comDD_GIT_COMMIT_AUTHOR_DATE (Optional)2021-08-16T15:41:45.000ZDD_GIT_COMMIT_COMMITTER_NAME (Optional)Jane SmithDD_GIT_COMMIT_COMMITTER_EMAIL (Optional)jane@example.comDD_GIT_COMMIT_COMMITTER_DATE (Optional)2021-08-16T15:41:45.000ZIf you set only repository, branch and commit, the plugin will try to extract the rest of the Git information from the .git folder.
An example of usage:
pipeline {
agent any
stages {
stage('Checkout') {
steps {
script {
def gitVars = git url:'https://github.com/my-org/my-repo.git', branch:'some/feature-branch'
// Setting Git information manually via environment variables.
env.DD_GIT_REPOSITORY_URL=gitVars.GIT_URL
env.DD_GIT_BRANCH=gitVars.GIT_BRANCH
env.DD_GIT_COMMIT_SHA=gitVars.GIT_COMMIT
}
}
}
stage('Test') {
steps {
// Execute the rest of the pipeline.
}
}
}
}You can configure the Jenkins Plugin to include or exclude specific pipelines:
DATADOG_JENKINS_PLUGIN_EXCLUDEDsusans-job,johns-.*,prod_folder/prod_releaseDATADOG_JENKINS_PLUGIN_INCLUDEDsusans-job,johns-.*,prod_folder/prod_releaseLists of included and excluded jobs can contain regular expressions, but not glob patterns. To include a job with a specific prefix, use prefix-.*—not prefix-*.
To report pipeline results, attach the default branch name (for example, main) to pipeline spans in an attribute called git.default_branch. This is usually done automatically, but in some cases the plugin cannot extract this information because it might not be provided by Jenkins.
If this happens, set the default branch manually using the DD_GIT_DEFAULT_BRANCH environment variable in your build. For example:
pipeline {
agent any
environment {
DD_GIT_DEFAULT_BRANCH = 'main'
...
}
stages {
...
}
}The Datadog plugin adds a datadog step that allows adding custom tags to your pipeline-based jobs.
In declarative pipelines, add the step to a top-level option block:
def DD_TYPE = "release"
pipeline {
agent any
options {
datadog(tags: ["team:backend", "type:${DD_TYPE}", "${DD_TYPE}:canary"])
}
stages {
stage('Example') {
steps {
echo "Hello world."
}
}
}
}In scripted pipelines, wrap the relevant section with the datadog step:
datadog(tags: ["team:backend", "release:canary"]){
node {
stage('Example') {
echo "Hello world."
}
}
}To display and filter the teams associated with your pipelines, add team:<your-team> as a custom tag. The custom tag name must match your Datadog Teams team handle exactly.
You can configure the Jenkins Plugin to send custom tags (like global tags and global job tags) in all pipeline traces:
DATADOG_JENKINS_PLUGIN_GLOBAL_TAGSkey1:value1,key2:${SOME_ENVVAR},${OTHER_ENVVAR}:value3$ symbol.DATADOG_JENKINS_PLUGIN_GLOBAL_JOB_TAGS(.*?)_job_(.*?)_release, owner:$1, release_env:$2, optional:Tag3Once the integration is successfully configured, both the CI Pipeline List and Executions pages populate with data after pipelines finish.
The CI Pipeline List page shows data for only the default branch of each repository. For more information, see Search and Manage CI Pipelines.
When reporting an issue to the Datadog support team, generate a plugin diagnostic flare and provide it along with the issue description.
To generate the flare do the following:
If you have any issues with the Datadog Plugin, you can set the logs for the plugin in DEBUG level. Using this level you are able to see stacktrace details if an exception is thrown.
org.datadog.jenkins.plugins.datadog.clients -> Log Level ALLorg.datadog.jenkins.plugins.datadog.traces -> Log Level ALLorg.datadog.jenkins.plugins.datadog.logs -> Log Level ALLorg.datadog.jenkins.plugins.datadog.model -> Log Level ALLorg.datadog.jenkins.plugins.datadog.listeners -> Log Level ALLYou may also want to split the loggers into different log recorders.
Once the log recorders are successfully configured, you can check the logs in the DEBUG mode by accessing the desired log recorder through Manage Jenkins > System log.
If you trigger a Jenkins pipeline, you can search for the message Send pipeline traces in the Datadog Plugin Logs. This message indicates that the plugin is sending CI Visibility data to the Datadog Agent.
Send pipeline traces.
...
Send pipeline traces.
...If your Jenkins instance is behind an HTTP proxy, go to Manage Jenkins > Manage Plugins > Advanced tab and make sure the proxy configuration is correct:
| Datadog site | Host value |
|---|---|
| US1 | datadoghq.com |
| US3 | us3.datadoghq.com |
| US5 | us5.datadoghq.com |
| EU1 | datadoghq.eu |
| AP1 | ap1.datadoghq.com |
If the following error message appears in the Jenkins Log, make sure that the plugin configuration is correct.
Error writing to serverIf you are using localhost as the hostname, change it to the server hostname instead.
If the Datadog plugin is configured to send data to a Datadog Agent, do the following:
If the Datadog Plugin section does not appear in Jenkins configuration section, make sure that the plugin is enabled. To do so:
/safeRestart URL path.If the CI Visibility option does not appear in the Datadog Plugin section, make sure that the correct version is installed and restart the Jenkins instance. To do so:
/safeRestart URL path.Make sure you have gone through the steps to correlate infrastructure metrics with Jenkins pipelines.
If, even after following the steps, infrastructure metrics are still not correlated with Jenkins pipelines, try restarting the Jenkins instance.
Additional helpful documentation, links, and articles:
| |