Send Custom Pipelines to Datadog

Overview

If your CI provider is not supported, you can send custom pipelines through HTTP using the public API endpoint. For more information about how pipeline executions are modeled, see Pipeline Data Model and Execution Types.

Compatibility

Configure CI Visibility

To send pipeline events programmatically to Datadog, ensure that your DD_API_KEY is configured.

1. Set the headers of your HTTP request:

   • DD-API-KEY: Your Datadog API key.
   • Content-Type: application/json.

2. Prepare the payload body by entering information about the pipeline execution in a cURL command:

   Parameter Name	Description	Example Value
   Unique ID	The UUID of the pipeline run. The ID has to be unique across retries and pipelines, including partial retries.	b3262537-a573-44eb-b777-4c0f37912b05
   Name	The name of the pipeline. All pipeline runs for the builds should have the same name.	Documentation Build
   Git Repository	The Git repository URL that triggered the pipeline.	https://github.com/Datadog/documentation
   Commit Author	The commit author email that triggered the pipeline.	contributor@github.com
   Commit SHA	The commit hash that triggered the pipeline.	cf852e17dea14008ac83036430843a1c
   Status	The final status of the pipeline. Allowed enum values: success, error, canceled, skipped, blocked, or running.	success
   Partial Retry	Whether or not the pipeline was a partial retry of a previous attempt. This field expects a boolean value (true or false). A partial retry is one which only runs a subset of the original jobs.	false
   Start	Time when the pipeline run started (it should not include any queue time). The time format must be RFC3339.	2024-08-22T11:36:29-07:00
   End	Time when the pipeline run finished. The time format must be RFC3339. The end time cannot be longer than 1 year after the start time.	2024-08-22T14:36:00-07:00
   URL	The URL to look at the pipeline in the CI provider UI.	http://your-ci-provider.com/pipeline/{pipeline-id}

   For example, this payload sends a CI pipeline event to Datadog:

    curl -X POST "https://api.datadoghq.com/api/v2/ci/pipeline" \
    -H "Content-Type: application/json" \
    -H "DD-API-KEY: <YOUR_API_KEY>" \
    -d @- << EOF
    {
    "data": {
    "attributes": {
    "provider_name": "<YOUR_CI_PROVIDER>",
    "resource": {
    "level": "pipeline",
    "unique_id": "b3262537-a573-44eb-b777-4c0f37912b05",
    "name": "Documentation Build",
    "git": {
    "repository_url": "https://github.com/Datadog/documentation",
    "author_email": "contributor@github.com",
    "sha": "cf852e17dea14008ac83036430843a1c"
    },
    "status": "success",
    "start": "2024-08-22T11:36:29-07:00",
    "end": "2024-08-22T14:36:00-07:00",
    "partial_retry": false,
    "url": ""
    }
    },
    "type": "cipipeline_resource_request"
    }
    }
    EOF
   

3. After sending your pipeline event to Datadog, you can integrate additional event types such as stage, job, and step. For more information, see the Send Pipeline Event endpoint.

Running pipelines

Pipeline events sent with the status set to running have the same unique_id as the final pipeline event. Running pipelines can be updated by adding more information while they are still running. A running pipeline consists of the following events:

1. The initial running pipeline event with the status set to running.
2. Optionally, N running pipeline events that update the pipeline with more information, with the same unique_id and the status set to running.
3. The final pipeline event without a running status and the same unique_id.

Note: The most recent value may not always be the one displayed in the UI when a field is updated. For example, if the tag my_tag is set to value1 in the first running pipeline, and then is updated to value2, you may see value1 instead of value2 in the UI. It is recommended to only update running pipelines by adding more fields instead modifying existing ones.

Running jobs

Job events can also be sent while a job is still running by setting the status to running. As with running pipelines, all events for the same job share the same id and pipeline_unique_id. A running job consists of the following events:

1. The initial running job event with the status set to running.
2. The final job event with a terminal status (such as success or error) and the same id and pipeline_unique_id.

A running job event does not require an end time. The end time is set when the final job event is sent.

Visualize pipeline data in Datadog

The CI Pipeline List and Executions pages populate with data after the pipelines are accepted for processing.

The CI Pipeline List page shows data for only the default branch of each repository. For more information, see Search and Manage CI Pipelines.

Further reading

Request a personalized demo