Set Up Test Parallelization

Docs > Test Optimization in Datadog > Test Parallelization > Set Up Test Parallelization

Join the Preview!

Test Parallelization is in Preview. Complete the form to request access.

Request Access

Prerequisites

Before setting up Test Parallelization:

Set up Test Optimization.
Use the datadog-ci gem version 1.31.0 or later.
Enable Test Impact Analysis for the test service when you want Test Parallelization to split only the tests affected by a code change.

Concepts

Runner: A program that runs tests. ddtest can run tests directly or write file lists for another runner.
CI node: One CI execution environment, such as a GitHub Actions job, CircleCI parallel container, Kubernetes pod, VM, or local machine.
Worker: A process started by ddtest to execute tests. One CI node can run one worker or multiple workers.
Plan: The generated .testoptimization/ directory. It contains the runnable test files, the selected parallelism, and per-node file lists used by ddtest run or another runner.
Selected parallelism: The CI node count or local worker count that ddtest chooses after estimating test file durations.

Install ddtest

Install the ddtest CLI in your CI job. Datadog publishes precompiled binaries in GitHub Releases.

- name:Download ddtest binaryrun:| mkdir -p bin
 gh release download --repo DataDog/ddtest --pattern "ddtest-linux-amd64" --dir bin
 mv bin/ddtest-linux-amd64 bin/ddtest
 chmod +x bin/ddtestenv:GH_TOKEN:${{ github.token }}

mkdir -p bin
curl -fsSL https://github.com/DataDog/ddtest/releases/latest/download/ddtest-linux-amd64 -o bin/ddtest
chmod +x bin/ddtest

Run on a single CI node

If you run your tests on a single CI node, run ddtest run:

bin/ddtest run --platform ruby --framework rspec

By default, ddtest can start one worker for each physical CPU core available on the node.

Run across multiple CI nodes

For multiple CI nodes, run ddtest plan once, share the .testoptimization/ directory with every CI node, and pass each node its zero-indexed CI node number:

bin/ddtest plan \
 --platform ruby \
 --framework rspec \
 --min-parallelism 1 \
 --max-parallelism 8
bin/ddtest run \
 --platform ruby \
 --framework rspec \
 --ci-node <CI_NODE_INDEX>

In CI-node mode, ddtest uses one local worker by default. To start multiple workers in each CI node, set --ci-node-workers to a positive integer or ncpu.

For a list of available environment variables, defaults, and examples, see Configuration.

CI examples

Use the following examples as starting points for GitHub Actions and CircleCI.

The plan job chooses the CI node count and emits a matrix. The test job downloads the .testoptimization/ artifact and runs only the files assigned to its matrix node.

name:CI with Test Parallelizationon:[push]env:DD_TEST_OPTIMIZATION_RUNNER_PLATFORM:rubyDD_TEST_OPTIMIZATION_RUNNER_FRAMEWORK:rspecDD_TEST_OPTIMIZATION_RUNNER_MIN_PARALLELISM:1DD_TEST_OPTIMIZATION_RUNNER_MAX_PARALLELISM:8jobs:dd_plan:runs-on:ubuntu-latestoutputs:matrix:${{ steps.dd_plan.outputs.matrix }}steps:- uses:actions/checkout@v4- name:Download ddtest binaryrun:| mkdir -p bin
 gh release download --repo DataDog/ddtest --pattern "ddtest-linux-amd64" --dir bin
 mv bin/ddtest-linux-amd64 bin/ddtest
 chmod +x bin/ddtestenv:GH_TOKEN:${{ github.token }}- name:Setup Rubyuses:ruby/setup-ruby@v1with:bundler-cache:true- name:Configure Datadog Test Optimizationuses:datadog/test-visibility-github-action@v2with:languages:rubyapi_key:${{ secrets.DD_API_KEY }}site:datadoghq.com- id:dd_planname:Plan test executionrun:bin/ddtest plan- uses:actions/upload-artifact@v4with:name:dd-artifactspath:.testoptimizationinclude-hidden-files:truedd_test:runs-on:ubuntu-latestneeds:[dd_plan]strategy:fail-fast:falsematrix:${{ fromJson(needs.dd_plan.outputs.matrix) }}steps:- uses:actions/checkout@v4- name:Download ddtest binaryrun:| mkdir -p bin
 gh release download --repo DataDog/ddtest --pattern "ddtest-linux-amd64" --dir bin
 mv bin/ddtest-linux-amd64 bin/ddtest
 chmod +x bin/ddtestenv:GH_TOKEN:${{ github.token }}- uses:actions/download-artifact@v4with:name:dd-artifactspath:.testoptimization- name:Setup Rubyuses:ruby/setup-ruby@v1with:bundler-cache:true- name:Configure Datadog Test Optimizationuses:datadog/test-visibility-github-action@v2with:languages:rubyapi_key:${{ secrets.DD_API_KEY }}site:datadoghq.com- name:Run testsrun:bin/ddtest run --ci-node ${{ matrix.ci_node_index }}

The setup workflow runs ddtest plan, stores .testoptimization/, and continues into a test workflow with the selected CI node count.

In .circleci/config.yml:

version:"2.1"setup:trueorbs:ruby:circleci/ruby@2test-optimization-circleci-orb:datadog/test-optimization-circleci-orb@1continuation:circleci/continuation@0.2.0jobs:plan:docker:- image:cimg/ruby:3.4.1steps:- checkout- ruby/install-deps- test-optimization-circleci-orb/autoinstrument:languages:rubysite:datadoghq.com- run:name:Download ddtestcommand:| mkdir -p bin
 curl -fsSL https://github.com/DataDog/ddtest/releases/latest/download/ddtest-linux-amd64 -o bin/ddtest
 chmod +x bin/ddtest- run:name:Plan testscommand:bin/ddtest plan --platform ruby --framework rspecenvironment:DD_TEST_OPTIMIZATION_RUNNER_MIN_PARALLELISM:1DD_TEST_OPTIMIZATION_RUNNER_MAX_PARALLELISM:8- save_cache:key:ddtest-plan-{{ .Revision }}paths:- .testoptimization- bin/ddtest- run:name:Continue with selected parallelismcommand:| desired=$(cat .testoptimization/runner/parallel-runners.txt 2>/dev/null || echo 1)
 printf '{"parallelism": %s}\n' "${desired}" > pipeline-parameters.json- continuation/continue:configuration_path:.circleci/test.ymlparameters:pipeline-parameters.jsonworkflows:plan:jobs:- plan

In .circleci/test.yml:

version:"2.1"parameters:parallelism:type:integerdefault:1orbs:ruby:circleci/ruby@2test-optimization-circleci-orb:datadog/test-optimization-circleci-orb@1jobs:test:parallelism:<< pipeline.parameters.parallelism >>docker:- image:cimg/ruby:3.4.1steps:- checkout- restore_cache:keys:- ddtest-plan-{{ .Revision }}- ruby/install-deps- test-optimization-circleci-orb/autoinstrument:languages:rubysite:datadoghq.com- run:name:Run testscommand:| export DD_TEST_SESSION_NAME="ruby-tests-${CIRCLE_NODE_INDEX:-0}"
 bin/ddtest run --platform ruby --framework rspec --ci-node "${CIRCLE_NODE_INDEX:-0}"workflows:test:jobs:- test

Use third-party test runners

Use ddtest plan files when you want ddtest to choose which files should run, but another runner should execute them.

To learn about the full contents of the plan directory, see Plan artifacts.

File	Use
`.testoptimization/runner/test-files.txt`	All runnable test files after Test Impact Analysis skips are applied.
`.testoptimization/runner/tests-split/runner-N`	Files assigned to CI node or worker `N`.

For example, use .testoptimization/runner/test-files.txt with Knapsack Pro:

KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE=.testoptimization/runner/test-files.txt bundle exec rake knapsack_pro:queue:rspec

URL: https://docs.datadoghq.com/tests/test_parallelization/setup/