Code Coverage in Datadog

Compatibility

• dd-trace>=4.45.0 and dd-trace>=5.21.0.
• jest>=24.8.0, only when run with jest-circus.
• mocha>=5.2.0.
• cucumber-js>=7.0.0.
• vitest>=2.0.0.

Mocha/Cucumber-js

Only Istanbul code coverage is supported for mocha and cucumber-js.

To report total code coverage from your mocha and cucumber-js test sessions, install nyc and wrap your test commands:

1. Install nyc:

npm install --save-dev nyc

2. Wrap your test commands with nyc:

{
 "scripts": {
 "test": "mocha",
 "coverage": "nyc npm run test"
 }
}

Jest

Jest includes Istanbul by default, so you don’t need to install nyc. Simply pass --coverage:

{
 "scripts": {
 "coverage": "jest --coverage"
 }
}

The only supported coverageProvider is babel, which is the default.

Vitest

Vitest requires extra dependencies for running with code coverage. See vitest docs for more information. After the dependencies are installed, pass --coverage to your test command:

{
 "scripts": {
 "coverage": "vitest run --coverage"
 }
}

After modifying your test commands, run your tests with the new coverage command:

NODE_OPTIONS="-r dd-trace/ci/init" DD_ENV=ci DD_SERVICE=my-javascript-service npm run coverage

Compatibility

• dd-trace>=2.31.0.

When code coverage is available, the Datadog Tracer (v2.31.0 or later) reports it under the test.code_coverage.lines_pct tag for your test sessions.

If you are using Coverlet to compute your code coverage, indicate the path to the report file in the DD_CIVISIBILITY_EXTERNAL_CODE_COVERAGE_PATH environment variable when running dd-trace. The report file must be in the OpenCover or Cobertura formats. Alternatively, you can enable the Datadog Tracer’s built-in code coverage calculation with the DD_CIVISIBILITY_CODE_COVERAGE_ENABLED=true environment variable.

Note: DD_CIVISIBILITY_EXTERNAL_CODE_COVERAGE_PATH is only used when the command instrumented by dd-trace ci run is dotnet test, dotnet vstest, or vstest.console. For example, the results in the variable are ignored when running dd-trace ci run -- coverlet TestAssembly.dll --target dotnet --targetargs "test TestAssembly.dll".

Advanced options

The Datadog Tracer’s built-in code coverage has support for both Coverlet and VS Code Coverage options through the .runsettings file.

File structure

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
 <DataCollectionRunSettings>
 <DataCollectors>
 <DataCollector friendlyName="DatadogCoverage">
 <Configuration>
 <!-- Datadog Code Coverage settings -->
 ...
 </Configuration>
 </DataCollector>
 </DataCollectors>
 </DataCollectionRunSettings>
</RunSettings>

Coverlet options

Attributes

You can exclude a method, an entire class, or assembly from code coverage by creating and applying the ExcludeFromCodeCoverage attribute present in the System.Diagnostics.CodeAnalysis namespace.

Exclude additional attributes with the ExcludeByAttribute property and the short name of the attribute (the type name without the namespace).

Source files

Exclude specific source files from code coverage with the ExcludeByFile property.

• Use a single or multiple paths, separated by comma.
• Use the file path or directory path with a wildcard (*), for example: dir1/*.cs.

Filters

Filters provide fine-grained control over what gets excluded using filter expressions with the following syntax:

[<ASSEMBLY_FILTER>]<TYPE_FILTER>

Wildcards are supported:

• * => matches zero or more characters
• ? => the prefixed character is optional

Examples:

• [*]* => Excludes all types in all assemblies (nothing is instrumented)
• [coverlet.*]Coverlet.Core.Coverage => Excludes the Coverage class in the Coverlet.Core namespace belonging to any assembly that matches coverlet.* (for example, coverlet.core)
• [*]Coverlet.Core.Instrumentation.* => Excludes all types belonging to the Coverlet.Core.Instrumentation namespace in any assembly
• [coverlet.*.tests?]* => Excludes all types in any assembly starting with coverlet. and ending with .test or .tests (the ? makes the s optional)
• [coverlet.*]*,[*]Coverlet.Core*\ => Excludes assemblies matching coverlet.* and excludes all types belonging to the Coverlet.Core namespace in any assembly

VS code coverage options

See Customize code coverage analysis in the Microsoft documentation for additional information.

Runsettings example

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
 <DataCollectionRunSettings>
 <DataCollectors>
 <DataCollector friendlyName="DatadogCoverage">
 <Configuration>
 <!-- Coverlet configuration -->
 <ExcludeByAttribute>CompilerGeneratedAttribute</ExcludeByAttribute>
 <ExcludeByFile>**/Fibonorial.cs</ExcludeByFile>
 <Exclude>[myproject.*.tests?]*</Exclude>
 <!-- VS Code Coverage configuration -->
 <CodeCoverage>
 <Attributes>
 <Exclude>
 <Attribute>^System\.ObsoleteAttribute$</Attribute>
 </Exclude>
 </Attributes>
 <Sources>
 <Exclude>
 <Source>^MyFile\.cs$</Source>
 </Exclude>
 </Sources>
 </CodeCoverage>
 </Configuration>
 </DataCollector>
 </DataCollectors>
 </DataCollectionRunSettings>
</RunSettings>

Compatibility

• dd-trace-java >= 1.24.2.

When code coverage is available, the Datadog Tracer reports it under the test.code_coverage.lines_pct tag for your test sessions.

Jacoco is supported as a code coverage library.

If your project already has Jacoco configured, the Datadog Tracer instruments it and reports the coverage data to Datadog automatically.

Otherwise, you can configure the SDK to add Jacoco to your test runs at runtime. Use DD_CIVISIBILITY_JACOCO_PLUGIN_VERSION environment variable to specify which version of Jacoco you want to have injected (for example: DD_CIVISIBILITY_JACOCO_PLUGIN_VERSION=0.8.11).

Compatibility

• dd-trace>=2.5.0.
• Python>=3.7.
• coverage>=4.4.2.
• pytest>=3.0.0.
• pytest-cov>=2.7.0.
• unittest>=3.8.
• Only coverage.py and pytest-cov code coverage are supported.

When tests are instrumented with coverage.py or pytest-cov, the Datadog Tracer reports code coverage under the test.code_coverage.lines_pct tag for your test sessions automatically.

To report total code coverage from your test sessions with coverage.py, follow these steps:

1. Install coverage:

python3 -m pip install coverage

2. Run your test with the new coverage command:

DD_ENV=ci DD_SERVICE=my-python-service coverage run -m pytest

Alternatively, to report total code coverage from your test sessions with pytest-cov, follow these steps:

1. Install pytest:

python3 -m pip install pytest

2. Install pytest-cov:

python3 -m pip install pytest-cov

3. Run your test by appending the --cov flag to your pytest command:

DD_ENV=ci DD_SERVICE=my-python-service pytest --cov

Compatibility

• datadog-ci-rb>=1.7.0
• simplecov>=0.18.0.

If your project has simplecov configured, the datadog-ci-rb library instruments it and reports the coverage data to Datadog automatically under the test.code_coverage.lines_pct tag for your test sessions.

This feature is enabled by default. Use DD_CIVISIBILITY_SIMPLECOV_INSTRUMENTATION_ENABLED environment variable to disable this feature (for example: DD_CIVISIBILITY_SIMPLECOV_INSTRUMENTATION_ENABLED=0).

Compatibility

• go test -cover

If your tests are executed with the -cover flag, the Datadog library instruments it and automatically reports the coverage data to Datadog under the test.code_coverage.lines_pct tag for your test sessions.

Compatibility

• dd-sdk-swift-testing>=2.5.3.
• Xcode>=14.3.

When code coverage is enabled, the Datadog Tracer reports it under the test.code_coverage.lines_pct tag for your test sessions.

To enable code coverage for Xcode projects you can follow this guide from Apple: Enable code coverage in your test plan.

For SPM tests, add the --enable-code-coverage parameter to your swift test invocation.

Compatibility

• datadog-ci>=2.17.2.

You can upload a code coverage percentage value when using JUnit Report uploads:

datadog-ci junit upload --service <service_name> --report-measures=test.code_coverage.lines_pct:85 <path>

In this example, 85 is the percentage of lines covered by your tests and needs to be generated with a different tool.

The code coverage report needs to be generated in a different process, otherwise the JUnit report uploads will not generate code coverage reports. The reported metric name must be test.code_coverage.lines_pct.