Support test automation exclusions in ScalaTest

[calvinbayer]

What Does This Do

Support test automation exclusions in ScalaTest. Wire test declaration metadata into Datadog policy selection and reporting, preserving assigned Test Management properties.

Motivation

Allow individual tests to prevent Datadog quarantine/disabling and exclude Datadog ATR, EFD and Attempt to Fix. Developer selection and framework retries retain their existing semantics.

Additional Notes

Stacked on codex/karate-automation-exclusions; review this PR against its base for the framework-specific change. Shared public API and policy are introduced by the first JUnit4 PR. No refactor commit was needed.

Validation: Module spotlessApply and instrumented ScalatestTest exclusion regression passed, including protected one-time failure, assignment tags, and developer-ignored test tags.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	55.52	56.55	$\color{green}{\blacktriangledown}$ -1.03	56.55	$\color{green}{\blacktriangledown}$ -1.03	131/371
agentEvpProxy	54.88	n/a	n/a	n/a	n/a	-

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	37.57	37.15	$\color{red}{\blacktriangle}$ +0.42	37.15	$\color{red}{\blacktriangle}$ +0.42	64/197
agentless	37.14	37.15	$\color{green}{\blacktriangledown}$ -0.01	36.42	$\color{red}{\blacktriangle}$ +0.72	64/170
agentlessCodeCoverage	45.90	46.30	$\color{green}{\blacktriangledown}$ -0.40	45.38	$\color{red}{\blacktriangle}$ +0.52	62/167
agentlessLineCoverage	56.16	56.55	$\color{green}{\blacktriangledown}$ -0.39	56.55	$\color{green}{\blacktriangledown}$ -0.39	61/166

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - pass4s

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	5.56	10.54	$\color{green}{\blacktriangledown}$ -4.98	10.33	$\color{green}{\blacktriangledown}$ -4.77	67/197
agentless	10.44	9.92	$\color{red}{\blacktriangle}$ +0.52	9.92	$\color{red}{\blacktriangle}$ +0.52	66/167
agentlessCodeCoverage	16.06	17.38	$\color{green}{\blacktriangledown}$ -1.32	17.73	$\color{green}{\blacktriangledown}$ -1.67	59/153

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - reactive-streams-jvm

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	-74.56	22.54	$\color{green}{\blacktriangledown}$ -97.10	22.54	$\color{green}{\blacktriangledown}$ -97.10	80/214
agentless	-74.59	19.59	$\color{green}{\blacktriangledown}$ -94.18	19.59	$\color{green}{\blacktriangledown}$ -94.18	77/179
agentlessCodeCoverage	-73.74	20.39	$\color{green}{\blacktriangledown}$ -94.13	19.99	$\color{green}{\blacktriangledown}$ -93.73	77/179
agentlessLineCoverage	-73.09	27.52	$\color{green}{\blacktriangledown}$ -100.61	27.52	$\color{green}{\blacktriangledown}$ -100.61	47/121

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	89.65	89.58	$\color{red}{\blacktriangle}$ +0.07	89.58	$\color{red}{\blacktriangle}$ +0.07	66/200
agentless	84.91	82.69	$\color{red}{\blacktriangle}$ +2.22	82.69	$\color{red}{\blacktriangle}$ +2.22	66/171
agentlessCodeCoverage	97.41	97.04	$\color{red}{\blacktriangle}$ +0.37	97.04	$\color{red}{\blacktriangle}$ +0.37	65/170
agentlessLineCoverage	113.93	113.88	$\color{red}{\blacktriangle}$ +0.05	113.88	$\color{red}{\blacktriangle}$ +0.05	65/170

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - heliboard

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	10.81	9.92	$\color{red}{\blacktriangle}$ +0.89	9.92	$\color{red}{\blacktriangle}$ +0.89	65/170

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	11.84	13.40	$\color{green}{\blacktriangledown}$ -1.56	13.40	$\color{green}{\blacktriangledown}$ -1.56	66/201
agentless	10.75	12.12	$\color{green}{\blacktriangledown}$ -1.37	12.12	$\color{green}{\blacktriangledown}$ -1.37	66/171
agentlessCodeCoverage	13.34	15.72	$\color{green}{\blacktriangledown}$ -2.38	15.41	$\color{green}{\blacktriangledown}$ -2.07	64/169
agentlessLineCoverage	14.88	17.38	$\color{green}{\blacktriangledown}$ -2.50	17.38	$\color{green}{\blacktriangledown}$ -2.50	61/166

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - jolokia

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	-88.21	95.12	$\color{green}{\blacktriangledown}$ -183.33	95.12	$\color{green}{\blacktriangledown}$ -183.33	67/200
agentless	-88.24	91.39	$\color{green}{\blacktriangledown}$ -179.63	89.58	$\color{green}{\blacktriangledown}$ -177.82	66/171
agentlessCodeCoverage	-87.49	101.00	$\color{green}{\blacktriangledown}$ -188.49	99.00	$\color{green}{\blacktriangledown}$ -186.49	67/172
agentlessLineCoverage	-87.29	103.04	$\color{green}{\blacktriangledown}$ -190.33	101.00	$\color{green}{\blacktriangledown}$ -188.29	75/186

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - okhttp

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	18.06	19.20	$\color{green}{\blacktriangledown}$ -1.14	19.59	$\color{green}{\blacktriangledown}$ -1.53	68/201
agentless	17.24	18.45	$\color{green}{\blacktriangledown}$ -1.21	19.20	$\color{green}{\blacktriangledown}$ -1.96	68/177
agentlessCodeCoverage	21.13	21.65	$\color{green}{\blacktriangledown}$ -0.52	22.54	$\color{green}{\blacktriangledown}$ -1.41	70/181
agentlessLineCoverage	37.38	37.91	$\color{green}{\blacktriangledown}$ -0.53	39.45	$\color{green}{\blacktriangledown}$ -2.07	64/172

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.99 s	13.92 s	[-0.2%; +1.2%] (no difference)
startup:insecure-bank:tracing:Agent	12.86 s	13.00 s	[-1.6%; -0.5%] (maybe better)
startup:petclinic:appsec:Agent	17.04 s	16.82 s	[+0.4%; +2.2%] (maybe worse)
startup:petclinic:iast:Agent	16.46 s	16.26 s	[-4.8%; +7.3%] (unstable)
startup:petclinic:profiling:Agent	16.69 s	16.58 s	[-0.5%; +1.9%] (no difference)
startup:petclinic:sca:Agent	16.40 s	16.88 s	[-8.0%; +2.4%] (unstable)
startup:petclinic:tracing:Agent	16.11 s	16.16 s	[-1.1%; +0.5%] (no difference)

Commit: 22ed8f1e · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - spring_boot

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	16.67	16.36	$\color{red}{\blacktriangle}$ +0.31	16.36	$\color{red}{\blacktriangle}$ +0.31	66/198
agentless	9.57	9.92	$\color{green}{\blacktriangledown}$ -0.35	9.92	$\color{green}{\blacktriangledown}$ -0.35	66/170
agentlessCodeCoverage	13.67	13.94	$\color{green}{\blacktriangledown}$ -0.27	13.67	$\color{gray}{\bullet}$ 0.00	66/172
agentlessLineCoverage	23.36	22.99	$\color{red}{\blacktriangle}$ +0.37	22.54	$\color{red}{\blacktriangle}$ +0.82	65/170

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-19T22:17:34.732623Z	1273e6b	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sonar-java

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	50.17	13.94	$\color{red}{\blacktriangle}$ +36.23	13.94	$\color{red}{\blacktriangle}$ +36.23	66/197
agentless	50.69	12.62	$\color{red}{\blacktriangle}$ +38.07	12.87	$\color{red}{\blacktriangle}$ +37.82	66/171
agentlessCodeCoverage	97.47	86.07	$\color{red}{\blacktriangle}$ +11.40	82.69	$\color{red}{\blacktriangle}$ +14.78	66/171
agentlessLineCoverage	125.01	120.92	$\color{red}{\blacktriangle}$ +4.09	123.36	$\color{red}{\blacktriangle}$ +1.65	61/167

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[datadog-prod-us1-4]

Running a ScalaTest test by name bypasses declaration-tag collection, allowing Datadog retries and failure suppression despite explicit exclusions.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 1273e6b}