Test JDK 27 as a non-default JVM

[sarahchen6]

What Does This Do

Run EA JDK 27 as a non-default JVM for the main CI test-suite, for arm64 tests, and for debugger tests. Skip all tests that fail on 27 for now.

Motivation

Start preparing to support JDK 27 which GAs mid-September 2026.

Additional Notes

Non-default JVM GitLab pipeline run that includes JDK 27 tests: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-java/-/pipelines/123389323

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Add your completed PR to the merge queue by commenting /merge. You can also:
  • Customize the commit message associated with the merge with /merge --commit-message "..."
  • Remove your PR from the merge queue with /merge -c
  • Skip all merge queue checks with /merge -f --reason "reason"; please use this judiciously, as some checks do not run at the PR-level (note: the PR still needs to be mergeable, this will only skip the pre-merge build)
  • Get more information in this doc

Jira ticket: [PROJ-IDENT]

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.03 s	13.91 s	[+0.2%; +1.5%] (maybe worse)
startup:insecure-bank:tracing:Agent	12.96 s	12.97 s	[-0.6%; +0.6%] (no difference)
startup:petclinic:appsec:Agent	17.52 s	17.29 s	[+0.5%; +2.3%] (maybe worse)
startup:petclinic:iast:Agent	17.50 s	17.58 s	[-1.2%; +0.3%] (no difference)
startup:petclinic:profiling:Agent	17.41 s	17.21 s	[-0.0%; +2.5%] (no difference)
startup:petclinic:sca:Agent	17.44 s	17.42 s	[-1.0%; +1.1%] (no difference)
startup:petclinic:tracing:Agent	16.53 s	16.70 s	[-2.2%; +0.2%] (no difference)

Commit: 847496ff · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[pr-commenter]

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1783460393	1783460739
end_time	2026-07-07T21:41:19	2026-07-07T21:47:05
git_branch	master	sarahchen6/test-jdk-27
git_commit_sha	f570544	847496f
start_time	2026-07-07T21:39:54	2026-07-07T21:45:40

See matching parameters

	Baseline	Candidate
ci_job_id	1839380238	1839380238
ci_pipeline_id	123393260	123393260
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1783459476	1783459476

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 5 unstable metrics.

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-16.601µs; +19.985µs] or [-5.691%; +6.852%]	unstable [-27.598µs; +33.002µs] or [-8.276%; +9.897%]	unstable [-35.995µs; +42.571µs] or [-10.343%; +12.232%]	unstable [-280.644µs; -81.841µs] or [-20.452%; -5.964%]	same
scenario:basic	same	same	same	unstable [-353.101µs; -66.328µs] or [-27.940%; -5.248%]	same
scenario:loop	unsure [-9.508µs; -2.838µs] or [-0.107%; -0.032%]	unsure [-13.016µs; -1.823µs] or [-0.145%; -0.020%]	unsure [-21.368µs; -9.987µs] or [-0.236%; -0.110%]	same	same

Request duration reports for reports

gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (333.448 µs) : 308, 359
.   : milestone, 333,
basic (296.962 µs) : 290, 304
.   : milestone, 297,
loop (8.989 ms) : 8984, 8994
.   : milestone, 8989,
section candidate
noprobe (336.151 µs) : 305, 368
.   : milestone, 336,
basic (297.074 µs) : 290, 304
.   : milestone, 297,
loop (8.982 ms) : 8976, 8987
.   : milestone, 8982,

Loading

• baseline results

Scenario	Request median duration [CI 0.99]
noprobe	333.448 µs [307.627 µs, 359.27 µs]
basic	296.962 µs [290.307 µs, 303.617 µs]
loop	8.989 ms [8.984 ms, 8.994 ms]

• candidate results

Scenario	Request median duration [CI 0.99]
noprobe	336.151 µs [304.646 µs, 367.655 µs]
basic	297.074 µs [289.73 µs, 304.419 µs]
loop	8.982 ms [8.976 ms, 8.987 ms]

[sarahchen6]

/merge -f --reason "Unblock master" --commit-message "Test JDK 27 as a non-default JVM"

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-07-08 12:29:52 UTC ℹ️ Start processing command /merge -f --reason "Unblock master" --commit-message "Test JDK 27 as a non-default JVM"

---

2026-07-08 12:29:57 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 0s (p90).

---

2026-07-08 12:30:08 UTC ℹ️ MergeQueue: This merge request was merged

Warning

This change was merged without running any pre merge CI checks

Reason: "Unblock master"

[datadog-datadog-prod-us1]

Missing global DEFAULT_TEST_JVMS update for JDK 27

Inconsistent test execution across CI matrix: amd64 test jobs will skip Java 27 on PR validation unless NON_DEFAULT_JVMS is explicitly set, while arm64 runs it by default. This defeats the stated goal of 'preparing to support JDK 27' — if tests don't run on PRs, integration issues won't be caught before master.

Assertion details

• Input: DEFAULT_TEST_JVMS variable at line 60 controls which JVMs run tests by default via CI rules: 'if: $testJvm =~ $DEFAULT_TEST_JVMS'. When Java 27 is added to .test_matrix (line 103), test jobs inheriting from .test_job without an explicit override will use the global DEFAULT_TEST_JVMS regex.
• Expected: Java 27 tests should run with consistent default treatment across all test jobs. The global DEFAULT_TEST_JVMS should either include 27 (matching the matrix), or all jobs using .test_matrix should explicitly override it.
• Actual: Global DEFAULT_TEST_JVMS at line 60 is /^(8|11|17|21|25|tip)$/ (no 27). When a job runs with testJvm=27, the CI rule 'if: $testJvm =~ $DEFAULT_TEST_JVMS' evaluates to false, causing the job to not run (or only run on master/mq branches). Individual overrides exist for .test_job_arm64 (line 748), muzzle-dep-report, and test_debugger, but base .test_job jobs without overrides inherit the stale global regex.

Was this helpful? React 👍 or 👎
_{🤖 Datadog Autotest · What is Autotest? · Any feedback? Reach out in #autotest}

[sarahchen6]

The reason we exclude it in DEFAULT_JVMs is to avoid extra CI running for PRs, though JDK 27 EA is still run for all tests on MQ and master pipelines. It's fine to have the JDK 27 EA image run extra, i.e. on PRs, especially for smaller test jobs such arm64 and debugger below.

[datadog-datadog-prod-us1]

This PR adds JDK 27 EA testing support but has a critical inconsistency: the global DEFAULT_TEST_JVMS variable (line 60) was not updated to include "27", causing amd64 tests to skip Java 27 on PRs while arm64 tests run it by default. Individual jobs (muzzle-dep-report, test_debugger, .test_job_arm64) were updated correctly, but the shared root variable controlling all .test_job rules was missed.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 847496f · What is Autotest? · Any feedback? Reach out in #autotest}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 847496ff4f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".