Pin system-tests to SHA with pinned download-artifact action by jandro996 · Pull Request #10889 · DataDog/dd-trace-java

jandro996 · 2026-03-18T14:46:41Z

Summary

Updates DataDog/system-tests pinned ref from 61f7ee29 to 50e03c1d ("Nccatoni/sha pin" Update the enablement logic for allocation and liveheap #6492)
The previous SHA used actions/download-artifact@v4 (non-SHA pin) internally, violating the DataDog enterprise GitHub Actions policy
The new SHA uses actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 (SHA-pinned), compliant with the policy
This fix unblocks all PRs targeting release/v1.60.x where "Check system tests success" was stuck as "Expected — Waiting"

Backport fix for #10722 to release/v1.60.x

🤖 Generated with Claude Code

github-actions · 2026-03-18T14:47:57Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

Update DataDog/system-tests ref from 61f7ee29e7432cba821a4d303c95929160bd23fd to ab6ccde3946ded935e7b1ffddf260b982275795e which pins actions/download-artifact to a full commit SHA, fixing the enterprise policy violation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

init Merge branch 'master' into mhlidd/update_telemetry_key_normalization Merge branch 'master' into mhlidd/update_telemetry_key_normalization Merge branch 'master' into mhlidd/update_telemetry_key_normalization Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>

pr-commenter · 2026-03-18T16:33:59Z

Kafka / producer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	release/v1.60.x	alejandro.gonzalez/fix-system-tests-sha-pin
git_commit_date	1773846743	1773849383
git_commit_sha	`c1e6a13`	`d96eff8`

See matching parameters

	Baseline	Candidate
ci_job_date	1773850540	1773850540
ci_job_id	1518071048	1518071048
ci_pipeline_id	103308887	103308887
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce	unsure [-61179.984op/s; -6599.424op/s] or [-3.335%; -0.360%]
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce	same

pr-commenter · 2026-03-18T16:45:35Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	alejandro.gonzalez/fix-system-tests-sha-pin
git_commit_date	1772492986	1773849383
git_commit_sha	`0f5d84f`	`d96eff8`
release_version	1.60.0~0f5d84f2c4	1.61.0-SNAPSHOT~d96eff8552

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773851228	1773851228
ci_job_id	1518071037	1518071037
ci_pipeline_id	103308887	103308887
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-yuvueikv 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-yuvueikv 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 11 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~d96eff8552, baseline=1.60.0~0f5d84f2c4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1067244
Total [baseline] (8.759 s) : 0, 8758621
Agent [candidate] (1.063 s) : 0, 1062839
Total [candidate] (8.716 s) : 0, 8715602
section iast
Agent [baseline] (1.239 s) : 0, 1238522
Total [baseline] (9.38 s) : 0, 9380108
Agent [candidate] (1.229 s) : 0, 1229376
Total [candidate] (9.348 s) : 0, 9347985

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.067 s	-
Agent	iast	1.239 s	171.278 ms (16.0%)
Total	tracing	8.759 s	-
Total	iast	9.38 s	621.488 ms (7.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.063 s	-
Agent	iast	1.229 s	166.537 ms (15.7%)
Total	tracing	8.716 s	-
Total	iast	9.348 s	632.383 ms (7.3%)

gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~d96eff8552, baseline=1.60.0~0f5d84f2c4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.2 ms) : 0, 1200
crashtracking [candidate] (1.214 ms) : 0, 1214
BytebuddyAgent [baseline] (628.277 ms) : 0, 628277
BytebuddyAgent [candidate] (626.037 ms) : 0, 626037
AgentMeter [baseline] (29.173 ms) : 0, 29173
AgentMeter [candidate] (29.059 ms) : 0, 29059
GlobalTracer [baseline] (258.518 ms) : 0, 258518
GlobalTracer [candidate] (257.253 ms) : 0, 257253
AppSec [baseline] (33.084 ms) : 0, 33084
AppSec [candidate] (32.954 ms) : 0, 32954
Debugger [baseline] (63.363 ms) : 0, 63363
Debugger [candidate] (62.688 ms) : 0, 62688
Remote Config [baseline] (610.677 µs) : 0, 611
Remote Config [candidate] (607.131 µs) : 0, 607
Telemetry [baseline] (12.196 ms) : 0, 12196
Telemetry [candidate] (9.839 ms) : 0, 9839
Flare Poller [baseline] (4.56 ms) : 0, 4560
Flare Poller [candidate] (6.931 ms) : 0, 6931
section iast
crashtracking [baseline] (1.22 ms) : 0, 1220
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (801.602 ms) : 0, 801602
BytebuddyAgent [candidate] (794.146 ms) : 0, 794146
AgentMeter [baseline] (11.551 ms) : 0, 11551
AgentMeter [candidate] (11.311 ms) : 0, 11311
GlobalTracer [baseline] (248.471 ms) : 0, 248471
GlobalTracer [candidate] (247.208 ms) : 0, 247208
AppSec [baseline] (34.137 ms) : 0, 34137
AppSec [candidate] (34.886 ms) : 0, 34886
Debugger [baseline] (65.47 ms) : 0, 65470
Debugger [candidate] (64.792 ms) : 0, 64792
Remote Config [baseline] (529.028 µs) : 0, 529
Remote Config [candidate] (532.073 µs) : 0, 532
Telemetry [baseline] (8.607 ms) : 0, 8607
Telemetry [candidate] (8.707 ms) : 0, 8707
Flare Poller [baseline] (3.476 ms) : 0, 3476
Flare Poller [candidate] (3.464 ms) : 0, 3464
IAST [baseline] (27.175 ms) : 0, 27175
IAST [candidate] (26.939 ms) : 0, 26939

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~d96eff8552, baseline=1.60.0~0f5d84f2c4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.072 s) : 0, 1071927
Total [baseline] (10.924 s) : 0, 10924136
Agent [candidate] (1.065 s) : 0, 1065086
Total [candidate] (10.764 s) : 0, 10763891
section appsec
Agent [baseline] (1.24 s) : 0, 1239929
Total [baseline] (11.079 s) : 0, 11078868
Agent [candidate] (1.236 s) : 0, 1236260
Total [candidate] (11.02 s) : 0, 11019922
section iast
Agent [baseline] (1.239 s) : 0, 1239255
Total [baseline] (11.178 s) : 0, 11177762
Agent [candidate] (1.23 s) : 0, 1230443
Total [candidate] (11.18 s) : 0, 11179623
section profiling
Agent [baseline] (1.2 s) : 0, 1200423
Total [baseline] (10.977 s) : 0, 10976985
Agent [candidate] (1.202 s) : 0, 1201528
Total [candidate] (11.094 s) : 0, 11094236

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.072 s	-
Agent	appsec	1.24 s	168.002 ms (15.7%)
Agent	iast	1.239 s	167.328 ms (15.6%)
Agent	profiling	1.2 s	128.496 ms (12.0%)
Total	tracing	10.924 s	-
Total	appsec	11.079 s	154.732 ms (1.4%)
Total	iast	11.178 s	253.625 ms (2.3%)
Total	profiling	10.977 s	52.849 ms (0.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	appsec	1.236 s	171.174 ms (16.1%)
Agent	iast	1.23 s	165.357 ms (15.5%)
Agent	profiling	1.202 s	136.443 ms (12.8%)
Total	tracing	10.764 s	-
Total	appsec	11.02 s	256.031 ms (2.4%)
Total	iast	11.18 s	415.732 ms (3.9%)
Total	profiling	11.094 s	330.345 ms (3.1%)

gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~d96eff8552, baseline=1.60.0~0f5d84f2c4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.196 ms) : 0, 1196
crashtracking [candidate] (1.188 ms) : 0, 1188
BytebuddyAgent [baseline] (631.289 ms) : 0, 631289
BytebuddyAgent [candidate] (627.417 ms) : 0, 627417
AgentMeter [baseline] (29.263 ms) : 0, 29263
AgentMeter [candidate] (29.07 ms) : 0, 29070
GlobalTracer [baseline] (259.097 ms) : 0, 259097
GlobalTracer [candidate] (257.946 ms) : 0, 257946
AppSec [baseline] (33.257 ms) : 0, 33257
AppSec [candidate] (33.133 ms) : 0, 33133
Debugger [baseline] (64.993 ms) : 0, 64993
Debugger [candidate] (65.937 ms) : 0, 65937
Remote Config [baseline] (611.213 µs) : 0, 611
Remote Config [candidate] (623.572 µs) : 0, 624
Telemetry [baseline] (10.689 ms) : 0, 10689
Telemetry [candidate] (9.93 ms) : 0, 9930
Flare Poller [baseline] (5.311 ms) : 0, 5311
Flare Poller [candidate] (3.713 ms) : 0, 3713
section appsec
crashtracking [baseline] (1.201 ms) : 0, 1201
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (658.713 ms) : 0, 658713
BytebuddyAgent [candidate] (655.015 ms) : 0, 655015
AgentMeter [baseline] (12.026 ms) : 0, 12026
AgentMeter [candidate] (11.913 ms) : 0, 11913
GlobalTracer [baseline] (258.888 ms) : 0, 258888
GlobalTracer [candidate] (258.382 ms) : 0, 258382
IAST [baseline] (25.391 ms) : 0, 25391
IAST [candidate] (25.432 ms) : 0, 25432
AppSec [baseline] (167.676 ms) : 0, 167676
AppSec [candidate] (167.902 ms) : 0, 167902
Debugger [baseline] (66.33 ms) : 0, 66330
Debugger [candidate] (66.664 ms) : 0, 66664
Remote Config [baseline] (668.572 µs) : 0, 669
Remote Config [candidate] (695.856 µs) : 0, 696
Telemetry [baseline] (9.256 ms) : 0, 9256
Telemetry [candidate] (9.308 ms) : 0, 9308
Flare Poller [baseline] (3.699 ms) : 0, 3699
Flare Poller [candidate] (3.729 ms) : 0, 3729
section iast
crashtracking [baseline] (1.207 ms) : 0, 1207
crashtracking [candidate] (1.186 ms) : 0, 1186
BytebuddyAgent [baseline] (801.403 ms) : 0, 801403
BytebuddyAgent [candidate] (794.32 ms) : 0, 794320
AgentMeter [baseline] (11.539 ms) : 0, 11539
AgentMeter [candidate] (11.291 ms) : 0, 11291
GlobalTracer [baseline] (248.7 ms) : 0, 248700
GlobalTracer [candidate] (247.556 ms) : 0, 247556
IAST [baseline] (27.197 ms) : 0, 27197
IAST [candidate] (27.067 ms) : 0, 27067
AppSec [baseline] (32.266 ms) : 0, 32266
AppSec [candidate] (32.186 ms) : 0, 32186
Debugger [baseline] (68.382 ms) : 0, 68382
Debugger [candidate] (68.225 ms) : 0, 68225
Remote Config [baseline] (533.221 µs) : 0, 533
Remote Config [candidate] (540.276 µs) : 0, 540
Telemetry [baseline] (8.519 ms) : 0, 8519
Telemetry [candidate] (8.619 ms) : 0, 8619
Flare Poller [baseline] (3.46 ms) : 0, 3460
Flare Poller [candidate] (3.432 ms) : 0, 3432
section profiling
crashtracking [baseline] (1.173 ms) : 0, 1173
crashtracking [candidate] (1.177 ms) : 0, 1177
BytebuddyAgent [baseline] (687.9 ms) : 0, 687900
BytebuddyAgent [candidate] (688.372 ms) : 0, 688372
AgentMeter [baseline] (8.633 ms) : 0, 8633
AgentMeter [candidate] (8.688 ms) : 0, 8688
GlobalTracer [baseline] (217.429 ms) : 0, 217429
GlobalTracer [candidate] (217.879 ms) : 0, 217879
AppSec [baseline] (32.711 ms) : 0, 32711
AppSec [candidate] (32.89 ms) : 0, 32890
Debugger [baseline] (67.605 ms) : 0, 67605
Debugger [candidate] (67.667 ms) : 0, 67667
Remote Config [baseline] (638.118 µs) : 0, 638
Remote Config [candidate] (639.88 µs) : 0, 640
Telemetry [baseline] (9.009 ms) : 0, 9009
Telemetry [candidate] (9.052 ms) : 0, 9052
Flare Poller [baseline] (3.83 ms) : 0, 3830
Flare Poller [candidate] (3.779 ms) : 0, 3779
ProfilingAgent [baseline] (100.252 ms) : 0, 100252
ProfilingAgent [candidate] (100.129 ms) : 0, 100129
Profiling [baseline] (100.847 ms) : 0, 100847
Profiling [candidate] (100.711 ms) : 0, 100711

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	alejandro.gonzalez/fix-system-tests-sha-pin
git_commit_date	1772492986	1773849383
git_commit_sha	`0f5d84f`	`d96eff8`
release_version	1.60.0~0f5d84f2c4	1.61.0-SNAPSHOT~d96eff8552

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773851816	1773851816
ci_job_id	1518071039	1518071039
ci_pipeline_id	103308887	103308887
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-08q3z0a1 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-08q3z0a1 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 3 performance regressions! Performance is the same for 15 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:no_agent:high_load	worse [+20.619µs; +92.008µs] or [+2.141%; +9.556%]	unstable [+55.159µs; +820.110µs] or [+2.093%; +31.126%]	unstable [-679.048op/s; +227.735op/s] or [-18.136%; +6.082%]	1019.166µs	3.072ms	3518.625op/s	962.852µs	2.635ms	3744.281op/s
scenario:load:insecure-bank:iast:high_load	better [-127.801µs; -51.456µs] or [-5.018%; -2.021%]	same [-343.128µs; +142.666µs] or [-4.662%; +1.938%]	unstable [-110.347op/s; +200.909op/s] or [-7.909%; +14.399%]	2.457ms	7.260ms	1440.562op/s	2.547ms	7.360ms	1395.281op/s
scenario:load:insecure-bank:iast_GLOBAL:high_load	worse [+147.287µs; +271.391µs] or [+5.467%; +10.074%]	worse [+200.287µs; +579.380µs] or [+2.592%; +7.498%]	unstable [-212.716op/s; +76.403op/s] or [-16.141%; +5.798%]	2.903ms	8.117ms	1249.688op/s	2.694ms	7.727ms	1317.844op/s
scenario:load:petclinic:tracing:high_load	better [-1462.222µs; -442.058µs] or [-7.927%; -2.397%]	unsure [-1764.861µs; -112.374µs] or [-5.931%; -0.378%]	unstable [-15.948op/s; +39.323op/s] or [-6.416%; +15.820%]	17.494ms	28.820ms	260.250op/s	18.446ms	29.759ms	248.562op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~d96eff8552, baseline=1.60.0~0f5d84f2c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.943 ms) : 17765, 18121
.   : milestone, 17943,
appsec (18.987 ms) : 18796, 19178
.   : milestone, 18987,
code_origins (17.537 ms) : 17361, 17714
.   : milestone, 17537,
iast (17.842 ms) : 17663, 18020
.   : milestone, 17842,
profiling (19.073 ms) : 18881, 19264
.   : milestone, 19073,
tracing (18.775 ms) : 18589, 18961
.   : milestone, 18775,
section candidate
no_agent (18.376 ms) : 18186, 18566
.   : milestone, 18376,
appsec (18.619 ms) : 18430, 18808
.   : milestone, 18619,
code_origins (17.57 ms) : 17394, 17747
.   : milestone, 17570,
iast (17.981 ms) : 17797, 18164
.   : milestone, 17981,
profiling (18.703 ms) : 18518, 18889
.   : milestone, 18703,
tracing (17.929 ms) : 17749, 18108
.   : milestone, 17929,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.943 ms [17.765 ms, 18.121 ms]	-
appsec	18.987 ms [18.796 ms, 19.178 ms]	1.044 ms (5.8%)
code_origins	17.537 ms [17.361 ms, 17.714 ms]	-406.129 µs (-2.3%)
iast	17.842 ms [17.663 ms, 18.02 ms]	-101.733 µs (-0.6%)
profiling	19.073 ms [18.881 ms, 19.264 ms]	1.129 ms (6.3%)
tracing	18.775 ms [18.589 ms, 18.961 ms]	831.618 µs (4.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.376 ms [18.186 ms, 18.566 ms]	-
appsec	18.619 ms [18.43 ms, 18.808 ms]	242.976 µs (1.3%)
code_origins	17.57 ms [17.394 ms, 17.747 ms]	-805.781 µs (-4.4%)
iast	17.981 ms [17.797 ms, 18.164 ms]	-395.374 µs (-2.2%)
profiling	18.703 ms [18.518 ms, 18.889 ms]	327.251 µs (1.8%)
tracing	17.929 ms [17.749 ms, 18.108 ms]	-447.539 µs (-2.4%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~d96eff8552, baseline=1.60.0~0f5d84f2c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.176 ms) : 1165, 1187
.   : milestone, 1176,
iast (3.281 ms) : 3232, 3330
.   : milestone, 3281,
iast_FULL (6.014 ms) : 5953, 6074
.   : milestone, 6014,
iast_GLOBAL (3.478 ms) : 3416, 3540
.   : milestone, 3478,
profiling (2.005 ms) : 1987, 2023
.   : milestone, 2005,
tracing (1.75 ms) : 1736, 1765
.   : milestone, 1750,
section candidate
no_agent (1.257 ms) : 1245, 1269
.   : milestone, 1257,
iast (3.176 ms) : 3134, 3218
.   : milestone, 3176,
iast_FULL (5.988 ms) : 5928, 6049
.   : milestone, 5988,
iast_GLOBAL (3.672 ms) : 3611, 3733
.   : milestone, 3672,
profiling (1.967 ms) : 1949, 1985
.   : milestone, 1967,
tracing (1.79 ms) : 1775, 1805
.   : milestone, 1790,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.176 ms [1.165 ms, 1.187 ms]	-
iast	3.281 ms [3.232 ms, 3.33 ms]	2.105 ms (179.0%)
iast_FULL	6.014 ms [5.953 ms, 6.074 ms]	4.838 ms (411.3%)
iast_GLOBAL	3.478 ms [3.416 ms, 3.54 ms]	2.302 ms (195.7%)
profiling	2.005 ms [1.987 ms, 2.023 ms]	829.149 µs (70.5%)
tracing	1.75 ms [1.736 ms, 1.765 ms]	574.353 µs (48.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.257 ms [1.245 ms, 1.269 ms]	-
iast	3.176 ms [3.134 ms, 3.218 ms]	1.919 ms (152.7%)
iast_FULL	5.988 ms [5.928 ms, 6.049 ms]	4.731 ms (376.4%)
iast_GLOBAL	3.672 ms [3.611 ms, 3.733 ms]	2.415 ms (192.1%)
profiling	1.967 ms [1.949 ms, 1.985 ms]	709.949 µs (56.5%)
tracing	1.79 ms [1.775 ms, 1.805 ms]	533.132 µs (42.4%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	alejandro.gonzalez/fix-system-tests-sha-pin
git_commit_date	1772492986	1773849383
git_commit_sha	`0f5d84f`	`d96eff8`
release_version	1.60.0~0f5d84f2c4	1.61.0-SNAPSHOT~d96eff8552

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1773851587	1773851587
ci_job_id	1518071040	1518071040
ci_pipeline_id	103308887	103308887
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-kcrt0l4b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-kcrt0l4b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~d96eff8552, baseline=1.60.0~0f5d84f2c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.481 ms) : 1469, 1492
.   : milestone, 1481,
appsec (2.536 ms) : 2481, 2591
.   : milestone, 2536,
iast (2.266 ms) : 2197, 2335
.   : milestone, 2266,
iast_GLOBAL (2.31 ms) : 2240, 2380
.   : milestone, 2310,
profiling (2.086 ms) : 2031, 2141
.   : milestone, 2086,
tracing (2.091 ms) : 2037, 2145
.   : milestone, 2091,
section candidate
no_agent (1.478 ms) : 1467, 1490
.   : milestone, 1478,
appsec (3.753 ms) : 3536, 3969
.   : milestone, 3753,
iast (2.261 ms) : 2192, 2330
.   : milestone, 2261,
iast_GLOBAL (2.317 ms) : 2247, 2386
.   : milestone, 2317,
profiling (2.119 ms) : 2062, 2176
.   : milestone, 2119,
tracing (2.083 ms) : 2029, 2137
.   : milestone, 2083,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.481 ms [1.469 ms, 1.492 ms]	-
appsec	2.536 ms [2.481 ms, 2.591 ms]	1.055 ms (71.3%)
iast	2.266 ms [2.197 ms, 2.335 ms]	785.092 µs (53.0%)
iast_GLOBAL	2.31 ms [2.24 ms, 2.38 ms]	829.215 µs (56.0%)
profiling	2.086 ms [2.031 ms, 2.141 ms]	604.994 µs (40.9%)
tracing	2.091 ms [2.037 ms, 2.145 ms]	610.356 µs (41.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.467 ms, 1.49 ms]	-
appsec	3.753 ms [3.536 ms, 3.969 ms]	2.274 ms (153.9%)
iast	2.261 ms [2.192 ms, 2.33 ms]	783.211 µs (53.0%)
iast_GLOBAL	2.317 ms [2.247 ms, 2.386 ms]	838.53 µs (56.7%)
profiling	2.119 ms [2.062 ms, 2.176 ms]	640.597 µs (43.3%)
tracing	2.083 ms [2.029 ms, 2.137 ms]	605.054 µs (40.9%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~d96eff8552, baseline=1.60.0~0f5d84f2c4
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.663 s) : 15663000, 15663000
.   : milestone, 15663000,
appsec (14.869 s) : 14869000, 14869000
.   : milestone, 14869000,
iast (18.364 s) : 18364000, 18364000
.   : milestone, 18364000,
iast_GLOBAL (17.838 s) : 17838000, 17838000
.   : milestone, 17838000,
profiling (15.026 s) : 15026000, 15026000
.   : milestone, 15026000,
tracing (14.803 s) : 14803000, 14803000
.   : milestone, 14803000,
section candidate
no_agent (15.288 s) : 15288000, 15288000
.   : milestone, 15288000,
appsec (15.057 s) : 15057000, 15057000
.   : milestone, 15057000,
iast (18.199 s) : 18199000, 18199000
.   : milestone, 18199000,
iast_GLOBAL (17.59 s) : 17590000, 17590000
.   : milestone, 17590000,
profiling (14.948 s) : 14948000, 14948000
.   : milestone, 14948000,
tracing (14.862 s) : 14862000, 14862000
.   : milestone, 14862000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.663 s [15.663 s, 15.663 s]	-
appsec	14.869 s [14.869 s, 14.869 s]	-794.0 ms (-5.1%)
iast	18.364 s [18.364 s, 18.364 s]	2.701 s (17.2%)
iast_GLOBAL	17.838 s [17.838 s, 17.838 s]	2.175 s (13.9%)
profiling	15.026 s [15.026 s, 15.026 s]	-637.0 ms (-4.1%)
tracing	14.803 s [14.803 s, 14.803 s]	-860.0 ms (-5.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.288 s [15.288 s, 15.288 s]	-
appsec	15.057 s [15.057 s, 15.057 s]	-231.0 ms (-1.5%)
iast	18.199 s [18.199 s, 18.199 s]	2.911 s (19.0%)
iast_GLOBAL	17.59 s [17.59 s, 17.59 s]	2.302 s (15.1%)
profiling	14.948 s [14.948 s, 14.948 s]	-340.0 ms (-2.2%)
tracing	14.862 s [14.862 s, 14.862 s]	-426.0 ms (-2.8%)

pr-commenter · 2026-03-18T16:45:50Z

Kafka / consumer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	release/v1.60.x	alejandro.gonzalez/fix-system-tests-sha-pin
git_commit_date	1773846743	1773849383
git_commit_sha	`c1e6a13`	`d96eff8`

See matching parameters

	Baseline	Candidate
ci_job_date	1773850559	1773850559
ci_job_id	1518071050	1518071050
ci_pipeline_id	103308887	103308887
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same

sarahchen6 · 2026-03-18T23:02:03Z

Instead of adjusted the pinned SHA, I opted to create a dd-trace-java-1.60.2-release branch in system-tests here (DataDog/system-tests#6535) that cherry-picks the changes we need to get the original pinned system tests SHA to pass.

Then, I pinned this dd-trace-java-1.60.2-release branch to the release/v1.60.x branch here: #10898 (with another cherry-pick to address gitlab system test failures).

Looks green now...

jandro996 · 2026-03-19T08:37:56Z

Instead of adjusted the pinned SHA, I opted to create a dd-trace-java-1.60.2-release branch in system-tests here (DataDog/system-tests#6535) that cherry-picks the changes we need to get the original pinned system tests SHA to pass.

Then, I pinned this dd-trace-java-1.60.2-release branch to the release/v1.60.x branch here: #10898 (with another cherry-pick to address gitlab system test failures).

Looks green now...

Thanks a lot @sarahchen6! I'll close this one

jandro996 requested a review from a team as a code owner March 18, 2026 14:46

jandro996 requested review from sarahchen6 and removed request for a team March 18, 2026 14:46

sarahchen6 approved these changes Mar 18, 2026

View reviewed changes

jandro996 added tag: no release notes Changes to exclude from release notes tag: dependencies Dependencies related changes labels Mar 18, 2026

jandro996 changed the title ~~🍒 Pin system-tests to SHA with pinned download-artifact action~~ Pin system-tests to SHA with pinned download-artifact action Mar 18, 2026

jandro996 added this to the 1.60.2 milestone Mar 18, 2026

jandro996 force-pushed the alejandro.gonzalez/fix-system-tests-sha-pin branch from b0ebc00 to 2983bc2 Compare March 18, 2026 15:23

update after run update_system_test_reference.sh

8077609

github-actions bot added the tag: ai generated Largely based on code generated by an AI or LLM label Mar 18, 2026

jandro996 requested a review from a team as a code owner March 18, 2026 15:56

jandro996 requested a review from manuel-alvarez-alvarez March 18, 2026 15:56

jandro996 closed this Mar 19, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pin system-tests to SHA with pinned download-artifact action#10889

Pin system-tests to SHA with pinned download-artifact action#10889
jandro996 wants to merge 3 commits intorelease/v1.60.xfrom
alejandro.gonzalez/fix-system-tests-sha-pin

jandro996 commented Mar 18, 2026

Uh oh!

github-actions bot commented Mar 18, 2026 •

edited

Loading

Uh oh!

pr-commenter bot commented Mar 18, 2026

Uh oh!

pr-commenter bot commented Mar 18, 2026

Uh oh!

pr-commenter bot commented Mar 18, 2026

Uh oh!

sarahchen6 commented Mar 18, 2026

Uh oh!

jandro996 commented Mar 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jandro996 commented Mar 18, 2026

Summary

Uh oh!

github-actions bot commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Mar 18, 2026

Kafka / producer-benchmark

Parameters

Summary

Uh oh!

pr-commenter bot commented Mar 18, 2026

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

pr-commenter bot commented Mar 18, 2026

Kafka / consumer-benchmark

Parameters

Summary

Uh oh!

sarahchen6 commented Mar 18, 2026

Uh oh!

jandro996 commented Mar 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Mar 18, 2026 •

edited

Loading