Skip to content

Run test jobs on non default jvms via commit message#11148

Open
bric3 wants to merge 2 commits intomasterfrom
bdu/run-non-default-jvms-on-demand
Open

Run test jobs on non default jvms via commit message#11148
bric3 wants to merge 2 commits intomasterfrom
bdu/run-non-default-jvms-on-demand

Conversation

@bric3
Copy link
Copy Markdown
Contributor

@bric3 bric3 commented Apr 17, 2026 •
edited
Loading

What Does This Do

Allow to request non default jvms to run in the pipeline by adding this line in the commit message:

[ci: NON_DEFAULT_JVMS]

Motivation

Previously the only way to run a CI job on a non default JVM was to trigger it via the CI pipeline interface which is cumbersome.

Additional information

I was also able to run a single jvm, but this proved a tad verbose in the rules (1 rule per test jvm) because Gitlab workflow rules is bit limited/constrained in that area. So I reverted those changes to keep something simple.

@bric3 bric3 requested a review from a team as a code owner April 17, 2026 13:20
@bric3 bric3 requested review from PerfectSlayer, TonyCTHsu, jbachorik and jpbempel and removed request for a team April 17, 2026 13:20
@bric3 bric3 added type: enhancement Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Apr 17, 2026
@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Apr 17, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/run-non-default-jvms-on-demand
git_commit_date 1776382763 1776431782
git_commit_sha 8b1580f 9a7cf40
release_version 1.62.0-SNAPSHOT~8b1580f2ad 1.62.0-SNAPSHOT~9a7cf40306
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1776433513 1776433513
ci_job_id 1605527330 1605527330
ci_pipeline_id 108231355 108231355
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-8fiqihml 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-8fiqihml 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 65 metrics, 6 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~9a7cf40306, baseline=1.62.0-SNAPSHOT~8b1580f2ad

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066402
Total [baseline] (8.854 s) : 0, 8854116
Agent [candidate] (1.066 s) : 0, 1066292
Total [candidate] (8.846 s) : 0, 8845532
section iast
Agent [baseline] (1.224 s) : 0, 1224338
Total [baseline] (9.544 s) : 0, 9544166
Agent [candidate] (1.222 s) : 0, 1221832
Total [candidate] (9.555 s) : 0, 9554760
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent iast 1.224 s 157.937 ms (14.8%)
Total tracing 8.854 s -
Total iast 9.544 s 690.05 ms (7.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent iast 1.222 s 155.54 ms (14.6%)
Total tracing 8.846 s -
Total iast 9.555 s 709.228 ms (8.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~9a7cf40306, baseline=1.62.0-SNAPSHOT~8b1580f2ad

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.256 ms) : 0, 1256
crashtracking [candidate] (1.236 ms) : 0, 1236
BytebuddyAgent [baseline] (640.312 ms) : 0, 640312
BytebuddyAgent [candidate] (639.547 ms) : 0, 639547
AgentMeter [baseline] (29.628 ms) : 0, 29628
AgentMeter [candidate] (29.687 ms) : 0, 29687
GlobalTracer [baseline] (250.363 ms) : 0, 250363
GlobalTracer [candidate] (250.627 ms) : 0, 250627
AppSec [baseline] (32.636 ms) : 0, 32636
AppSec [candidate] (32.7 ms) : 0, 32700
Debugger [baseline] (59.546 ms) : 0, 59546
Debugger [candidate] (59.288 ms) : 0, 59288
Remote Config [baseline] (600.787 µs) : 0, 601
Remote Config [candidate] (589.517 µs) : 0, 590
Telemetry [baseline] (8.152 ms) : 0, 8152
Telemetry [candidate] (8.088 ms) : 0, 8088
Flare Poller [baseline] (7.485 ms) : 0, 7485
Flare Poller [candidate] (8.249 ms) : 0, 8249
section iast
crashtracking [baseline] (1.255 ms) : 0, 1255
crashtracking [candidate] (1.226 ms) : 0, 1226
BytebuddyAgent [baseline] (801.835 ms) : 0, 801835
BytebuddyAgent [candidate] (799.556 ms) : 0, 799556
AgentMeter [baseline] (11.414 ms) : 0, 11414
AgentMeter [candidate] (11.398 ms) : 0, 11398
GlobalTracer [baseline] (239.046 ms) : 0, 239046
GlobalTracer [candidate] (238.524 ms) : 0, 238524
IAST [baseline] (25.757 ms) : 0, 25757
IAST [candidate] (25.697 ms) : 0, 25697
AppSec [baseline] (30.438 ms) : 0, 30438
AppSec [candidate] (32.392 ms) : 0, 32392
Debugger [baseline] (65.049 ms) : 0, 65049
Debugger [candidate] (63.461 ms) : 0, 63461
Remote Config [baseline] (547.638 µs) : 0, 548
Remote Config [candidate] (550.401 µs) : 0, 550
Telemetry [baseline] (9.286 ms) : 0, 9286
Telemetry [candidate] (9.369 ms) : 0, 9369
Flare Poller [baseline] (3.492 ms) : 0, 3492
Flare Poller [candidate] (3.537 ms) : 0, 3537
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~9a7cf40306, baseline=1.62.0-SNAPSHOT~8b1580f2ad

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1056557
Total [baseline] (11.081 s) : 0, 11080710
Agent [candidate] (1.062 s) : 0, 1062354
Total [candidate] (11.155 s) : 0, 11155355
section appsec
Agent [baseline] (1.256 s) : 0, 1256227
Total [baseline] (11.129 s) : 0, 11129447
Agent [candidate] (1.247 s) : 0, 1247405
Total [candidate] (11.069 s) : 0, 11069250
section iast
Agent [baseline] (1.229 s) : 0, 1229109
Total [baseline] (11.304 s) : 0, 11304361
Agent [candidate] (1.231 s) : 0, 1231340
Total [candidate] (11.379 s) : 0, 11378549
section profiling
Agent [baseline] (1.193 s) : 0, 1192906
Total [baseline] (11.159 s) : 0, 11158728
Agent [candidate] (1.192 s) : 0, 1192252
Total [candidate] (10.995 s) : 0, 10995092
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.256 s 199.67 ms (18.9%)
Agent iast 1.229 s 172.552 ms (16.3%)
Agent profiling 1.193 s 136.349 ms (12.9%)
Total tracing 11.081 s -
Total appsec 11.129 s 48.737 ms (0.4%)
Total iast 11.304 s 223.651 ms (2.0%)
Total profiling 11.159 s 78.017 ms (0.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent appsec 1.247 s 185.051 ms (17.4%)
Agent iast 1.231 s 168.986 ms (15.9%)
Agent profiling 1.192 s 129.898 ms (12.2%)
Total tracing 11.155 s -
Total appsec 11.069 s -86.105 ms (-0.8%)
Total iast 11.379 s 223.193 ms (2.0%)
Total profiling 10.995 s -160.264 ms (-1.4%) 
gantt
    title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~9a7cf40306, baseline=1.62.0-SNAPSHOT~8b1580f2ad

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.229 ms) : 0, 1229
crashtracking [candidate] (1.231 ms) : 0, 1231
BytebuddyAgent [baseline] (631.949 ms) : 0, 631949
BytebuddyAgent [candidate] (636.622 ms) : 0, 636622
AgentMeter [baseline] (29.399 ms) : 0, 29399
AgentMeter [candidate] (29.645 ms) : 0, 29645
GlobalTracer [baseline] (248.403 ms) : 0, 248403
GlobalTracer [candidate] (250.252 ms) : 0, 250252
AppSec [baseline] (32.374 ms) : 0, 32374
AppSec [candidate] (32.692 ms) : 0, 32692
Debugger [baseline] (60.141 ms) : 0, 60141
Debugger [candidate] (60.279 ms) : 0, 60279
Remote Config [baseline] (591.977 µs) : 0, 592
Remote Config [candidate] (597.533 µs) : 0, 598
Telemetry [baseline] (8.048 ms) : 0, 8048
Telemetry [candidate] (8.117 ms) : 0, 8117
Flare Poller [baseline] (8.225 ms) : 0, 8225
Flare Poller [candidate] (6.675 ms) : 0, 6675
section appsec
crashtracking [baseline] (1.229 ms) : 0, 1229
crashtracking [candidate] (1.215 ms) : 0, 1215
BytebuddyAgent [baseline] (666.531 ms) : 0, 666531
BytebuddyAgent [candidate] (660.483 ms) : 0, 660483
AgentMeter [baseline] (12.196 ms) : 0, 12196
AgentMeter [candidate] (12.045 ms) : 0, 12045
GlobalTracer [baseline] (250.155 ms) : 0, 250155
GlobalTracer [candidate] (248.401 ms) : 0, 248401
IAST [baseline] (24.609 ms) : 0, 24609
IAST [candidate] (24.51 ms) : 0, 24510
AppSec [baseline] (186.088 ms) : 0, 186088
AppSec [candidate] (185.914 ms) : 0, 185914
Debugger [baseline] (66.28 ms) : 0, 66280
Debugger [candidate] (66.064 ms) : 0, 66064
Remote Config [baseline] (612.957 µs) : 0, 613
Remote Config [candidate] (617.989 µs) : 0, 618
Telemetry [baseline] (8.487 ms) : 0, 8487
Telemetry [candidate] (8.336 ms) : 0, 8336
Flare Poller [baseline] (3.55 ms) : 0, 3550
Flare Poller [candidate] (3.522 ms) : 0, 3522
section iast
crashtracking [baseline] (1.232 ms) : 0, 1232
crashtracking [candidate] (1.241 ms) : 0, 1241
BytebuddyAgent [baseline] (806.395 ms) : 0, 806395
BytebuddyAgent [candidate] (807.147 ms) : 0, 807147
AgentMeter [baseline] (11.478 ms) : 0, 11478
AgentMeter [candidate] (11.513 ms) : 0, 11513
GlobalTracer [baseline] (238.758 ms) : 0, 238758
GlobalTracer [candidate] (239.589 ms) : 0, 239589
IAST [baseline] (26.559 ms) : 0, 26559
IAST [candidate] (25.831 ms) : 0, 25831
AppSec [baseline] (31.938 ms) : 0, 31938
AppSec [candidate] (33.522 ms) : 0, 33522
Debugger [baseline] (63.152 ms) : 0, 63152
Debugger [candidate] (62.703 ms) : 0, 62703
Remote Config [baseline] (542.581 µs) : 0, 543
Remote Config [candidate] (552.039 µs) : 0, 552
Telemetry [baseline] (9.257 ms) : 0, 9257
Telemetry [candidate] (9.422 ms) : 0, 9422
Flare Poller [baseline] (3.421 ms) : 0, 3421
Flare Poller [candidate] (3.537 ms) : 0, 3537
section profiling
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.202 ms) : 0, 1202
BytebuddyAgent [baseline] (696.87 ms) : 0, 696870
BytebuddyAgent [candidate] (696.652 ms) : 0, 696652
AgentMeter [baseline] (9.144 ms) : 0, 9144
AgentMeter [candidate] (9.139 ms) : 0, 9139
GlobalTracer [baseline] (208.717 ms) : 0, 208717
GlobalTracer [candidate] (208.401 ms) : 0, 208401
AppSec [baseline] (33.014 ms) : 0, 33014
AppSec [candidate] (33.01 ms) : 0, 33010
Debugger [baseline] (65.938 ms) : 0, 65938
Debugger [candidate] (65.788 ms) : 0, 65788
Remote Config [baseline] (580.974 µs) : 0, 581
Remote Config [candidate] (577.355 µs) : 0, 577
Telemetry [baseline] (7.836 ms) : 0, 7836
Telemetry [candidate] (7.798 ms) : 0, 7798
Flare Poller [baseline] (3.529 ms) : 0, 3529
Flare Poller [candidate] (3.518 ms) : 0, 3518
ProfilingAgent [baseline] (94.334 ms) : 0, 94334
ProfilingAgent [candidate] (94.366 ms) : 0, 94366
Profiling [baseline] (94.901 ms) : 0, 94901
Profiling [candidate] (94.928 ms) : 0, 94928
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/run-non-default-jvms-on-demand
git_commit_date 1776382763 1776431782
git_commit_sha 8b1580f 9a7cf40
release_version 1.62.0-SNAPSHOT~8b1580f2ad 1.62.0-SNAPSHOT~9a7cf40306
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1776434074 1776434074
ci_job_id 1605527332 1605527332
ci_pipeline_id 108231355 108231355
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-5fjwqd8o 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-5fjwqd8o 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 1 performance regressions! Performance is the same for 17 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load better
[-223.057µs; -89.272µs] or [-7.326%; -2.932%]		 better
[-725.931µs; -259.063µs] or [-8.542%; -3.048%]		 unstable
[-59.611op/s; +204.173op/s] or [-4.983%; +17.066%]		 2.889ms 8.006ms 1268.625op/s 3.045ms 8.498ms 1196.344op/s
scenario:load:petclinic:profiling:high_load worse
[+0.810ms; +1.675ms] or [+4.543%; +9.390%]		 unsure
[+0.239ms; +1.885ms] or [+0.813%; +6.420%]		 unstable
[-36.579op/s; +9.891op/s] or [-14.343%; +3.878%]		 19.077ms 30.417ms 241.688op/s 17.835ms 29.356ms 255.031op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~9a7cf40306, baseline=1.62.0-SNAPSHOT~8b1580f2ad
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.244 ms) : 1232, 1256
.   : milestone, 1244,
iast (3.371 ms) : 3326, 3417
.   : milestone, 3371,
iast_FULL (6.26 ms) : 6196, 6324
.   : milestone, 6260,
iast_GLOBAL (3.836 ms) : 3772, 3901
.   : milestone, 3836,
profiling (2.002 ms) : 1985, 2020
.   : milestone, 2002,
tracing (1.892 ms) : 1876, 1908
.   : milestone, 1892,
section candidate
no_agent (1.24 ms) : 1229, 1252
.   : milestone, 1240,
iast (3.434 ms) : 3385, 3484
.   : milestone, 3434,
iast_FULL (6.06 ms) : 5998, 6121
.   : milestone, 6060,
iast_GLOBAL (3.615 ms) : 3563, 3666
.   : milestone, 3615,
profiling (2.136 ms) : 2116, 2155
.   : milestone, 2136,
tracing (1.849 ms) : 1833, 1864
.   : milestone, 1849,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.244 ms [1.232 ms, 1.256 ms] -
iast 3.371 ms [3.326 ms, 3.417 ms] 2.127 ms (171.0%)
iast_FULL 6.26 ms [6.196 ms, 6.324 ms] 5.016 ms (403.2%)
iast_GLOBAL 3.836 ms [3.772 ms, 3.901 ms] 2.592 ms (208.4%)
profiling 2.002 ms [1.985 ms, 2.02 ms] 758.307 µs (61.0%)
tracing 1.892 ms [1.876 ms, 1.908 ms] 647.805 µs (52.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.24 ms [1.229 ms, 1.252 ms] -
iast 3.434 ms [3.385 ms, 3.484 ms] 2.194 ms (176.9%)
iast_FULL 6.06 ms [5.998 ms, 6.121 ms] 4.819 ms (388.5%)
iast_GLOBAL 3.615 ms [3.563 ms, 3.666 ms] 2.374 ms (191.4%)
profiling 2.136 ms [2.116 ms, 2.155 ms] 895.283 µs (72.2%)
tracing 1.849 ms [1.833 ms, 1.864 ms] 608.077 µs (49.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~9a7cf40306, baseline=1.62.0-SNAPSHOT~8b1580f2ad
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.534 ms) : 19337, 19732
.   : milestone, 19534,
appsec (18.991 ms) : 18802, 19180
.   : milestone, 18991,
code_origins (17.786 ms) : 17611, 17962
.   : milestone, 17786,
iast (17.842 ms) : 17666, 18018
.   : milestone, 17842,
profiling (18.298 ms) : 18114, 18481
.   : milestone, 18298,
tracing (17.853 ms) : 17679, 18028
.   : milestone, 17853,
section candidate
no_agent (19.498 ms) : 19298, 19699
.   : milestone, 19498,
appsec (18.607 ms) : 18416, 18798
.   : milestone, 18607,
code_origins (17.958 ms) : 17775, 18141
.   : milestone, 17958,
iast (17.959 ms) : 17778, 18140
.   : milestone, 17959,
profiling (19.313 ms) : 19120, 19506
.   : milestone, 19313,
tracing (17.542 ms) : 17373, 17710
.   : milestone, 17542,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.534 ms [19.337 ms, 19.732 ms] -
appsec 18.991 ms [18.802 ms, 19.18 ms] -543.627 µs (-2.8%)
code_origins 17.786 ms [17.611 ms, 17.962 ms] -1.748 ms (-8.9%)
iast 17.842 ms [17.666 ms, 18.018 ms] -1.692 ms (-8.7%)
profiling 18.298 ms [18.114 ms, 18.481 ms] -1.237 ms (-6.3%)
tracing 17.853 ms [17.679 ms, 18.028 ms] -1.681 ms (-8.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.498 ms [19.298 ms, 19.699 ms] -
appsec 18.607 ms [18.416 ms, 18.798 ms] -890.937 µs (-4.6%)
code_origins 17.958 ms [17.775 ms, 18.141 ms] -1.54 ms (-7.9%)
iast 17.959 ms [17.778 ms, 18.14 ms] -1.539 ms (-7.9%)
profiling 19.313 ms [19.12 ms, 19.506 ms] -185.439 µs (-1.0%)
tracing 17.542 ms [17.373 ms, 17.71 ms] -1.957 ms (-10.0%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/run-non-default-jvms-on-demand
git_commit_date 1776382763 1776431782
git_commit_sha 8b1580f 9a7cf40
release_version 1.62.0-SNAPSHOT~8b1580f2ad 1.62.0-SNAPSHOT~9a7cf40306
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1776433790 1776433790
ci_job_id 1605527335 1605527335
ci_pipeline_id 108231355 108231355
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-dfddl1vq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-dfddl1vq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.461ms; -1.112ms] or [-38.342%; -29.174%]		 2.524ms 3.811ms
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~9a7cf40306, baseline=1.62.0-SNAPSHOT~8b1580f2ad
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.488 ms) : 1477, 1500
.   : milestone, 1488,
appsec (3.811 ms) : 3588, 4034
.   : milestone, 3811,
iast (2.273 ms) : 2204, 2342
.   : milestone, 2273,
iast_GLOBAL (2.308 ms) : 2238, 2377
.   : milestone, 2308,
profiling (2.096 ms) : 2041, 2150
.   : milestone, 2096,
tracing (2.08 ms) : 2026, 2133
.   : milestone, 2080,
section candidate
no_agent (1.486 ms) : 1474, 1497
.   : milestone, 1486,
appsec (2.524 ms) : 2470, 2579
.   : milestone, 2524,
iast (2.268 ms) : 2199, 2337
.   : milestone, 2268,
iast_GLOBAL (2.312 ms) : 2243, 2382
.   : milestone, 2312,
profiling (2.094 ms) : 2039, 2149
.   : milestone, 2094,
tracing (2.072 ms) : 2018, 2125
.   : milestone, 2072,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.488 ms [1.477 ms, 1.5 ms] -
appsec 3.811 ms [3.588 ms, 4.034 ms] 2.323 ms (156.1%)
iast 2.273 ms [2.204 ms, 2.342 ms] 784.941 µs (52.7%)
iast_GLOBAL 2.308 ms [2.238 ms, 2.377 ms] 819.592 µs (55.1%)
profiling 2.096 ms [2.041 ms, 2.15 ms] 607.568 µs (40.8%)
tracing 2.08 ms [2.026 ms, 2.133 ms] 591.727 µs (39.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.486 ms [1.474 ms, 1.497 ms] -
appsec 2.524 ms [2.47 ms, 2.579 ms] 1.039 ms (69.9%)
iast 2.268 ms [2.199 ms, 2.337 ms] 782.099 µs (52.6%)
iast_GLOBAL 2.312 ms [2.243 ms, 2.382 ms] 826.961 µs (55.7%)
profiling 2.094 ms [2.039 ms, 2.149 ms] 608.147 µs (40.9%)
tracing 2.072 ms [2.018 ms, 2.125 ms] 586.339 µs (39.5%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~9a7cf40306, baseline=1.62.0-SNAPSHOT~8b1580f2ad
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.561 s) : 15561000, 15561000
.   : milestone, 15561000,
appsec (14.508 s) : 14508000, 14508000
.   : milestone, 14508000,
iast (18.185 s) : 18185000, 18185000
.   : milestone, 18185000,
iast_GLOBAL (17.85 s) : 17850000, 17850000
.   : milestone, 17850000,
profiling (14.951 s) : 14951000, 14951000
.   : milestone, 14951000,
tracing (15.067 s) : 15067000, 15067000
.   : milestone, 15067000,
section candidate
no_agent (15.339 s) : 15339000, 15339000
.   : milestone, 15339000,
appsec (14.71 s) : 14710000, 14710000
.   : milestone, 14710000,
iast (18.219 s) : 18219000, 18219000
.   : milestone, 18219000,
iast_GLOBAL (18.199 s) : 18199000, 18199000
.   : milestone, 18199000,
profiling (15.494 s) : 15494000, 15494000
.   : milestone, 15494000,
tracing (14.756 s) : 14756000, 14756000
.   : milestone, 14756000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.561 s [15.561 s, 15.561 s] -
appsec 14.508 s [14.508 s, 14.508 s] -1.053 s (-6.8%)
iast 18.185 s [18.185 s, 18.185 s] 2.624 s (16.9%)
iast_GLOBAL 17.85 s [17.85 s, 17.85 s] 2.289 s (14.7%)
profiling 14.951 s [14.951 s, 14.951 s] -610.0 ms (-3.9%)
tracing 15.067 s [15.067 s, 15.067 s] -494.0 ms (-3.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.339 s [15.339 s, 15.339 s] -
appsec 14.71 s [14.71 s, 14.71 s] -629.0 ms (-4.1%)
iast 18.219 s [18.219 s, 18.219 s] 2.88 s (18.8%)
iast_GLOBAL 18.199 s [18.199 s, 18.199 s] 2.86 s (18.6%)
profiling 15.494 s [15.494 s, 15.494 s] 155.0 ms (1.0%)
tracing 14.756 s [14.756 s, 14.756 s] -583.0 ms (-3.8%)

@jbachorik
Copy link
Copy Markdown
Contributor

jbachorik commented Apr 17, 2026

What about the run-tests:* labels? Is this the same, just different input?

@bric3
Copy link
Copy Markdown
Contributor Author

bric3 commented Apr 17, 2026

What about the run-tests:* labels? Is this the same, just different input?

@jbachorik The labels don't work anymore since we moved away from circle-ci. I believe this is the closest we can do today without heavier machinery. So consider this different mechanism.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jbachorik jbachorik Awaiting requested review from jbachorik

@TonyCTHsu TonyCTHsu Awaiting requested review from TonyCTHsu TonyCTHsu is a code owner automatically assigned from DataDog/apm-release-platform

@jpbempel jpbempel Awaiting requested review from jpbempel

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes type: enhancement Enhancements and improvements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@bric3 @jbachorik