Skip to content

🪞 9979 - Instrumentation test error visibility#10300

Draft
mcculls wants to merge 6 commits intomasterfrom
community-pr-9979
Draft

🪞 9979 - Instrumentation test error visibility#10300
mcculls wants to merge 6 commits intomasterfrom
community-pr-9979

Conversation

@mcculls
Copy link
Copy Markdown
Contributor

@mcculls mcculls commented Jan 7, 2026

This PR mirrors the changes from the original community contribution to enable CI testing with maintainer privileges.

Original PR: #9979
Original Author: @deejgregor
Original Branch: deejgregor/dd-trace-java:feature-instrumentation-test-error-visibility

Closes #9979

This is an automated mirror created to run CI checks. See tooling/mirror-community-pull-request.sh for details.

@mcculls mcculls requested a review from a team as a code owner January 7, 2026 10:36
@mcculls mcculls added the tag: community Community contribution label Jan 7, 2026
@mcculls mcculls requested a review from amarziali January 7, 2026 10:36
@mcculls mcculls added the tag: do not merge Do not merge changes label Jan 7, 2026
@mcculls
Copy link
Copy Markdown
Contributor Author

mcculls commented Jan 7, 2026

Created this to mainly run CI checks - there are a couple of things to clean up before this can be merged.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jan 7, 2026

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@mcculls mcculls added type: enhancement Enhancements and improvements comp: tooling Build & Tooling labels Jan 7, 2026
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Jan 7, 2026 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master community-pr-9979
git_commit_date 1777012226 1777017142
git_commit_sha 7c1b47b f696e30
release_version 1.62.0-SNAPSHOT~7c1b47b5d5 1.62.0-SNAPSHOT~f696e30012
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1777019000 1777019000
ci_job_id 1627135508 1627135508
ci_pipeline_id 109480985 109480985
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-h5uiyyua 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-h5uiyyua 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 metrics, 9 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~f696e30012, baseline=1.62.0-SNAPSHOT~7c1b47b5d5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1053960
Total [baseline] (8.822 s) : 0, 8821896
Agent [candidate] (1.054 s) : 0, 1053914
Total [candidate] (8.836 s) : 0, 8835989
section iast
Agent [baseline] (1.233 s) : 0, 1232771
Total [baseline] (9.534 s) : 0, 9534234
Agent [candidate] (1.234 s) : 0, 1233550
Total [candidate] (9.554 s) : 0, 9553992
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent iast 1.233 s 178.811 ms (17.0%)
Total tracing 8.822 s -
Total iast 9.534 s 712.338 ms (8.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent iast 1.234 s 179.635 ms (17.0%)
Total tracing 8.836 s -
Total iast 9.554 s 718.003 ms (8.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~f696e30012, baseline=1.62.0-SNAPSHOT~7c1b47b5d5

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.241 ms) : 0, 1241
crashtracking [candidate] (1.216 ms) : 0, 1216
BytebuddyAgent [baseline] (634.178 ms) : 0, 634178
BytebuddyAgent [candidate] (634.078 ms) : 0, 634078
AgentMeter [baseline] (29.342 ms) : 0, 29342
AgentMeter [candidate] (29.498 ms) : 0, 29498
GlobalTracer [baseline] (248.275 ms) : 0, 248275
GlobalTracer [candidate] (248.507 ms) : 0, 248507
AppSec [baseline] (32.388 ms) : 0, 32388
AppSec [candidate] (32.253 ms) : 0, 32253
Debugger [baseline] (59.546 ms) : 0, 59546
Debugger [candidate] (59.384 ms) : 0, 59384
Remote Config [baseline] (597.613 µs) : 0, 598
Remote Config [candidate] (597.505 µs) : 0, 598
Telemetry [baseline] (8.053 ms) : 0, 8053
Telemetry [candidate] (8.764 ms) : 0, 8764
Flare Poller [baseline] (4.256 ms) : 0, 4256
Flare Poller [candidate] (3.526 ms) : 0, 3526
section iast
crashtracking [baseline] (1.23 ms) : 0, 1230
crashtracking [candidate] (1.223 ms) : 0, 1223
BytebuddyAgent [baseline] (809.949 ms) : 0, 809949
BytebuddyAgent [candidate] (811.412 ms) : 0, 811412
AgentMeter [baseline] (11.393 ms) : 0, 11393
AgentMeter [candidate] (11.385 ms) : 0, 11385
GlobalTracer [baseline] (238.966 ms) : 0, 238966
GlobalTracer [candidate] (239.729 ms) : 0, 239729
IAST [baseline] (27.384 ms) : 0, 27384
IAST [candidate] (29.118 ms) : 0, 29118
AppSec [baseline] (28.733 ms) : 0, 28733
AppSec [candidate] (28.405 ms) : 0, 28405
Debugger [baseline] (67.2 ms) : 0, 67200
Debugger [candidate] (64.529 ms) : 0, 64529
Remote Config [baseline] (540.66 µs) : 0, 541
Remote Config [candidate] (538.221 µs) : 0, 538
Telemetry [baseline] (7.848 ms) : 0, 7848
Telemetry [candidate] (7.77 ms) : 0, 7770
Flare Poller [baseline] (3.482 ms) : 0, 3482
Flare Poller [candidate] (3.419 ms) : 0, 3419
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~f696e30012, baseline=1.62.0-SNAPSHOT~7c1b47b5d5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.062 s) : 0, 1062375
Total [baseline] (11.001 s) : 0, 11000702
Agent [candidate] (1.061 s) : 0, 1061400
Total [candidate] (11.066 s) : 0, 11065644
section appsec
Agent [baseline] (1.267 s) : 0, 1266670
Total [baseline] (11.008 s) : 0, 11008011
Agent [candidate] (1.262 s) : 0, 1261624
Total [candidate] (11.142 s) : 0, 11142497
section iast
Agent [baseline] (1.233 s) : 0, 1232638
Total [baseline] (11.239 s) : 0, 11239257
Agent [candidate] (1.232 s) : 0, 1232069
Total [candidate] (11.339 s) : 0, 11338682
section profiling
Agent [baseline] (1.193 s) : 0, 1193333
Total [baseline] (11.04 s) : 0, 11040060
Agent [candidate] (1.194 s) : 0, 1193660
Total [candidate] (10.978 s) : 0, 10977566
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent appsec 1.267 s 204.295 ms (19.2%)
Agent iast 1.233 s 170.263 ms (16.0%)
Agent profiling 1.193 s 130.958 ms (12.3%)
Total tracing 11.001 s -
Total appsec 11.008 s 7.31 ms (0.1%)
Total iast 11.239 s 238.555 ms (2.2%)
Total profiling 11.04 s 39.359 ms (0.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent appsec 1.262 s 200.223 ms (18.9%)
Agent iast 1.232 s 170.669 ms (16.1%)
Agent profiling 1.194 s 132.259 ms (12.5%)
Total tracing 11.066 s -
Total appsec 11.142 s 76.852 ms (0.7%)
Total iast 11.339 s 273.038 ms (2.5%)
Total profiling 10.978 s -88.078 ms (-0.8%) 
gantt
    title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~f696e30012, baseline=1.62.0-SNAPSHOT~7c1b47b5d5

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.236 ms) : 0, 1236
crashtracking [candidate] (1.225 ms) : 0, 1225
BytebuddyAgent [baseline] (639.021 ms) : 0, 639021
BytebuddyAgent [candidate] (638.079 ms) : 0, 638079
AgentMeter [baseline] (29.549 ms) : 0, 29549
AgentMeter [candidate] (29.488 ms) : 0, 29488
GlobalTracer [baseline] (250.089 ms) : 0, 250089
GlobalTracer [candidate] (249.189 ms) : 0, 249189
AppSec [baseline] (32.49 ms) : 0, 32490
AppSec [candidate] (32.558 ms) : 0, 32558
Debugger [baseline] (60.608 ms) : 0, 60608
Debugger [candidate] (60.932 ms) : 0, 60932
Remote Config [baseline] (603.295 µs) : 0, 603
Remote Config [candidate] (602.435 µs) : 0, 602
Telemetry [baseline] (8.861 ms) : 0, 8861
Telemetry [candidate] (8.848 ms) : 0, 8848
Flare Poller [baseline] (3.618 ms) : 0, 3618
Flare Poller [candidate] (4.388 ms) : 0, 4388
section appsec
crashtracking [baseline] (1.225 ms) : 0, 1225
crashtracking [candidate] (1.212 ms) : 0, 1212
BytebuddyAgent [baseline] (677.482 ms) : 0, 677482
BytebuddyAgent [candidate] (674.864 ms) : 0, 674864
AgentMeter [baseline] (12.294 ms) : 0, 12294
AgentMeter [candidate] (12.234 ms) : 0, 12234
GlobalTracer [baseline] (250.024 ms) : 0, 250024
GlobalTracer [candidate] (248.252 ms) : 0, 248252
AppSec [baseline] (187.966 ms) : 0, 187966
AppSec [candidate] (187.383 ms) : 0, 187383
Debugger [baseline] (65.211 ms) : 0, 65211
Debugger [candidate] (65.182 ms) : 0, 65182
Remote Config [baseline] (566.982 µs) : 0, 567
Remote Config [candidate] (576.553 µs) : 0, 577
Telemetry [baseline] (7.774 ms) : 0, 7774
Telemetry [candidate] (7.892 ms) : 0, 7892
Flare Poller [baseline] (3.388 ms) : 0, 3388
Flare Poller [candidate] (3.434 ms) : 0, 3434
IAST [baseline] (24.306 ms) : 0, 24306
IAST [candidate] (24.211 ms) : 0, 24211
section iast
crashtracking [baseline] (1.226 ms) : 0, 1226
crashtracking [candidate] (1.215 ms) : 0, 1215
BytebuddyAgent [baseline] (809.974 ms) : 0, 809974
BytebuddyAgent [candidate] (809.67 ms) : 0, 809670
AgentMeter [baseline] (11.418 ms) : 0, 11418
AgentMeter [candidate] (11.404 ms) : 0, 11404
GlobalTracer [baseline] (239.11 ms) : 0, 239110
GlobalTracer [candidate] (238.845 ms) : 0, 238845
AppSec [baseline] (31.242 ms) : 0, 31242
AppSec [candidate] (28.409 ms) : 0, 28409
Debugger [baseline] (63.632 ms) : 0, 63632
Debugger [candidate] (63.878 ms) : 0, 63878
Remote Config [baseline] (538.574 µs) : 0, 539
Remote Config [candidate] (533.263 µs) : 0, 533
Telemetry [baseline] (7.827 ms) : 0, 7827
Telemetry [candidate] (7.794 ms) : 0, 7794
Flare Poller [baseline] (3.43 ms) : 0, 3430
Flare Poller [candidate] (3.462 ms) : 0, 3462
IAST [baseline] (28.225 ms) : 0, 28225
IAST [candidate] (30.785 ms) : 0, 30785
section profiling
ProfilingAgent [baseline] (93.664 ms) : 0, 93664
ProfilingAgent [candidate] (93.42 ms) : 0, 93420
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.189 ms) : 0, 1189
BytebuddyAgent [baseline] (697.583 ms) : 0, 697583
BytebuddyAgent [candidate] (698.243 ms) : 0, 698243
AgentMeter [baseline] (9.102 ms) : 0, 9102
AgentMeter [candidate] (9.068 ms) : 0, 9068
GlobalTracer [baseline] (208.841 ms) : 0, 208841
GlobalTracer [candidate] (208.888 ms) : 0, 208888
AppSec [baseline] (32.824 ms) : 0, 32824
AppSec [candidate] (33.055 ms) : 0, 33055
Debugger [baseline] (66.146 ms) : 0, 66146
Debugger [candidate] (65.766 ms) : 0, 65766
Remote Config [baseline] (577.889 µs) : 0, 578
Remote Config [candidate] (577.488 µs) : 0, 577
Telemetry [baseline] (7.783 ms) : 0, 7783
Telemetry [candidate] (7.752 ms) : 0, 7752
Flare Poller [baseline] (3.542 ms) : 0, 3542
Flare Poller [candidate] (3.499 ms) : 0, 3499
Profiling [baseline] (94.23 ms) : 0, 94230
Profiling [candidate] (93.983 ms) : 0, 93983
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master community-pr-9979
git_commit_date 1777012226 1777017142
git_commit_sha 7c1b47b f696e30
release_version 1.62.0-SNAPSHOT~7c1b47b5d5 1.62.0-SNAPSHOT~f696e30012
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1777019400 1777019400
ci_job_id 1627135509 1627135509
ci_pipeline_id 109480985 109480985
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-zjw586ii 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-zjw586ii 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 18 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:profiling:high_load worse
[+127.192µs; +271.008µs] or [+6.810%; +14.510%]		 unstable
[+514.320µs; +1422.111µs] or [+9.706%; +26.836%]		 unstable
[-463.099op/s; -79.339op/s] or [-23.506%; -4.027%]		 2.067ms 6.267ms 1698.938op/s 1.868ms 5.299ms 1970.156op/s
scenario:load:petclinic:tracing:high_load worse
[+467.606µs; +1441.419µs] or [+2.570%; +7.922%]		 unsure
[+0.146ms; +2.143ms] or [+0.487%; +7.151%]		 unstable
[-35.270op/s; +13.208op/s] or [-14.038%; +5.257%]		 19.150ms 31.111ms 240.219op/s 18.195ms 29.966ms 251.250op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~f696e30012, baseline=1.62.0-SNAPSHOT~7c1b47b5d5
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.608 ms) : 17434, 17781
.   : milestone, 17608,
appsec (19.105 ms) : 18910, 19300
.   : milestone, 19105,
code_origins (18.838 ms) : 18644, 19032
.   : milestone, 18838,
iast (18.853 ms) : 18659, 19046
.   : milestone, 18853,
profiling (18.883 ms) : 18693, 19073
.   : milestone, 18883,
tracing (18.575 ms) : 18391, 18759
.   : milestone, 18575,
section candidate
no_agent (18.505 ms) : 18316, 18694
.   : milestone, 18505,
appsec (19.276 ms) : 19081, 19470
.   : milestone, 19276,
code_origins (18.728 ms) : 18539, 18917
.   : milestone, 18728,
iast (18.508 ms) : 18321, 18694
.   : milestone, 18508,
profiling (18.855 ms) : 18668, 19041
.   : milestone, 18855,
tracing (19.432 ms) : 19230, 19633
.   : milestone, 19432,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.608 ms [17.434 ms, 17.781 ms] -
appsec 19.105 ms [18.91 ms, 19.3 ms] 1.497 ms (8.5%)
code_origins 18.838 ms [18.644 ms, 19.032 ms] 1.23 ms (7.0%)
iast 18.853 ms [18.659 ms, 19.046 ms] 1.245 ms (7.1%)
profiling 18.883 ms [18.693 ms, 19.073 ms] 1.275 ms (7.2%)
tracing 18.575 ms [18.391 ms, 18.759 ms] 967.007 µs (5.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.505 ms [18.316 ms, 18.694 ms] -
appsec 19.276 ms [19.081 ms, 19.47 ms] 770.688 µs (4.2%)
code_origins 18.728 ms [18.539 ms, 18.917 ms] 222.855 µs (1.2%)
iast 18.508 ms [18.321 ms, 18.694 ms] 2.608 µs (0.0%)
profiling 18.855 ms [18.668 ms, 19.041 ms] 349.573 µs (1.9%)
tracing 19.432 ms [19.23 ms, 19.633 ms] 926.602 µs (5.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~f696e30012, baseline=1.62.0-SNAPSHOT~7c1b47b5d5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.357 ms) : 1344, 1370
.   : milestone, 1357,
iast (3.57 ms) : 3522, 3618
.   : milestone, 3570,
iast_FULL (6.393 ms) : 6326, 6459
.   : milestone, 6393,
iast_GLOBAL (4.05 ms) : 3978, 4122
.   : milestone, 4050,
profiling (2.3 ms) : 2277, 2323
.   : milestone, 2300,
tracing (2.075 ms) : 2058, 2093
.   : milestone, 2075,
section candidate
no_agent (1.387 ms) : 1372, 1402
.   : milestone, 1387,
iast (3.616 ms) : 3561, 3671
.   : milestone, 3616,
iast_FULL (6.405 ms) : 6338, 6472
.   : milestone, 6405,
iast_GLOBAL (3.933 ms) : 3862, 4004
.   : milestone, 3933,
profiling (2.678 ms) : 2651, 2706
.   : milestone, 2678,
tracing (2.064 ms) : 2047, 2081
.   : milestone, 2064,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.357 ms [1.344 ms, 1.37 ms] -
iast 3.57 ms [3.522 ms, 3.618 ms] 2.213 ms (163.1%)
iast_FULL 6.393 ms [6.326 ms, 6.459 ms] 5.036 ms (371.2%)
iast_GLOBAL 4.05 ms [3.978 ms, 4.122 ms] 2.693 ms (198.5%)
profiling 2.3 ms [2.277 ms, 2.323 ms] 942.73 µs (69.5%)
tracing 2.075 ms [2.058 ms, 2.093 ms] 718.59 µs (53.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.387 ms [1.372 ms, 1.402 ms] -
iast 3.616 ms [3.561 ms, 3.671 ms] 2.229 ms (160.7%)
iast_FULL 6.405 ms [6.338 ms, 6.472 ms] 5.018 ms (361.8%)
iast_GLOBAL 3.933 ms [3.862 ms, 4.004 ms] 2.546 ms (183.5%)
profiling 2.678 ms [2.651 ms, 2.706 ms] 1.291 ms (93.1%)
tracing 2.064 ms [2.047 ms, 2.081 ms] 676.91 µs (48.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master community-pr-9979
git_commit_date 1777012226 1777017142
git_commit_sha 7c1b47b f696e30
release_version 1.62.0-SNAPSHOT~7c1b47b5d5 1.62.0-SNAPSHOT~f696e30012
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1777019245 1777019245
ci_job_id 1627135510 1627135510
ci_pipeline_id 109480985 109480985
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-lnt7guvw 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-lnt7guvw 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.471ms; -1.117ms] or [-38.090%; -28.931%]		 2.568ms 3.862ms
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~f696e30012, baseline=1.62.0-SNAPSHOT~7c1b47b5d5
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.476 s) : 15476000, 15476000
.   : milestone, 15476000,
appsec (15.036 s) : 15036000, 15036000
.   : milestone, 15036000,
iast (18.911 s) : 18911000, 18911000
.   : milestone, 18911000,
iast_GLOBAL (18.22 s) : 18220000, 18220000
.   : milestone, 18220000,
profiling (14.851 s) : 14851000, 14851000
.   : milestone, 14851000,
tracing (14.901 s) : 14901000, 14901000
.   : milestone, 14901000,
section candidate
no_agent (14.813 s) : 14813000, 14813000
.   : milestone, 14813000,
appsec (14.885 s) : 14885000, 14885000
.   : milestone, 14885000,
iast (18.902 s) : 18902000, 18902000
.   : milestone, 18902000,
iast_GLOBAL (17.83 s) : 17830000, 17830000
.   : milestone, 17830000,
profiling (14.924 s) : 14924000, 14924000
.   : milestone, 14924000,
tracing (14.694 s) : 14694000, 14694000
.   : milestone, 14694000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.476 s [15.476 s, 15.476 s] -
appsec 15.036 s [15.036 s, 15.036 s] -440.0 ms (-2.8%)
iast 18.911 s [18.911 s, 18.911 s] 3.435 s (22.2%)
iast_GLOBAL 18.22 s [18.22 s, 18.22 s] 2.744 s (17.7%)
profiling 14.851 s [14.851 s, 14.851 s] -625.0 ms (-4.0%)
tracing 14.901 s [14.901 s, 14.901 s] -575.0 ms (-3.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.813 s [14.813 s, 14.813 s] -
appsec 14.885 s [14.885 s, 14.885 s] 72.0 ms (0.5%)
iast 18.902 s [18.902 s, 18.902 s] 4.089 s (27.6%)
iast_GLOBAL 17.83 s [17.83 s, 17.83 s] 3.017 s (20.4%)
profiling 14.924 s [14.924 s, 14.924 s] 111.0 ms (0.7%)
tracing 14.694 s [14.694 s, 14.694 s] -119.0 ms (-0.8%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~f696e30012, baseline=1.62.0-SNAPSHOT~7c1b47b5d5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.499 ms) : 1488, 1511
.   : milestone, 1499,
appsec (3.862 ms) : 3636, 4087
.   : milestone, 3862,
iast (2.296 ms) : 2226, 2366
.   : milestone, 2296,
iast_GLOBAL (2.341 ms) : 2270, 2411
.   : milestone, 2341,
profiling (2.109 ms) : 2054, 2164
.   : milestone, 2109,
tracing (2.105 ms) : 2051, 2158
.   : milestone, 2105,
section candidate
no_agent (1.496 ms) : 1484, 1508
.   : milestone, 1496,
appsec (2.568 ms) : 2512, 2623
.   : milestone, 2568,
iast (2.296 ms) : 2226, 2365
.   : milestone, 2296,
iast_GLOBAL (2.331 ms) : 2261, 2401
.   : milestone, 2331,
profiling (2.108 ms) : 2053, 2162
.   : milestone, 2108,
tracing (2.087 ms) : 2034, 2141
.   : milestone, 2087,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.499 ms [1.488 ms, 1.511 ms] -
appsec 3.862 ms [3.636 ms, 4.087 ms] 2.362 ms (157.6%)
iast 2.296 ms [2.226 ms, 2.366 ms] 796.512 µs (53.1%)
iast_GLOBAL 2.341 ms [2.27 ms, 2.411 ms] 841.251 µs (56.1%)
profiling 2.109 ms [2.054 ms, 2.164 ms] 609.36 µs (40.6%)
tracing 2.105 ms [2.051 ms, 2.158 ms] 605.215 µs (40.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.496 ms [1.484 ms, 1.508 ms] -
appsec 2.568 ms [2.512 ms, 2.623 ms] 1.072 ms (71.6%)
iast 2.296 ms [2.226 ms, 2.365 ms] 799.591 µs (53.4%)
iast_GLOBAL 2.331 ms [2.261 ms, 2.401 ms] 834.537 µs (55.8%)
profiling 2.108 ms [2.053 ms, 2.162 ms] 611.541 µs (40.9%)
tracing 2.087 ms [2.034 ms, 2.141 ms] 591.191 µs (39.5%)

@mcculls mcculls marked this pull request as draft January 30, 2026 15:03
deejgregor and others added 6 commits April 24, 2026 09:17
@deejgregor @amarziali
ExceptionHandlers: Fix code in bytecode comment
0bd5aab
@deejgregor @amarziali
InstrumentationContext: be more explicit about the reasons for failure
f86b353
@deejgregor @amarziali
Add instrumentation error details on test failures
0c9d123 
This puts the details front and center in the test summary and is
much easier than hunting back through log messages.

Note: In InstrumentationErrors, errors are only recorded when
enableRecordingAndReset has been called. This is only done from the
InstrumentationSpecification test specification, so we don't need
to worry about the ArrayList growing without bounds in production.
I considered adding a limit to the ArrayList, but opted not to for
simplicity. I also wanted to avoid silently discarding some errors.
@deejgregor @amarziali
Check for blocked instrumentation in tests
2162702 
If there is a blocked instrumentation, the test will fail and the
failure message will include the same information that is logged
from MuzzleCheck.
@deejgregor @amarziali
Fail fast on muzzle or instrumentation errors
fafc220
@amarziali
revert asserting muzzle failures in instrumentation tests
f696e30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@amarziali amarziali Awaiting requested review from amarziali amarziali is a code owner automatically assigned from DataDog/apm-java

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

comp: tooling Build & Tooling tag: community Community contribution tag: do not merge Do not merge changes type: enhancement Enhancements and improvements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mcculls @deejgregor @amarziali