wip by amarziali · Pull Request #8964 · DataDog/dd-trace-java

amarziali · 2025-06-11T16:28:08Z

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-06-11T17:05:31Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	andrea.marziali/test
git_commit_date	1749638913	1749659259
git_commit_sha	`089cf8f`	`cb9add0`
release_version	1.50.0-SNAPSHOT~089cf8fd47	1.50.0-SNAPSHOT~cb9add0142

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1749661484	1749661484
ci_job_id	977104362	977104362
ci_pipeline_id	67447080	67447080
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-63wmvl1-project-304-concurrent-0-t1p4gmmt 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-63wmvl1-project-304-concurrent-0-t1p4gmmt 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 12 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~cb9add0142, baseline=1.50.0-SNAPSHOT~089cf8fd47

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.026 s) : 0, 1025536
Total [baseline] (8.552 s) : 0, 8552026
Agent [candidate] (1.023 s) : 0, 1022977
Total [candidate] (8.54 s) : 0, 8539792
section iast
Agent [baseline] (1.15 s) : 0, 1149569
Total [baseline] (9.215 s) : 0, 9215056
Agent [candidate] (1.148 s) : 0, 1147844
Total [candidate] (9.202 s) : 0, 9201735
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.155 s) : 0, 1155210
Total [baseline] (9.162 s) : 0, 9162394
Agent [candidate] (1.152 s) : 0, 1151834
Total [candidate] (9.171 s) : 0, 9170566
section iast_TELEMETRY_OFF
Agent [baseline] (1.147 s) : 0, 1146758
Total [baseline] (9.218 s) : 0, 9217922
Agent [candidate] (1.157 s) : 0, 1157015
Total [candidate] (9.31 s) : 0, 9310280

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.026 s	-
Agent	iast	1.15 s	124.033 ms (12.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.155 s	129.674 ms (12.6%)
Agent	iast_TELEMETRY_OFF	1.147 s	121.222 ms (11.8%)
Total	tracing	8.552 s	-
Total	iast	9.215 s	663.03 ms (7.8%)
Total	iast_HARDCODED_SECRET_DISABLED	9.162 s	610.369 ms (7.1%)
Total	iast_TELEMETRY_OFF	9.218 s	665.897 ms (7.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.023 s	-
Agent	iast	1.148 s	124.867 ms (12.2%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.152 s	128.857 ms (12.6%)
Agent	iast_TELEMETRY_OFF	1.157 s	134.038 ms (13.1%)
Total	tracing	8.54 s	-
Total	iast	9.202 s	661.943 ms (7.8%)
Total	iast_HARDCODED_SECRET_DISABLED	9.171 s	630.774 ms (7.4%)
Total	iast_TELEMETRY_OFF	9.31 s	770.488 ms (9.0%)

gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~cb9add0142, baseline=1.50.0-SNAPSHOT~089cf8fd47

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.052 ms) : 0, 684052
BytebuddyAgent [candidate] (683.474 ms) : 0, 683474
GlobalTracer [baseline] (240.809 ms) : 0, 240809
GlobalTracer [candidate] (240.469 ms) : 0, 240469
AppSec [baseline] (56.421 ms) : 0, 56421
AppSec [candidate] (56.468 ms) : 0, 56468
Debugger [baseline] (6.206 ms) : 0, 6206
Debugger [candidate] (6.228 ms) : 0, 6228
Remote Config [baseline] (734.757 µs) : 0, 735
Remote Config [candidate] (740.646 µs) : 0, 741
Telemetry [baseline] (13.747 ms) : 0, 13747
Telemetry [candidate] (12.053 ms) : 0, 12053
section iast
BytebuddyAgent [baseline] (801.511 ms) : 0, 801511
BytebuddyAgent [candidate] (800.403 ms) : 0, 800403
GlobalTracer [baseline] (230.389 ms) : 0, 230389
GlobalTracer [candidate] (230.364 ms) : 0, 230364
IAST [baseline] (26.029 ms) : 0, 26029
IAST [candidate] (26.02 ms) : 0, 26020
AppSec [baseline] (53.438 ms) : 0, 53438
AppSec [candidate] (53.125 ms) : 0, 53125
Debugger [baseline] (6.028 ms) : 0, 6028
Debugger [candidate] (6.002 ms) : 0, 6002
Remote Config [baseline] (621.508 µs) : 0, 622
Remote Config [candidate] (598.683 µs) : 0, 599
Telemetry [baseline] (7.993 ms) : 0, 7993
Telemetry [candidate] (7.876 ms) : 0, 7876
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (804.852 ms) : 0, 804852
BytebuddyAgent [candidate] (803.418 ms) : 0, 803418
GlobalTracer [baseline] (231.208 ms) : 0, 231208
GlobalTracer [candidate] (230.468 ms) : 0, 230468
IAST [baseline] (25.67 ms) : 0, 25670
IAST [candidate] (26.986 ms) : 0, 26986
AppSec [baseline] (55.149 ms) : 0, 55149
AppSec [candidate] (52.847 ms) : 0, 52847
Debugger [baseline] (6.085 ms) : 0, 6085
Debugger [candidate] (6.011 ms) : 0, 6011
Remote Config [baseline] (625.993 µs) : 0, 626
Remote Config [candidate] (608.558 µs) : 0, 609
Telemetry [baseline] (8.012 ms) : 0, 8012
Telemetry [candidate] (7.975 ms) : 0, 7975
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (798.83 ms) : 0, 798830
BytebuddyAgent [candidate] (807.428 ms) : 0, 807428
GlobalTracer [baseline] (230.529 ms) : 0, 230529
GlobalTracer [candidate] (231.674 ms) : 0, 231674
IAST [baseline] (25.791 ms) : 0, 25791
IAST [candidate] (28.061 ms) : 0, 28061
AppSec [baseline] (53.672 ms) : 0, 53672
AppSec [candidate] (51.455 ms) : 0, 51455
Debugger [baseline] (5.978 ms) : 0, 5978
Debugger [candidate] (6.064 ms) : 0, 6064
Remote Config [baseline] (601.086 µs) : 0, 601
Remote Config [candidate] (622.889 µs) : 0, 623
Telemetry [baseline] (7.817 ms) : 0, 7817
Telemetry [candidate] (7.995 ms) : 0, 7995

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~cb9add0142, baseline=1.50.0-SNAPSHOT~089cf8fd47

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.024 s) : 0, 1023595
Total [baseline] (11.102 s) : 0, 11101719
Agent [candidate] (1.023 s) : 0, 1022707
Total [candidate] (11.123 s) : 0, 11122678
section appsec
Agent [baseline] (1.173 s) : 0, 1173027
Total [baseline] (11.248 s) : 0, 11247829
Agent [candidate] (1.179 s) : 0, 1179136
Total [candidate] (11.304 s) : 0, 11304342
section iast
Agent [baseline] (1.15 s) : 0, 1150177
Total [baseline] (11.333 s) : 0, 11332732
Agent [candidate] (1.159 s) : 0, 1158600
Total [candidate] (11.442 s) : 0, 11442477
section profiling
Agent [baseline] (1.271 s) : 0, 1271358
Total [baseline] (11.492 s) : 0, 11492218
Agent [candidate] (1.267 s) : 0, 1267145
Total [candidate] (11.509 s) : 0, 11508755

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.024 s	-
Agent	appsec	1.173 s	149.432 ms (14.6%)
Agent	iast	1.15 s	126.583 ms (12.4%)
Agent	profiling	1.271 s	247.763 ms (24.2%)
Total	tracing	11.102 s	-
Total	appsec	11.248 s	146.11 ms (1.3%)
Total	iast	11.333 s	231.013 ms (2.1%)
Total	profiling	11.492 s	390.499 ms (3.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.023 s	-
Agent	appsec	1.179 s	156.429 ms (15.3%)
Agent	iast	1.159 s	135.893 ms (13.3%)
Agent	profiling	1.267 s	244.438 ms (23.9%)
Total	tracing	11.123 s	-
Total	appsec	11.304 s	181.663 ms (1.6%)
Total	iast	11.442 s	319.799 ms (2.9%)
Total	profiling	11.509 s	386.077 ms (3.5%)

gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~cb9add0142, baseline=1.50.0-SNAPSHOT~089cf8fd47

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (683.261 ms) : 0, 683261
BytebuddyAgent [candidate] (683.111 ms) : 0, 683111
GlobalTracer [baseline] (240.526 ms) : 0, 240526
GlobalTracer [candidate] (240.686 ms) : 0, 240686
AppSec [baseline] (57.902 ms) : 0, 57902
AppSec [candidate] (57.086 ms) : 0, 57086
Debugger [baseline] (6.19 ms) : 0, 6190
Debugger [candidate] (6.171 ms) : 0, 6171
Remote Config [baseline] (736.521 µs) : 0, 737
Remote Config [candidate] (730.89 µs) : 0, 731
Telemetry [baseline] (11.366 ms) : 0, 11366
Telemetry [candidate] (11.371 ms) : 0, 11371
section appsec
BytebuddyAgent [baseline] (704.149 ms) : 0, 704149
BytebuddyAgent [candidate] (708.28 ms) : 0, 708280
GlobalTracer [baseline] (234.217 ms) : 0, 234217
GlobalTracer [candidate] (235.383 ms) : 0, 235383
IAST [baseline] (21.847 ms) : 0, 21847
IAST [candidate] (21.923 ms) : 0, 21923
AppSec [baseline] (175.501 ms) : 0, 175501
AppSec [candidate] (175.944 ms) : 0, 175944
Debugger [baseline] (5.908 ms) : 0, 5908
Debugger [candidate] (5.976 ms) : 0, 5976
Remote Config [baseline] (616.895 µs) : 0, 617
Remote Config [candidate] (617.854 µs) : 0, 618
Telemetry [baseline] (7.27 ms) : 0, 7270
Telemetry [candidate] (7.355 ms) : 0, 7355
section iast
BytebuddyAgent [baseline] (801.994 ms) : 0, 801994
BytebuddyAgent [candidate] (806.704 ms) : 0, 806704
GlobalTracer [baseline] (230.646 ms) : 0, 230646
GlobalTracer [candidate] (233.085 ms) : 0, 233085
IAST [baseline] (26.052 ms) : 0, 26052
IAST [candidate] (28.867 ms) : 0, 28867
AppSec [baseline] (53.587 ms) : 0, 53587
AppSec [candidate] (51.483 ms) : 0, 51483
Debugger [baseline] (5.918 ms) : 0, 5918
Debugger [candidate] (6.084 ms) : 0, 6084
Remote Config [baseline] (595.236 µs) : 0, 595
Remote Config [candidate] (608.884 µs) : 0, 609
Telemetry [baseline] (7.828 ms) : 0, 7828
Telemetry [candidate] (8.077 ms) : 0, 8077
section profiling
BytebuddyAgent [baseline] (678.639 ms) : 0, 678639
BytebuddyAgent [candidate] (676.312 ms) : 0, 676312
GlobalTracer [baseline] (361.258 ms) : 0, 361258
GlobalTracer [candidate] (360.874 ms) : 0, 360874
AppSec [baseline] (61.987 ms) : 0, 61987
AppSec [candidate] (61.199 ms) : 0, 61199
Debugger [baseline] (6.178 ms) : 0, 6178
Debugger [candidate] (6.085 ms) : 0, 6085
Remote Config [baseline] (672.522 µs) : 0, 673
Remote Config [candidate] (666.951 µs) : 0, 667
Telemetry [baseline] (8.164 ms) : 0, 8164
Telemetry [candidate] (8.125 ms) : 0, 8125
ProfilingAgent [baseline] (103.269 ms) : 0, 103269
ProfilingAgent [candidate] (102.972 ms) : 0, 102972
Profiling [baseline] (103.293 ms) : 0, 103293
Profiling [candidate] (102.996 ms) : 0, 102996

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-06-11T16:38:54	2025-06-11T16:48:52
git_branch	master	andrea.marziali/test
git_commit_date	1749638913	1749659259
git_commit_sha	`089cf8f`	`cb9add0`
release_version	1.50.0-SNAPSHOT~089cf8fd47	1.50.0-SNAPSHOT~cb9add0142
start_time	2025-06-11T16:38:34	2025-06-11T16:48:31

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1749660948	1749660948
ci_job_id	977104363	977104363
ci_pipeline_id	67447080	67447080
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-63wmvl1-project-304-concurrent-1-hyeiac4b 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-63wmvl1-project-304-concurrent-1-hyeiac4b 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
thresholds_or_results	results	results
variant	iast_FULL	iast_FULL

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 5 metrics, 14 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:code_origins	worse [+26.062ms; +28.302ms] or [+61.958%; +67.282%]	unstable [-58.643op/s; -32.159op/s] or [-51.995%; -28.514%]	69.247ms	67.385op/s	42.065ms	112.786op/s
scenario:load:petclinic:iast	worse [+26.960ms; +28.497ms] or [+77.107%; +81.501%]	unstable [-78.999op/s; -50.681op/s] or [-56.492%; -36.242%]	62.693ms	75.000op/s	34.965ms	139.840op/s
scenario:load:petclinic:tracing	worse [+2.320ms; +2.475ms] or [+25.165%; +26.839%]	unstable [-144.548op/s; -78.567op/s] or [-26.922%; -14.633%]	11.617ms	425.355op/s	9.220ms	536.913op/s

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	andrea.marziali/test
git_commit_date	1749638913	1749659259
git_commit_sha	`089cf8f`	`cb9add0`
release_version	1.50.0-SNAPSHOT~089cf8fd47	1.50.0-SNAPSHOT~cb9add0142

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1749661191	1749661191
ci_job_id	977104364	977104364
ci_pipeline_id	67447080	67447080
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-pz8nwqvr-project-304-concurrent-0-slwbbvtb 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-pz8nwqvr-project-304-concurrent-0-slwbbvtb 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~cb9add0142, baseline=1.50.0-SNAPSHOT~089cf8fd47
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.231 s) : 15231000, 15231000
.   : milestone, 15231000,
appsec (14.702 s) : 14702000, 14702000
.   : milestone, 14702000,
iast (18.635 s) : 18635000, 18635000
.   : milestone, 18635000,
iast_GLOBAL (17.924 s) : 17924000, 17924000
.   : milestone, 17924000,
profiling (15.177 s) : 15177000, 15177000
.   : milestone, 15177000,
tracing (14.783 s) : 14783000, 14783000
.   : milestone, 14783000,
section candidate
no_agent (14.731 s) : 14731000, 14731000
.   : milestone, 14731000,
appsec (14.882 s) : 14882000, 14882000
.   : milestone, 14882000,
iast (18.869 s) : 18869000, 18869000
.   : milestone, 18869000,
iast_GLOBAL (18.016 s) : 18016000, 18016000
.   : milestone, 18016000,
profiling (15.187 s) : 15187000, 15187000
.   : milestone, 15187000,
tracing (14.992 s) : 14992000, 14992000
.   : milestone, 14992000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.231 s [15.231 s, 15.231 s]	-
appsec	14.702 s [14.702 s, 14.702 s]	-529.0 ms (-3.5%)
iast	18.635 s [18.635 s, 18.635 s]	3.404 s (22.3%)
iast_GLOBAL	17.924 s [17.924 s, 17.924 s]	2.693 s (17.7%)
profiling	15.177 s [15.177 s, 15.177 s]	-54.0 ms (-0.4%)
tracing	14.783 s [14.783 s, 14.783 s]	-448.0 ms (-2.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.731 s [14.731 s, 14.731 s]	-
appsec	14.882 s [14.882 s, 14.882 s]	151.0 ms (1.0%)
iast	18.869 s [18.869 s, 18.869 s]	4.138 s (28.1%)
iast_GLOBAL	18.016 s [18.016 s, 18.016 s]	3.285 s (22.3%)
profiling	15.187 s [15.187 s, 15.187 s]	456.0 ms (3.1%)
tracing	14.992 s [14.992 s, 14.992 s]	261.0 ms (1.8%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~cb9add0142, baseline=1.50.0-SNAPSHOT~089cf8fd47
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.482 ms) : 1471, 1494
.   : milestone, 1482,
appsec (2.41 ms) : 2361, 2458
.   : milestone, 2410,
iast (2.202 ms) : 2141, 2263
.   : milestone, 2202,
iast_GLOBAL (2.222 ms) : 2161, 2283
.   : milestone, 2222,
profiling (2.031 ms) : 1982, 2079
.   : milestone, 2031,
tracing (2.01 ms) : 1963, 2057
.   : milestone, 2010,
section candidate
no_agent (1.479 ms) : 1468, 1491
.   : milestone, 1479,
appsec (2.399 ms) : 2351, 2448
.   : milestone, 2399,
iast (2.195 ms) : 2133, 2256
.   : milestone, 2195,
iast_GLOBAL (2.244 ms) : 2183, 2306
.   : milestone, 2244,
profiling (2.494 ms) : 2320, 2667
.   : milestone, 2494,
tracing (2.03 ms) : 1983, 2078
.   : milestone, 2030,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.482 ms [1.471 ms, 1.494 ms]	-
appsec	2.41 ms [2.361 ms, 2.458 ms]	927.646 µs (62.6%)
iast	2.202 ms [2.141 ms, 2.263 ms]	719.963 µs (48.6%)
iast_GLOBAL	2.222 ms [2.161 ms, 2.283 ms]	739.903 µs (49.9%)
profiling	2.031 ms [1.982 ms, 2.079 ms]	548.827 µs (37.0%)
tracing	2.01 ms [1.963 ms, 2.057 ms]	528.212 µs (35.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.479 ms [1.468 ms, 1.491 ms]	-
appsec	2.399 ms [2.351 ms, 2.448 ms]	920.05 µs (62.2%)
iast	2.195 ms [2.133 ms, 2.256 ms]	715.609 µs (48.4%)
iast_GLOBAL	2.244 ms [2.183 ms, 2.306 ms]	764.898 µs (51.7%)
profiling	2.494 ms [2.32 ms, 2.667 ms]	1.014 ms (68.6%)
tracing	2.03 ms [1.983 ms, 2.078 ms]	550.987 µs (37.2%)

wip

cb9add0

amarziali closed this Jun 12, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wip#8964

wip#8964
amarziali wants to merge 1 commit intomasterfrom
andrea.marziali/test

amarziali commented Jun 11, 2025

Uh oh!

pr-commenter bot commented Jun 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

amarziali commented Jun 11, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Jun 11, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant