Improve benchmark readme by sarahchen6 · Pull Request #9038 · DataDog/dd-trace-java

sarahchen6 · 2025-06-25T21:27:45Z

What Does This Do

Improves the benchmark README file a bit

Motivation

Improve documentation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

github-actions · 2025-06-25T21:27:56Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

pr-commenter · 2025-06-25T21:59:17Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/update-benchmark-readme
git_commit_date	1750865797	1750886772
git_commit_sha	`d6d3d21`	`2aceb0f`
release_version	1.51.0-SNAPSHOT~d6d3d210b1	1.51.0-SNAPSHOT~2aceb0fcd7

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750888612	1750888612
ci_job_id	999152158	999152158
ci_pipeline_id	68795276	68795276
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-vvzzugvx-project-304-concurrent-0-wiodoke5 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-vvzzugvx-project-304-concurrent-0-wiodoke5 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 8 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~2aceb0fcd7, baseline=1.51.0-SNAPSHOT~d6d3d210b1

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.005 s) : 0, 1004530
Total [baseline] (10.756 s) : 0, 10756468
Agent [candidate] (997.138 ms) : 0, 997138
Total [candidate] (10.764 s) : 0, 10763949
section appsec
Agent [baseline] (1.175 s) : 0, 1174685
Total [baseline] (10.81 s) : 0, 10810300
Agent [candidate] (1.174 s) : 0, 1173990
Total [candidate] (10.738 s) : 0, 10738385
section iast
Agent [baseline] (1.133 s) : 0, 1132839
Total [baseline] (10.836 s) : 0, 10836343
Agent [candidate] (1.14 s) : 0, 1139795
Total [candidate] (10.861 s) : 0, 10861247
section profiling
Agent [baseline] (1.26 s) : 0, 1259861
Total [baseline] (11.099 s) : 0, 11099377
Agent [candidate] (1.255 s) : 0, 1254833
Total [candidate] (11.033 s) : 0, 11032519

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.005 s	-
Agent	appsec	1.175 s	170.155 ms (16.9%)
Agent	iast	1.133 s	128.308 ms (12.8%)
Agent	profiling	1.26 s	255.331 ms (25.4%)
Total	tracing	10.756 s	-
Total	appsec	10.81 s	53.832 ms (0.5%)
Total	iast	10.836 s	79.875 ms (0.7%)
Total	profiling	11.099 s	342.909 ms (3.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	997.138 ms	-
Agent	appsec	1.174 s	176.852 ms (17.7%)
Agent	iast	1.14 s	142.657 ms (14.3%)
Agent	profiling	1.255 s	257.695 ms (25.8%)
Total	tracing	10.764 s	-
Total	appsec	10.738 s	-25.564 ms (-0.2%)
Total	iast	10.861 s	97.298 ms (0.9%)
Total	profiling	11.033 s	268.569 ms (2.5%)

gantt
    title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~2aceb0fcd7, baseline=1.51.0-SNAPSHOT~d6d3d210b1

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (693.084 ms) : 0, 693084
BytebuddyAgent [candidate] (687.835 ms) : 0, 687835
GlobalTracer [baseline] (244.87 ms) : 0, 244870
GlobalTracer [candidate] (242.889 ms) : 0, 242889
AppSec [baseline] (30.463 ms) : 0, 30463
AppSec [candidate] (30.462 ms) : 0, 30462
Debugger [baseline] (6.102 ms) : 0, 6102
Debugger [candidate] (6.086 ms) : 0, 6086
Remote Config [baseline] (673.331 µs) : 0, 673
Remote Config [candidate] (681.549 µs) : 0, 682
Telemetry [baseline] (8.273 ms) : 0, 8273
Telemetry [candidate] (8.254 ms) : 0, 8254
section appsec
BytebuddyAgent [baseline] (710.573 ms) : 0, 710573
BytebuddyAgent [candidate] (710.296 ms) : 0, 710296
GlobalTracer [baseline] (236.366 ms) : 0, 236366
GlobalTracer [candidate] (236.077 ms) : 0, 236077
AppSec [baseline] (170.267 ms) : 0, 170267
AppSec [candidate] (170.278 ms) : 0, 170278
Debugger [baseline] (5.785 ms) : 0, 5785
Debugger [candidate] (5.783 ms) : 0, 5783
Remote Config [baseline] (611.018 µs) : 0, 611
Remote Config [candidate] (607.653 µs) : 0, 608
Telemetry [baseline] (8.155 ms) : 0, 8155
Telemetry [candidate] (8.134 ms) : 0, 8134
IAST [baseline] (22.045 ms) : 0, 22045
IAST [candidate] (21.978 ms) : 0, 21978
section iast
BytebuddyAgent [baseline] (808.618 ms) : 0, 808618
BytebuddyAgent [candidate] (812.993 ms) : 0, 812993
GlobalTracer [baseline] (233.384 ms) : 0, 233384
GlobalTracer [candidate] (234.851 ms) : 0, 234851
AppSec [baseline] (26.85 ms) : 0, 26850
AppSec [candidate] (29.829 ms) : 0, 29829
Debugger [baseline] (5.835 ms) : 0, 5835
Debugger [candidate] (5.882 ms) : 0, 5882
Remote Config [baseline] (584.293 µs) : 0, 584
Remote Config [candidate] (595.026 µs) : 0, 595
Telemetry [baseline] (7.894 ms) : 0, 7894
Telemetry [candidate] (8.089 ms) : 0, 8089
IAST [baseline] (28.737 ms) : 0, 28737
IAST [candidate] (26.602 ms) : 0, 26602
section profiling
BytebuddyAgent [baseline] (688.07 ms) : 0, 688070
BytebuddyAgent [candidate] (683.508 ms) : 0, 683508
GlobalTracer [baseline] (364.356 ms) : 0, 364356
GlobalTracer [candidate] (364.431 ms) : 0, 364431
AppSec [baseline] (32.542 ms) : 0, 32542
AppSec [candidate] (31.973 ms) : 0, 31973
Debugger [baseline] (10.645 ms) : 0, 10645
Debugger [candidate] (10.64 ms) : 0, 10640
Remote Config [baseline] (1.4 ms) : 0, 1400
Remote Config [candidate] (664.606 µs) : 0, 665
Telemetry [baseline] (8.891 ms) : 0, 8891
Telemetry [candidate] (10.337 ms) : 0, 10337
ProfilingAgent [baseline] (104.572 ms) : 0, 104572
ProfilingAgent [candidate] (104.033 ms) : 0, 104033
Profiling [baseline] (104.597 ms) : 0, 104597
Profiling [candidate] (104.058 ms) : 0, 104058

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~2aceb0fcd7, baseline=1.51.0-SNAPSHOT~d6d3d210b1

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (999.469 ms) : 0, 999469
Total [baseline] (8.592 s) : 0, 8591981
Agent [candidate] (997.788 ms) : 0, 997788
Total [candidate] (8.573 s) : 0, 8572748
section iast
Agent [baseline] (1.143 s) : 0, 1143432
Total [baseline] (9.325 s) : 0, 9324867
Agent [candidate] (1.136 s) : 0, 1135631
Total [candidate] (9.32 s) : 0, 9319843

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	999.469 ms	-
Agent	iast	1.143 s	143.963 ms (14.4%)
Total	tracing	8.592 s	-
Total	iast	9.325 s	732.886 ms (8.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	997.788 ms	-
Agent	iast	1.136 s	137.843 ms (13.8%)
Total	tracing	8.573 s	-
Total	iast	9.32 s	747.095 ms (8.7%)

gantt
    title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~2aceb0fcd7, baseline=1.51.0-SNAPSHOT~d6d3d210b1

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (689.867 ms) : 0, 689867
BytebuddyAgent [candidate] (688.434 ms) : 0, 688434
GlobalTracer [baseline] (243.141 ms) : 0, 243141
GlobalTracer [candidate] (243.024 ms) : 0, 243024
AppSec [baseline] (30.516 ms) : 0, 30516
AppSec [candidate] (30.446 ms) : 0, 30446
Debugger [baseline] (6.06 ms) : 0, 6060
Debugger [candidate] (6.067 ms) : 0, 6067
Remote Config [baseline] (670.116 µs) : 0, 670
Remote Config [candidate] (670.58 µs) : 0, 671
Telemetry [baseline] (8.26 ms) : 0, 8260
Telemetry [candidate] (8.246 ms) : 0, 8246
section iast
BytebuddyAgent [baseline] (816.209 ms) : 0, 816209
BytebuddyAgent [candidate] (811.02 ms) : 0, 811020
GlobalTracer [baseline] (235.806 ms) : 0, 235806
GlobalTracer [candidate] (233.845 ms) : 0, 233845
AppSec [baseline] (28.702 ms) : 0, 28702
AppSec [candidate] (26.765 ms) : 0, 26765
Debugger [baseline] (5.809 ms) : 0, 5809
Debugger [candidate] (5.82 ms) : 0, 5820
Remote Config [baseline] (591.102 µs) : 0, 591
Remote Config [candidate] (579.988 µs) : 0, 580
Telemetry [baseline] (8.053 ms) : 0, 8053
Telemetry [candidate] (7.994 ms) : 0, 7994
IAST [baseline] (27.352 ms) : 0, 27352
IAST [candidate] (28.745 ms) : 0, 28745

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/update-benchmark-readme
git_commit_date	1750865797	1750886772
git_commit_sha	`d6d3d21`	`2aceb0f`
release_version	1.51.0-SNAPSHOT~d6d3d210b1	1.51.0-SNAPSHOT~2aceb0fcd7

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750888287	1750888287
ci_job_id	999152160	999152160
ci_pipeline_id	68795276	68795276
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-hvkztcm7-project-304-concurrent-0-41bw25nj 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-hvkztcm7-project-304-concurrent-0-41bw25nj 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 4 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load	worse [+221.963µs; +616.686µs] or [+2.268%; +6.303%]	unstable [-75.773op/s; +37.148op/s] or [-15.964%; +7.826%]	10.204ms	455.344op/s	9.785ms	474.656op/s
scenario:load:insecure-bank:no_agent:high_load	worse [+119.458µs; +227.849µs] or [+2.769%; +5.282%]	unstable [-158.021op/s; +76.833op/s] or [-14.859%; +7.225%]	4.488ms	1022.906op/s	4.314ms	1063.500op/s
scenario:load:petclinic:profiling:high_load	worse [+1.361ms; +2.246ms] or [+2.988%; +4.930%]	unstable [-10.913op/s; +3.088op/s] or [-10.627%; +3.007%]	47.367ms	98.775op/s	45.563ms	102.688op/s
scenario:load:petclinic:code_origins:high_load	worse [+1.411ms; +2.235ms] or [+3.198%; +5.063%]	unstable [-11.440op/s; +3.115op/s] or [-10.794%; +2.939%]	45.961ms	101.825op/s	44.138ms	105.987op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~2aceb0fcd7, baseline=1.51.0-SNAPSHOT~d6d3d210b1
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.786 ms) : 36491, 37082
.   : milestone, 36786,
appsec (47.52 ms) : 47105, 47936
.   : milestone, 47520,
code_origins (44.138 ms) : 43768, 44508
.   : milestone, 44138,
iast (42.716 ms) : 42337, 43096
.   : milestone, 42716,
profiling (45.563 ms) : 45158, 45969
.   : milestone, 45563,
tracing (43.321 ms) : 42958, 43684
.   : milestone, 43321,
section candidate
no_agent (36.303 ms) : 36007, 36598
.   : milestone, 36303,
appsec (47.112 ms) : 46704, 47520
.   : milestone, 47112,
code_origins (45.961 ms) : 45566, 46356
.   : milestone, 45961,
iast (43.79 ms) : 43402, 44177
.   : milestone, 43790,
profiling (47.367 ms) : 46950, 47784
.   : milestone, 47367,
tracing (43.537 ms) : 43168, 43907
.   : milestone, 43537,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.786 ms [36.491 ms, 37.082 ms]	-
appsec	47.52 ms [47.105 ms, 47.936 ms]	10.734 ms (29.2%)
code_origins	44.138 ms [43.768 ms, 44.508 ms]	7.352 ms (20.0%)
iast	42.716 ms [42.337 ms, 43.096 ms]	5.93 ms (16.1%)
profiling	45.563 ms [45.158 ms, 45.969 ms]	8.777 ms (23.9%)
tracing	43.321 ms [42.958 ms, 43.684 ms]	6.535 ms (17.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.303 ms [36.007 ms, 36.598 ms]	-
appsec	47.112 ms [46.704 ms, 47.52 ms]	10.81 ms (29.8%)
code_origins	45.961 ms [45.566 ms, 46.356 ms]	9.658 ms (26.6%)
iast	43.79 ms [43.402 ms, 44.177 ms]	7.487 ms (20.6%)
profiling	47.367 ms [46.95 ms, 47.784 ms]	11.065 ms (30.5%)
tracing	43.537 ms [43.168 ms, 43.907 ms]	7.235 ms (19.9%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~2aceb0fcd7, baseline=1.51.0-SNAPSHOT~d6d3d210b1
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.314 ms) : 4264, 4364
.   : milestone, 4314,
iast (9.055 ms) : 8900, 9211
.   : milestone, 9055,
iast_FULL (14.115 ms) : 13830, 14401
.   : milestone, 14115,
iast_GLOBAL (9.785 ms) : 9608, 9962
.   : milestone, 9785,
profiling (8.955 ms) : 8817, 9092
.   : milestone, 8955,
tracing (7.616 ms) : 7510, 7722
.   : milestone, 7616,
section candidate
no_agent (4.488 ms) : 4437, 4538
.   : milestone, 4488,
iast (9.098 ms) : 8951, 9246
.   : milestone, 9098,
iast_FULL (13.8 ms) : 13528, 14072
.   : milestone, 13800,
iast_GLOBAL (10.204 ms) : 10015, 10394
.   : milestone, 10204,
profiling (8.805 ms) : 8662, 8947
.   : milestone, 8805,
tracing (7.625 ms) : 7519, 7730
.   : milestone, 7625,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.314 ms [4.264 ms, 4.364 ms]	-
iast	9.055 ms [8.9 ms, 9.211 ms]	4.741 ms (109.9%)
iast_FULL	14.115 ms [13.83 ms, 14.401 ms]	9.801 ms (227.2%)
iast_GLOBAL	9.785 ms [9.608 ms, 9.962 ms]	5.471 ms (126.8%)
profiling	8.955 ms [8.817 ms, 9.092 ms]	4.641 ms (107.6%)
tracing	7.616 ms [7.51 ms, 7.722 ms]	3.302 ms (76.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.488 ms [4.437 ms, 4.538 ms]	-
iast	9.098 ms [8.951 ms, 9.246 ms]	4.611 ms (102.7%)
iast_FULL	13.8 ms [13.528 ms, 14.072 ms]	9.312 ms (207.5%)
iast_GLOBAL	10.204 ms [10.015 ms, 10.394 ms]	5.716 ms (127.4%)
profiling	8.805 ms [8.662 ms, 8.947 ms]	4.317 ms (96.2%)
tracing	7.625 ms [7.519 ms, 7.73 ms]	3.137 ms (69.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/update-benchmark-readme
git_commit_date	1750865797	1750886772
git_commit_sha	`d6d3d21`	`2aceb0f`
release_version	1.51.0-SNAPSHOT~d6d3d210b1	1.51.0-SNAPSHOT~2aceb0fcd7

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1750888720	1750888720
ci_job_id	999152162	999152162
ci_pipeline_id	68795276	68795276
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-vvzzugvx-project-304-concurrent-1-47az0azn 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-vvzzugvx-project-304-concurrent-1-47az0azn 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~2aceb0fcd7, baseline=1.51.0-SNAPSHOT~d6d3d210b1
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.478 ms) : 1467, 1490
.   : milestone, 1478,
appsec (2.393 ms) : 2344, 2442
.   : milestone, 2393,
iast (2.196 ms) : 2134, 2258
.   : milestone, 2196,
iast_GLOBAL (2.239 ms) : 2176, 2302
.   : milestone, 2239,
profiling (2.027 ms) : 1978, 2077
.   : milestone, 2027,
tracing (2.009 ms) : 1961, 2057
.   : milestone, 2009,
section candidate
no_agent (1.479 ms) : 1467, 1491
.   : milestone, 1479,
appsec (2.407 ms) : 2357, 2456
.   : milestone, 2407,
iast (2.194 ms) : 2132, 2256
.   : milestone, 2194,
iast_GLOBAL (2.231 ms) : 2168, 2293
.   : milestone, 2231,
profiling (2.039 ms) : 1989, 2089
.   : milestone, 2039,
tracing (2.014 ms) : 1965, 2062
.   : milestone, 2014,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.467 ms, 1.49 ms]	-
appsec	2.393 ms [2.344 ms, 2.442 ms]	915.14 µs (61.9%)
iast	2.196 ms [2.134 ms, 2.258 ms]	717.889 µs (48.6%)
iast_GLOBAL	2.239 ms [2.176 ms, 2.302 ms]	761.121 µs (51.5%)
profiling	2.027 ms [1.978 ms, 2.077 ms]	549.106 µs (37.1%)
tracing	2.009 ms [1.961 ms, 2.057 ms]	530.785 µs (35.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.479 ms [1.467 ms, 1.491 ms]	-
appsec	2.407 ms [2.357 ms, 2.456 ms]	927.51 µs (62.7%)
iast	2.194 ms [2.132 ms, 2.256 ms]	714.636 µs (48.3%)
iast_GLOBAL	2.231 ms [2.168 ms, 2.293 ms]	751.423 µs (50.8%)
profiling	2.039 ms [1.989 ms, 2.089 ms]	559.699 µs (37.8%)
tracing	2.014 ms [1.965 ms, 2.062 ms]	534.519 µs (36.1%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~2aceb0fcd7, baseline=1.51.0-SNAPSHOT~d6d3d210b1
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.812 s) : 14812000, 14812000
.   : milestone, 14812000,
appsec (14.7 s) : 14700000, 14700000
.   : milestone, 14700000,
iast (18.264 s) : 18264000, 18264000
.   : milestone, 18264000,
iast_GLOBAL (18.01 s) : 18010000, 18010000
.   : milestone, 18010000,
profiling (15.226 s) : 15226000, 15226000
.   : milestone, 15226000,
tracing (14.796 s) : 14796000, 14796000
.   : milestone, 14796000,
section candidate
no_agent (14.922 s) : 14922000, 14922000
.   : milestone, 14922000,
appsec (14.832 s) : 14832000, 14832000
.   : milestone, 14832000,
iast (18.101 s) : 18101000, 18101000
.   : milestone, 18101000,
iast_GLOBAL (18.004 s) : 18004000, 18004000
.   : milestone, 18004000,
profiling (15.16 s) : 15160000, 15160000
.   : milestone, 15160000,
tracing (14.887 s) : 14887000, 14887000
.   : milestone, 14887000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.812 s [14.812 s, 14.812 s]	-
appsec	14.7 s [14.7 s, 14.7 s]	-112.0 ms (-0.8%)
iast	18.264 s [18.264 s, 18.264 s]	3.452 s (23.3%)
iast_GLOBAL	18.01 s [18.01 s, 18.01 s]	3.198 s (21.6%)
profiling	15.226 s [15.226 s, 15.226 s]	414.0 ms (2.8%)
tracing	14.796 s [14.796 s, 14.796 s]	-16.0 ms (-0.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.922 s [14.922 s, 14.922 s]	-
appsec	14.832 s [14.832 s, 14.832 s]	-90.0 ms (-0.6%)
iast	18.101 s [18.101 s, 18.101 s]	3.179 s (21.3%)
iast_GLOBAL	18.004 s [18.004 s, 18.004 s]	3.082 s (20.7%)
profiling	15.16 s [15.16 s, 15.16 s]	238.0 ms (1.6%)
tracing	14.887 s [14.887 s, 14.887 s]	-35.0 ms (-0.2%)

PerfectSlayer

Looks good to me. cc @bric3 if you would like to have a look too

Update benchmark readme

2aceb0f

sarahchen6 requested a review from a team as a code owner June 25, 2025 21:27

sarahchen6 requested review from PerfectSlayer and removed request for a team June 25, 2025 21:27

sarahchen6 added comp: testing Testing type: documentation tag: no release notes Changes to exclude from release notes labels Jun 25, 2025

sarahchen6 changed the title ~~Update benchmark readme~~ Improve benchmark readme Jun 25, 2025

PerfectSlayer approved these changes Jun 26, 2025

View reviewed changes

smola approved these changes Jun 26, 2025

View reviewed changes

bric3 approved these changes Jun 26, 2025

View reviewed changes

sarahchen6 merged commit 6353d83 into master Jun 26, 2025
512 of 514 checks passed

sarahchen6 deleted the sarahchen6/update-benchmark-readme branch June 26, 2025 16:04

github-actions Bot added this to the 1.51.0 milestone Jun 26, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve benchmark readme#9038

Improve benchmark readme#9038
sarahchen6 merged 1 commit into
masterfrom
sarahchen6/update-benchmark-readme

sarahchen6 commented Jun 25, 2025 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 25, 2025

Uh oh!

pr-commenter Bot commented Jun 25, 2025

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

sarahchen6 commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions Bot commented Jun 25, 2025

Uh oh!

pr-commenter Bot commented Jun 25, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sarahchen6 commented Jun 25, 2025 •

edited

Loading