Open
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~491533bc5f, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1063954
Total [baseline] (8.827 s) : 0, 8827127
Agent [candidate] (1.065 s) : 0, 1064546
Total [candidate] (8.86 s) : 0, 8860284
section iast
Agent [baseline] (1.247 s) : 0, 1247278
Total [baseline] (9.522 s) : 0, 9522184
Agent [candidate] (1.241 s) : 0, 1241144
Total [candidate] (9.533 s) : 0, 9532897
gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~491533bc5f, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.223 ms) : 0, 1223
crashtracking [candidate] (1.225 ms) : 0, 1225
BytebuddyAgent [baseline] (636.134 ms) : 0, 636134
BytebuddyAgent [candidate] (636.167 ms) : 0, 636167
AgentMeter [baseline] (29.397 ms) : 0, 29397
AgentMeter [candidate] (29.394 ms) : 0, 29394
GlobalTracer [baseline] (248.776 ms) : 0, 248776
GlobalTracer [candidate] (248.906 ms) : 0, 248906
AppSec [baseline] (32.828 ms) : 0, 32828
AppSec [candidate] (32.757 ms) : 0, 32757
Debugger [baseline] (60.045 ms) : 0, 60045
Debugger [candidate] (59.966 ms) : 0, 59966
Remote Config [baseline] (608.052 µs) : 0, 608
Remote Config [candidate] (608.447 µs) : 0, 608
Telemetry [baseline] (9.889 ms) : 0, 9889
Telemetry [candidate] (8.358 ms) : 0, 8358
Flare Poller [baseline] (9.015 ms) : 0, 9015
Flare Poller [candidate] (11.23 ms) : 0, 11230
section iast
crashtracking [baseline] (1.234 ms) : 0, 1234
crashtracking [candidate] (1.21 ms) : 0, 1210
BytebuddyAgent [baseline] (824.563 ms) : 0, 824563
BytebuddyAgent [candidate] (821.633 ms) : 0, 821633
AgentMeter [baseline] (11.413 ms) : 0, 11413
AgentMeter [candidate] (11.281 ms) : 0, 11281
GlobalTracer [baseline] (239.431 ms) : 0, 239431
GlobalTracer [candidate] (237.548 ms) : 0, 237548
AppSec [baseline] (32.479 ms) : 0, 32479
AppSec [candidate] (30.588 ms) : 0, 30588
Debugger [baseline] (62.689 ms) : 0, 62689
Debugger [candidate] (62.168 ms) : 0, 62168
Remote Config [baseline] (522.555 µs) : 0, 523
Remote Config [candidate] (513.202 µs) : 0, 513
Telemetry [baseline] (7.897 ms) : 0, 7897
Telemetry [candidate] (7.899 ms) : 0, 7899
Flare Poller [baseline] (3.348 ms) : 0, 3348
Flare Poller [candidate] (3.348 ms) : 0, 3348
IAST [baseline] (27.561 ms) : 0, 27561
IAST [candidate] (29.002 ms) : 0, 29002
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~491533bc5f, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1064569
Total [baseline] (11.04 s) : 0, 11040175
Agent [candidate] (1.066 s) : 0, 1065929
Total [candidate] (11.031 s) : 0, 11030501
section appsec
Agent [baseline] (1.271 s) : 0, 1270692
Total [baseline] (11.156 s) : 0, 11155977
Agent [candidate] (1.265 s) : 0, 1265180
Total [candidate] (11.052 s) : 0, 11051624
section iast
Agent [baseline] (1.25 s) : 0, 1249592
Total [baseline] (11.278 s) : 0, 11278173
Agent [candidate] (1.254 s) : 0, 1253616
Total [candidate] (11.321 s) : 0, 11320819
section profiling
Agent [baseline] (1.187 s) : 0, 1187311
Total [baseline] (11.067 s) : 0, 11066698
Agent [candidate] (1.185 s) : 0, 1185496
Total [candidate] (11.058 s) : 0, 11057786
gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~491533bc5f, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.215 ms) : 0, 1215
crashtracking [candidate] (1.218 ms) : 0, 1218
BytebuddyAgent [baseline] (635.54 ms) : 0, 635540
BytebuddyAgent [candidate] (635.361 ms) : 0, 635361
AgentMeter [baseline] (29.478 ms) : 0, 29478
AgentMeter [candidate] (29.269 ms) : 0, 29269
GlobalTracer [baseline] (248.557 ms) : 0, 248557
GlobalTracer [candidate] (249.567 ms) : 0, 249567
AppSec [baseline] (32.731 ms) : 0, 32731
AppSec [candidate] (33.015 ms) : 0, 33015
Debugger [baseline] (60.591 ms) : 0, 60591
Debugger [candidate] (61.067 ms) : 0, 61067
Remote Config [baseline] (606.544 µs) : 0, 607
Remote Config [candidate] (610.794 µs) : 0, 611
Telemetry [baseline] (8.359 ms) : 0, 8359
Telemetry [candidate] (9.963 ms) : 0, 9963
Flare Poller [baseline] (11.448 ms) : 0, 11448
Flare Poller [candidate] (9.887 ms) : 0, 9887
section appsec
crashtracking [baseline] (1.227 ms) : 0, 1227
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (677.819 ms) : 0, 677819
BytebuddyAgent [candidate] (675.14 ms) : 0, 675140
AgentMeter [baseline] (12.32 ms) : 0, 12320
AgentMeter [candidate] (12.315 ms) : 0, 12315
GlobalTracer [baseline] (250.303 ms) : 0, 250303
GlobalTracer [candidate] (249.197 ms) : 0, 249197
AppSec [baseline] (185.862 ms) : 0, 185862
AppSec [candidate] (184.836 ms) : 0, 184836
Debugger [baseline] (65.227 ms) : 0, 65227
Debugger [candidate] (64.566 ms) : 0, 64566
Remote Config [baseline] (567.303 µs) : 0, 567
Remote Config [candidate] (566.8 µs) : 0, 567
Telemetry [baseline] (7.897 ms) : 0, 7897
Telemetry [candidate] (7.729 ms) : 0, 7729
Flare Poller [baseline] (8.051 ms) : 0, 8051
Flare Poller [candidate] (6.171 ms) : 0, 6171
IAST [baseline] (24.694 ms) : 0, 24694
IAST [candidate] (24.606 ms) : 0, 24606
section iast
crashtracking [baseline] (1.229 ms) : 0, 1229
crashtracking [candidate] (1.225 ms) : 0, 1225
BytebuddyAgent [baseline] (827.354 ms) : 0, 827354
BytebuddyAgent [candidate] (830.429 ms) : 0, 830429
AgentMeter [baseline] (11.344 ms) : 0, 11344
AgentMeter [candidate] (11.408 ms) : 0, 11408
GlobalTracer [baseline] (238.247 ms) : 0, 238247
GlobalTracer [candidate] (239.193 ms) : 0, 239193
AppSec [baseline] (30.002 ms) : 0, 30002
AppSec [candidate] (30.675 ms) : 0, 30675
Debugger [baseline] (63.67 ms) : 0, 63670
Debugger [candidate] (63.571 ms) : 0, 63571
Remote Config [baseline] (524.381 µs) : 0, 524
Remote Config [candidate] (528.012 µs) : 0, 528
Telemetry [baseline] (7.993 ms) : 0, 7993
Telemetry [candidate] (7.974 ms) : 0, 7974
Flare Poller [baseline] (3.365 ms) : 0, 3365
Flare Poller [candidate] (3.354 ms) : 0, 3354
IAST [baseline] (29.784 ms) : 0, 29784
IAST [candidate] (29.09 ms) : 0, 29090
section profiling
crashtracking [baseline] (1.186 ms) : 0, 1186
crashtracking [candidate] (1.183 ms) : 0, 1183
BytebuddyAgent [baseline] (693.344 ms) : 0, 693344
BytebuddyAgent [candidate] (691.457 ms) : 0, 691457
AgentMeter [baseline] (8.946 ms) : 0, 8946
AgentMeter [candidate] (8.868 ms) : 0, 8868
GlobalTracer [baseline] (207.937 ms) : 0, 207937
GlobalTracer [candidate] (207.656 ms) : 0, 207656
AppSec [baseline] (32.674 ms) : 0, 32674
AppSec [candidate] (32.782 ms) : 0, 32782
Debugger [baseline] (65.761 ms) : 0, 65761
Debugger [candidate] (65.753 ms) : 0, 65753
Remote Config [baseline] (567.771 µs) : 0, 568
Remote Config [candidate] (572.159 µs) : 0, 572
Telemetry [baseline] (8.116 ms) : 0, 8116
Telemetry [candidate] (8.07 ms) : 0, 8070
Flare Poller [baseline] (3.551 ms) : 0, 3551
Flare Poller [candidate] (3.552 ms) : 0, 3552
ProfilingAgent [baseline] (93.758 ms) : 0, 93758
ProfilingAgent [candidate] (94.321 ms) : 0, 94321
Profiling [baseline] (94.338 ms) : 0, 94338
Profiling [candidate] (94.899 ms) : 0, 94899
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 1 performance regressions! Performance is the same for 16 metrics, 19 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~491533bc5f, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (1.246 ms) : 1234, 1258
. : milestone, 1246,
iast (3.304 ms) : 3256, 3352
. : milestone, 3304,
iast_FULL (6.017 ms) : 5956, 6078
. : milestone, 6017,
iast_GLOBAL (3.665 ms) : 3594, 3735
. : milestone, 3665,
profiling (2.266 ms) : 2244, 2288
. : milestone, 2266,
tracing (1.899 ms) : 1883, 1914
. : milestone, 1899,
section candidate
no_agent (1.27 ms) : 1257, 1284
. : milestone, 1270,
iast (3.355 ms) : 3306, 3403
. : milestone, 3355,
iast_FULL (6.114 ms) : 6051, 6176
. : milestone, 6114,
iast_GLOBAL (3.65 ms) : 3587, 3713
. : milestone, 3650,
profiling (2.339 ms) : 2317, 2362
. : milestone, 2339,
tracing (1.89 ms) : 1874, 1906
. : milestone, 1890,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~491533bc5f, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (17.026 ms) : 16858, 17193
. : milestone, 17026,
appsec (18.595 ms) : 18405, 18785
. : milestone, 18595,
code_origins (17.83 ms) : 17654, 18006
. : milestone, 17830,
iast (17.828 ms) : 17652, 18004
. : milestone, 17828,
profiling (18.234 ms) : 18056, 18413
. : milestone, 18234,
tracing (18.261 ms) : 18079, 18443
. : milestone, 18261,
section candidate
no_agent (17.261 ms) : 17092, 17430
. : milestone, 17261,
appsec (18.65 ms) : 18462, 18838
. : milestone, 18650,
code_origins (18.835 ms) : 18646, 19023
. : milestone, 18835,
iast (17.769 ms) : 17595, 17943
. : milestone, 17769,
profiling (18.357 ms) : 18173, 18542
. : milestone, 18357,
tracing (18.027 ms) : 17849, 18205
. : milestone, 18027,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~491533bc5f, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (15.497 s) : 15497000, 15497000
. : milestone, 15497000,
appsec (14.557 s) : 14557000, 14557000
. : milestone, 14557000,
iast (18.188 s) : 18188000, 18188000
. : milestone, 18188000,
iast_GLOBAL (17.884 s) : 17884000, 17884000
. : milestone, 17884000,
profiling (15.197 s) : 15197000, 15197000
. : milestone, 15197000,
tracing (14.645 s) : 14645000, 14645000
. : milestone, 14645000,
section candidate
no_agent (15.502 s) : 15502000, 15502000
. : milestone, 15502000,
appsec (14.683 s) : 14683000, 14683000
. : milestone, 14683000,
iast (18.257 s) : 18257000, 18257000
. : milestone, 18257000,
iast_GLOBAL (18.076 s) : 18076000, 18076000
. : milestone, 18076000,
profiling (15.252 s) : 15252000, 15252000
. : milestone, 15252000,
tracing (14.806 s) : 14806000, 14806000
. : milestone, 14806000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~491533bc5f, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (1.485 ms) : 1473, 1496
. : milestone, 1485,
appsec (3.863 ms) : 3638, 4088
. : milestone, 3863,
iast (2.285 ms) : 2215, 2356
. : milestone, 2285,
iast_GLOBAL (2.337 ms) : 2266, 2408
. : milestone, 2337,
profiling (2.102 ms) : 2047, 2157
. : milestone, 2102,
tracing (2.094 ms) : 2039, 2148
. : milestone, 2094,
section candidate
no_agent (1.491 ms) : 1480, 1503
. : milestone, 1491,
appsec (3.817 ms) : 3594, 4039
. : milestone, 3817,
iast (2.283 ms) : 2212, 2353
. : milestone, 2283,
iast_GLOBAL (2.328 ms) : 2257, 2399
. : milestone, 2328,
profiling (2.102 ms) : 2047, 2157
. : milestone, 2102,
tracing (2.089 ms) : 2034, 2143
. : milestone, 2089,
|
mcculls
reviewed
Apr 27, 2026
| @Override | ||
| public void flush() { | ||
| OtlpPayload payload = collector.collectTraces(); | ||
| if (payload.getContentLength() > 0) { |
Contributor
There was a problem hiding this comment.
Suggested change
| if (payload.getContentLength() > 0) { | |
| if (payload != OtlpPayload.EMPTY) { |
for consistency with OtlpMetricsService (just in case in the future we have payloads that don't declare their content length up-front)
mtoffl01
commented
Apr 27, 2026
mtoffl01
commented
Apr 27, 2026
mtoffl01
commented
Apr 27, 2026
|
|
||
| @Override | ||
| public void onDroppedTrace(int spanCount) { | ||
| // TODO: surface drop counts via HealthMetrics |
Contributor
Author
There was a problem hiding this comment.
I've purposefully left this out as its not in the current spec
Contributor
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
mtoffl01
changed the title
mtoffl01
added
the
tag: ai generated
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4333c2d0ac
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Comment on lines
+129
to
+131
| protocol == OtlpConfig.Protocol.GRPC | ||
| ? new OtlpGrpcSender( | ||
| endpoint, TRACES_SIGNAL_PATH, headers, timeoutMillis, compression) |
There was a problem hiding this comment.
In OtlpWriterBuilder.build, the gRPC branch passes TRACES_SIGNAL_PATH (/v1/traces) to OtlpGrpcSender, but that sender emits application/grpc requests and needs a gRPC method path (as shown by metrics using /opentelemetry.proto.collector.metrics.v1.MetricsService/Export). With the current path, OTLP collectors will reject the request (typically 404/UNIMPLEMENTED), so trace export fails whenever otlp.traces.protocol=grpc is used.
Useful? React with 👍 / 👎.
Contributor
There was a problem hiding this comment.
Yes - I believe the GRPC sender should use
"/opentelemetry.proto.collector.trace.v1.TraceService/Export"
while the HTTP sender should use
"/v1/traces"
|
|
||
| private static boolean shouldExport(CoreSpan<?> span) { | ||
| // trace-level sampling priority | ||
| if (span.samplingPriority() > 0) { |
There was a problem hiding this comment.
shouldExport drops any span whose samplingPriority() is not > 0 unless span-sampling tags are present. In configurations that legitimately publish traces without priority sampling (for example DD_PRIORITY_SAMPLING=false with AllSampler), spans are published but keep UNSET priority, so this filter discards every span and OTLP emits nothing. Since tracing already decided to publish upstream, UNSET should not be treated as an automatic drop here.
Useful? React with 👍 / 👎.
Contributor
There was a problem hiding this comment.
The default payload dispatcher doesn't have this extra sampling check - it just assumes that anything reaching the dispatcher should be exported. Is there a reason we have this additional check for OTLP?
Contributor
Author
There was a problem hiding this comment.
As per the spec:
only export spans that have been sampled
| private final OtlpSender sender; | ||
|
|
||
| OtlpPayloadDispatcher(OtlpSender sender) { | ||
| this(sender, OtlpTraceProtoCollector.INSTANCE); |
There was a problem hiding this comment.
The default dispatcher constructor reuses OtlpTraceProtoCollector.INSTANCE for all OtlpWriter instances, but that collector maintains mutable state and is documented as single-threaded. If more than one OTLP writer exists in-process (e.g., multiple tracers or MultiWriter compositions), serializer threads will concurrently mutate shared collector state, which can corrupt payload assembly or mix traces across writers.
Useful? React with 👍 / 👎.
Contributor
Author
There was a problem hiding this comment.
I don't believe we ever have a scenario where multiple OtlpWriters are initialized 🤔
Contributor
There was a problem hiding this comment.
Correct - there should only ever be at most one OtlpWriter initialized.
But since OtlpTraceProtoCollector is only ever used inside this class we could always drop the OtlpTraceProtoCollector.INSTANCE field (as we don't need to share the instance across classes) and construct OtlpTraceProtoCollector in this constructor.
mtoffl01
added
type: enhancement
mtoffl01
changed the title
mcculls
reviewed
Apr 28, 2026
| } | ||
| } | ||
| if (sampled != null) { | ||
| collector.addTrace(sampled); |
Contributor
There was a problem hiding this comment.
Is there a reason we can't just call collector.addTrace(trace) like the non-OTLP dispatcher does?
mcculls
reviewed
Apr 28, 2026
Contributor
mcculls
left a comment
mcculls
left a comment
There was a problem hiding this comment.
Just one Q about the extra sampling in OtlpPayloadDispatcher (if this is not needed then removing it may help with the E2E test) - and the GRPC signal path needs fixing.
…, plus related tests
mtoffl01
force-pushed
the
mtoff/otlp-writer
branch
from
f7b12c1 to
491533b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Adds a new
OtlpWriterthat sends Datadog traces via OTLP -- HTTP/protobuf (default) or gRPC -- to a collector endpoint. This new writer type is auto-selected whenOTEL_TRACES_EXPORTER=otlpis set and is otherwise off by default.New components:
OtlpWriter+OtlpPayloadDispatcherWriterFactorydriven byConfig.getOtlpTraces*()gettersConfigplumbing fortraceOtelExporter→OtlpWriterselectionMotivation
This PR builds on existing work that serializes Datadog spans in OTLP format by providing an "Otlp" writer mechanism for forwarding the data to a configurable endpoint.
Additional Notes
Much of the code was AI generated but based on a relatively detailed plan that I pre-approved before starting the code generation. This PR passes system tests: DataDog/system-tests#6819
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.