Provide optimized writers for OpenTelemetry's "trace.proto" wire protocol#11120
Open
Provide optimized writers for OpenTelemetry's "trace.proto" wire protocol#11120
Conversation
mcculls
added
tag: do not merge
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~f5eae6e507, baseline=1.62.0-SNAPSHOT~d5d2097cb9
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1058335
Total [baseline] (8.859 s) : 0, 8859052
Agent [candidate] (1.059 s) : 0, 1058768
Total [candidate] (8.872 s) : 0, 8871675
section iast
Agent [baseline] (1.229 s) : 0, 1229364
Total [baseline] (9.581 s) : 0, 9580947
Agent [candidate] (1.233 s) : 0, 1233021
Total [candidate] (9.596 s) : 0, 9595985
gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~f5eae6e507, baseline=1.62.0-SNAPSHOT~d5d2097cb9
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.251 ms) : 0, 1251
crashtracking [candidate] (1.245 ms) : 0, 1245
BytebuddyAgent [baseline] (633.791 ms) : 0, 633791
BytebuddyAgent [candidate] (634.093 ms) : 0, 634093
AgentMeter [baseline] (29.438 ms) : 0, 29438
AgentMeter [candidate] (29.628 ms) : 0, 29628
GlobalTracer [baseline] (249.234 ms) : 0, 249234
GlobalTracer [candidate] (249.875 ms) : 0, 249875
AppSec [baseline] (32.604 ms) : 0, 32604
AppSec [candidate] (32.477 ms) : 0, 32477
Debugger [baseline] (59.644 ms) : 0, 59644
Debugger [candidate] (59.203 ms) : 0, 59203
Remote Config [baseline] (591.738 µs) : 0, 592
Remote Config [candidate] (597.966 µs) : 0, 598
Telemetry [baseline] (8.11 ms) : 0, 8110
Telemetry [candidate] (8.026 ms) : 0, 8026
Flare Poller [baseline] (7.492 ms) : 0, 7492
Flare Poller [candidate] (7.336 ms) : 0, 7336
section iast
crashtracking [baseline] (1.25 ms) : 0, 1250
crashtracking [candidate] (1.232 ms) : 0, 1232
BytebuddyAgent [baseline] (805.814 ms) : 0, 805814
BytebuddyAgent [candidate] (805.76 ms) : 0, 805760
AgentMeter [baseline] (11.617 ms) : 0, 11617
AgentMeter [candidate] (11.787 ms) : 0, 11787
GlobalTracer [baseline] (239.391 ms) : 0, 239391
GlobalTracer [candidate] (242.044 ms) : 0, 242044
AppSec [baseline] (31.285 ms) : 0, 31285
AppSec [candidate] (31.555 ms) : 0, 31555
Debugger [baseline] (64.339 ms) : 0, 64339
Debugger [candidate] (64.741 ms) : 0, 64741
Remote Config [baseline] (536.951 µs) : 0, 537
Remote Config [candidate] (545.724 µs) : 0, 546
Telemetry [baseline] (9.319 ms) : 0, 9319
Telemetry [candidate] (9.379 ms) : 0, 9379
Flare Poller [baseline] (3.589 ms) : 0, 3589
Flare Poller [candidate] (3.631 ms) : 0, 3631
IAST [baseline] (25.864 ms) : 0, 25864
IAST [candidate] (26.176 ms) : 0, 26176
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~f5eae6e507, baseline=1.62.0-SNAPSHOT~d5d2097cb9
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066011
Total [baseline] (11.13 s) : 0, 11129697
Agent [candidate] (1.058 s) : 0, 1058455
Total [candidate] (11.134 s) : 0, 11133956
section appsec
Agent [baseline] (1.25 s) : 0, 1249692
Total [baseline] (11.033 s) : 0, 11032701
Agent [candidate] (1.257 s) : 0, 1256956
Total [candidate] (11.168 s) : 0, 11168222
section iast
Agent [baseline] (1.228 s) : 0, 1227782
Total [baseline] (11.289 s) : 0, 11288754
Agent [candidate] (1.226 s) : 0, 1226457
Total [candidate] (11.313 s) : 0, 11312519
section profiling
Agent [baseline] (1.187 s) : 0, 1186530
Total [baseline] (11.039 s) : 0, 11039321
Agent [candidate] (1.184 s) : 0, 1184278
Total [candidate] (10.948 s) : 0, 10947831
gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~f5eae6e507, baseline=1.62.0-SNAPSHOT~d5d2097cb9
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.258 ms) : 0, 1258
crashtracking [candidate] (1.231 ms) : 0, 1231
BytebuddyAgent [baseline] (637.379 ms) : 0, 637379
BytebuddyAgent [candidate] (632.403 ms) : 0, 632403
AgentMeter [baseline] (29.708 ms) : 0, 29708
AgentMeter [candidate] (29.472 ms) : 0, 29472
GlobalTracer [baseline] (250.241 ms) : 0, 250241
GlobalTracer [candidate] (249.472 ms) : 0, 249472
AppSec [baseline] (32.639 ms) : 0, 32639
AppSec [candidate] (32.344 ms) : 0, 32344
Debugger [baseline] (60.53 ms) : 0, 60530
Debugger [candidate] (59.742 ms) : 0, 59742
Remote Config [baseline] (603.21 µs) : 0, 603
Remote Config [candidate] (620.5 µs) : 0, 620
Telemetry [baseline] (8.158 ms) : 0, 8158
Telemetry [candidate] (7.985 ms) : 0, 7985
Flare Poller [baseline] (9.165 ms) : 0, 9165
Flare Poller [candidate] (8.985 ms) : 0, 8985
section appsec
crashtracking [baseline] (1.227 ms) : 0, 1227
crashtracking [candidate] (1.223 ms) : 0, 1223
BytebuddyAgent [baseline] (662.025 ms) : 0, 662025
BytebuddyAgent [candidate] (665.396 ms) : 0, 665396
AgentMeter [baseline] (12.295 ms) : 0, 12295
AgentMeter [candidate] (12.386 ms) : 0, 12386
GlobalTracer [baseline] (249.392 ms) : 0, 249392
GlobalTracer [candidate] (250.995 ms) : 0, 250995
AppSec [baseline] (185.272 ms) : 0, 185272
AppSec [candidate] (186.591 ms) : 0, 186591
Debugger [baseline] (65.811 ms) : 0, 65811
Debugger [candidate] (66.379 ms) : 0, 66379
Remote Config [baseline] (606.745 µs) : 0, 607
Remote Config [candidate] (602.249 µs) : 0, 602
Telemetry [baseline] (8.519 ms) : 0, 8519
Telemetry [candidate] (8.641 ms) : 0, 8641
Flare Poller [baseline] (3.555 ms) : 0, 3555
Flare Poller [candidate] (3.588 ms) : 0, 3588
IAST [baseline] (24.522 ms) : 0, 24522
IAST [candidate] (24.672 ms) : 0, 24672
section iast
crashtracking [baseline] (1.235 ms) : 0, 1235
crashtracking [candidate] (1.218 ms) : 0, 1218
BytebuddyAgent [baseline] (803.882 ms) : 0, 803882
BytebuddyAgent [candidate] (801.347 ms) : 0, 801347
AgentMeter [baseline] (11.597 ms) : 0, 11597
AgentMeter [candidate] (11.639 ms) : 0, 11639
GlobalTracer [baseline] (239.099 ms) : 0, 239099
GlobalTracer [candidate] (240.002 ms) : 0, 240002
AppSec [baseline] (30.625 ms) : 0, 30625
AppSec [candidate] (33.711 ms) : 0, 33711
Debugger [baseline] (65.746 ms) : 0, 65746
Debugger [candidate] (62.631 ms) : 0, 62631
Remote Config [baseline] (532.964 µs) : 0, 533
Remote Config [candidate] (526.874 µs) : 0, 527
Telemetry [baseline] (9.349 ms) : 0, 9349
Telemetry [candidate] (9.638 ms) : 0, 9638
Flare Poller [baseline] (3.67 ms) : 0, 3670
Flare Poller [candidate] (3.648 ms) : 0, 3648
IAST [baseline] (25.733 ms) : 0, 25733
IAST [candidate] (25.851 ms) : 0, 25851
section profiling
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.185 ms) : 0, 1185
BytebuddyAgent [baseline] (693.128 ms) : 0, 693128
BytebuddyAgent [candidate] (691.641 ms) : 0, 691641
AgentMeter [baseline] (9.19 ms) : 0, 9190
AgentMeter [candidate] (9.186 ms) : 0, 9186
GlobalTracer [baseline] (207.031 ms) : 0, 207031
GlobalTracer [candidate] (207.196 ms) : 0, 207196
AppSec [baseline] (32.863 ms) : 0, 32863
AppSec [candidate] (32.77 ms) : 0, 32770
Debugger [baseline] (65.898 ms) : 0, 65898
Debugger [candidate] (65.412 ms) : 0, 65412
Remote Config [baseline] (579.049 µs) : 0, 579
Remote Config [candidate] (581.216 µs) : 0, 581
Telemetry [baseline] (7.775 ms) : 0, 7775
Telemetry [candidate] (7.735 ms) : 0, 7735
Flare Poller [baseline] (3.53 ms) : 0, 3530
Flare Poller [candidate] (3.529 ms) : 0, 3529
ProfilingAgent [baseline] (94.003 ms) : 0, 94003
ProfilingAgent [candidate] (93.718 ms) : 0, 93718
Profiling [baseline] (94.562 ms) : 0, 94562
Profiling [candidate] (94.283 ms) : 0, 94283
LoadParameters
See matching parameters
SummaryFound 4 performance improvements and 0 performance regressions! Performance is the same for 15 metrics, 17 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~f5eae6e507, baseline=1.62.0-SNAPSHOT~d5d2097cb9
dateFormat X
axisFormat %s
section baseline
no_agent (18.256 ms) : 18065, 18447
. : milestone, 18256,
appsec (18.651 ms) : 18462, 18840
. : milestone, 18651,
code_origins (18.113 ms) : 17932, 18295
. : milestone, 18113,
iast (17.627 ms) : 17454, 17801
. : milestone, 17627,
profiling (18.569 ms) : 18387, 18752
. : milestone, 18569,
tracing (18.772 ms) : 18584, 18961
. : milestone, 18772,
section candidate
no_agent (17.467 ms) : 17291, 17643
. : milestone, 17467,
appsec (18.555 ms) : 18370, 18739
. : milestone, 18555,
code_origins (17.665 ms) : 17492, 17838
. : milestone, 17665,
iast (18.108 ms) : 17927, 18289
. : milestone, 18108,
profiling (18.371 ms) : 18190, 18553
. : milestone, 18371,
tracing (17.722 ms) : 17547, 17897
. : milestone, 17722,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~f5eae6e507, baseline=1.62.0-SNAPSHOT~d5d2097cb9
dateFormat X
axisFormat %s
section baseline
no_agent (1.26 ms) : 1248, 1272
. : milestone, 1260,
iast (3.369 ms) : 3320, 3419
. : milestone, 3369,
iast_FULL (5.99 ms) : 5929, 6052
. : milestone, 5990,
iast_GLOBAL (3.805 ms) : 3739, 3871
. : milestone, 3805,
profiling (2.184 ms) : 2163, 2205
. : milestone, 2184,
tracing (1.948 ms) : 1931, 1965
. : milestone, 1948,
section candidate
no_agent (1.244 ms) : 1232, 1257
. : milestone, 1244,
iast (3.408 ms) : 3360, 3455
. : milestone, 3408,
iast_FULL (5.966 ms) : 5906, 6027
. : milestone, 5966,
iast_GLOBAL (3.677 ms) : 3617, 3736
. : milestone, 3677,
profiling (1.972 ms) : 1956, 1988
. : milestone, 1972,
tracing (1.89 ms) : 1874, 1906
. : milestone, 1890,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~f5eae6e507, baseline=1.62.0-SNAPSHOT~d5d2097cb9
dateFormat X
axisFormat %s
section baseline
no_agent (1.487 ms) : 1475, 1498
. : milestone, 1487,
appsec (3.852 ms) : 3630, 4073
. : milestone, 3852,
iast (2.272 ms) : 2203, 2342
. : milestone, 2272,
iast_GLOBAL (2.316 ms) : 2246, 2386
. : milestone, 2316,
profiling (2.099 ms) : 2044, 2154
. : milestone, 2099,
tracing (2.086 ms) : 2032, 2140
. : milestone, 2086,
section candidate
no_agent (1.483 ms) : 1471, 1494
. : milestone, 1483,
appsec (3.829 ms) : 3606, 4051
. : milestone, 3829,
iast (2.27 ms) : 2200, 2340
. : milestone, 2270,
iast_GLOBAL (2.318 ms) : 2248, 2388
. : milestone, 2318,
profiling (2.099 ms) : 2044, 2154
. : milestone, 2099,
tracing (2.095 ms) : 2041, 2149
. : milestone, 2095,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~f5eae6e507, baseline=1.62.0-SNAPSHOT~d5d2097cb9
dateFormat X
axisFormat %s
section baseline
no_agent (15.21 s) : 15210000, 15210000
. : milestone, 15210000,
appsec (14.8 s) : 14800000, 14800000
. : milestone, 14800000,
iast (18.397 s) : 18397000, 18397000
. : milestone, 18397000,
iast_GLOBAL (18.29 s) : 18290000, 18290000
. : milestone, 18290000,
profiling (14.756 s) : 14756000, 14756000
. : milestone, 14756000,
tracing (14.684 s) : 14684000, 14684000
. : milestone, 14684000,
section candidate
no_agent (14.912 s) : 14912000, 14912000
. : milestone, 14912000,
appsec (14.99 s) : 14990000, 14990000
. : milestone, 14990000,
iast (18.55 s) : 18550000, 18550000
. : milestone, 18550000,
iast_GLOBAL (17.94 s) : 17940000, 17940000
. : milestone, 17940000,
profiling (15.103 s) : 15103000, 15103000
. : milestone, 15103000,
tracing (14.859 s) : 14859000, 14859000
. : milestone, 14859000,
|
mcculls
force-pushed
the
mcculls/otlp-traces-proto
branch
2 times, most recently
from
583dc0c to
4adb56e
Compare
mcculls
changed the title
mtoffl01
reviewed
Apr 16, 2026
Comment on lines
+59
to
+65
| /** | ||
| * Collects trace spans and marshalls them into a chunked payload. | ||
| * | ||
| * <p>This payload is only valid for the calling thread until the next collection. | ||
| */ | ||
| @Override | ||
| public OtlpPayload collectSpans(List<DDSpan> spans) { |
Contributor
There was a problem hiding this comment.
Is List<DDSpan> spans expected to be spans from a single trace? If so, each collectSpans call produces a full TracesData envelope with resource and scope wrappers per trace. This doesn't seem optimal and differs from the Datadog/msgpack implementation? Unless the expectation is that the eventual OtlpWriter will accumulated completed traces and call this once per flush cycle with a combined span list (although that can't be right based on the MetaWriter, which expects just a single trace at a time).
Contributor
Author
There was a problem hiding this comment.
Very good point - on reflection I'll change this to add a flush method so we can accumulate trace chunks over multiple calls.
Contributor
Author
There was a problem hiding this comment.
OK, I've updated the collector API so it has two methods:
addTrace(spans)which adds a trace to the collectorcollectTraces()which marshals the collected spans into a payload
This should allow its use as a replacement PayloadDispatcher, which means we can re-use more of the existing remote writer code.
…send them as first-class links (likewise turn off legacy baggage injection)
mcculls
force-pushed
the
mcculls/otlp-traces-proto
branch
from
ab2ef0b to
7cdfed7
Compare
dougqh
reviewed
Apr 17, 2026
| writeSpanTag(buf, PROCESS_TAGS_KEY, metadata.processTags()); | ||
| } | ||
|
|
||
| metadata.getTags().forEach(tagEntry -> writeSpanTag(buf, tagEntry)); |
Contributor
There was a problem hiding this comment.
Full-disclosure, I did just quickly run this through Claude.
It did catch that this is introducing a capturing lambda.
TagMap has helper routines to avoid the lambda capture by doing...
forEach(buf, (b, tagEntry) -> ...);
I suppose to avoid capture entirely. You need two variables, but that is also supported.
- Per-span lambda allocations — OtlpTraceProto.java:312 and :182-186
metadata.getTags().forEach(tagEntry -> writeSpanTag(buf, tagEntry));
This is a capturing lambda (captures buf); it allocates a fresh object on every accept() call. MetaWriter already holds buf as a field — hoist a
Consumer<TagMap.EntryReader> into the instance and reuse it. Same applies to the BiConsumer inside recordSpanLinkMessage. On hot paths this compounds with
the per-span spanLink.attributes().asMap() which likely allocates an EntrySet wrapper + iterator per span.
Contributor
Author
There was a problem hiding this comment.
interesting, thanks!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Uses a single temporary buffer as in #10983 to prepare message chunks at different nesting levels (resource / scope / span)
First we chunk all nested messages, i.e. span-links, for a given span. Once the span is complete we add the first part of the span message and its chunked links to the scoped chunks. Once the scope is complete we add the first part of the scoped spans message and all its chunks (span messages and their links) to the payload. Once all the span data has been chunked we add the enclosing resource metrics message to the start of the payload.
Multiple traces can be added to the collector before collecting them into a payload. Note that this payload is only valid for the calling thread until the next collection. Adding traces after collection automatically starts a new payload.
Motivation
Avoids need to use full protobuf library while keeping intermediate array creation to a minimum.
Additional Notes
OtlpTraceProtoTestwas created with the help of Claude.Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.