Add Kafka cluster ID and offset tags to producer/consumer spans#11107
Draft
piochelepiotr wants to merge 1 commit intomasterfrom
Draft
Add Kafka cluster ID and offset tags to producer/consumer spans#11107piochelepiotr wants to merge 1 commit intomasterfrom
piochelepiotr wants to merge 1 commit intomasterfrom
Conversation
Tag producer spans with messaging.kafka.cluster.id, topic, and broker-assigned partition/offset (via callback). Tag consumer spans with messaging.kafka.cluster.id, topic, consumer group, and offset. These tags enable linking from a span directly to a specific Kafka message. Previously the cluster ID was only extracted when Data Streams was enabled; MetadataState is now always initialized on the producer so cluster ID flows through regardless of DSM configuration. tag: ai generated Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Kafka / producer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 12 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~f2a32a7836, baseline=1.62.0-SNAPSHOT~9f89a0b26c
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057127
Total [baseline] (8.826 s) : 0, 8825777
Agent [candidate] (1.054 s) : 0, 1053897
Total [candidate] (8.8 s) : 0, 8799632
section iast
Agent [baseline] (1.223 s) : 0, 1223310
Total [baseline] (9.503 s) : 0, 9503018
Agent [candidate] (1.221 s) : 0, 1221376
Total [candidate] (9.548 s) : 0, 9548084
gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~f2a32a7836, baseline=1.62.0-SNAPSHOT~9f89a0b26c
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.251 ms) : 0, 1251
crashtracking [candidate] (1.224 ms) : 0, 1224
BytebuddyAgent [baseline] (634.592 ms) : 0, 634592
BytebuddyAgent [candidate] (633.892 ms) : 0, 633892
AgentMeter [baseline] (29.394 ms) : 0, 29394
AgentMeter [candidate] (29.289 ms) : 0, 29289
GlobalTracer [baseline] (248.046 ms) : 0, 248046
GlobalTracer [candidate] (247.74 ms) : 0, 247740
AppSec [baseline] (31.995 ms) : 0, 31995
AppSec [candidate] (31.868 ms) : 0, 31868
Debugger [baseline] (58.863 ms) : 0, 58863
Debugger [candidate] (58.679 ms) : 0, 58679
Remote Config [baseline] (591.015 µs) : 0, 591
Remote Config [candidate] (601.391 µs) : 0, 601
Telemetry [baseline] (8.002 ms) : 0, 8002
Telemetry [candidate] (8.005 ms) : 0, 8005
Flare Poller [baseline] (8.155 ms) : 0, 8155
Flare Poller [candidate] (6.499 ms) : 0, 6499
section iast
crashtracking [baseline] (1.236 ms) : 0, 1236
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (801.726 ms) : 0, 801726
BytebuddyAgent [candidate] (800.269 ms) : 0, 800269
AgentMeter [baseline] (11.333 ms) : 0, 11333
AgentMeter [candidate] (11.395 ms) : 0, 11395
GlobalTracer [baseline] (238.433 ms) : 0, 238433
GlobalTracer [candidate] (238.617 ms) : 0, 238617
AppSec [baseline] (32.51 ms) : 0, 32510
AppSec [candidate] (32.57 ms) : 0, 32570
Debugger [baseline] (57.974 ms) : 0, 57974
Debugger [candidate] (58.027 ms) : 0, 58027
Remote Config [baseline] (521.705 µs) : 0, 522
Remote Config [candidate] (1.116 ms) : 0, 1116
Telemetry [baseline] (13.834 ms) : 0, 13834
Telemetry [candidate] (12.605 ms) : 0, 12605
Flare Poller [baseline] (3.475 ms) : 0, 3475
Flare Poller [candidate] (3.612 ms) : 0, 3612
IAST [baseline] (25.718 ms) : 0, 25718
IAST [candidate] (25.698 ms) : 0, 25698
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~f2a32a7836, baseline=1.62.0-SNAPSHOT~9f89a0b26c
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.056 s) : 0, 1056480
Total [baseline] (11.072 s) : 0, 11071546
Agent [candidate] (1.052 s) : 0, 1052121
Total [candidate] (10.988 s) : 0, 10988436
section appsec
Agent [baseline] (1.244 s) : 0, 1244281
Total [baseline] (11.063 s) : 0, 11063121
Agent [candidate] (1.244 s) : 0, 1244125
Total [candidate] (11.156 s) : 0, 11155816
section iast
Agent [baseline] (1.223 s) : 0, 1222877
Total [baseline] (11.367 s) : 0, 11367439
Agent [candidate] (1.226 s) : 0, 1226463
Total [candidate] (11.204 s) : 0, 11204243
section profiling
Agent [baseline] (1.181 s) : 0, 1180891
Total [baseline] (10.984 s) : 0, 10983989
Agent [candidate] (1.186 s) : 0, 1185555
Total [candidate] (11.184 s) : 0, 11184332
gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~f2a32a7836, baseline=1.62.0-SNAPSHOT~9f89a0b26c
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.209 ms) : 0, 1209
crashtracking [candidate] (1.209 ms) : 0, 1209
BytebuddyAgent [baseline] (631.056 ms) : 0, 631056
BytebuddyAgent [candidate] (629.59 ms) : 0, 629590
AgentMeter [baseline] (29.294 ms) : 0, 29294
AgentMeter [candidate] (29.276 ms) : 0, 29276
GlobalTracer [baseline] (247.842 ms) : 0, 247842
GlobalTracer [candidate] (247.282 ms) : 0, 247282
AppSec [baseline] (31.975 ms) : 0, 31975
AppSec [candidate] (31.792 ms) : 0, 31792
Debugger [baseline] (59.816 ms) : 0, 59816
Debugger [candidate] (59.477 ms) : 0, 59477
Remote Config [baseline] (591.874 µs) : 0, 592
Remote Config [candidate] (588.822 µs) : 0, 589
Telemetry [baseline] (8.071 ms) : 0, 8071
Telemetry [candidate] (8.009 ms) : 0, 8009
Flare Poller [baseline] (10.544 ms) : 0, 10544
Flare Poller [candidate] (9.007 ms) : 0, 9007
section appsec
crashtracking [baseline] (1.23 ms) : 0, 1230
crashtracking [candidate] (1.22 ms) : 0, 1220
BytebuddyAgent [baseline] (659.43 ms) : 0, 659430
BytebuddyAgent [candidate] (660.717 ms) : 0, 660717
AgentMeter [baseline] (12.008 ms) : 0, 12008
AgentMeter [candidate] (12.02 ms) : 0, 12020
GlobalTracer [baseline] (247.999 ms) : 0, 247999
GlobalTracer [candidate] (247.701 ms) : 0, 247701
AppSec [baseline] (184.362 ms) : 0, 184362
AppSec [candidate] (183.557 ms) : 0, 183557
Debugger [baseline] (65.799 ms) : 0, 65799
Debugger [candidate] (65.459 ms) : 0, 65459
Remote Config [baseline] (609.714 µs) : 0, 610
Remote Config [candidate] (592.771 µs) : 0, 593
Telemetry [baseline] (8.498 ms) : 0, 8498
Telemetry [candidate] (8.569 ms) : 0, 8569
Flare Poller [baseline] (3.568 ms) : 0, 3568
Flare Poller [candidate] (3.472 ms) : 0, 3472
IAST [baseline] (24.511 ms) : 0, 24511
IAST [candidate] (24.509 ms) : 0, 24509
section iast
crashtracking [baseline] (1.225 ms) : 0, 1225
crashtracking [candidate] (1.224 ms) : 0, 1224
BytebuddyAgent [baseline] (799.371 ms) : 0, 799371
BytebuddyAgent [candidate] (803.019 ms) : 0, 803019
AgentMeter [baseline] (11.36 ms) : 0, 11360
AgentMeter [candidate] (11.349 ms) : 0, 11349
GlobalTracer [baseline] (238.578 ms) : 0, 238578
GlobalTracer [candidate] (238.32 ms) : 0, 238320
AppSec [baseline] (32.761 ms) : 0, 32761
AppSec [candidate] (31.201 ms) : 0, 31201
Debugger [baseline] (60.35 ms) : 0, 60350
Debugger [candidate] (60.671 ms) : 0, 60671
Remote Config [baseline] (538.322 µs) : 0, 538
Remote Config [candidate] (526.627 µs) : 0, 527
Telemetry [baseline] (12.677 ms) : 0, 12677
Telemetry [candidate] (14.309 ms) : 0, 14309
Flare Poller [baseline] (3.502 ms) : 0, 3502
Flare Poller [candidate] (3.636 ms) : 0, 3636
IAST [baseline] (26.056 ms) : 0, 26056
IAST [candidate] (26.001 ms) : 0, 26001
section profiling
ProfilingAgent [baseline] (94.016 ms) : 0, 94016
ProfilingAgent [candidate] (94.367 ms) : 0, 94367
crashtracking [baseline] (1.174 ms) : 0, 1174
crashtracking [candidate] (1.176 ms) : 0, 1176
BytebuddyAgent [baseline] (688.843 ms) : 0, 688843
BytebuddyAgent [candidate] (691.005 ms) : 0, 691005
AgentMeter [baseline] (9.063 ms) : 0, 9063
AgentMeter [candidate] (9.155 ms) : 0, 9155
GlobalTracer [baseline] (206.5 ms) : 0, 206500
GlobalTracer [candidate] (208.2 ms) : 0, 208200
AppSec [baseline] (32.523 ms) : 0, 32523
AppSec [candidate] (32.74 ms) : 0, 32740
Debugger [baseline] (65.565 ms) : 0, 65565
Debugger [candidate] (65.659 ms) : 0, 65659
Remote Config [baseline] (572.806 µs) : 0, 573
Remote Config [candidate] (566.209 µs) : 0, 566
Telemetry [baseline] (7.925 ms) : 0, 7925
Telemetry [candidate] (7.884 ms) : 0, 7884
Flare Poller [baseline] (3.65 ms) : 0, 3650
Flare Poller [candidate] (3.616 ms) : 0, 3616
Profiling [baseline] (94.578 ms) : 0, 94578
Profiling [candidate] (94.925 ms) : 0, 94925
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 2 performance regressions! Performance is the same for 17 metrics, 17 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~f2a32a7836, baseline=1.62.0-SNAPSHOT~9f89a0b26c
dateFormat X
axisFormat %s
section baseline
no_agent (19.137 ms) : 18941, 19333
. : milestone, 19137,
appsec (18.458 ms) : 18273, 18644
. : milestone, 18458,
code_origins (17.947 ms) : 17767, 18126
. : milestone, 17947,
iast (17.816 ms) : 17637, 17995
. : milestone, 17816,
profiling (18.069 ms) : 17892, 18246
. : milestone, 18069,
tracing (18.708 ms) : 18523, 18893
. : milestone, 18708,
section candidate
no_agent (19.364 ms) : 19166, 19561
. : milestone, 19364,
appsec (18.86 ms) : 18669, 19051
. : milestone, 18860,
code_origins (17.642 ms) : 17468, 17815
. : milestone, 17642,
iast (17.604 ms) : 17434, 17774
. : milestone, 17604,
profiling (19.944 ms) : 19732, 20156
. : milestone, 19944,
tracing (18.059 ms) : 17881, 18237
. : milestone, 18059,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~f2a32a7836, baseline=1.62.0-SNAPSHOT~9f89a0b26c
dateFormat X
axisFormat %s
section baseline
no_agent (1.265 ms) : 1253, 1277
. : milestone, 1265,
iast (3.259 ms) : 3210, 3308
. : milestone, 3259,
iast_FULL (6.34 ms) : 6274, 6407
. : milestone, 6340,
iast_GLOBAL (3.617 ms) : 3566, 3668
. : milestone, 3617,
profiling (2.265 ms) : 2244, 2287
. : milestone, 2265,
tracing (1.875 ms) : 1859, 1891
. : milestone, 1875,
section candidate
no_agent (1.267 ms) : 1255, 1279
. : milestone, 1267,
iast (3.352 ms) : 3302, 3402
. : milestone, 3352,
iast_FULL (6.185 ms) : 6121, 6248
. : milestone, 6185,
iast_GLOBAL (3.576 ms) : 3532, 3621
. : milestone, 3576,
profiling (2.193 ms) : 2172, 2215
. : milestone, 2193,
tracing (1.852 ms) : 1836, 1868
. : milestone, 1852,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~f2a32a7836, baseline=1.62.0-SNAPSHOT~9f89a0b26c
dateFormat X
axisFormat %s
section baseline
no_agent (15.657 s) : 15657000, 15657000
. : milestone, 15657000,
appsec (15.012 s) : 15012000, 15012000
. : milestone, 15012000,
iast (18.806 s) : 18806000, 18806000
. : milestone, 18806000,
iast_GLOBAL (18.04 s) : 18040000, 18040000
. : milestone, 18040000,
profiling (15.314 s) : 15314000, 15314000
. : milestone, 15314000,
tracing (14.984 s) : 14984000, 14984000
. : milestone, 14984000,
section candidate
no_agent (15.547 s) : 15547000, 15547000
. : milestone, 15547000,
appsec (14.941 s) : 14941000, 14941000
. : milestone, 14941000,
iast (18.376 s) : 18376000, 18376000
. : milestone, 18376000,
iast_GLOBAL (18.208 s) : 18208000, 18208000
. : milestone, 18208000,
profiling (14.76 s) : 14760000, 14760000
. : milestone, 14760000,
tracing (15.204 s) : 15204000, 15204000
. : milestone, 15204000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~f2a32a7836, baseline=1.62.0-SNAPSHOT~9f89a0b26c
dateFormat X
axisFormat %s
section baseline
no_agent (1.485 ms) : 1473, 1497
. : milestone, 1485,
appsec (3.86 ms) : 3637, 4083
. : milestone, 3860,
iast (2.275 ms) : 2205, 2344
. : milestone, 2275,
iast_GLOBAL (2.323 ms) : 2253, 2394
. : milestone, 2323,
profiling (2.112 ms) : 2056, 2168
. : milestone, 2112,
tracing (2.096 ms) : 2041, 2150
. : milestone, 2096,
section candidate
no_agent (1.485 ms) : 1473, 1497
. : milestone, 1485,
appsec (3.829 ms) : 3607, 4051
. : milestone, 3829,
iast (2.264 ms) : 2194, 2333
. : milestone, 2264,
iast_GLOBAL (2.313 ms) : 2243, 2382
. : milestone, 2313,
profiling (2.105 ms) : 2049, 2160
. : milestone, 2105,
tracing (2.073 ms) : 2019, 2127
. : milestone, 2073,
|
Kafka / consumer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
messaging.kafka.cluster.idtag to both producer and consumer Kafka spans, always (not gated on Data Streams)partitionandoffsettags to producer spans via the async callback (usingRecordMetadata)partitionandoffset; no change thereisDataStreamsEnabled()guard fromextractClusterId()inKafkaConsumerInstrumentationHelper(both 0.11 and 3.8 modules)MetadataStateinProducerConstructorAdviceso cluster ID is available at produce time regardless of DSM configThese tags enable linking a trace span directly to a specific Kafka message (cluster + topic + partition + offset).
Notes
Futurewill not have the offset tagkafka-clients-0.11andkafka-clients-3.8modules updated in parallelTest plan
./gradlew :dd-java-agent:instrumentation:kafka:kafka-clients-0.11:test./gradlew :dd-java-agent:instrumentation:kafka:kafka-clients-3.8:testmessaging.kafka.cluster.id,partition,offset,kafka.group, andmessaging.destination.nametagsmessaging.kafka.cluster.id,partition,offset, andmessaging.destination.nametagstag: no release note
tag: ai generated
🤖 Generated with Claude Code