fix(trace-stats)!: add grpc_method to aggregation key by mabdinur · Pull Request #2151 · DataDog/libdatadog

mabdinur · 2026-06-22T17:37:59Z

What does this PR do?

Removes `grpc_method` / `rpc.method` entirely from the OTLP trace metrics pipeline.

Previously `grpc_method` was a sidecar on `GroupedStats` — present in the OTLP metric payload as an explicit attribute, but not part of the aggregation key. This PR removes it from both:

`FixedAggregationKey` — it is no longer an aggregation dimension.
`OtlpExactGroup` — it is no longer carried in the per-cell sidecar.
`build_attributes()` — `rpc.method` is no longer emitted on OTLP histogram data points.

Rationale

For gRPC spans, `span.resource` already carries the full method path (e.g. `/package.Service/Method`). Emitting `rpc.method` as a separate OTLP attribute is redundant — `span.resource` (mapped to `span.name` on the data point) is the authoritative carrier of gRPC method identity and is already an aggregation dimension.

Adding `rpc.method` as a separate dimension would also increase cardinality for gRPC services without providing any information not already present on the metric.

Note: `rpc.response.status_code` (from `grpc.status.code`) is kept as a first-class aggregation dimension — it is not redundant with any existing field.

Motivation

Correctness and cleanliness for the OTLP trace metrics feature landed in #2067.

Risk

Breaking changes (major semver):

`FixedAggregationKey` loses its `grpc_method: T` field. Callers constructing the struct by name must remove the field.
`OtlpExactGroup` loses its `grpc_method` field.
`build_attributes()` loses the `grpc_method` parameter.

Blast radius is limited because:

The OTLP trace metrics feature (feat(data-pipeline)!: export client-computed span stats as OTLP trace metrics #2067) landed on `main` after the v36 release (June 19). No released version is affected — this only exists in unreleased code.
The OTLP metrics path is opt-in: the sidecar only activates it when `set_otlp_metrics_endpoint` is configured. No SDK enables this today.
Agent `/v0.6/stats` protobuf wire format is unchanged — `ClientGroupedStats` has no `grpc_method` field.

Urgency: low. No live production systems are affected. This must land before the next release that ships the OTLP metrics feature.

How to test the change?

cargo nextest run -p libdd-trace-stats -p datadog-ipc -p libdd-data-pipeline

Spans with different gRPC methods were previously merged into the same stats group (only the first span's method was kept). Adding grpc_method to FixedAggregationKey ensures each method gets a separate bucket. The OtlpExactGroup.grpc_method field is now sourced from the key rather than a GroupedStats sidecar. The agent /v0.6/stats protobuf wire format is unchanged (no grpc_method field in ClientGroupedStats). SHM_VERSION bumped to 2 because FixedAggregationKey<StringRef> is #[repr(C)] and the new field changes the layout; mismatched sidecar/worker pairs will safely fail with a version-mismatch error. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

github-actions · 2026-06-22T17:41:13Z

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

Base Branch: origin/main
PR Branch: origin/munir/grpc-method-aggregation-key

Summary by Rule

Rule	Base Branch	PR Branch	Change
unwrap_used	1	1	No change (0%)
Total	1	1	No change (0%)

Annotation Counts by File

File	Base Branch	PR Branch	Change
`libdd-data-pipeline/src/otlp/metrics.rs`	1	1	No change (0%)

Annotation Stats by Crate

Crate	Base Branch	PR Branch	Change
`clippy-annotation-reporter`	5	5	No change (0%)
`datadog-ffe-ffi`	1	1	No change (0%)
`datadog-ipc`	22	22	No change (0%)
`datadog-live-debugger`	4	4	No change (0%)
`datadog-live-debugger-ffi`	10	10	No change (0%)
`datadog-profiling-replayer`	4	4	No change (0%)
`datadog-sidecar`	45	45	No change (0%)
`libdd-common`	13	13	No change (0%)
`libdd-common-ffi`	12	12	No change (0%)
`libdd-data-pipeline`	6	6	No change (0%)
`libdd-ddsketch`	2	2	No change (0%)
`libdd-dogstatsd-client`	1	1	No change (0%)
`libdd-profiling`	13	13	No change (0%)
`libdd-remote-config`	3	3	No change (0%)
`libdd-telemetry`	20	20	No change (0%)
`libdd-tinybytes`	4	4	No change (0%)
`libdd-trace-normalization`	2	2	No change (0%)
`libdd-trace-obfuscation`	3	3	No change (0%)
`libdd-trace-stats`	1	1	No change (0%)
`libdd-trace-utils`	11	11	No change (0%)
Total	182	182	No change (0%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be18c4394d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

datadog-prod-us1-4 · 2026-06-22T17:43:30Z

Tests

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

Metadata, docs and deps | dependency-check

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 73.93% (-0.02%)

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: f54e2e7 | Docs | Datadog PR Page | Give us feedback!}

dd-octo-sts · 2026-06-22T18:16:47Z

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	84.87 MB	84.86 MB	-0% (-8.07 KB) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	7.82 MB	7.76 MB	--.79% (-64.00 KB) 💪

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	96.00 MB	95.99 MB	-0% (-8.34 KB) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.44 MB	10.44 MB	-0% (-272 B) 👌

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	25.05 MB	25.05 MB	-0% (-2.50 KB) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	87.68 KB	87.68 KB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	182.80 MB	182.80 MB	0% (0 B) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	936.10 MB	936.08 MB	-0% (-18.82 KB) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	8.20 MB	8.20 MB	--.03% (-3.00 KB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	87.68 KB	87.68 KB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	24.23 MB	24.23 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	48.34 MB	48.34 MB	--.01% (-6.92 KB) 💪

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	21.71 MB	21.71 MB	--.01% (-3.00 KB) 💪
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	89.06 KB	89.06 KB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	186.80 MB	186.78 MB	--.01% (-24.00 KB) 💪
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	924.73 MB	924.71 MB	-0% (-18.95 KB) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	6.33 MB	6.32 MB	--.01% (-1.00 KB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	89.06 KB	89.06 KB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	25.99 MB	25.99 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	45.96 MB	45.96 MB	-0% (-4.66 KB) 👌

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	75.64 MB	75.63 MB	-0% (-6.43 KB) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	8.68 MB	8.68 MB	0% (0 B) 👌

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	91.09 MB	91.09 MB	-0% (-6.24 KB) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.57 MB	10.57 MB	-0% (-168 B) 👌

Bincode encodes struct fields positionally. Inserting grpc_method before http_status_code shifted all subsequent field positions, breaking IPC fallback decoding (OwnedShmSpanInput) between mismatched worker/sidecar versions. Moving it to the end of the struct preserves all existing field positions, so old-format IPC messages are decoded correctly up to the grpc_method field; the decode then fails with EOF rather than silently misinterpreting existing fields. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

VianneyRuhlmann

Some concerns regarding cardinality for non-otlp exporter otherwise LGTM

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…xport span.resource already carries the full gRPC method path for gRPC spans (e.g. /package.Service/Method). Adding grpc_method as a separate aggregation dimension or OTLP attribute is redundant and adds cardinality. Removes: - grpc_method from FixedAggregationKey - grpc_method from OtlpExactGroup - rpc.method from the OTLP data-point attributes emitted by build_attributes - GRPC_METHOD_FIELD constant and get_grpc_method() helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mabdinur requested review from a team as code owners June 22, 2026 17:38

chatgpt-codex-connector Bot reviewed Jun 22, 2026

View reviewed changes

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated

mabdinur changed the title ~~fix(trace-stats): add grpc_method to aggregation key~~ fix(trace-stats)!: add grpc_method to aggregation key Jun 22, 2026

ci: trigger fresh workflow run

eb3cfc7

mabdinur commented Jun 22, 2026

View reviewed changes

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated

mabdinur commented Jun 22, 2026

View reviewed changes

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs

VianneyRuhlmann approved these changes Jun 23, 2026

View reviewed changes

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated

mabdinur and others added 2 commits June 25, 2026 15:10

style(trace-stats): remove stale comments

6aa1463

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mabdinur requested a review from a team as a code owner June 25, 2026 20:37

mabdinur requested review from vpellan and removed request for a team June 25, 2026 20:37

github-actions Bot added the data-pipeline label Jun 25, 2026

Merge branch 'main' into munir/grpc-method-aggregation-key

f54e2e7

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: waiting mergequeue-status: queued mergequeue-status: in_progress mergequeue-status: rejected and removed mergequeue-status: waiting mergequeue-status: queued mergequeue-status: in_progress mergequeue-status: rejected labels Jun 26, 2026

gh-worker-dd-devflow-36fce6 Bot added the mergequeue-status: in_progress label Jun 26, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit 53e20b5 into main Jun 26, 2026
172 of 173 checks passed

gh-worker-dd-mergequeue-cf854d Bot deleted the munir/grpc-method-aggregation-key branch June 26, 2026 21:30

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: done and removed mergequeue-status: in_progress labels Jun 26, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(trace-stats)!: add grpc_method to aggregation key#2151

fix(trace-stats)!: add grpc_method to aggregation key#2151
gh-worker-dd-mergequeue-cf854d[bot] merged 6 commits into
mainfrom
munir/grpc-method-aggregation-key

mabdinur commented Jun 22, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 22, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

datadog-prod-us1-4 Bot commented Jun 22, 2026 •

edited by datadog-official Bot

Loading

Uh oh!

dd-octo-sts Bot commented Jun 22, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

VianneyRuhlmann left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

mabdinur commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Rationale

Motivation

Risk

How to test the change?

Uh oh!

github-actions Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Clippy Allow Annotation Report

Summary by Rule

Annotation Counts by File

Annotation Stats by Crate

About This Report

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

datadog-prod-us1-4 Bot commented Jun 22, 2026 • edited by datadog-official Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

dd-octo-sts Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Artifact Size Benchmark Report

Uh oh!

Uh oh!

Uh oh!

VianneyRuhlmann left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mabdinur commented Jun 22, 2026 •

edited

Loading

github-actions Bot commented Jun 22, 2026 •

edited

Loading

datadog-prod-us1-4 Bot commented Jun 22, 2026 •

edited by datadog-official Bot

Loading

dd-octo-sts Bot commented Jun 22, 2026 •

edited

Loading