Skip to content

fix(aggregate transform): Fix incorrectly dropped events bug in metrics aggregate transform#25486

Merged
ArunPiduguDD merged 9 commits into
masterfrom
fixed-dropped-events-aggregate
May 29, 2026
Merged

fix(aggregate transform): Fix incorrectly dropped events bug in metrics aggregate transform#25486
ArunPiduguDD merged 9 commits into
masterfrom
fixed-dropped-events-aggregate

Conversation

@ArunPiduguDD

@ArunPiduguDD ArunPiduguDD commented May 22, 2026

Copy link
Copy Markdown
Contributor

Summary

There appears to be a major bug in the aggregate transform. According to the official docs for certain mode types incremental or absolute metric kinds should be ignored (e.g. in Sum mode absolute metrics should be ignored). However it seems the current behavior is to silently drop the events (in the example absolute metrics would be dropped if the mode is Sum).

Makes a fix to stop dropping non matching events and pass them through.

Vector configuration

sources:
    otel_in:
      type: opentelemetry
      grpc:
        address: "0.0.0.0:4317"

  transforms:
    agg:
      type: aggregate
      inputs: [otel_in.metrics]
      interval_ms: 5000
      mode: sum

  sinks:
    console_out:
      type: console
      inputs: [agg]
      encoding:
        codec: json

How did you test this PR?

Tested with above config

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details on the dd-rust-license-tool.

@github-actions github-actions Bot added the domain: transforms Anything related to Vector's transform components label May 22, 2026
@ArunPiduguDD ArunPiduguDD changed the title fix(aggregate): Fix incorrectly dropped events bug in metrics aggregate processor fix(aggregate): Fix incorrectly dropped events bug in metrics aggregate transform May 22, 2026
@ArunPiduguDD ArunPiduguDD changed the title fix(aggregate): Fix incorrectly dropped events bug in metrics aggregate transform fix(aggregate transform): Fix incorrectly dropped events bug in metrics aggregate transform May 22, 2026
@ArunPiduguDD ArunPiduguDD force-pushed the fixed-dropped-events-aggregate branch from 1bc9183 to e69682d Compare May 22, 2026 17:21
@ArunPiduguDD ArunPiduguDD marked this pull request as ready for review May 22, 2026 17:21
@ArunPiduguDD ArunPiduguDD requested a review from a team as a code owner May 22, 2026 17:21
@ArunPiduguDD ArunPiduguDD force-pushed the fixed-dropped-events-aggregate branch from e69682d to 517371f Compare May 22, 2026 17:25
ArunPiduguDD and others added 2 commits May 22, 2026 17:37
Originally wanted to avoid deconstructing and reconstructing the Metric
Event for ignored metrics, but the other solution was not ideal
Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@bruceg bruceg self-requested a review May 22, 2026 20:00
@bruceg bruceg added type: bug transform: aggregate Anything `aggregate` transform related labels May 22, 2026

@bruceg bruceg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

Comment thread src/transforms/aggregate.rs
Comment thread src/transforms/aggregate.rs Outdated

@pront pront left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also improve the docs, for example "Sums incremental metrics, ignores absolute." -> "Sums incremental metrics, absolute metrics pass through unchanged."

Comment thread changelog.d/aggregate_passthrough_ignored_kind.fix.md Outdated
@ArunPiduguDD ArunPiduguDD requested a review from pront May 26, 2026 14:40
@ArunPiduguDD

Copy link
Copy Markdown
Contributor Author

Let's also improve the docs, for example "Sums incremental metrics, ignores absolute." -> "Sums incremental metrics, absolute metrics pass through unchanged."

@pront Just realized I undid this, updating

@ArunPiduguDD ArunPiduguDD requested a review from a team as a code owner May 28, 2026 18:57
@github-actions github-actions Bot added the domain: external docs Anything related to Vector's external, public documentation label May 28, 2026
@ArunPiduguDD

Copy link
Copy Markdown
Contributor Author

Let's also improve the docs, for example "Sums incremental metrics, ignores absolute." -> "Sums incremental metrics, absolute metrics pass through unchanged."

@pront Just realized I undid this, updating

Updated now

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 496fd61c0d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/transforms/aggregate.rs
Comment thread src/transforms/aggregate.rs
@ArunPiduguDD ArunPiduguDD force-pushed the fixed-dropped-events-aggregate branch from b8341bb to 1d62287 Compare May 29, 2026 14:41
Comment thread changelog.d/aggregate_passthrough_ignored_kind.fix.md Outdated
@ArunPiduguDD ArunPiduguDD added this pull request to the merge queue May 29, 2026
Merged via the queue into master with commit f7e394f May 29, 2026
61 checks passed
@ArunPiduguDD ArunPiduguDD deleted the fixed-dropped-events-aggregate branch May 29, 2026 19:30
@github-actions github-actions Bot locked and limited conversation to collaborators May 29, 2026
@pront pront removed the type: bug label May 29, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

domain: external docs Anything related to Vector's external, public documentation domain: transforms Anything related to Vector's transform components transform: aggregate Anything `aggregate` transform related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants