Skip to content

fix(metrics): restore IndexMap for unbounded MetricSet to fix CPU regression#25493

Merged
thomasqueirozb merged 4 commits into
masterfrom
perf/metric-set-lrucache-indexmap
May 29, 2026
Merged

fix(metrics): restore IndexMap for unbounded MetricSet to fix CPU regression#25493
thomasqueirozb merged 4 commits into
masterfrom
perf/metric-set-lrucache-indexmap

Conversation

@thomasqueirozb

@thomasqueirozb thomasqueirozb commented May 23, 2026

Copy link
Copy Markdown
Member

Summary

MetricSet was changed from IndexMap to LruCache in #23374 to support the incremental_to_absolute transform's capacity eviction feature. However, LruCache::get_mut unconditionally updates LRU bookkeeping (pointer manipulation in a doubly-linked list) on every metric lookup, even when no capacity policy is configured. For high-throughput metrics pipelines (e.g. prometheus_remote_write sinks processing thousands of metrics/second) this per-metric overhead accumulates significantly on the hot path and is the root cause of the CPU regression reported in #24058 that persists in v0.55.0 after the buffer hot-path fix (#24995) and file source fix (#25064).

MetricSet now uses an either IndexMap or LruCache if needed, meaning that performance should decrease unless the user is using the incremental_to_absolute transform.

Vector configuration

NA

How did you test this PR?

NA

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

@github-actions github-actions Bot added the domain: sinks Anything related to the Vector's sinks label May 23, 2026
@thomasqueirozb thomasqueirozb changed the title perf(metrics): restore IndexMap for unbounded MetricSet to fix CPU regression fix(metrics): restore IndexMap for unbounded MetricSet to fix CPU regression May 23, 2026
@thomasqueirozb thomasqueirozb force-pushed the perf/metric-set-lrucache-indexmap branch from ddc6088 to 4de1fa7 Compare May 23, 2026 00:34
@thomasqueirozb thomasqueirozb force-pushed the perf/metric-set-lrucache-indexmap branch from e545bce to ce224f0 Compare May 23, 2026 01:03
@thomasqueirozb thomasqueirozb marked this pull request as ready for review May 28, 2026 14:54
@thomasqueirozb thomasqueirozb requested a review from a team as a code owner May 28, 2026 14:54
@thomasqueirozb thomasqueirozb added the domain: performance Anything related to Vector's performance label May 28, 2026
Comment thread src/sinks/util/buffer/metrics/normalize.rs
…nforce_capacity_policy

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@thomasqueirozb thomasqueirozb enabled auto-merge May 29, 2026 03:19
@thomasqueirozb thomasqueirozb added this pull request to the merge queue May 29, 2026
Merged via the queue into master with commit 08b6975 May 29, 2026
60 checks passed
@thomasqueirozb thomasqueirozb deleted the perf/metric-set-lrucache-indexmap branch May 29, 2026 04:03
@github-actions github-actions Bot locked and limited conversation to collaborators May 29, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

domain: performance Anything related to Vector's performance domain: sinks Anything related to the Vector's sinks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants