enhancement(tag_cardinality_limit): Add per-tag cache_size_per_key override in probabilistic mode

[ArunPiduguDD]

Summary

In probabilistic mode, the bloom filter cache size (cache_size_per_key) was a single global/per-metric setting inherited by every tag. However, when specifying a per-tag override, it doesn't really make sense to inherit this value, especially in cases where the per-tag override specifies a much higher limit than the global/per-metric limit.

This adds an optional cache_size_per_key field to per-tag limit_override entries. When set, it overrides the bloom filter size for that tag only. When omitted, it inherits from the enclosing per-metric or global config as before. Setting it in exact mode causes a validation error.

Example:

mode: probabilistic
cache_size_per_key: 5120
per_tag_limits:
  trace_id:
    mode: limit_override
    value_limit: 1000
    cache_size_per_key: 32768  # bigger filter for this tag only

Vector configuration

See example above. Works in both top-level per_tag_limits and inside per_metric_limits.

How did you test this PR?

• Added unit tests for apply_cache_size_override covering probabilistic+Some, probabilistic+None, and exact+Some (no-op).
• Added YAML deserialization tests for both per_metric_limits and global per_tag_limits scopes.
• All existing 41 tag cardinality tests continue to pass.

Change Type

• Bug fix
• New feature
• Dependencies
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ce0e240502

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[pront]

codex generated

Mostly yes, but I’d ask for one small follow-up before merging:

Finding

• [P2] cache_size_per_key validation only rejects inherited exact, not inherited excluded. In src/transforms/tag_cardinality_limit/config.rs, per-metric validation only checks OverrideMode::Exact; a per-metric config with mode: excluded plus a per-tag cache_size_per_key would still build, even though the changelog/docs now say the option is only valid in probabilistic mode. Tiny fix: reject any per-metric inherited mode that is not OverrideMode::Probabilistic(_), or explicitly document that excluded metrics ignore nested per-tag settings.

Minor polish:

• The PR body still says exact mode is “silently ignored,” but the code/changelog now say it errors.
• A couple comments/docs still imply all per-tag settings are inherited, e.g. src/transforms/tag_cardinality_limit/mod.rs. Worth rewording so cache_size_per_key is called out as the exception.

Verification: cargo test -p vector --lib --no-default-features --features transforms-tag_cardinality_limit transforms::tag_cardinality_limit:: passes: 45/45. git diff --check is clean. The earlier generated-docs churn also looks fixed now.

[ArunPiduguDD]

Updated PR with requested changes

[pront]

Updated PR with requested changes

@ArunPiduguDD thanks. Please also fix the merge conflicts and I will take a final look.

[ArunPiduguDD]

Updated PR with requested changes

@ArunPiduguDD thanks. Please also fix the merge conflicts and I will take a final look.

@pront Rebased PR & fixed merged conflicts (+signed earlier unsigned commits)