enhancement(tag_cardinality_limit transform): add exclude_tags option to bypass cardinality limiting (#25316)

* enhancement(tag_cardinality_limit transform): add top-level per_tag_limits

Adds a `per_tag_limits` map at the top level of the `tag_cardinality_limit`
config, reusing the `PerTagConfig` / `PerTagMode` types introduced in #25360
for the per-metric variant. Each entry uses the same `mode: limit_override`
(with a custom `value_limit`) or `mode: excluded` shape, applied to every
metric that does not match a `per_metric_limits` entry.

Per-metric overrides shadow the global per-tag map: a matched metric only
consults its own `per_tag_limits` (consistent with how the rest of the
transform resolves per-metric vs. global config).

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(tag_cardinality_limit transform): preserve untracked passthrough when value_limit is zero

`tag_limit_exceeded` previously returned `true` for any missing-bucket lookup when
`value_limit == 0`, which under `drop_event` caused events to be rejected before
`record_tag_value` could detect that `max_tracked_keys` was exhausted. New
(metric, tag-key) pairs that cannot be allocated must instead pass through
unchecked and emit `tag_cardinality_untracked_events_total`. Guard the
`value_limit == 0` rejection on `can_allocate_new_key()` so the untracked path
runs whenever the pair would not actually be tracked.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(tag_cardinality_limit transform): explain global per_tag_limits with example

Address review feedback from @pront: the field doc comment was correct but
dense. Move the precedence rules and a worked YAML example into a new
"Per-tag overrides" section under "How it works", and trim the field
doc-comment to a one-line description that points at the new section.

Also fix CRLF → LF line endings in the changelog fragment caught by
`vdev check fmt`.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

Author: 3 people

changelog.d/tag_cardinality_limit_global_per_tag.enhancement.md

@@ -0,0 +1,6 @@
+The `tag_cardinality_limit` transform now accepts a top-level `per_tag_limits` map,
+mirroring the per-metric one: `mode: limit_override` to set a per-tag cap, or
+`mode: excluded` to bypass cardinality tracking for that tag on every metric without a
+`per_metric_limits` entry.
+
+authors: kaarolch

src/transforms/tag_cardinality_limit/config.rs

@@ -49,6 +49,19 @@ pub struct Config {
     )]
     #[serde(default)]
     pub per_metric_limits: HashMap<String, PerMetricConfig>,
+
+    /// Global per-tag-key overrides, applied to every metric that does not match a
+    /// `per_metric_limits` entry. Each entry sets `mode: limit_override` (with a
+    /// per-tag `value_limit`) or `mode: excluded` (bypass tracking for that tag).
+    ///
+    /// See the "Per-tag overrides" section under "How it works" for a worked example
+    /// and the precedence rules.
+    #[configurable(
+        derived,
+        metadata(docs::additional_props_description = "An individual tag configuration.")
+    )]
+    #[serde(default)]
+    pub per_tag_limits: HashMap<String, PerTagConfig>,
 }
 
 /// Controls how tag tracking state is partitioned across metrics.
@@ -311,6 +324,7 @@ impl GenerateConfig for Config {
             tracking_scope: TrackingScope::default(),
             max_tracked_keys: None,
             per_metric_limits: HashMap::default(),
+            per_tag_limits: HashMap::default(),
         })
         .unwrap()
     }

src/transforms/tag_cardinality_limit/mod.rs

@@ -87,24 +87,30 @@ impl TagCardinalityLimit {
     /// Per-tag entries support two modes:
     /// - `mode: limit_override` — uses the per-tag `value_limit`; all other settings
     ///   (`mode`, `cache_size_per_key`, `limit_exceeded_action`, `internal_metrics`)
-    ///   are inherited from the per-metric config.
+    ///   are inherited from the enclosing per-metric (or, for global overrides, the
+    ///   global) config.
     /// - `mode: excluded` — opts the tag out entirely; all values pass through.
     ///
     /// Per-metric exclusion is blanket: `mode: excluded` on a per-metric entry opts out
     /// every tag on that metric and `per_tag_limits` is ignored.
+    ///
+    /// Per-metric `per_tag_limits` take precedence over the top-level
+    /// `Config::per_tag_limits`: when a metric matches a per-metric entry, the global
+    /// per-tag overrides are not consulted for that metric.
     fn get_config_for_metric_tag(
         &self,
         metric_key: Option<&MetricId>,
         tag_key: &str,
     ) -> TagSettings {
-        // No matching per-metric override → use the global config as-is.
+        // No matching per-metric override → use the global config, with global
+        // per-tag overrides layered on top.
         let Some((metric_namespace, metric_name)) = metric_key else {
-            return TagSettings::Tracked(self.config.global);
+            return self.apply_global_per_tag(tag_key);
         };
         let Some((_, per_metric)) = self.config.per_metric_limits.iter().find(|(name, cfg)| {
             *name == metric_name && (cfg.namespace.is_none() || cfg.namespace == *metric_namespace)
         }) else {
-            return TagSettings::Tracked(self.config.global);
+            return self.apply_global_per_tag(tag_key);
         };
 
         // Per-metric exclusion is blanket — per-tag overrides do not apply.
@@ -140,6 +146,20 @@ impl TagCardinalityLimit {
         })
     }
 
+    /// Apply the top-level `per_tag_limits` (if any) on top of the global `Inner`.
+    /// Used for metrics that do not match any `per_metric_limits` entry.
+    fn apply_global_per_tag(&self, tag_key: &str) -> TagSettings {
+        let global = self.config.global;
+        match self.config.per_tag_limits.get(tag_key).map(|c| c.mode) {
+            Some(PerTagMode::Excluded) => TagSettings::Excluded,
+            Some(PerTagMode::LimitOverride { value_limit }) => TagSettings::Tracked(Inner {
+                value_limit,
+                ..global
+            }),
+            None => TagSettings::Tracked(global),
+        }
+    }
+
     /// Returns the `limit_exceeded_action` that applies to this metric. Decided once per event:
     /// per-metric override if any, else global.
     fn metric_action(&self, metric_key: Option<&MetricId>) -> LimitExceededAction {
@@ -237,9 +257,12 @@ impl TagCardinalityLimit {
             Some(value_set) if value_set.contains(value) => false,
             // Adding this value would push us at or past the configured cap. Treat a
             // missing bucket as an empty set so `value_limit: 0` correctly rejects
-            // the first occurrence too.
+            // the first occurrence too — but only when the (metric, tag) pair would
+            // actually be tracked. If `max_tracked_keys` is exhausted, `record_tag_value`
+            // will pass the tag through unchecked and emit `TagCardinalityLimitUntracked`,
+            // so we must not pre-empt that path by reporting the limit as exceeded here.
             Some(value_set) => value_set.len() >= resolved.value_limit,
-            None => resolved.value_limit == 0,
+            None => resolved.value_limit == 0 && self.can_allocate_new_key(),
         }
     }