enhancement(tag_cardinality_limit transform): Add more fine grained controls tag cardinality

[ArunPiduguDD]

Summary

Vector configuration

How did you test this PR?

Change Type

• Bug fix
• New feature
• Dependencies
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

References

Notes

• Please read our Vector contributor resources.
• Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
• Some CI checks run only after we manually approve them.
  • We recommend adding a pre-push hook, please see this template.
  • Alternatively, we recommend running the following locally before pushing to the remote branch:
    • make fmt
    • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
    • make test
• After a review is requested, please avoid force pushes to help us review incrementally.
  • Feel free to push as many commits as you want. They will be squashed into one before merging.
  • For example, you can run git merge origin master and git push.
• If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
  run make build-licenses to regenerate the license inventory and commit the changes (if any). More details on the dd-rust-license-tool.

[kaarolch]

btw. Do you have any use case why include metrics exclusion to cardinality_tags would be better compare to current possible solution: route + remap which should be more sufficient instead scan cardinality list?

#name: metrics_route_cardinality_limi
type: route
inputs:
  - vector_ingress
route:
  cardinality_bypass: .tags.cardinality_bypass == "true"
...  
type: tag_cardinality_limit
inputs:
  - metrics_route_cardinality_limit._unmatched
.....

If you connect it with enrichment table you can manage metrics from external csv.

:FYI another PR for tags exclusion: #25316.

[ArunPiduguDD]

@kaarolch This would be if you wanted to different limits for tags in a metric (or if only one tag on a metric should be excluded from the limit)

[ArunPiduguDD]

Took a look at your PR, if we add support for per tag limits I wonder if exclude_tags might be integrated with this approach