Skip to content

feat(azure_blob sink): support blob index tags and metadata#25545

Open
Danielku15 wants to merge 4 commits into
vectordotdev:masterfrom
Danielku15:feature/azure-sink-tags
Open

feat(azure_blob sink): support blob index tags and metadata#25545
Danielku15 wants to merge 4 commits into
vectordotdev:masterfrom
Danielku15:feature/azure-sink-tags

Conversation

@Danielku15

Copy link
Copy Markdown

Summary

Adds blob tags and metadata support to the azure_blob sink. This updates request building and Azure common config handling so tags/metadata are forwarded correctly, and adds unit + Azurite-backed integration coverage for the new behavior.

Vector configuration

Example config used during testing:

sinks:
  azure_blob:
    type: azure_blob
    inputs:
      - my_source
    connection_string: ${AZURE_STORAGE_CONNECTION_STRING}
    container_name: logs
    blob_prefix: vector/
    encoding:
      codec: text
    tags:
      env: prod
      team: platform
    metadata:
      source: vector
      region: us-east-1

How did you test this PR?

Ran:

  • cargo fmt --all --check
  • cargo clippy --no-default-features --features sinks-azure_blob --tests -- -D warnings
  • cargo test --no-default-features --features sinks-azure_blob azure_blob_build_request_with_blob_ -- --nocapture
  • cargo test --no-default-features --features sinks-azure_blob,azure-blob-integration-tests sinks::azure_blob::integration_tests:: -- --nocapture
  • cargo vdev check generated-docs
  • cargo vdev check markdown
  • ./scripts/check_changelog_fragments.sh

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Closes: #22316

Notes

No notes from my side.

@Danielku15 Danielku15 requested a review from a team as a code owner June 1, 2026 11:56
@github-actions github-actions Bot added the domain: sinks Anything related to the Vector's sinks label Jun 1, 2026
@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@Danielku15

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@Danielku15

Copy link
Copy Markdown
Author

Any of the maintainers around who can give feedback this feature contribution? I'd like to also implement the AppendBlob support soon-ish and this makes most sense with this feature also in place.

The two features would be quite crucial for us to use vector.dev in product, without them we need to look for some alternative solutions. No intention to be pushy, just trying to understand the general release cycle and policies on external contributions 😉

@Danielku15

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eb7b08e504

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/sinks/azure_blob/config.rs
Comment thread src/sinks/azure_blob/config.rs
@Danielku15 Danielku15 requested a review from a team as a code owner June 15, 2026 17:26
@github-actions github-actions Bot added docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. domain: external docs Anything related to Vector's external, public documentation labels Jun 15, 2026
@rtrieu rtrieu self-requested a review June 17, 2026 19:44

@rtrieu rtrieu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @Danielku15 left you some editorial suggestions for your consideration.

Comment thread website/cue/reference/components/sinks/generated/azure_blob.cue Outdated
Comment thread website/cue/reference/components/sinks/generated/azure_blob.cue Outdated
Comment thread website/cue/reference/components/sinks/generated/azure_blob.cue Outdated
Comment thread website/cue/reference/components/sinks/generated/azure_blob.cue Outdated
Comment thread website/cue/reference/components/sinks/generated/azure_blob.cue Outdated
Comment thread website/cue/reference/components/sinks/generated/azure_blob.cue Outdated
Comment thread website/cue/reference/components/sinks/generated/azure_blob.cue Outdated
Comment thread website/cue/reference/components/sinks/generated/azure_blob.cue Outdated
Comment thread website/cue/reference/components/sinks/generated/azure_blob.cue Outdated
Comment thread website/cue/reference/components/sinks/generated/azure_blob.cue Outdated
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
@Danielku15

Copy link
Copy Markdown
Author

@rtrieu Thanks for the feedback. I've applied all changes.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 89d143a6f8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/sinks/azure_blob/config.rs Outdated
Comment thread src/sinks/azure_blob/config.rs Outdated
@jlaundry

Copy link
Copy Markdown
Contributor

Hi @Danielku15, I'm not a maintainer, but have done a bunch of Azure sink stuff recently.

This looks good! Two comments:

  1. Given the 10 tag limit, as well as a 128 character key limit and 256 character value limit, should these be checked during the config build, so that there's an clear error for the user to fix?
  2. FYI, it's been on my list to move the Blob-specific code out of azure_common, and I've opened chore(azure sinks): refactor Blob code out of azure_common #25689 this morning. I'm a bit late with this, and I'm happy to rebase my PR if this merges first.

@jlaundry jlaundry mentioned this pull request Jun 26, 2026
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

support tags in Azure Blob Storage sink

3 participants