Skip to content

Support overwriting x-ms-version header in azure_blob sink #25986

Description

@opencow

A note for the community

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

When the Azure Blob sink sends requests to a storage service that supports only older Azure Storage API versions, the service can reject them with:

400 InvalidHeaderValue
HeaderName: x-ms-version
HeaderValue: 2025-11-05

This occurs when the x-ms-version header is newer than what's accepted by the storage service.

Attempted Solutions

We tried using different authentication methods for azure_blob sink, and all of them run into the same issue.

Proposal

The Azure Blob sink should provide an api_version configuration option that lets users select a version supported by their storage service, eg api_version: "2021-08-06"

References

https://learn.microsoft.com/en-us/rest/api/storageservices/versioning-for-the-azure-storage-services#invalidheadervalue-error-message

Version

0.57

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions