update azure blob upload support to use azure_storage_blob by demoray · Pull Request #762 · microsoft/avml

demoray · 2026-03-17T19:33:33Z

No description provided.

Copilot

Pull request overview

Updates the Azure Blob Storage upload implementation to use the newer azure_storage_blob crate/API, aligning the blobstore uploader with the updated Azure Rust SDK ecosystem.

Changes:

Replace azure_storage_blobs usage with azure_storage_blob (stage_block + commit_block_list) in the blob uploader.
Refactor the uploader to share a BlockBlobClient via Arc for concurrent block uploads.
Update Cargo dependencies to azure_core = 0.33 and azure_storage_blob = 0.10.

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

File	Description
`src/upload/blobstore.rs`	Migrates block upload/finalize calls to `azure_storage_blob` APIs and updates client handling for concurrency.
`Cargo.toml`	Swaps dependency from `azure_storage_blobs` to `azure_storage_blob` and bumps `azure_core`.
`Cargo.lock`	Lockfile refresh reflecting the dependency migration and transitive dependency changes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot

Pull request overview

Updates the Azure Blob Storage upload implementation to use the newer azure_storage_blob crate (and newer azure_core) rather than azure_storage_blobs, aligning the blob upload path with the current Azure SDK crates.

Changes:

Migrated blob upload/commit logic from put_block/put_block_list to stage_block/commit_block_list using azure_storage_blob.
Updated BlobUploader to share the BlobClient via Arc for concurrent upload workers.
Bumped Azure SDK dependencies and adjusted feature flags accordingly in Cargo.toml (with lockfile updates).

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

File	Description
src/upload/blobstore.rs	Ports the uploader to `azure_storage_blob` APIs and updates concurrency/client sharing accordingly.
Cargo.toml	Switches dependencies from `azure_storage_blobs` to `azure_storage_blob` and updates Azure SDK versions/features.
Cargo.lock	Lockfile refresh reflecting updated Azure SDK crates and new transitive dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot

Pull request overview

Updates the Azure Blob upload implementation to use the newer azure_storage_blob crate APIs, replacing the previous manual block upload approach.

Changes:

Replace queued/manual block uploads with BlobClient::upload(...) using SDK upload options.
Add a SeekableStream-backed FileStream adapter to support SDK retries/resets and progress reporting.
Update Cargo features/dependencies and add a unit test for stream reset behavior.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 5 comments.

File	Description
src/upload/status.rs	Adds `Status::reset()` to support rewinding progress on stream reset during blob uploads.
src/upload/blobstore.rs	Reworks blob upload logic to SDK `upload`, introduces `FileStream` implementing `SeekableStream`, and adds a reset test.
Cargo.toml	Switches Azure crate dependencies/features to `azure_storage_blob` + `async-trait`, and adjusts default features.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot

Pull request overview

This PR migrates the Azure Blob upload implementation to the newer azure_storage_blob crate and adapts the upload path to use the SDK’s built-in parallel upload with a seekable body.

Changes:

Replace the previous block-queue uploader with BlobClient::upload using BlobClientUploadOptions (parallelism + partition size).
Introduce a SeekableStream-backed FileStream to support SDK retries/reset while streaming from disk.
Update Cargo features/dependencies to use azure_storage_blob/azure_core versions and remove async-channel.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

File	Description
`src/upload/status.rs`	Adds a `reset()` hook so progress reporting can be reset when the SDK retries/rewinds the stream.
`src/upload/blobstore.rs`	Reworks blob upload to the new SDK API and adds `FileStream` implementing `SeekableStream` + `AsyncRead`.
`Cargo.toml`	Swaps Azure blob dependencies/features to `azure_storage_blob` and updates TLS feature wiring.
`Cargo.lock`	Updates lockfile for the new Azure SDK dependency graph.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot

Pull request overview

This PR migrates the Azure Blob upload implementation to the newer azure_storage_blob crate and adapts the upload path to use the SDK’s built-in parallel upload with a seekable request body.

Changes:

Replaced manual block-queue/concurrent uploader logic with BlobClient::upload(...) + BlobClientUploadOptions.
Added a seekable, cloneable FileStream wrapper to support SDK retry/reset semantics.
Updated Cargo features/dependencies to use azure_storage_blob, newer azure_core, and async-trait.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.

File	Description
src/upload/status.rs	Adds a `reset()` hook (real + no-op) used by blob upload stream retry/reset logic.
src/upload/blobstore.rs	Reworks upload implementation to `azure_storage_blob` and introduces `FileStream` for seekable uploads.
Cargo.toml	Swaps Azure SDK crates, updates feature flags, and adds `async-trait` for async trait impl.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot

Pull request overview

This PR migrates Azure Blob uploads from azure_storage_blobs to azure_storage_blob, updating the uploader implementation and related CLI/config surfaces accordingly.

Changes:

Replaced the previous block-queue uploader with azure_storage_blob::BlobClient::upload() and a seekable, retry-friendly file stream.
Switched block size/concurrency parameters to NonZeroUsize and adjusted defaulting logic + tests around upload parameter calculation.
Updated feature flags/dependencies and build script to run doc tests with all features.

Reviewed changes

Copilot reviewed 5 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/upload/status.rs	Adds `Status::reset()` for blobstore use and a no-op stub for non-status builds.
src/upload/blobstore.rs	Migrates uploader to `azure_storage_blob`, introduces `FileStream` (seekable/retryable), updates sizing/concurrency logic and tests.
src/bin/avml.rs	Updates CLI arg types for blobstore block size/concurrency to `NonZeroUsize` options.
src/bin/avml-upload.rs	Updates upload CLI arg types for block size/concurrency to `NonZeroUsize` options.
eng/build.sh	Adds `cargo test --doc` run for all features.
README.md	Updates help text to reflect non-zero requirements for blobstore args.
Cargo.toml	Bumps crate version, swaps Azure blob dependency, updates feature flags and TLS wiring.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

# Conflicts: # Cargo.lock

Copilot

Pull request overview

This PR migrates AVML’s Azure Blob Storage upload implementation to the newer azure_storage_blob crate (and newer azure_core), updating the uploader API and CLI surface accordingly.

Changes:

Replaced the prior block-based uploader implementation with BlobClient::upload(...) using azure_storage_blob, including a SeekableStream wrapper to report progress.
Updated block-size/concurrency handling to use NonZero* types and adjusted CLI flags/help text to enforce “> 0”.
Updated build automation to run doc tests and refreshed dependency graph/versioning.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`src/upload/status.rs`	Adds progress-bar reset support (used for retry/rewind behavior in blob uploads) and derives `Debug`.
`src/upload/blobstore.rs`	Reimplements Azure blob upload using `azure_storage_blob` + `BlobClient::upload`, adds progress stream wrapper, and rewrites sizing/concurrency logic + tests.
`src/bin/avml.rs`	Updates CLI options/types for SAS block size/concurrency to `NonZero*` (optional overrides).
`src/bin/avml-upload.rs`	Updates the standalone upload tool CLI to the new `NonZero*` option types and library API.
`eng/build.sh`	Adds `cargo test --doc` to CI/build script to validate doctests with all features.
`README.md`	Updates CLI help text to reflect the new “must be > 0” constraints (and removes the displayed default).
`Cargo.toml`	Bumps crate version and swaps dependencies/features from `azure_storage_blobs` to `azure_storage_blob` (+ `async-trait`).
`Cargo.lock`	Locks updated transitive dependencies from the Azure SDK and related crates.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

demoray added 2 commits March 17, 2026 19:32

update azure blob upload support to use azure_storage_blob

3f08bd5

fmt

f026b1f

demoray requested a review from Copilot March 17, 2026 19:35

Copilot started reviewing on behalf of demoray March 17, 2026 19:36 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

Comment thread Cargo.toml Outdated

demoray added 2 commits March 17, 2026 20:01

address lint

1e5781f

reduce features used from azure_core

4efbc04

demoray requested a review from Copilot March 17, 2026 20:14

Copilot started reviewing on behalf of demoray March 17, 2026 20:16 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

Comment thread Cargo.toml Outdated

demoray added 2 commits March 18, 2026 11:19

use the Azure SDK's built-in parallel upload functionality

37892b4

address copilot PR feedback

8af2084

demoray requested a review from Copilot March 18, 2026 11:24

Copilot AI reviewed Mar 18, 2026

View reviewed changes

Comment thread src/upload/blobstore.rs Outdated

Comment thread src/upload/blobstore.rs Outdated

Comment thread src/upload/blobstore.rs Outdated

Comment thread Cargo.toml Outdated

Comment thread src/upload/blobstore.rs Outdated

Copilot started reviewing on behalf of demoray March 18, 2026 11:37 View session

demoray added 2 commits March 18, 2026 12:00

address feedback

8b54936

remove status as a default feature

8c9bda5

demoray requested a review from Copilot March 18, 2026 12:02

Copilot started reviewing on behalf of demoray March 18, 2026 12:06 View session

Copilot AI reviewed Mar 18, 2026

View reviewed changes

Comment thread src/upload/blobstore.rs Outdated

Comment thread src/upload/blobstore.rs Outdated

don't expose setting the size for BlobUploader, as this isn't used

8f29971

demoray requested a review from Copilot March 18, 2026 12:18

Copilot AI reviewed Mar 18, 2026

View reviewed changes

Comment thread src/upload/blobstore.rs Outdated

Comment thread src/upload/blobstore.rs Outdated

Comment thread src/upload/blobstore.rs Outdated

Copilot started reviewing on behalf of demoray March 18, 2026 12:31 View session

demoray added 2 commits March 18, 2026 13:18

bump rev

b1bacec

use NonZeroUsize for sizes

d2615ab

demoray requested a review from Copilot March 19, 2026 14:35

Copilot AI reviewed Mar 19, 2026

View reviewed changes

Comment thread src/upload/blobstore.rs Outdated

Comment thread src/upload/blobstore.rs Outdated

Comment thread README.md Outdated

demoray added 2 commits April 27, 2026 17:08

Merge remote-tracking branch 'origin/main' into update-azure-sdk

a3be64e

# Conflicts: # Cargo.lock

continued updates towards new azure sdk

4e29d6d

update

e138f74

demoray requested a review from Copilot April 27, 2026 18:50

Copilot AI reviewed Apr 27, 2026

View reviewed changes

Comment thread src/upload/blobstore.rs Outdated

Comment thread src/upload/blobstore.rs Outdated

Comment thread src/upload/blobstore.rs

Comment thread src/upload/blobstore.rs Outdated

Comment thread src/upload/blobstore.rs Outdated

Comment thread Cargo.toml Outdated

demoray added 2 commits April 27, 2026 19:07

address feedback

f177e65

address feedback

6f9eb93

demoray requested a review from Copilot April 27, 2026 19:28

Copilot AI reviewed Apr 27, 2026

View reviewed changes

Comment thread src/upload/blobstore.rs

Comment thread src/upload/blobstore.rs Outdated

demoray added 2 commits April 27, 2026 19:37

update dependency

049cf4e

address feedback

6a25933

demoray requested a review from Copilot April 27, 2026 19:44

demoray enabled auto-merge (squash) April 27, 2026 19:45

cduplantisms approved these changes Apr 27, 2026

View reviewed changes

Copilot AI reviewed Apr 27, 2026

View reviewed changes

Comment thread src/upload/blobstore.rs

Comment thread src/bin/avml.rs

Comment thread src/bin/avml-upload.rs

Comment thread README.md

demoray requested a review from cduplantisms April 29, 2026 15:36

cduplantisms approved these changes Apr 29, 2026

View reviewed changes

demoray merged commit bff7783 into main Apr 29, 2026
12 checks passed

demoray deleted the update-azure-sdk branch April 29, 2026 15:41

Conversation

demoray commented Mar 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development