Skip to content

Add fall back to all exceptions encountered when presigned URL is not available for upload and download in FilesExt#1115

Merged
parthban-db merged 4 commits into
databricks:mainfrom
yuanjieding-db:yuanjie/fix-delta-sharing-download
Nov 20, 2025
Merged

Add fall back to all exceptions encountered when presigned URL is not available for upload and download in FilesExt#1115
parthban-db merged 4 commits into
databricks:mainfrom
yuanjieding-db:yuanjie/fix-delta-sharing-download

Conversation

@yuanjieding-db

Copy link
Copy Markdown
Collaborator

What changes are proposed in this pull request?

WHAT

  • Add fall back to all exceptions encountered when presigned URL is not available for upload and download in FilesExt

WHY

  • This is to eliminate the possibility of any potential regression introduced by upload and download through presigned URL.

How is this tested?

Unit tests are updated to reflect the change.

@parthban-db parthban-db changed the title Yuanjie/fix delta sharing download Add fall back to all exceptions encountered when presigned URL is not available for upload and download in FilesExt Nov 13, 2025

@parthban-db parthban-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also add this change in the changelog?

len(pre_read_buffer),
session_token,
BytesIO(pre_read_buffer),
is_first_part=True,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to pass this information? Every error returned by this will fall back to FilesAPI, right?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do need this information for two reasons:

  • Whenever it is not possible to use Presigned URL, the SDK should fail at the first part. Otherwise client should retry.
  • For uploading of unseekable streams, if the function fails at any part other than the first part, the previous parts would been dropped from memory and thus not possible to perform a fallback. In that case the SDK has no option but throw an exception, and the client should perform a retry.

@github-actions

Copy link
Copy Markdown

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-py

Inputs:

  • PR number: 1115
  • Commit SHA: 3151fe51be2712f542813f77c164325baa250e69

Checks will be approved automatically on success.

@parthban-db parthban-db added this pull request to the merge queue Nov 20, 2025
Merged via the queue into databricks:main with commit a26962a Nov 20, 2025
24 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants