Surface errors when files are modified during request#6405
Merged
alextwoods merged 6 commits intomasterfrom Sep 9, 2025
Merged
Surface errors when files are modified during request#6405alextwoods merged 6 commits intomasterfrom
alextwoods merged 6 commits intomasterfrom
Conversation
…propogated correctly.
zoewangg
requested changes
Sep 8, 2025
|
zoewangg
approved these changes
Sep 9, 2025
davidh44
approved these changes
Sep 9, 2025
|
This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Surface errors from FileAsyncRequestBody when files are modified during request.
Motivation and Context
This PR addresses three related issues:
We have logic in the FileAsyncRequestBody that detected changes in file modification time and signaled IOExceptions - but these were not being propagated.
The
NettyRequestExecutor(which, indirectly, is subscribed to theFileAsyncRequestBody.FilePublisher) callscancelwhen Netty detects that it has read the full expected content-length (see logic here). Oncecancelhas been called, the Reactive Streams spec specifies that errors should no longer be signaled (and our code respects that). The check for file modification time changes is only done inonCompletewhich is only called after Netty cancels the subscription, meaning errors will never be propagated. The solution to this is to move the validation logic into the onComplete during read, specifically a check here before the call tosignalOnNext(which will cause Netty to call cancel).The second issue is that the exception raised is an IOException, which is considered retryable by default. File modification during request should be terminal.
The third issue is that we were recording the initial modification time and file size when the Publisher is created - which is done per execution attempt / per part in split, so if the file is modified between attempts or between parts, it was not being detected.
Modifications
There are three changes, which address each of the issues listed above.
modifiedTimeAtStartandsizeAtStartwhen theFileAsyncRequestBodyis constructured, ensuring that it stays consistent between retries/splits.Testing
Added new tests to cover behavior.
Manual testing of behavior:
Behavior of putObject/uploadFile with different setups when file modified:
Low level clients:
TransferManagers:
Example exception:
Screenshots (if appropriate)
Types of changes
Checklist
mvn installsucceedsscripts/new-changescript and following the instructions. Commit the new file created by the script in.changes/next-releasewith your changes.License