HDDS-15167. Broaden commit conflict detection for conditional writes#10183
Open
peterxcli wants to merge 6 commits intoapache:masterfrom
Open
HDDS-15167. Broaden commit conflict detection for conditional writes#10183peterxcli wants to merge 6 commits intoapache:masterfrom
peterxcli wants to merge 6 commits intoapache:masterfrom
Conversation
Signed-off-by: peterxcli <peterxcli@gmail.com>
peterxcli
changed the title
Member
Author
|
cc @YutaLin |
…ction Signed-off-by: peterxcli <peterxcli@gmail.com>
chungen0126
reviewed
May 4, 2026
Contributor
chungen0126
left a comment
chungen0126
left a comment
There was a problem hiding this comment.
Thanks @peterxcli for the patch. Left some comments.
Comment on lines
+1354
to
+1375
| protected void validateAtomicRewriteAtCommit(OmKeyInfo existing, | ||
| OmKeyInfo toCommit, Map<String, String> auditMap) throws OMException { | ||
| validateAtomicRewriteAtCommit(existing, | ||
| toCommit.getExpectedDataGeneration(), auditMap); | ||
| } | ||
|
|
||
| /** | ||
| * Validates an already admitted condition at serialized commit time. | ||
| * | ||
| * This form is for callers that must check the admitted generation before | ||
| * building the final key info to commit. | ||
| */ | ||
| protected void validateAtomicRewriteAtCommit(OmKeyInfo existing, | ||
| Long expectedGen, Map<String, String> auditMap) throws OMException { | ||
| if (expectedGen == null) { | ||
| return; | ||
| } | ||
| validateExpectedDataGeneration(existing, expectedGen, auditMap, | ||
| AtomicRewritePhase.COMMIT); | ||
| } | ||
|
|
||
| private void validateExpectedDataGeneration(OmKeyInfo existing, |
Contributor
There was a problem hiding this comment.
I feel like splitting the validation logic into these three methods might be a bit of over-engineering. Could we simplify it?
Comment on lines
+1323
to
+1328
| KeyArgs resolvedKeyArgs = validateAndRewriteIfMatchAsExpectedGeneration( | ||
| keyArgs, dbKeyInfo); | ||
| if (resolvedKeyArgs.hasExpectedDataGeneration()) { | ||
| addRewriteGenerationToAuditMap( | ||
| resolvedKeyArgs.getExpectedDataGeneration(), auditMap); | ||
| } |
Contributor
There was a problem hiding this comment.
A few quick suggestions to streamline the logic:
- Could we make
validateAndRewriteIfMatchAsExpectedGenerationprivate if it's not used elsewhere? - The
keyArgs.hasExpectedDataGeneration()check insidevalidateAndRewriteIfMatchAsExpectedGenerationis redundant, as the caller already filters it out. - Consider moving
addRewriteGenerationToAuditMapintovalidateAndRewriteIfMatchAsExpectedGeneration. This will make the logic much cleaner.
Member
Author
|
And I think we should apply this concept to conditional put, too.
cc @ivandika3 |
…, only commit validate as commit Signed-off-by: peterxcli <peterxcli@gmail.com>
Signed-off-by: peterxcli <peterxcli@gmail.com>
peterxcli
changed the title
peterxcli
changed the title
…ditional-mpu-complete Signed-off-by: peterxcli <peterxcli@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Relates to: #10164
This patch broadens the conditional write conflict handling from conditional
CompleteMultipartUploadto the shared conditional write flow used by conditionalPutObjectand the existing atomic key rewrite path.The main change is to split conditional write handling into two phases:
412 PreconditionFailed) when the caller's condition is already false, such as an existing destination forIf-None-Match: *, a missing destination forIf-Match, a missing ETag, or an ETag mismatch;409 ConditionalRequestConflictwhen an admitted conditional write loses a concurrent commit race before it becomes visible.Concretely, this patch:
CommitKeyhandling, including FSO, so stale admitted generations fail withATOMIC_WRITE_CONFLICTat commit time;ConditionalRequestConflictwhile keeping admission failures asPreconditionFailed;After a
409 ConditionalRequestConflict, clients should fetch the latest object ETag, start a new write or multipart upload as needed, and retry using a condition based on the latest object view.What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15167
How was this patch tested?
UT/IT
https://github.com/peterxcli/ozone/actions/runs/25326550113