Skip to content

HDDS-13117. Add Conditional Writes, Reads, Copies, Deletes and CompleteMultipartUpload Design#10010

Merged
peterxcli merged 9 commits into
apache:masterfrom
peterxcli:docs/s3-conditional-copy-get-delete
Apr 26, 2026
Merged

HDDS-13117. Add Conditional Writes, Reads, Copies, Deletes and CompleteMultipartUpload Design#10010
peterxcli merged 9 commits into
apache:masterfrom
peterxcli:docs/s3-conditional-copy-get-delete

Conversation

@peterxcli

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

Add conditional writes, reads, copies, and deletes design

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-13117

How was this patch tested?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds/expands the design document for AWS S3-style conditional requests in Ozone’s S3 Gateway/OM paths, covering intended semantics and an incremental delivery plan aligned to HDDS-13117.

Changes:

  • Expands the specification sections for conditional writes/reads/copies and adds conditional deletes coverage.
  • Adds proposed gateway/OM implementation flows, including atomicity and error-mapping discussions.
  • Adds references and clarifies current state + phased delivery plan.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
@peterxcli peterxcli marked this pull request as ready for review April 1, 2026 18:36
@peterxcli peterxcli self-assigned this Apr 7, 2026
@peterxcli peterxcli added the s3 S3 Gateway label Apr 10, 2026
@peterxcli

Copy link
Copy Markdown
Member Author

@ivandika3 @jojochuang Please take a look when you have time, thanks!

@jojochuang jojochuang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AWS S3 supports conditional requests for multipart upload. Specifically, If-Match, If-None-Match for CompleteMultipartUpload.

Is that being considered or is it left out internationally? Just like to know what else remains unsupported once this design and implementation is complete.

@peterxcli

peterxcli commented Apr 21, 2026

Copy link
Copy Markdown
Member Author

Just like to know what else remains unsupported once this design and implementation is complete.

  • DeleteObjects conditional support

@peterxcli peterxcli changed the title HDDS-13117. Add conditional writes, reads, copies, and deletes design HDDS-13117. Add conditional writes, reads, copies, deletes and complete mpu design Apr 21, 2026
@peterxcli peterxcli changed the title HDDS-13117. Add conditional writes, reads, copies, deletes and complete mpu design HDDS-13117. Add Conditional Writes, Reads, Copies, Deletes and CompleteMultipartUpload Design Apr 21, 2026
@peterxcli

Copy link
Copy Markdown
Member Author

AWS S3 supports conditional requests for multipart upload. Specifically, If-Match, If-None-Match for CompleteMultipartUpload.

thanks, I updated the design and raised a jira for it: https://issues.apache.org/jira/browse/HDDS-15067

@peterxcli peterxcli requested a review from jojochuang April 22, 2026 08:49

@ivandika3 ivandika3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @peterxcli , overall LGTM. Left some comments.

I think you can start on the implementation. I believe that as long as we can check the implementation against https://github.com/peterxcli/ozone-s3-compatibility, it should be OK.

Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md Outdated
Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
Comment thread hadoop-hdds/docs/content/design/s3-conditional-requests.md
Signed-off-by: peterxcli <peterxcli@gmail.com>
@peterxcli peterxcli requested a review from ivandika3 April 26, 2026 09:22

@ivandika3 ivandika3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1.

@peterxcli peterxcli merged commit e0e821f into apache:master Apr 26, 2026
16 checks passed
@peterxcli peterxcli deleted the docs/s3-conditional-copy-get-delete branch April 26, 2026 09:47
@peterxcli

Copy link
Copy Markdown
Member Author

Thanks @ivandika3, @jojochuang for the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

design s3 S3 Gateway

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants