Skip to content

feat: presigned URLs with extra query params and signed headers (SignedUrlOptions)#771

Open
zfarrell wants to merge 6 commits into
apache:mainfrom
zfarrell:feat/signed-url-with-query-params
Open

feat: presigned URLs with extra query params and signed headers (SignedUrlOptions)#771
zfarrell wants to merge 6 commits into
apache:mainfrom
zfarrell:feat/signed-url-with-query-params

Conversation

@zfarrell

Copy link
Copy Markdown

Which issue does this PR close?

Closes #271
Closes #270

Also related to #620 (versionId) and #318.

Rationale for this change

Right now Signer::signed_url can only presign a bare URL. There's no way to add extra query parameters or to sign request headers, and S3 needs both.

The big one is multipart uploads. To presign an UploadPart request you have to include partNumber and uploadId as query parameters, and they have to be part of the signature. There's no way to do that today, so you can't hand out presigned URLs for the parts of a multipart upload.

Signing headers matters for the same reason. If you want S3 to enforce a checksum, or pin the Content-Type, or require SSE, those headers have to be baked into the signature when you presign. Otherwise the client can send whatever it wants.

What changes are included in this PR?

  • Adds SignedUrlOptions and a new Signer::signed_url_opts method that takes extra query params and signed (name, value) header pairs along with the usual method/path/expiry.
  • signed_url_opts has a default impl, and the existing signed_url now just calls it with no options. So nothing changes for existing callers and Azure doesn't have to implement anything.
  • Implemented for S3 (SigV4) and GCS (GOOG4). Azure keeps the default — its SAS model can't sign arbitrary params, and there's an inline comment saying so.
  • Fixes two signing bugs the new code surfaced: query values were being encoded with + instead of %20, which doesn't match the canonical query, and the canonical query now sorts by the encoded (key, value) pair the way the spec wants.

Are there any user-facing changes?

Yes, but only additions. SignedUrlOptions and signed_url_opts are new. signed_url behaves exactly as before. No breaking changes — the trait method is defaulted, and the internal signing methods that got reworked were all pub(crate).

This was tested end-to-end against real S3 (and MinIO): a full multipart round-trip with a sub-5MiB final part, checksum enforcement (accepts a matching body, rejects a tampered one, rejects a missing one), Content-Type binding, tampering with partNumber/uploadId/signed headers all getting a 403, percent-encoded keys, expiry, and If-None-Match conditional create.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow presigning object_store multipart uploads Allow specifying Content-Type when signing object_store PUT requests

1 participant