Skip to content

feat: Add presigned URL download support to S3AsyncClient and S3 Transfer Manager#7115

Open
jencymaryjoseph wants to merge 40 commits into
masterfrom
feature/master/pre-signed-url-getobject
Open

feat: Add presigned URL download support to S3AsyncClient and S3 Transfer Manager#7115
jencymaryjoseph wants to merge 40 commits into
masterfrom
feature/master/pre-signed-url-getobject

Conversation

@jencymaryjoseph

@jencymaryjoseph jencymaryjoseph commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Motivation and Context

Adds the ability to download S3 objects using pre-signed URLs through the SDK's async client pipeline. This enables customers who receive pre-signed URLs to download objects with SDK features without needing AWS credentials configured.

Modifications

Public API

  • AsyncPresignedUrlExtension interface on S3AsyncClient (accessed via s3AsyncClient.presignedUrlExtension())
  • PresignedUrlDownloadRequest model for presigned URL downloads
  • S3TransferManager.downloadFileWithPresignedUrl() and downloadWithPresignedUrl() with progress tracking support
  • PresignedDownloadFileRequest, PresignedDownloadRequest<T>, PresignedFileDownload Transfer Manager models

Core SDK

  • SKIP_ENDPOINT_RESOLUTION execution attribute in SdkInternalExecutionAttribute
  • skipEndpointResolution() utility method in AwsEndpointProviderUtils
  • SplittingTransformerConfiguration.responseMapper() for metadata rewriting in multipart downloads
  • DefaultChecksumAlgorithm.values() for dynamic checksum algorithm registration
  • EndpointResolutionStage updated to respect SKIP_ENDPOINT_RESOLUTION

S3 Internal Implementation

  • DefaultAsyncPresignedUrlExtension — executes GetObject via presigned URL with NoOpSigner
  • PresignedUrlDownloadRequestMarshaller — custom marshaller using the presigned URL directly
  • ParallelPresignedUrlMultipartDownloaderSubscriber — parallel Range-header multipart downloads
  • PresignedUrlMultipartDownloaderSubscriber — sequential multipart downloader
  • MultipartAsyncPresignedUrlExtension — multipart-enabled variant

Code Generation

  • presignedUrlExtensionSupported customization flag to generate presignedUrlExtension() on async clients

Testing

  • Unit tests for all new classes
  • Reactive Streams TCK compliance tests for both multipart subscribers
  • WireMock integration tests for multipart, retry, checksum validation, and progress listener scenarios
  • Integration tests against real S3 (single-part and multipart)
  • Transfer Manager integration tests
  • All existing S3, sdk-core, aws-core, and codegen tests passing

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

…6216)

* Add presigned URL request models and tests

* Pre-signed URL API and Get Object Request definitions

* JavaDoc changes, String to URL change

* Delete services/s3/src/main/java/software/amazon/awssdk/services/s3/internal/presignedurl/DefaultPresignedUrlManager.java

* Delete services/s3/src/main/java/software/amazon/awssdk/services/s3/internal/presignedurl/PresignedUrlGetObjectRequestMarshaller.java

* Delete services/s3/src/test/java/software/amazon/awssdk/services/s3/presignedurl/model/PresignedUrlTest.java

* Format PresignedUrlGetObjectRequestWrapper.java

* Format PresignedUrlManager.java

* Format PresignedUrlGetObjectRequest.java

* Format PresignedUrlGetObjectRequestWrapperTest.java

* Format PresignedUrlGetObjectRequestTest.java

* sonarcube issues fixed

* Delete .idea/codeStyles/Project.xml

* added consumer builder, javadoc for builder, assert range value test

* removing codestle from gitignore
* presigned url get object request custom marshaller

* used protocolFactory and fixed sonar cube issues

* removed unused test method, removed overriden method
…gned URL (#6241)

* getobject implementation and tests

* corrected the getObject params in test:

* fixed build failure-dependency issue with Retrystrategy in test

* fixed sonarcube issues

* addressed sonarcube issue

* edited javadocs, added invalidurl test
…ed URLs (#6255)

* async presignedurlmanager api and implementation

* Refactor DefaultAsyncPresignedUrlManager: move config to constructor, declare variables inline

* change operation name
* async s3 client codegen changes

* minimised service-2.json
…Endpoint, enhance AsyncPresignedUrlManager API (#6275)

* modify operation name, null check for resolvedendpoint, add getObject APIs

* added null check for resolvedendpointHeader, doc updates, test additions

* fixed checkstyle issue
…iew (#6286)

* surface api name changes

* rename files, remove consumer builder from downloadrequest

* some nitpicks

* renamed variables
* Amazon DataZone Update: This release adds support for 1) highlighting relevant text in returned results for Search and SearchListings APIs and 2) returning aggregated counts of values for specified attributes for SearchListings API.

* Amazon Omics Update: Add Git integration and README support for HealthOmics workflows

* Updated endpoints.json and partitions.json.

* Release 2.32.8. Updated CHANGELOG.md, README.md and all pom.xml.

* Update to next snapshot version: 2.32.9-SNAPSHOT

* Amazon AppIntegrations Service Update: Amazon AppIntegrations introduces new configuration capabilities to enable customers to manage iframe permissions, control application refresh behavior (per contact or per browser/cross-contact), and run background applications (service).

* AWS Key Management Service Update: Doc only update: fixed grammatical errors.

* AWS Config Update: Documentation improvements have been made to the EvaluationModel and DescribeConfigurationRecorders APIs.

* AWS End User Messaging Social Update: This release introduces new WhatsApp template management APIs that enable customers to programmatically create and submit templates for approval, monitor approval status, and manage the complete template lifecycle

* AWS Budgets Update: Adds IPv6 and PrivateLink support for AWS Budgets in IAD.

* Amazon Simple Queue Service Update: Documentation updates for Amazon SQS fair queues feature.

* AWS Elemental MediaPackage v2 Update: This release adds support for specifying a preferred input for channels using CMAF ingest.

* Amazon Elastic Compute Cloud Update: Transit Gateway native integration with AWS Network Firewall. Adding new enum value for the new Transit Gateway Attachment type.

* Release 2.32.9. Updated CHANGELOG.md, README.md and all pom.xml.

* Update to next snapshot version: 2.32.10-SNAPSHOT

* AWS IoT SiteWise Update: Add support for native anomaly detection in IoT SiteWise using new Computation Model APIs

* Amazon OpenSearch Ingestion Update: Add Pipeline Role Arn as an optional parameter to the create / update pipeline APIs as an alternative to passing in the pipeline configuration body

* AWS Direct Connect Update: Enable MACSec support and features on Interconnects.

* Updated endpoints.json and partitions.json.

* Release 2.32.10. Updated CHANGELOG.md, README.md and all pom.xml.

* Update to next snapshot version: 2.32.11-SNAPSHOT

* Rebase #6269: Update PresignedUrlManager to PresignedUrlExtension naming

* Integ tests

* Refactor AsyncPresignedUrlManager integration tests to extend S3IntegrationTestBase

* updated test method names to follow testing guidelines

* simplified tests

* added checksum test for largeobjects

* rename in test files, address nit comment, fix rebase issues

---------

Co-authored-by: AWS <>
#6318)

* Add multipart presigned URL download support with range-based routing

* Renamed AsyncPresignedUrlExtension variable

* Commented the failing test with TODO
* Transfer Manager presigned URL download request models

* Fixed checkstyle issues
)

* Range based multipart download subscriber

* Refactor code and add tests

* Fixed sonarcube issue

* Address PR comments: use Long type, inline variables, allow empty files

* Refactored subscriber and tests

* delete testutil

* Added validation tests

* Fixed failing tests for IfMatch header

* Refactor: split validatePartAndRequestMore into separate methods

* Fixed validation tests

* Remove onComplete after subscription cancel

* Make validatePart boolean function
* Fix check-then-act race condition in parallel subscriber

* address comments
* Add presigned URL download support to S3 Transfer Manager

* Add presigned URL download methods to DelegatingS3TransferManager

* Address PR Comments
…dd listener tests (#6977)

* fix progress tracking for ranged requests

* Add WireMock listener tests
…aws/aws-sdk-java-v2 into feature/master/pre-signed-url-getobject
)

* Add PresignedFileDownload return type for presigned URL downloads

* Address PR Comments
…aws/aws-sdk-java-v2 into feature/master/pre-signed-url-getobject
…t subscriber (#7013)

* Fix first part validation and refactor

* Use String.format for exceptions
* Enable checksum validation for presigned URL downloads

* Add public method to return all checksum algorithms

* assert checksum validation status
… paths (#7077)

* Add response rewrite infrastructure for multipart download metadata

* Fix presigned URL multipart download response metadata and 416 fallback

* Fix normal multipart download response metadata

* Consolidate normal path integration tests with parameterized tests

* Address review: Builder ctor, split(config,mapper) interface method, parameterized tests

* Fix transfer manager presigned serial wrapper to use correct progress wrapper

* Move responseMapper to SplittingTransformerConfiguration and simplify split wiring

* Revert normal object path changes and add presigned custom transformer metadata test

* Use single splitWithResponseRewrite

* Clarify responseMapper Javadoc on SplittingTransformerConfiguration
@jencymaryjoseph jencymaryjoseph added the api-surface-area-approved-by-team Indicate API surface area introduced by this PR has been approved by team label Jul 8, 2026
@jencymaryjoseph jencymaryjoseph marked this pull request as ready for review July 8, 2026 16:47
@jencymaryjoseph jencymaryjoseph requested a review from a team as a code owner July 8, 2026 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api-surface-area-approved-by-team Indicate API surface area introduced by this PR has been approved by team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant