Allow specifying transferDirectoryMaxConcurrency in S3TransferManager

[AndreKurait]

Describe the feature

Currently there is a hardcoded value for DEFAULT_DIRECTORY_TRANSFER_MAX_CONCURRENCY in TransferConfigurationOption of 100 added in #5031. When using the S3TransferManager with limited java heap allocation, this can cause OutOfMemory errors due to number of concurrent file downloads.

Customers would like the ability to specify transferDirectoryMaxConcurrency during creation of the S3TransferManager.

Use Case

Currently on smaller heap sizes, we can get OutOfMemory Errors with downloadDirectory. See #6323

Proposed Solution

Add parameter to specify transferDirectoryMaxConcurrency during creation of the S3TransferManager. See https://github.com/aws/aws-sdk-java-v2/pull/6329/files

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

AWS Java SDK version used

2.32.18

JDK version used

17

Operating System and version

AmazonLinux2023

[AndreKurait]

Implemented in #6329

[AndreKurait]

Released in 2.32.22, thanks @alextwoods

Customers can specify this with .transferDirectoryMaxConcurrency(<max-num-of-parallel-file-downloads>) in the transfer manager builder!

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.