Skip to content

PDJB-NONE: Inject singleton S3Presigner to fix native memory leak#1341

Open
Travis-Softwire wants to merge 1 commit into
mainfrom
fix/PDJB-NONE-s3-presigner-native-memory-leak
Open

PDJB-NONE: Inject singleton S3Presigner to fix native memory leak#1341
Travis-Softwire wants to merge 1 commit into
mainfrom
fix/PDJB-NONE-s3-presigner-native-memory-leak

Conversation

@Travis-Softwire
Copy link
Copy Markdown
Collaborator

Ticket number

PDJB-NONE

Goal of change

Stop AwsS3FileDownloader leaking native memory by reusing a single S3Presigner instead of constructing a new one per request.

Description of main change(s)

  • AwsS3FileDownloader now injects the singleton S3Presigner auto-configured by spring-cloud-aws-starter-s3, instead of calling S3Presigner.create() on every getDownloadUrl call.
  • Each presigner owns an underlying SDK HTTP client, connection pool, and direct ByteBuffers. Creating one per call (and never closing it) leaked off-heap memory until ECS killed the task for exceeding its container memory limit. Reusing a single thread-safe instance removes the leak and matches the pattern already used for S3Client / S3TransferManager in the other AwsS3* services.

Anything you'd like to highlight to the reviewer?

  • The other AwsS3* services (AwsS3FileUploader, AwsS3SafeFileDeleter, AwsS3QuarantinedFileDeleter, AwsS3DequarantiningFileCopier) were checked and already inject singleton SDK clients - no change needed there.
  • We should separately add JVM-level diagnostics (-XX:NativeMemoryTracking=summary, -XX:MaxDirectMemorySize, -XX:+HeapDumpOnOutOfMemoryError, a sensible -XX:MaxRAMPercentage) so the next native-memory incident is debuggable rather than just an ECS kill. Those flags live in the deployment infra repo, not here.

Checklist

  • Unit tests for new logic (e.g. new service methods) have been added — N/A, no behaviour change beyond reusing the bean; AwsS3FileDownloader has no existing tests

Copy link
Copy Markdown
Contributor

@JasminConterioSW JasminConterioSW left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds plausible 🤞

@samyou-softwire
Copy link
Copy Markdown
Contributor

out of interest, what documents that injecting S3Presigner will inject a singleton? this something in the library docs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants