Skip to content

S3 sink server-side encryption with KMS#6655

Merged
dlvenable merged 2 commits into
opensearch-project:mainfrom
dlvenable:6528-s3-sink-kms
Mar 20, 2026
Merged

S3 sink server-side encryption with KMS#6655
dlvenable merged 2 commits into
opensearch-project:mainfrom
dlvenable:6528-s3-sink-kms

Conversation

@dlvenable

Copy link
Copy Markdown
Member

Description

Adds new configuration for encryption options in the S3 sink. Allow configuring a custom KMS key for S3 server-side encryption. Support SSE-KMS and DSSE-KMS.

Includes integration test updates to verify. I ran these locally.

./gradlew :data-prepper-plugins:s3-sink:integrationTest -Dtests.s3sink.region=us-east-1 -Dtests.s3sink.bucket=my-bucket -Dtests.s3sink.kms_key=arn:aws:kms:us-east-1:123456789012:key/acf789f2-6ec6-22f2-aabb-71256718862c

Issues Resolved

Resolves #6528.

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Adds new configuration for encryption options in the S3 sink. Allow configuring a custom KMS key for S3 server-side encryption. Support SSE-KMS and DSSE-KMS.

Resolves opensearch-project#6528.

Signed-off-by: David Venable <dlv@amazon.com>
Comment on lines +29 to 32
final BucketOwnerProvider bucketOwnerProvider,
final ServerSideEncryptionConfig serverSideEncryptionConfig) {
return new MultipartBuffer(new S3OutputStream(s3Client, bucketSupplier, keySupplier, defaultBucket, bucketOwnerProvider));
}

@oeyh oeyh Mar 20, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This serverSideEncryptionConfig is not passed down to MultipartBuffer, so data will not be encrypted as configured when using MultipartBuffer.

@oeyh

oeyh commented Mar 20, 2026

Copy link
Copy Markdown
Collaborator

Build is also failing due to unused imports

Signed-off-by: David Venable <dlv@amazon.com>
@dlvenable dlvenable merged commit d4b8363 into opensearch-project:main Mar 20, 2026
70 of 72 checks passed
@dlvenable dlvenable deleted the 6528-s3-sink-kms branch March 23, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

S3 sink server-side encryption with KMS

3 participants