Is your feature request related to a problem? Please describe.
The S3 output plugin doesn't currently support Server-Side Encryption (SSE) headers. If you need SSE with a specific KMS key, you're stuck relying on bucket-level defaults, which doesn't always work for our use case.
We need to:
- Use SSE-KMS with our own customer-managed keys
- Control encryption at the Fluent Bit level instead of depending on bucket policies
Describe the solution you'd like
Add two new config options to the S3 output plugin:
-
sse - Server-side encryption type. Accepted values:
AES256 - S3-managed keys (SSE-S3)
aws:kms - AWS KMS-managed keys (SSE-KMS)
aws:kms:dsse - Dual-layer server-side encryption with KMS (DSSE-KMS)
-
sse_kms_key_id - AWS key ARN for SSE-KMS/DSSE-KMS encryption (optional). If not specified when using aws:kms or aws:kms:dsse, the default AWS-managed KMS key for S3 is used.
Example configuration:
[OUTPUT]
Name s3
Match *
bucket my-bucket
region eu-west-1
sse aws:kms
sse_kms_key_id arn:aws:kms:us-west-2:123456789012:key/my-key-id
This adds the following headers to S3 uploads:
x-amz-server-side-encryption: encryption type
x-amz-server-side-encryption-aws-kms-key-id: KMS key ARN (only for aws:kms or aws:kms:dsse)
Describe alternatives you've considered
- S3 bucket default encryption - Works but we can't control it from Fluent Bit and it's not flexible enough for our setup
- Post-processing with Lambda - Tried this but re-encrypting after upload is slow and leaves data unencrypted temporarily
Additional context
Is your feature request related to a problem? Please describe.
The S3 output plugin doesn't currently support Server-Side Encryption (SSE) headers. If you need SSE with a specific KMS key, you're stuck relying on bucket-level defaults, which doesn't always work for our use case.
We need to:
Describe the solution you'd like
Add two new config options to the S3 output plugin:
sse- Server-side encryption type. Accepted values:AES256- S3-managed keys (SSE-S3)aws:kms- AWS KMS-managed keys (SSE-KMS)aws:kms:dsse- Dual-layer server-side encryption with KMS (DSSE-KMS)sse_kms_key_id- AWS key ARN for SSE-KMS/DSSE-KMS encryption (optional). If not specified when usingaws:kmsoraws:kms:dsse, the default AWS-managed KMS key for S3 is used.Example configuration:
[OUTPUT] Name s3 Match * bucket my-bucket region eu-west-1 sse aws:kms sse_kms_key_id arn:aws:kms:us-west-2:123456789012:key/my-key-idThis adds the following headers to S3 uploads:
x-amz-server-side-encryption: encryption typex-amz-server-side-encryption-aws-kms-key-id: KMS key ARN (only for aws:kms or aws:kms:dsse)Describe alternatives you've considered
Additional context