Skip to content

expose storage_class for s3 #1659

@msporleder-work

Description

@msporleder-work

Is your feature request related to a problem? Please describe.

s3 supports many storage classes. Far more than even mentioned in https://github.com/fluent/fluent-plugin-s3/blob/master/docs/output.md?plain=1#L332

I am specifically after INTELLIGENT_TIERING. I think that if the aws sdk is new enough it will just work to use the value.

Describe the solution you'd like

diff --git a/templates/conf/fluent.conf.erb b/templates/conf/fluent.conf.erb
index f8eaced8..3b762a1f 100644
--- a/templates/conf/fluent.conf.erb
+++ b/templates/conf/fluent.conf.erb
@@ -131,6 +131,7 @@
   s3_bucket "#{ENV['S3_BUCKET_NAME']}"
   s3_region "#{ENV['S3_BUCKET_REGION']}"
   s3_object_key_format "#{ENV['S3_OBJECT_KEY_FORMAT'] || '%{path}%Y/%m/%d/cluster-log-%{index}.%{file_extension}'}"
+  storage_class "#{ENV['S3_STORAGE_CLASS'] || 'STANDARD'}"
   <inject>
     time_key time
     tag_key tag

Describe alternatives you've considered

Lifecycle policies on the bucket have a transition cost. Going IT from upload is, probably, a better default than even STANDARD.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions