feat(bucket): manage S3 lifecycle rules#2
Open
munin92 wants to merge 2 commits into
Open
Conversation
Adds an optional `lifecycle` list to BucketSpec, so a bucket's expiration rules
become part of the same declarative object as its quota and versioning.
Motivation: a pull-through cache bucket that no client ever prunes. The application
writing it could not GC its own cache, and every external cleaner we tried needed a
LIST — which is exactly what fails once the volume is full, so the cleanup broke
precisely when it was needed. A server-side lifecycle rule is the only thing that
still works under pressure, and it was the one bucket property the operator could not
express, forcing an imperative PUT ?lifecycle job alongside otherwise declarative CRs.
rc-core already exposes get/set/delete_bucket_lifecycle on ObjectStore; this only
surfaces them.
Semantics follow the existing quota/versioning fields:
- unset -> unmanaged: the operator never even READS the bucket's lifecycle,
so adopting a bucket that already has rules cannot wipe them
- empty list -> "no rules": DELETEs the configuration (S3 rejects a PUT of an
empty one)
- non-empty -> replaces the configuration when it differs
The CRD models a deliberate subset (expiration by age, abort incomplete multipart
uploads). Transitions and non-current-version rules are omitted: they need storage
tiers / versioning that RustFS does not offer, and a CRD field that silently does
nothing is worse than no field. Drift is detected on the serialized form because
rc-core's LifecycleRule has no PartialEq — comparing only the modelled fields would
call a bucket in sync while an unmodelled rule still differed.
Tests: 4 unit tests (apply / no-op when matching / empty list deletes / unset is
untouched) and the integration test now round-trips a rule through a real RustFS
container, asserting the rule comes back with its prefix, expiration and
abort-multipart days, and is gone after delete.
feat(bucket): manage S3 lifecycle rules
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an optional
lifecyclelist toBucketSpec, so a bucket's expiration rulesbecome part of the same declarative object as its quota and versioning.
rc-corealready exposesget/set/delete_bucket_lifecycleonObjectStore; thisonly surfaces them through the CRD and reconcile loop.
Why
A pull-through cache bucket that no client ever prunes. The application writing it
cannot GC its own cache, and every external cleaner needs a
LIST— which is exactlywhat fails once the volume is full, so the cleanup breaks precisely when it is needed.
A server-side lifecycle rule is the only thing that still works under pressure, and it
was the one bucket property the operator could not express — forcing an imperative
PUT ?lifecyclejob alongside otherwise declarative CRs.Semantics
Follows the existing quota/versioning fields:
adopting a bucket that already has rules cannot wipe them
DELETEs the configuration (S3 rejects aPUTof anempty one)
Scope
The CRD models a deliberate subset — expiration by age and abort incomplete
multipart uploads. Transitions and non-current-version rules are omitted: they need
storage tiers / versioning that RustFS does not offer, and a CRD field that silently
does nothing is worse than no field.
Drift is detected on the serialized form because
rc-core'sLifecycleRulehas noPartialEq— comparing only the modelled fields would call a bucket in sync while anunmodelled rule still differed.
Tests
comes back with its prefix, expiration and abort-multipart days, and is gone after
delete
CI green on the fork; integration test additionally verified locally against a real
RustFS container.