Cap GC threshold at store filesystem capacity#394
Open
vlast3k wants to merge 1 commit into
Open
Conversation
When a cell is redeployed with a smaller disk, the thresholder computes a GC threshold from the new disk's available space that can exceed the existing GrootFS XFS backing store capacity. This results in the threshold never being reached and garbage collection never firing, eventually leading to ENOSPC. After computing the threshold via the calculator, cap it at the actual store filesystem total capacity obtained via statfs(2). This ensures threshold_bytes <= store size regardless of disk resize history. Adds Meter.GetTotalCapacity() using Blocks*Bsize from statfs, with unit tests and an integration test that creates a small XFS store inside the test filesystem to verify the cap fires correctly. Closes: CFAR-1378
cf-foundation-community-automation
Bot
moved this to Inbox
in Application Runtime Platform Working Group
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ai-assisted=yes
Summary
The thresholder computes
threshold_bytesfrom disk available space, but doesn't check whether that value exceeds the actual GrootFS XFS store capacity. After a redeploy with a smaller disk, the store keeps its original (smaller) capacity while the threshold is recalculated against the new disk — can end up with threshold > store, so GC never fires and the store fills up.This adds a statfs check on the store path after threshold calculation and caps at
Blocks * Bsizeif the computed threshold would exceed the store's total capacity.Tests: new integration test creates a 100M XFS store inside the 20G test filesystem, verifies the cap kicks in. All 25 existing specs still pass.
Backward Compatibility
Breaking Change? No
Only fires when the store already exists at a smaller capacity than the computed threshold. Normal deploys unaffected.