feat: add required cache-key input, native GC, and server-driven config for v2#108
Conversation
Replace the hardcoded GITHUB_REPO_NAME sticky disk key with a required cache-key input. Users must explicitly declare which cache entity their build targets, preventing cache thrashing in multi-target repos. The cache-key is saved to action state and passed through to the commit phase so the same key is used for both expose and commit RPCs. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Remove max-cache-size-mb input and manual buildctl prune from the post phase. Enable BuildKit's native garbage collection with a time-based policy (keepDuration=192h / 8 days). Layers unused for longer than the retention period are automatically cleaned up; active layers survive indefinitely regardless of total cache size. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ooks Add BuildkitdConfig and PreCommitHook interfaces (server-config.ts) that define the contract for server-driven cache lifecycle management. The action reads BuildkitdConfig from the GetStickyDisk RPC response (with hardcoded fallback for Phase 1) and runs pre-commit hooks from the PrepareCommit RPC response before committing. Refactor writeBuildkitdTomlFile to accept a BuildkitdConfig parameter instead of hardcoding GC policy values. This makes it trivial to switch from hardcoded defaults to server-provided config when the proto is updated (Phase 2). Add runPreCommitHooks dispatcher that executes an ordered list of opaque shell commands with timeout and failure mode handling. The commit flow now goes through prepareCommit -> runHooks -> conditionalCommit, with Phase 1 returning unconditional commit + empty hooks. Phase 1 (now): hardcoded gc=true, keepDuration=192h, no hooks Phase 2: server returns BuildkitdConfig in GetStickyDiskResponse Phase 3: server returns hooks via PrepareCommit RPC Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Remove rollup.config.js and rollup devDependencies (@rollup/plugin-node-resolve, @rollup/plugin-typescript, rollup). The repo switched to ncc for bundling in commit 2ffcb3d; rollup artifacts were never cleaned up. The dist/main.bundle.mjs file was already deleted in a prior commit. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
There was a problem hiding this comment.
I think if we're going to turn GC back on we should probably remove the relevant code in our buildkit fork that alters the GC behaviour here that we added to support the previous usecase: useblacksmith/buildkit@e27a9da
|
Good call. For context on the interaction with our v2 GC config: Our policy is if len(cr.refs) > 0 && (!opt.all || opt.keepBytes == 0) {
cr.mu.Unlock()
continue // still skips layers with refs
}evaluates to That said, agreeing it's worth reverting to stay closer to upstream. The other two fork commits (v0.29.1 sort fix, v0.29.2 debug logging) are independent and could be evaluated separately. This is a BuildKit fork change though, not an action change — flagging for @piotr to decide whether to handle as part of this PR or separately. |
|
We will have to support both action versions concurrently for a while. If the same version behaves well enough for both, I'd delay the cleanup. However, if the patched binary behaves observably worse for the new setup, we need a way for actions at different versions to reference two different binaries. Devin, please analyze. |
|
Analyzed all three fork patches (v0.29.1–v0.29.3) against both action versions. The same binary works correctly for both v1 and v2 — no observable difference for either. Per-patch analysis
Why one binary is fineThe v0.29.3 behavioral change is gated by v1 needs the patch active because manual prune uses Sorting change (v0.29.1) is beneficial for bothThe sort fix moves Recommendation: keep the current binary for both versions. No cleanup needed until v1 is fully deprecated. |
Summary
v2 of
setup-docker-builderwith three changes:1. Required
cache-keyinput replaces hardcodedGITHUB_REPO_NAMERepos with multiple Dockerfiles (e.g.,
services/api/Dockerfile+services/web/Dockerfile) currently thrash each other's cache due to last-writer-wins commit semantics on a shared sticky disk key.cache-keymakes the cache entity explicit — different Dockerfiles get different keys, eliminating cross-target thrashing.stateHelper.setCacheKey()/getCacheKey()persists the value across setup → post phases, andcommitStickyDisk()now passes it to the RPC.2. BuildKit native GC replaces manual
buildctl prunebuildkitd.toml:gc = truewithgcPolicy = [{ keepDuration = "192h", all = true }](8-day retention)max-cache-size-mbinput and the entire manual prune block from post phasewriteBuildkitdTomlFilenow accepts an optionalBuildkitdConfigparameter (falls back toDEFAULT_BUILDKITD_CONFIG)3. Server-driven config infrastructure (
src/server-config.ts)Two primitives for future backend control without action updates:
BuildkitdConfig: structured config for daemon startup (gc on/off,gcPolicy[],maxParallelism). Currently hardcoded; Phase 2 will read fromGetStickyDiskRPC response.PreCommitHook[]+runPreCommitHooks(): ordered opaque shell commands run before commit. Each hook hascommand,timeoutSeconds, andfailureMode(skip_commit|commit_anyway|abort). Currently empty list; Phase 3PrepareCommitRPC will populate.Post-phase flow:
4. Cleanup: removed unused rollup build artifacts
rollup.config.jsand rollup devDependencies removed — the repo switched tonccbundling previously but never cleaned these up.dist/main.bundle.mjs(rollup output) was already absent.Design doc: FastActions/fa#4249</n Public docs: useblacksmith/docs#228</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n">
Link to Devin session: https://app.devin.ai/sessions/1054a9452e8a49ccb891391e2327a612
Requested by: @piob-io
Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled.Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled. (Staging)