Skip to content

feat: add required cache-key input, native GC, and server-driven config for v2#108

Merged
piob-io merged 4 commits into
release/v2from
devin/v2-cache-key
Jun 23, 2026
Merged

feat: add required cache-key input, native GC, and server-driven config for v2#108
piob-io merged 4 commits into
release/v2from
devin/v2-cache-key

Conversation

@piob-io

@piob-io piob-io commented Jun 23, 2026

Copy link
Copy Markdown

Summary

v2 of setup-docker-builder with three changes:

1. Required cache-key input replaces hardcoded GITHUB_REPO_NAME

Repos with multiple Dockerfiles (e.g., services/api/Dockerfile + services/web/Dockerfile) currently thrash each other's cache due to last-writer-wins commit semantics on a shared sticky disk key. cache-key makes the cache entity explicit — different Dockerfiles get different keys, eliminating cross-target thrashing.

# before (v1): implicit key = repo name → all targets share one cache
- uses: useblacksmith/setup-docker-builder@v1

# after (v2): explicit key per Dockerfile
- uses: useblacksmith/setup-docker-builder@v2
  with:
    cache-key: services/api/Dockerfile

stateHelper.setCacheKey() / getCacheKey() persists the value across setup → post phases, and commitStickyDisk() now passes it to the RPC.

2. BuildKit native GC replaces manual buildctl prune

  • buildkitd.toml: gc = true with gcPolicy = [{ keepDuration = "192h", all = true }] (8-day retention)
  • Removed max-cache-size-mb input and the entire manual prune block from post phase
  • writeBuildkitdTomlFile now accepts an optional BuildkitdConfig parameter (falls back to DEFAULT_BUILDKITD_CONFIG)

3. Server-driven config infrastructure (src/server-config.ts)

Two primitives for future backend control without action updates:

  • BuildkitdConfig: structured config for daemon startup (gc on/off, gcPolicy[], maxParallelism). Currently hardcoded; Phase 2 will read from GetStickyDisk RPC response.
  • PreCommitHook[] + runPreCommitHooks(): ordered opaque shell commands run before commit. Each hook has command, timeoutSeconds, and failureMode (skip_commit | commit_anyway | abort). Currently empty list; Phase 3 PrepareCommit RPC will populate.

Post-phase flow:

PrepareCommitResponse { shouldCommit, hooks[] }
  → run hooks (if any, with timeout + failure handling)
  → if shouldCommit: commitStickyDisk(exposeId, size, cacheKey)

4. Cleanup: removed unused rollup build artifacts

rollup.config.js and rollup devDependencies removed — the repo switched to ncc bundling previously but never cleaned these up. dist/main.bundle.mjs (rollup output) was already absent.

Design doc: FastActions/fa#4249</n Public docs: useblacksmith/docs#228</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n</n">

Link to Devin session: https://app.devin.ai/sessions/1054a9452e8a49ccb891391e2327a612
Requested by: @piob-io


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled. (Staging)

piob-io and others added 4 commits June 22, 2026 21:37
Replace the hardcoded GITHUB_REPO_NAME sticky disk key with a required
cache-key input. Users must explicitly declare which cache entity their
build targets, preventing cache thrashing in multi-target repos.

The cache-key is saved to action state and passed through to the commit
phase so the same key is used for both expose and commit RPCs.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Remove max-cache-size-mb input and manual buildctl prune from the post
phase. Enable BuildKit's native garbage collection with a time-based
policy (keepDuration=192h / 8 days). Layers unused for longer than the
retention period are automatically cleaned up; active layers survive
indefinitely regardless of total cache size.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ooks

Add BuildkitdConfig and PreCommitHook interfaces (server-config.ts) that
define the contract for server-driven cache lifecycle management. The
action reads BuildkitdConfig from the GetStickyDisk RPC response (with
hardcoded fallback for Phase 1) and runs pre-commit hooks from the
PrepareCommit RPC response before committing.

Refactor writeBuildkitdTomlFile to accept a BuildkitdConfig parameter
instead of hardcoding GC policy values. This makes it trivial to switch
from hardcoded defaults to server-provided config when the proto is
updated (Phase 2).

Add runPreCommitHooks dispatcher that executes an ordered list of opaque
shell commands with timeout and failure mode handling. The commit flow
now goes through prepareCommit -> runHooks -> conditionalCommit, with
Phase 1 returning unconditional commit + empty hooks.

Phase 1 (now): hardcoded gc=true, keepDuration=192h, no hooks
Phase 2: server returns BuildkitdConfig in GetStickyDiskResponse
Phase 3: server returns hooks via PrepareCommit RPC

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Remove rollup.config.js and rollup devDependencies (@rollup/plugin-node-resolve,
@rollup/plugin-typescript, rollup). The repo switched to ncc for bundling in
commit 2ffcb3d; rollup artifacts were never cleaned up. The dist/main.bundle.mjs
file was already deleted in a prior commit.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@piob-io piob-io self-assigned this Jun 23, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@piob-io piob-io marked this pull request as ready for review June 23, 2026 17:47

@bruce-y bruce-y left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if we're going to turn GC back on we should probably remove the relevant code in our buildkit fork that alters the GC behaviour here that we added to support the previous usecase: useblacksmith/buildkit@e27a9da

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Good call. For context on the interaction with our v2 GC config:

Our policy is { keepDuration: "192h", all: true } — no keepBytes. So in pruneOnce, your commit's key condition:

if len(cr.refs) > 0 && (!opt.all || opt.keepBytes == 0) {
    cr.mu.Unlock()
    continue  // still skips layers with refs
}

evaluates to !true || 0 == 0true, meaning layers with active refs are still skipped. So the behavioral change in e27a9dac doesn't actually activate with our current config — it only kicks in when all: true AND keepBytes > 0 (size-based GC).

That said, agreeing it's worth reverting to stay closer to upstream. The other two fork commits (v0.29.1 sort fix, v0.29.2 debug logging) are independent and could be evaluated separately.

This is a BuildKit fork change though, not an action change — flagging for @piotr to decide whether to handle as part of this PR or separately.

@piob-io

piob-io commented Jun 23, 2026

Copy link
Copy Markdown
Author

We will have to support both action versions concurrently for a while. If the same version behaves well enough for both, I'd delay the cleanup. However, if the patched binary behaves observably worse for the new setup, we need a way for actions at different versions to reference two different binaries. Devin, please analyze.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Analyzed all three fork patches (v0.29.1–v0.29.3) against both action versions. The same binary works correctly for both v1 and v2 — no observable difference for either.

Per-patch analysis

Patch What it does v1 (gc: false, manual buildctl prune --keep-storage=X) v2 (gc: true, keepDuration=192h, all=true, no keepBytes)
v0.29.1 (always sort) Moves sortDeleteRecords outside the if gcMode block so candidates are sorted in non-GC mode too ✅ Needed — fixes random deletion order during manual prune Neutral — in GC mode, sorting already happened before this patch (batchSize=1 picks the first sorted record)
v0.29.2 (debug logging) Adds Infof for skipped-ref layers and first deletion candidate Removed by v0.29.3 — no longer in the codebase N/A
v0.29.3 (refs + --all) When all=true AND keepBytes > 0: includes ref'd layers in candidate evaluation instead of skipping them ✅ Needed — v1's --keep-storage sets keepBytes > 0, so ref'd layers are correctly counted toward size budget No-op — v2 has keepBytes=0, so the guard !opt.all || opt.keepBytes == 0true, and ref'd layers are still skipped (identical to upstream)

Why one binary is fine

The v0.29.3 behavioral change is gated by keepBytes > 0. v2 never sets keepBytes (our policy is purely time-based), so the patched code path is never reached. v2's pruneOnce executes identically to upstream BuildKit for our config.

v1 needs the patch active because manual prune uses --keep-storage which sets keepBytes > 0.

Sorting change (v0.29.1) is beneficial for both

The sort fix moves sortDeleteRecords(toDelete) to run unconditionally (when there are candidates), rather than only in GC mode. For v2 this is harmless (GC mode processes 1 record at a time anyway, sorting is just "pick the oldest"). For v1 it's a correctness fix.

Recommendation: keep the current binary for both versions. No cleanup needed until v1 is fully deprecated.

@piob-io piob-io merged commit 426bee8 into release/v2 Jun 23, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants