Skip to content

ci: cache native builds, merge per-platform artifacts, cancel superseded runs#375

Merged
kdroidFilter merged 1 commit into
mainfrom
ci/native-build-cache
Jul 19, 2026
Merged

ci: cache native builds, merge per-platform artifacts, cancel superseded runs#375
kdroidFilter merged 1 commit into
mainfrom
ci/native-build-cache

Conversation

@kdroidFilter

Copy link
Copy Markdown
Collaborator

Problem

Native binaries were recompiled from scratch on every workflow invocation, even for pure-Kotlin PRs:

  • each PR push ran build-natives.yaml in parallel (pre-merge + test-graalvm + test-packaging) = 12 native-build runner jobs per push
  • a release tag ran it more (publish-maven, publish-plugin, release-desktop, release-graalvm)
  • no cache anywhere, while the outputs are a pure function of src/main/native/**
  • every consumer job carried ~25 copy-pasted download-artifact steps (~1400 duplicated YAML lines across 8 jobs)
  • pre-merge.yaml had no concurrency group, so superseded runs kept burning runners

Changes

build-natives.yaml

  • each platform job restores an output cache keyed on hashFiles('**/src/main/native/**', '.github/workflows/build-natives.yaml'); on a hit, all toolchain/compile/fetch steps are skipped — the job only verifies the restored tree and re-publishes (~1-2 min instead of 10-20)
  • per-module verify + upload steps collapsed into one full-list verify step and one merged artifact per platform: natives-windows, natives-macos, natives-linux-{x64,aarch64} (paths repo-root-relative)
  • key prefix natives-v1 can be bumped to force a rebuild

Consumers (7 workflows) — the download blocks become a single step:

- name: Download native artifacts
  uses: actions/download-artifact@v4
  with:
    pattern: 'natives-*'
    merge-multiple: true

The "Verify all natives present" EXPECTED checks in pre-merge/publish-maven are kept as-is.

pre-merge.yaml — concurrency group with cancel-in-progress on PR events only (main pushes are never cancelled: they save the caches every other run keys off, since tag runs can read default-branch caches — releases hit the cache built on main).

CLAUDE.md — "Adding a Native JNI Module" procedure updated: a new module needs a gated build step + verify entries in build-natives.yaml only; consumer workflows need no changes anymore.

Net: -1806 lines of YAML, and after this PR's first run populates the caches, an unchanged-natives PR spends ~5 min of runner time on natives instead of ~1.5-2 h aggregate.

Validation

  • actionlint clean (only pre-existing shellcheck warnings)
  • this PR's own CI exercises the full new flow: cache miss → build → merged artifacts → single-download consumers; a re-run then exercises the cache-hit path

The native binaries are a pure function of the src/main/native sources, yet
every workflow invocation recompiled all of them from scratch: each PR push
triggered three parallel full rebuilds (pre-merge + test-graalvm +
test-packaging), and a release tag four more.

build-natives.yaml now caches each platform's output tree keyed on
hashFiles('**/src/main/native/**') plus the workflow itself; on a hit every
toolchain/compile step is skipped and the job only verifies and re-publishes.
Per-module verify/upload steps are collapsed into one verify step and one
merged artifact per platform (natives-windows, natives-macos,
natives-linux-{x64,aarch64}), so the ~25 copy-pasted download steps in each
of the 7 consumer jobs become a single 'pattern: natives-*' download.

pre-merge.yaml also gains a concurrency group cancelling superseded PR runs
(pushes to main are never cancelled: their run saves the shared caches).
@kdroidFilter
kdroidFilter merged commit ea0a411 into main Jul 19, 2026
34 checks passed
@kdroidFilter
kdroidFilter deleted the ci/native-build-cache branch July 19, 2026 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant