fix(ci): rebuild Go backends on linked pkg/ changes and on matrix entry edits#10988
Merged
Conversation
…ry edits PR #10975 taught the backend matrix filter about shared build inputs, but left two paths that still rebuild nothing. Go backends link code from the main tree. `go list -deps ./backend/go/...` resolves to exactly six pkg subtrees (audio, grpc incl. base/grpcerrors/proto, httpclient, sound, store, utils), identical for GOOS/GOARCH in {linux,darwin} x {amd64,arm64}. Editing any of them changes the shipped binary, but they sit outside every backend directory so the prefix match never saw them. Enumerating those six rather than taking all of pkg/ is the point: all of pkg/ changes in ~8.6% of commits, these six in 2.0% — the same order as the already accepted scripts/build/ rule (1.9%). Blast radius 199/417 Linux, 26/56 Darwin; the ~21 core-server-only pkg subtrees still rebuild nothing, and neither do _test.go files. .github/backend-matrix.yml was excluded wholesale because matching its path would rebuild all 417 entries on every new-backend PR. That hid a real hole: editing an existing entry's base-image, build-type or cuda version changes the image it produces while touching no file the filter can see. Since the change is within a structured file, compare it against the base revision and rebuild only the entries whose fields actually differ — 1 entry for a base-image edit, 0 for a comment or whitespace edit, and all 417 only when the previous revision cannot be resolved. This also closes a third hole: a new matrix entry for an existing backend (a new CUDA variant, say) touches nothing under that backend's directory and previously rebuilt nothing. changed-backends.js fetches the base revision via the contents API, and only when the changed-file list actually names the matrix file, so the common path costs no extra request. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#10975 taught the backend matrix filter about shared build inputs but deliberately left two paths that still rebuild nothing. Both are closed here.
Gap 1: Go backends link
pkg/, butpkg/changes rebuilt nothingExcluded from #10975 on the assumption that including
pkg/would fire a Go matrix on nearly every commit. The import analysis says otherwise.go list -deps ./backend/go/...resolves to exactly six subtrees —pkg/audio,pkg/grpc(incl.base,grpcerrors,proto),pkg/httpclient,pkg/sound,pkg/store,pkg/utils— identical for GOOS/GOARCH in {linux,darwin} × {amd64,arm64}. That is 6 of 27pkg/subtrees;pkg/model,pkg/downloader,pkg/xioand the rest are core-server-only and never reach a backend binary.pkg/scripts/build/(accepted in #10975)Makefile(rejected in #10975)Enumerating puts this at the same frequency as a rule #10975 already accepted at comparable radius, and below the one it rejected. The "199 images on every commit" concern is really "199 images on 1 commit in 50".
_test.gofiles are excluded (they never reach a binary — mirrors the existing*_test.shcarve-out), and Darwin's bespoke C++ builders that carrylang: goonly for runner selection are correctly skipped.Measured: 199/417 Linux, 26/56 Darwin. Unlinked subtrees and
_test.go: 0/417, 0/56.Gap 2: editing an existing matrix entry rebuilt nothing
.github/backend-matrix.ymlwas excluded wholesale because matching its path would rebuild all 417 entries on every new-backend PR. That hid a real hole: changing an entry'sbase-image,build-typeor CUDA version changes the image it produces while touching no file the prefix match can see.The change is within a structured file, so the filter now compares it against the base revision and rebuilds only the entries whose fields differ. Entries are keyed on
(tag-suffix, platform-tag)— verified unique across all 417 Linux and 56 Darwin entries — and fingerprinted order-independently, so reindenting or reordering compares equal.base-imageeditedvllmentries addedchanged-backends.jsfetches the base revision via the contents API, and only when the changed-file list names the matrix file, so the common path costs no extra request. The filter itself stays pure. An unresolvable base revision rebuilds everything rather than claiming nothing changed — that is precisely the #10946 failure mode this family of fixes exists to prevent.This also closes a third hole neither gap described: adding a new matrix entry for an existing backend (a new CUDA variant of vllm, say) touches nothing under that backend's directory and previously rebuilt nothing. It now rebuilds exactly the new entries.
Tests
13 new cases in
scripts/lib/backend-filter_test.mjs(node --test, zero deps, run bymake test-ci-scriptsfrom thebuild-scriptsjob).Red-first: 8 fail before the change — exactly the new positive triggers. The 5 negative pins (unlinked
pkg/subtree,_test.go, no-op matrix edit, entry removal, previous-ignored-when-file-unchanged) pass both before and after, which is what proves the new triggers are not over-broad — the same property that validated #10975.Final: 28/28 pass.
make lint: 0 issues.🤖 Generated with Claude Code