Skip to content

Commit 6bfe7f8

Browse files
committed
ci(image-merge): apply the keepalive+ci-cache source fix to image_merge
Mirror of 8521af1 (which fixed backend_merge.yml) for image_merge.yml. Today's master-push run 25823024353 failed the gpu-vulkan-image-merge job with the exact same error pattern the backend merge had on v4.2.2: ERROR: quay.io/go-skynet/local-ai@sha256:68b22611...: not found Same root cause: image_build.yml pushes the per-arch manifest to quay.io/go-skynet/local-ai with push-by-digest=true (no tag), then the merge runs minutes-to-hours later, by which time quay's per-repo manifest GC has reaped the untagged digest from local-ai. The blob still lives in quay's storage but local-ai@<digest> no longer resolves. Three matching edits: 1. image_build.yml: anchor each per-arch digest into ci-cache immediately after the push, reusing .github/scripts/anchor-digest-in-cache.sh with SOURCE_IMAGE=quay.io/go-skynet/local-ai and TAG_SUFFIX defaulting to "-core" for the core image (matches the artifact-name convention). 2. image_merge.yml: change the quay merge source from local-ai@<digest> to ci-cache@<digest>. Same correctness argument as backend_merge.yml — the manifest content is alive in ci-cache; buildx imagetools create republishes it into local-ai and writes the user-facing manifest list pointing at it. End state in local-ai is self-contained. 3. image_merge.yml: add a sparse `actions/checkout@v6` (only .github/scripts) so cleanup-keepalive-tags.sh is available, plus the cleanup step itself with TAG_SUFFIX matching the anchor's "-core" placeholder. v4.2.3's image.yml run completed successfully (~50 min between push and merge — beat quay's GC). This commit closes the race for future releases and master pushes regardless of run length. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
1 parent 5a42dbf commit 6bfe7f8

2 files changed

Lines changed: 39 additions & 1 deletion

File tree

.github/workflows/image_build.yml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,19 @@ jobs:
185185
digest="${{ steps.build.outputs.digest }}"
186186
touch "/tmp/digests/${digest#sha256:}"
187187
188+
# See .github/scripts/anchor-digest-in-cache.sh for why this is needed
189+
# and how it interacts with image_merge.yml's cleanup step. Mirrors the
190+
# same anchor in backend_build.yml — quay's per-repo manifest GC reaps
191+
# untagged manifests in local-ai before the merge runs.
192+
- name: Anchor digest in ci-cache so quay GC won't reap before merge
193+
if: github.event_name != 'pull_request'
194+
env:
195+
TAG_SUFFIX: ${{ inputs.tag-suffix == '' && '-core' || inputs.tag-suffix }}
196+
PLATFORM_TAG: ${{ inputs.platform-tag || 'single' }}
197+
DIGEST: ${{ steps.build.outputs.digest }}
198+
SOURCE_IMAGE: quay.io/go-skynet/local-ai
199+
run: .github/scripts/anchor-digest-in-cache.sh
200+
188201
- name: Upload digest artifact
189202
if: github.event_name != 'pull_request'
190203
uses: actions/upload-artifact@v7

.github/workflows/image_merge.yml

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,15 @@ jobs:
3333
env:
3434
quay_username: ${{ secrets.quayUsername }}
3535
steps:
36+
# Sparse checkout: needed for .github/scripts/ (the keepalive cleanup
37+
# script). Skips the rest of the source tree.
38+
- name: Checkout (.github/scripts only)
39+
uses: actions/checkout@v6
40+
with:
41+
sparse-checkout: |
42+
.github/scripts
43+
sparse-checkout-cone-mode: false
44+
3645
- name: Download digests
3746
uses: actions/download-artifact@v8
3847
with:
@@ -72,6 +81,13 @@ jobs:
7281
latest=${{ inputs.tag-latest }}
7382
suffix=${{ inputs.tag-suffix }},onlatest=true
7483
84+
# Source from ci-cache, not local-ai. See backend_merge.yml for the
85+
# detailed rationale — quay's manifest GC is per-repository, so the
86+
# untagged digest in local-ai gets reaped while the same content lives
87+
# tagged under ci-cache (anchored by image_build.yml). buildx imagetools
88+
# create copies the manifest into local-ai (blobs already cross-mounted)
89+
# and publishes the manifest list with user-facing tags. End state in
90+
# local-ai is self-contained; no embedded reference to ci-cache.
7591
- name: Create manifest list and push (quay)
7692
working-directory: /tmp/digests
7793
run: |
@@ -82,7 +98,7 @@ jobs:
8298
else
8399
# shellcheck disable=SC2086
84100
docker buildx imagetools create $tags \
85-
$(printf 'quay.io/go-skynet/local-ai@sha256:%s ' *)
101+
$(printf 'quay.io/go-skynet/ci-cache@sha256:%s ' *)
86102
fi
87103
88104
- name: Create manifest list and push (dockerhub)
@@ -107,6 +123,15 @@ jobs:
107123
docker buildx imagetools inspect "$first_tag"
108124
fi
109125
126+
# See .github/scripts/cleanup-keepalive-tags.sh for the best-effort
127+
# semantics — fails soft when the registry credential isn't OAuth-scoped.
128+
- name: Cleanup keepalive tags in ci-cache
129+
if: github.event_name != 'pull_request' && success()
130+
env:
131+
TAG_SUFFIX: ${{ inputs.tag-suffix == '' && '-core' || inputs.tag-suffix }}
132+
QUAY_TOKEN: ${{ secrets.quayPassword }}
133+
run: .github/scripts/cleanup-keepalive-tags.sh
134+
110135
- name: Job summary
111136
run: |
112137
set -euo pipefail

0 commit comments

Comments
 (0)