fix(ci): repair nightly backend dep bumps for renamed localai-org repos#11012
Merged
Conversation
The "Bump Backend dependencies" workflow has failed every night for over ten days. Four upstreams — ced.cpp, moss-transcribe.cpp, voice-detect.cpp and rf-detr.cpp — moved from the mudler org to localai-org, so the GitHub API answers 301 for the old slugs. ced.cpp additionally renamed its default branch to main. bump_deps.sh fetched without -L or -f and never checked the response, so the redirect's JSON body was passed straight to sed, which died with "unterminated `s' command". The loud failure was luck: an error body without slashes would have been substituted into the Makefile as the new pin, silently corrupting the version and shipping it in a bump PR. Point the matrix at the new slugs and branch, and harden the script so a bad response can never reach sed: follow redirects, fail on HTTP errors, and require a bare 40-hex SHA before rewriting anything. Also refresh the now-stale repository URLs in the backend Makefiles, test scripts, backend/index.yaml and the docs. Verified all 25 matrix entries resolve to a commit SHA and that the four previously-failing jobs run end to end against the real API. Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Collaborator
Author
|
Per .agents/ai-coding-assistants.md, an AI agent MUST NOT add a To make DCO green, a human maintainer needs to sign off, e.g.: |
mudler
pushed a commit
that referenced
this pull request
Jul 22, 2026
The "Bump Backend dependencies" workflow has failed every night for the last two weeks. #11012 fixed one cause (repos renamed under localai-org); what is left is rate limiting. bump_deps.sh fans out to ~25 parallel matrix jobs that each query api.github.com anonymously. Anonymous calls are capped at 60/hour per source IP and GitHub-hosted runners egress through shared NAT addresses, so a random handful of jobs draw HTTP 403 and die at curl exit 22 with an empty response. Last night that hit ggml-org/whisper.cpp and mudler/depth-anything.cpp -- both public and resolvable, nothing wrong with either pin. Route every bump script through a shared gh_curl helper that sends GITHUB_TOKEN when present (1000/hour instead of 60) and retries transient failures, including the 403s that plain --retry ignores. The helper suppresses xtrace around the call so the Authorization header cannot land in a public job log. bump_docs.sh had a sharper version of the same bug: it piped an unchecked response into `jq -r .tag_name`, so a throttled request resolved to the string "null" and would have been published as the docs version. It now refuses to write anything it cannot resolve to a tag. Verified locally by running all four scripts end to end against their real upstreams: correct SHAs/tags written, exit 0; a nonexistent repo now fails with a named diagnostic instead of a bare exit 22 and leaves the pinned file untouched; the token is absent from the xtrace output; and the scripts still work unauthenticated. Assisted-by: Claude:opus-4.8 [Claude Code] Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Bump Backend dependencies workflow has been red every night for over ten days (since ~Jul 11). Four of its matrix jobs fail consistently:
ced.cpp,moss-transcribe.cpp,voice-detect.cpp,rf-detr.cpp.Root cause
Those four upstreams moved from the
mudlerorg tolocalai-org, so the GitHub API now answers301 Moved Permanentlyfor the old slugs.ced.cppadditionally renamed its default branchmaster→main..github/bump_deps.shfetched the commit SHA with a barecurl -s— no-L, no-f, and no validation of the result. The redirect body was piped straight intosed:The loud failure was luck. It only crashed because the JSON contains slashes. An error body without slashes — a rate-limit message, a plain-text error — would have been substituted in as the new pin, silently corrupting the version and shipping it in an auto-merged bump PR.
Changes
localai-org/*, and fixced.cpp's branch tomain.bump_deps.shso a bad response can never reachsed:curl -sfL(follow redirects, fail on HTTP errors) plus a^[0-9a-f]{40}$guard that aborts with a clear message instead of rewriting the Makefile.backend/index.yaml, README and docs.gallery/index.yamlis deliberately untouched — those are metadata links that redirect fine, and it is rewritten constantly by bots, so touching it here would only invite conflicts.Verification
All 25 matrix entries were checked against the live API — every one resolves to a commit SHA, and every referenced Makefile contains the named variable:
The four previously-failing jobs were then run end to end through the real script against the live API — all exit 0 and write a valid SHA.
ced.cpphad a genuine pending bump that the broken job had been missing:The guard was also confirmed to fail closed: pointing the hardened script at a slug that errors exits non-zero and leaves the Makefile untouched.