Skip to content

Commit 8b3477f

Browse files
huangdihdluigimak
andauthored
ci: download ccache once via setup job + artifact (fix parallel 504 throttling) (WildKernels#231)
* ci: download ccache once via setup job + artifact (fix parallel 504 throttling) Every matrix build job curl'd the same GitHub raw ccache URL at once; with ~30 parallel jobs GitHub's edge rate-limits the CI IPs and returns repeated 504s, failing the whole matrix (the URL is fine in a browser -- it is the parallel hammering that trips the throttle). The existing in-step `--retry 5` fires in lockstep across all jobs and stays throttled. Add a `prepare_ccache` job that downloads the binary ONCE (retries with jittered backoff) and shares it as an artifact; build jobs download that artifact instead of hitting the URL. If the single download still fails, build jobs fall back to the apt `ccache` already installed in "Install Minimal Dependencies". Co-authored-by: luigimak <luigimak@hotmail.it>
1 parent b358f0d commit 8b3477f

1 file changed

Lines changed: 59 additions & 11 deletions

File tree

.github/workflows/build-kernel-release.yml

Lines changed: 59 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -528,9 +528,46 @@ jobs:
528528
uses: ./.github/workflows/mirror-toolchains.yml
529529
secrets: inherit
530530

531+
prepare_ccache:
532+
name: Prepare ccache binary (download once)
533+
runs-on: ubuntu-latest
534+
steps:
535+
# Download the custom ccache binary ONCE for the whole matrix and share it as
536+
# an artifact. Previously every parallel matrix job curl'd the same GitHub raw
537+
# URL simultaneously, which made GitHub's edge rate-limit / 504 the CI IPs and
538+
# fail the whole build (the URL is fine in a browser -- it's the ~30x parallel
539+
# hammering that triggers the throttle).
540+
- name: Download custom ccache once
541+
run: |
542+
set -uo pipefail
543+
echo "::group::Download ccache"
544+
url="https://raw.githubusercontent.com/WildKernels/kernel_patches/refs/heads/main/ccache/ccache-x86-64"
545+
ok=0
546+
for attempt in 1 2 3 4 5 6; do
547+
if curl -LfsS --connect-timeout 30 --max-time 120 -H "User-Agent: Mozilla/5.0" "$url" -o ccache && [ -s ccache ]; then
548+
ok=1; break
549+
fi
550+
echo "attempt $attempt failed (GitHub raw 504/throttle); backing off..."
551+
sleep $(( attempt * 5 + RANDOM % 6 ))
552+
done
553+
if [ "$ok" = 1 ]; then
554+
echo "✅ downloaded ccache once for all matrix jobs"
555+
else
556+
echo "::warning::custom ccache download failed; matrix jobs will fall back to the apt ccache"
557+
rm -f ccache
558+
fi
559+
echo "::endgroup::"
560+
- name: Upload ccache artifact
561+
uses: actions/upload-artifact@v7
562+
with:
563+
name: ccache-binary
564+
path: ccache
565+
retention-days: 1
566+
if-no-files-found: warn
567+
531568
build:
532569
name: build (${{ matrix.model }}, ${{ matrix.soc }}, ${{ matrix.branch }}, ${{ matrix.manifest }}, ${{ matrix.android_version }}, ${{ matrix.kernel_version }}, ${{ matrix.os_version }}, ${{ matrix.ksu_type }})
533-
needs: [set-op-model, mirror_toolchain]
570+
needs: [set-op-model, mirror_toolchain, prepare_ccache]
534571
if: |
535572
!cancelled() &&
536573
needs.set-op-model.result == 'success' &&
@@ -565,17 +602,28 @@ jobs:
565602
echo "✅ Dependencies installed"
566603
echo "::endgroup::"
567604
568-
- name: Install ccache with ECS by cctv18
605+
- name: Get prebuilt ccache (downloaded once by prepare_ccache)
606+
uses: actions/download-artifact@v8
607+
with:
608+
name: ccache-binary
609+
path: ccache-dl
610+
continue-on-error: true
611+
612+
- name: Install ccache (ECS by cctv18, shared via artifact)
569613
run: |
570-
# Install ccache with ECS by cctv18
571-
set -euo pipefail
572-
echo "::group::Install ccache with ECS"
573-
curl -LfsS --retry 5 --retry-delay 5 --retry-all-errors --connect-timeout 30 --tcp-fastopen -H "User-Agent: Mozilla/5.0" "https://github.com/WildKernels/kernel_patches/raw/refs/heads/main/ccache/ccache-x86-64" -o ccache
574-
sudo cp -f ./ccache /usr/bin/ccache
575-
sudo chmod +x /usr/bin/ccache
576-
rm -f ./ccache
577-
578-
echo "[DEBUG] Ccache version : $(ccache --version)"
614+
set -uo pipefail
615+
echo "::group::Install ccache"
616+
if [ -s ccache-dl/ccache ]; then
617+
sudo cp -f ccache-dl/ccache /usr/bin/ccache
618+
sudo chmod +x /usr/bin/ccache
619+
echo "✅ installed custom ccache from shared artifact"
620+
else
621+
# prepare_ccache could not fetch it (504/throttle); keep the apt ccache
622+
# already installed in 'Install Minimal Dependencies'.
623+
echo "::warning::shared ccache artifact unavailable; using apt ccache"
624+
fi
625+
rm -rf ccache-dl
626+
echo "[DEBUG] Ccache version : $(ccache --version | head -1)"
579627
echo "::endgroup::"
580628
581629
- name: ♻️ Configure ccache & LTO cache (bounded)

0 commit comments

Comments
 (0)