Skip to content

Commit a187130

Browse files
authored
chore(ci): make sccache fail gracefully (#4208)
## Description Wraps the sccache config into a check so if infra fails for some reason it retries and if that fails too it moves on to complete the job without it. ## Breaking Changes <!-- Optional, if there are any breaking changes document them, including how to migrate older code. --> ## Notes & open questions <!-- Any notes, remarks or open questions you have to make about the PR. --> ## Change checklist <!-- Remove any that are not relevant. --> - [ ] Self-review. - [ ] Documentation updates following the [style guide](https://rust-lang.github.io/rfcs/1574-more-api-documentation-conventions.html#appendix-a-full-conventions-text), if relevant. - [ ] Tests if relevant. - [ ] All breaking changes documented. - [ ] List all breaking changes in the above "Breaking Changes" section. - [ ] Open an issue or PR on any number0 repos that are affected by this breaking change. Give guidance on how the updates should be handled or do the actual updates themselves. The major ones are: - [ ] [`quic-rpc`](https://github.com/n0-computer/quic-rpc) - [ ] [`iroh-gossip`](https://github.com/n0-computer/iroh-gossip) - [ ] [`iroh-blobs`](https://github.com/n0-computer/iroh-blobs) - [ ] [`dumbpipe`](https://github.com/n0-computer/dumbpipe) - [ ] [`sendme`](https://github.com/n0-computer/sendme)
1 parent 639754c commit a187130

6 files changed

Lines changed: 72 additions & 0 deletions

File tree

.github/workflows/ci.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -250,6 +250,8 @@ jobs:
250250
fetch-depth: 0
251251
- name: Install sccache
252252
uses: mozilla-actions/sccache-action@v0.0.9
253+
continue-on-error: true
254+
- uses: ./.github/workflows/sccache-probe
253255

254256
- name: Setup Environment (PR)
255257
if: ${{ github.event_name == 'pull_request' }}
@@ -282,6 +284,8 @@ jobs:
282284
with:
283285
components: rustfmt
284286
- uses: mozilla-actions/sccache-action@v0.0.9
287+
continue-on-error: true
288+
- uses: ./.github/workflows/sccache-probe
285289
- uses: taiki-e/install-action@cargo-make
286290
- run: cargo make format-check
287291

@@ -302,6 +306,8 @@ jobs:
302306
toolchain: nightly-2025-10-09
303307
- name: Install sccache
304308
uses: mozilla-actions/sccache-action@v0.0.9
309+
continue-on-error: true
310+
- uses: ./.github/workflows/sccache-probe
305311

306312
- name: Docs
307313
run: cargo doc --workspace --all-features --no-deps --document-private-items
@@ -321,6 +327,8 @@ jobs:
321327
components: clippy
322328
- name: Install sccache
323329
uses: mozilla-actions/sccache-action@v0.0.9
330+
continue-on-error: true
331+
- uses: ./.github/workflows/sccache-probe
324332

325333
# TODO: We have a bunch of platform-dependent code so should
326334
# probably run this job on the full platform matrix
@@ -371,6 +379,8 @@ jobs:
371379
toolchain: ${{ env.MSRV }}
372380
- name: Install sccache
373381
uses: mozilla-actions/sccache-action@v0.0.9
382+
continue-on-error: true
383+
- uses: ./.github/workflows/sccache-probe
374384

375385
- name: Check MSRV all features
376386
run: |

.github/workflows/docs.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,8 @@ jobs:
3535
toolchain: nightly-2025-10-09
3636
- name: Install sccache
3737
uses: mozilla-actions/sccache-action@v0.0.9
38+
continue-on-error: true
39+
- uses: ./.github/workflows/sccache-probe
3840

3941
- name: Generate Docs
4042
run: cargo doc --workspace --all-features --no-deps

.github/workflows/netsim_runner.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,8 @@ jobs:
114114

115115
- name: Install sccache
116116
uses: mozilla-actions/sccache-action@v0.0.9
117+
continue-on-error: true
118+
- uses: ./.github/workflows/sccache-probe
117119

118120
- name: Build iroh
119121
run: |

.github/workflows/patchbay.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@ jobs:
3131
- uses: actions/checkout@v6
3232
- uses: dtolnay/rust-toolchain@stable
3333
- uses: mozilla-actions/sccache-action@v0.0.9
34+
continue-on-error: true
35+
- uses: ./.github/workflows/sccache-probe
3436
- name: Install cargo-make and cargo-nextest
3537
uses: taiki-e/install-action@v2
3638
with:
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
name: Probe sccache backend
2+
description: >
3+
Verify the sccache backend is reachable; if not, unset RUSTC_WRAPPER so
4+
the job runs without sccache rather than failing. Handles transient
5+
outages of GHA cache (Blacksmith proxy 502, GitHub cache hiccups, etc.)
6+
without requiring a re-run.
7+
8+
runs:
9+
using: composite
10+
steps:
11+
- name: Probe sccache backend
12+
# Skip on Windows: our Windows runners are self-hosted hetz with
13+
# local-disk sccache (no GHA backend → no 502s to handle), and
14+
# `shell: bash` on those runners resolves to the WSL launcher
15+
# which fails outright. Probe only matters where SCCACHE_GHA_ENABLED
16+
# is in play, i.e. Linux self-hosted (Blacksmith) and ubuntu-latest.
17+
if: env.RUSTC_WRAPPER == 'sccache' && runner.os != 'Windows'
18+
shell: bash
19+
env:
20+
# Cap each --start-server attempt at 5s in case sccache hangs on
21+
# backend init. Empirically 502s from Blacksmith's GHA proxy
22+
# already fail in ~5s; this just bounds pathological hangs.
23+
# Two attempts × 5s = ~10s worst case. Portable across
24+
# Linux/macOS/Windows bash (no coreutils `timeout` needed).
25+
SCCACHE_STARTUP_NOTIFY_TIMEOUT: "5"
26+
run: |
27+
# If the sccache binary isn't installed (e.g. mozilla-actions/sccache-action
28+
# failed to download from GitHub releases), bail out fast.
29+
if ! command -v sccache >/dev/null 2>&1; then
30+
echo "::warning::sccache binary not found (install likely failed); running this job without sccache"
31+
echo "RUSTC_WRAPPER=" >> "$GITHUB_ENV"
32+
echo "SCCACHE_GHA_ENABLED=" >> "$GITHUB_ENV"
33+
exit 0
34+
fi
35+
# sccache --start-server triggers a backend probe (.sccache_check
36+
# read). On failure (e.g. Blacksmith's GHA cache proxy returning
37+
# 502 Bad Gateway), the daemon won't start and every rustc call
38+
# fails. One quick retry, then fall through to plain rustc —
39+
# slower, but the job will still succeed.
40+
for attempt in 1 2; do
41+
if sccache --start-server >/dev/null 2>&1; then
42+
echo "sccache backend OK (attempt $attempt)"
43+
exit 0
44+
fi
45+
echo "sccache start failed (attempt $attempt/2)"
46+
done
47+
echo "::warning::sccache backend unreachable; running this job without sccache"
48+
# mozilla-actions/sccache-action exports these via $GITHUB_ENV — override.
49+
echo "RUSTC_WRAPPER=" >> "$GITHUB_ENV"
50+
echo "SCCACHE_GHA_ENABLED=" >> "$GITHUB_ENV"

.github/workflows/tests.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,8 @@ jobs:
117117

118118
- name: Install sccache
119119
uses: mozilla-actions/sccache-action@v0.0.9
120+
continue-on-error: true
121+
- uses: ./.github/workflows/sccache-probe
120122

121123
- name: Select features
122124
run: |
@@ -262,6 +264,8 @@ jobs:
262264
263265
- name: Install sccache
264266
uses: mozilla-actions/sccache-action@v0.0.9
267+
continue-on-error: true
268+
- uses: ./.github/workflows/sccache-probe
265269

266270
- uses: msys2/setup-msys2@v2
267271
with:
@@ -362,6 +366,8 @@ jobs:
362366
363367
- name: Install sccache
364368
uses: mozilla-actions/sccache-action@v0.0.9
369+
continue-on-error: true
370+
- uses: ./.github/workflows/sccache-probe
365371

366372
- uses: msys2/setup-msys2@v2
367373
with:

0 commit comments

Comments
 (0)