Skip to content

Commit 019c56c

Browse files
joshuacolvin0claude
joshuacolvin0
and
claude
committed
Overhaul CI setup caching and harden build reproducibility
Rework the ci-setup composite action to add proper caching for node_modules, solidity build artifacts, Go build/modules, and cbrotli, with validation and corruption detection on cache restore. Key changes: - Add dedicated cache steps for node_modules, solidity, Go, and cbrotli with carefully designed cache keys that include tool versions and all relevant source/config files via hashFiles() - Add self-validating cache key inputs: extract hashFiles() patterns from the action YAML and verify each pattern matches at least one file - Add cache-hit validation that checks cached directories exist and are non-empty before touching Make sentinels to skip rebuilds - Handle solidity/node_modules cache split-brain: purge solidity artifacts when node_modules cache is evicted to ensure consistent rebuilds - Add RETRY macro for npm/yarn installs to handle transient registry errors, with optional node_modules cleanup between attempts - Pin cbindgen version and skip reinstall when cached version matches - Use frozen lockfiles (npm ci / yarn --frozen-lockfile) in CI - Centralize node_modules and solidity directory lists as single source of truth consumed by both cache steps and validation - Add lint step that warns about cache-sensitive files not covered by any hashFiles() pattern - Harden install-rust action with set -euo pipefail and quoted outputs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent e89bc08 commit 019c56c

4 files changed

Lines changed: 360 additions & 17 deletions

File tree

‎.github/actions/ci-setup/action.yml‎

Lines changed: 333 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -7,25 +7,94 @@ runs:
77

88
steps:
99
- name: Install dependencies
10-
run: sudo apt update && sudo apt install -y wabt gotestsum
10+
run: |
11+
set -euo pipefail
12+
sudo apt-get update && sudo apt-get install -y wabt gotestsum
13+
shell: bash
14+
15+
- name: Set Node.js version
16+
id: node-version
17+
run: echo "major=24" >> "$GITHUB_OUTPUT"
18+
shell: bash
19+
20+
- name: Set Foundry version
21+
id: foundry-version
22+
run: echo "version=v1.0.0" >> "$GITHUB_OUTPUT"
23+
shell: bash
24+
25+
# Single source of truth for node_modules directories. Consumed by
26+
# cache-node-modules and the validation step. Add new workspaces ONLY here.
27+
- name: Set node_modules directory list
28+
run: |
29+
set -euo pipefail
30+
dirs=(
31+
contracts/node_modules
32+
contracts-legacy/node_modules
33+
safe-smart-account/node_modules
34+
)
35+
{
36+
echo "NODE_MODULES_CACHE_PATHS<<NM_EOF"
37+
printf '%s\n' "${dirs[@]}"
38+
echo "NM_EOF"
39+
} >> "$GITHUB_ENV"
40+
echo "NODE_MODULES_DIRS=${dirs[*]}" >> "$GITHUB_ENV"
41+
shell: bash
42+
43+
# Single source of truth for solidity cache directories. Consumed by
44+
# cache-solidity (SOLIDITY_CACHE_PATHS) and the validate/purge step
45+
# (SOLIDITY_DIRS). Add new solidity cache directories ONLY here.
46+
- name: Set solidity cache directory list
47+
run: |
48+
set -euo pipefail
49+
# Space-separated; used by SOLIDITY_DIRS (word splitting) and
50+
# SOLIDITY_CACHE_PATHS (newline-separated for actions/cache).
51+
dirs=(
52+
contracts/build
53+
contracts/out
54+
contracts-legacy/build
55+
contracts-legacy/out
56+
contracts-local/out
57+
safe-smart-account/build
58+
solgen/go
59+
)
60+
{
61+
echo "SOLIDITY_CACHE_PATHS<<SOLIDITY_EOF"
62+
printf '%s\n' "${dirs[@]}"
63+
echo "SOLIDITY_EOF"
64+
} >> "$GITHUB_ENV"
65+
echo "SOLIDITY_DIRS=${dirs[*]}" >> "$GITHUB_ENV"
1166
shell: bash
1267

68+
# setup-node only caches yarn's global tarball cache, not node_modules.
69+
# No restore-keys: for safe-smart-account, npm ci wipes and reinstalls
70+
# node_modules (wasting any partial restore); for yarn workspaces,
71+
# additive installs over a stale restore could leave orphaned packages.
72+
- name: Cache node_modules
73+
id: cache-node-modules
74+
uses: actions/cache@v5
75+
with:
76+
path: ${{ env.NODE_MODULES_CACHE_PATHS }}
77+
key: ${{ runner.os }}-node-modules-node${{ steps.node-version.outputs.major }}-${{ hashFiles('contracts/yarn.lock', 'contracts-legacy/yarn.lock', 'safe-smart-account/package-lock.json') }}
78+
1379
- name: Setup Node.js
1480
uses: actions/setup-node@v5
1581
with:
16-
node-version: "24"
82+
node-version: ${{ steps.node-version.outputs.major }}
1783
cache: yarn
1884
cache-dependency-path: "**/yarn.lock"
1985

2086
- name: Setup Go
87+
id: setup-go
2188
uses: actions/setup-go@v6
2289
with:
2390
go-version-file: "go.mod"
91+
cache: false # See "Cache Go build and modules" step for restore-keys fallback
2492

2593
- name: Install wasm-ld
2694
run: |
27-
sudo apt-get update && sudo apt-get install -y lld-14
28-
sudo ln -s /usr/bin/wasm-ld-14 /usr/local/bin/wasm-ld
95+
set -euo pipefail
96+
sudo apt-get install -y lld-14
97+
sudo ln -sf /usr/bin/wasm-ld-14 /usr/local/bin/wasm-ld
2998
shell: bash
3099

31100
- name: Install rust
@@ -35,20 +104,210 @@ runs:
35104
- name: Setup Foundry
36105
uses: foundry-rs/foundry-toolchain@v1
37106
with:
38-
cache: false
39-
version: v1.0.0
107+
cache: true
108+
version: ${{ steps.foundry-version.outputs.version }}
40109

41-
- name: Install cbindgen
42-
run: cargo install --force cbindgen
110+
- name: Validate cache key inputs
111+
env:
112+
GO_VERSION: ${{ steps.setup-go.outputs.go-version }}
113+
RUST_VERSION: ${{ steps.install-rust.outputs.version }}
114+
FOUNDRY_VERSION: ${{ steps.foundry-version.outputs.version }}
115+
NODE_VERSION: ${{ steps.node-version.outputs.major }}
116+
run: |
117+
set -euo pipefail
118+
# Version strings used in cache keys must not be empty
119+
for pair in "Go:$GO_VERSION" "Rust:$RUST_VERSION" "Foundry:$FOUNDRY_VERSION" "Node:$NODE_VERSION"; do
120+
name="${pair%%:*}"; val="${pair#*:}"
121+
if [ -z "$val" ]; then
122+
echo "ERROR: $name version is empty -- cache keys will collide across versions"; exit 1
123+
fi
124+
if ! [[ "$val" =~ ^[a-zA-Z0-9._v-]+$ ]]; then
125+
echo "ERROR: $name version looks malformed: '$val' -- cache keys may not work correctly"; exit 1
126+
fi
127+
done
128+
# Auto-extract hashFiles() arguments from this file so the validation
129+
# stays in sync without maintaining a separate list.
130+
shopt -s globstar nullglob
131+
self="${GITHUB_ACTION_PATH}/action.yml"
132+
args=$(grep -oP "hashFiles\(\K[^)]+(?=\))" "$self" | tr "," "\n" | sed "s/^[[:space:]]*'//;s/'[[:space:]]*$//" | sort -u)
133+
# Fail-closed: if the regex stops matching (e.g., multi-line reformat),
134+
# surface it immediately rather than silently skipping validation.
135+
if [ -z "$args" ]; then
136+
echo "ERROR: no hashFiles() patterns extracted from $self -- grep regex may need updating"; exit 1
137+
fi
138+
# The line-by-line regex silently misses multi-line hashFiles() calls.
139+
# Guard against this by comparing call count vs extracted group count.
140+
# Only count 'key:' lines (where GHA hashFiles expressions live) to
141+
# exclude shell scripts and comments that mention hashFiles().
142+
expected=$(grep -cP '^\s*key:.*hashFiles\(' "$self")
143+
actual=$(grep -oP "hashFiles\(\K[^)]+(?=\))" "$self" | wc -l)
144+
if [ "$actual" -ne "$expected" ]; then
145+
echo "ERROR: found $expected hashFiles() calls but only extracted $actual -- possible multi-line hashFiles() call"; exit 1
146+
fi
147+
# Export for the lint step so the regex is not duplicated.
148+
{
149+
echo "HASHFILES_PATTERNS<<HASHFILES_EOF"
150+
echo "$args"
151+
echo "HASHFILES_EOF"
152+
} >> "$GITHUB_ENV"
153+
# Disable globbing during iteration so nullglob does not silently
154+
# remove unmatched patterns from the list; re-enable inside the
155+
# body for intentional expansion.
156+
set -f
157+
for pattern in $args; do
158+
set +f
159+
# shellcheck disable=SC2206 # intentional glob expansion (globstar enabled above)
160+
matches=( $pattern )
161+
set -f
162+
if [ ${#matches[@]} -eq 0 ]; then
163+
echo "ERROR: hashFiles pattern '$pattern' matches nothing -- update .github/actions/ci-setup/action.yml"; exit 1
164+
fi
165+
done
166+
set +f
43167
shell: bash
44168

45-
- name: Cache Go build
169+
# Best-effort lint: warn about cache-sensitive files not covered by
170+
# any hashFiles() pattern. Produces ::warning annotations only --
171+
# never fails the build. Separate from validation because this is
172+
# advisory, not a correctness gate.
173+
- name: Lint cache key coverage
174+
run: |
175+
set -euo pipefail
176+
shopt -s globstar nullglob
177+
# Reuse patterns extracted by the validate step (avoid duplicating the regex).
178+
args="${HASHFILES_PATTERNS:-}"
179+
if [ -z "$args" ]; then
180+
echo "::warning::HASHFILES_PATTERNS is empty -- validate step may have failed or been skipped"; exit 0
181+
fi
182+
# shellcheck disable=SC2317 # function used below
183+
file_covered_by() {
184+
local file="$1"; shift
185+
for hp in "$@"; do
186+
set +f
187+
# shellcheck disable=SC2206 # intentional glob expansion (globstar enabled)
188+
local expanded=( $hp )
189+
set -f
190+
local m
191+
for m in "${expanded[@]}"; do
192+
if [ "$m" = "$file" ]; then return 0; fi
193+
done
194+
done
195+
return 1
196+
}
197+
skip_re='^(.*node_modules.*|.*/lib/.*|.*/build/.*|.*/out/.*|.*/dist/.*|nitro-testnode/.*|go-ethereum/.*|crates/.*|.*typechain.*|.*/certora/.*)$'
198+
# Pre-expand args into an array with globbing off so patterns are
199+
# preserved as literals for file_covered_by to expand on demand.
200+
set -f
201+
# shellcheck disable=SC2206 # intentional word splitting
202+
args_arr=( $args )
203+
set +f
204+
# Check lockfiles, configs, and build files.
205+
for file in **/yarn.lock **/package-lock.json **/foundry.toml **/hardhat.config.ts; do
206+
[[ "$file" =~ $skip_re || "$file" == */src/* ]] && continue
207+
file_covered_by "$file" "${args_arr[@]}" ||
208+
echo "::warning::$file is not in any hashFiles() cache key in .github/actions/ci-setup/action.yml -- may need adding"
209+
done
210+
# Check for .sol source files not covered by any hashFiles() glob.
211+
sol_patterns=()
212+
for p in "${args_arr[@]}"; do
213+
[[ "$p" == *.sol ]] && sol_patterns+=("$p")
214+
done
215+
if [ ${#sol_patterns[@]} -gt 0 ]; then
216+
uncovered_sol=()
217+
for file in **/*.sol; do
218+
[[ "$file" =~ $skip_re || "$file" == */test/* ]] && continue
219+
file_covered_by "$file" "${sol_patterns[@]}" || uncovered_sol+=("$file")
220+
done
221+
if [ ${#uncovered_sol[@]} -gt 0 ]; then
222+
echo "::warning::${#uncovered_sol[@]} .sol file(s) not covered by any hashFiles() sol glob -- may need adding: ${uncovered_sol[*]}"
223+
fi
224+
fi
225+
shell: bash
226+
227+
# restore-keys prefix fallback: Go build cache is content-addressed
228+
# (stale entries ignored), module cache self-heals by re-downloading.
229+
- name: Cache Go build and modules
230+
id: cache-go
231+
uses: actions/cache@v5
232+
with:
233+
path: |
234+
~/.cache/go-build
235+
~/go/pkg/mod
236+
key: ${{ runner.os }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.mod', 'go.sum') }}
237+
restore-keys: |
238+
${{ runner.os }}-go-${{ steps.setup-go.outputs.go-version }}-
239+
240+
# No restore-keys: partial Solidity restores (stale Go bindings with
241+
# new ABIs) would produce inconsistent artifacts.
242+
- name: Cache contract build artifacts
243+
id: cache-solidity
46244
uses: actions/cache@v5
47245
with:
48-
path: ~/.cache/go-build
49-
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
246+
path: ${{ env.SOLIDITY_CACHE_PATHS }}
247+
key: ${{ runner.os }}-solidity-go${{ steps.setup-go.outputs.go-version }}-node${{ steps.node-version.outputs.major }}-foundry-${{ steps.foundry-version.outputs.version }}-${{ hashFiles('contracts/src/**/*.sol', 'contracts-legacy/src/**/*.sol', 'contracts-local/src/**/*.sol', 'contracts/foundry.toml', 'contracts-legacy/foundry.toml', 'contracts-local/foundry.toml', 'contracts/hardhat.config.ts', 'contracts-legacy/hardhat.config.ts', 'safe-smart-account/hardhat.config.ts', 'contracts-local/Makefile', 'safe-smart-account/contracts/**/*.sol', 'safe-smart-account/package.json', 'contracts/yarn.lock', 'contracts-legacy/yarn.lock', 'safe-smart-account/package-lock.json', 'solgen/gen.go', 'go-ethereum/accounts/abi/abigen/**/*.go', 'go.mod', 'go.sum') }}
248+
249+
# On solidity cache hit: validate artifacts, then either touch Make
250+
# sentinels (if node_modules also hit) or purge and rebuild (if not).
251+
- name: Validate and mark contract builds as up-to-date on cache hit
252+
if: steps.cache-solidity.outputs.cache-hit == 'true'
253+
env:
254+
NODE_MODULES_HIT: ${{ steps.cache-node-modules.outputs.cache-hit }}
255+
run: |
256+
set -euo pipefail
257+
258+
set -f
259+
for dir in $SOLIDITY_DIRS; do
260+
if [ ! -d "$dir" ]; then
261+
echo "ERROR: cached directory missing: $dir"; exit 1
262+
fi
263+
if [ -z "$(ls -A "$dir")" ]; then
264+
echo "ERROR: cached directory empty: $dir"; exit 1
265+
fi
266+
done
267+
set +f
268+
# Spot-check a known solgen output file to catch partial cache restores
269+
# (e.g., interrupted extraction) that pass the non-empty dir check above.
270+
if [ ! -f "solgen/go/bridgegen/bridgegen.go" ]; then
271+
echo "ERROR: solgen/go output looks incomplete despite cache hit -- possible partial restore"; exit 1
272+
fi
50273
274+
# Touch Make sentinels only when both solidity AND node_modules hit.
275+
mkdir -p .make
276+
if [ "$NODE_MODULES_HIT" = "true" ]; then
277+
if [ -z "${NODE_MODULES_DIRS:-}" ]; then
278+
echo "ERROR: NODE_MODULES_DIRS is empty or unset"; exit 1
279+
fi
280+
set -f
281+
for dir in $NODE_MODULES_DIRS; do
282+
test -d "$dir" || { echo "ERROR: $dir missing despite node_modules cache hit"; exit 1; }
283+
if [ -z "$(ls -A "$dir")" ]; then
284+
echo "ERROR: $dir is empty despite node_modules cache hit -- cache may be corrupted"; exit 1
285+
fi
286+
done
287+
set +f
288+
touch .make/yarndeps .make/solidity .make/solgen
289+
echo "INFO: All sentinels touched; Make will skip solidity/solgen/yarndeps targets"
290+
else
291+
# Solidity cache hit but node_modules missed -- purge cached
292+
# solidity artifacts so Make rebuilds with fresh node_modules.
293+
# One-time cost: node_modules saves on this run, so next run
294+
# both caches hit.
295+
echo "::warning::node_modules cache missed (likely evicted) -- purging cached solidity artifacts to rebuild (one-time cost)"
296+
if [ -z "${SOLIDITY_DIRS:-}" ]; then
297+
echo "ERROR: SOLIDITY_DIRS is empty or unset"; exit 1
298+
fi
299+
set -f # disable globbing for safe word splitting on $SOLIDITY_DIRS
300+
for dir in $SOLIDITY_DIRS; do
301+
rm -rf "$dir"
302+
done
303+
set +f
304+
fi
305+
shell: bash
306+
307+
# ORDERING: Must appear before "Cache cbrotli" -- the Rust cache
308+
# includes all of target/ which overlaps cbrotli paths.
51309
- name: Cache Rust build
310+
id: cache-rust
52311
uses: actions/cache@v5
53312
with:
54313
path: |
@@ -59,7 +318,23 @@ runs:
59318
target/etc/initial-machine-cache/
60319
/home/runner/.rustup/toolchains/
61320
key: ${{ runner.os }}-cargo-${{ steps.install-rust.outputs.version }}-${{ hashFiles('Cargo.lock') }}
321+
restore-keys: |
322+
${{ runner.os }}-cargo-${{ steps.install-rust.outputs.version }}-
323+
324+
- name: Install cbindgen
325+
run: |
326+
set -euo pipefail
327+
CBINDGEN_VERSION="0.24.3"
328+
actual=$(cbindgen --version 2>/dev/null) || true
329+
if [ "$actual" != "cbindgen $CBINDGEN_VERSION" ]; then
330+
echo "Installing cbindgen $CBINDGEN_VERSION (found: '${actual:-not installed}')"
331+
cargo install cbindgen --version "$CBINDGEN_VERSION" --force
332+
else
333+
echo "cbindgen $CBINDGEN_VERSION already installed"
334+
fi
335+
shell: bash
62336

337+
# ORDERING: Must appear after "Cache Rust build" -- see comment there.
63338
- name: Cache cbrotli
64339
id: cache-cbrotli
65340
uses: actions/cache@v5
@@ -70,7 +345,17 @@ runs:
70345
target/lib/libbrotlicommon-static.a
71346
target/lib/libbrotlienc-static.a
72347
target/lib/libbrotlidec-static.a
73-
key: ${{ runner.os }}-brotli-${{ hashFiles('scripts/build-brotli.sh') }}
348+
key: ${{ runner.os }}-brotli-rust${{ steps.install-rust.outputs.version }}-${{ hashFiles('scripts/build-brotli.sh') }}
349+
350+
# A partial Rust restore may place stale cbrotli artifacts in target/.
351+
# Remove them before rebuilding.
352+
- name: Clean stale cbrotli paths on cache miss
353+
if: steps.cache-cbrotli.outputs.cache-hit != 'true'
354+
run: |
355+
set -euo pipefail
356+
rm -rf target/include/brotli target/lib-wasm
357+
rm -f target/lib/libbrotli{common,enc,dec}-static.a
358+
shell: bash
74359

75360
- name: Build cbrotli-local
76361
if: steps.cache-cbrotli.outputs.cache-hit != 'true'
@@ -81,3 +366,39 @@ runs:
81366
if: steps.cache-cbrotli.outputs.cache-hit != 'true'
82367
run: ./scripts/build-brotli.sh -w -d
83368
shell: bash
369+
370+
# Runs after both cache-restore and build paths. Do not split into
371+
# separate "validate on hit" / "validate after build" steps -- identical
372+
# validation applies regardless of source, and splitting invites drift.
373+
- name: Validate cbrotli artifacts
374+
run: |
375+
set -euo pipefail
376+
for f in target/lib/libbrotli{common,enc,dec}-static.a \
377+
target/lib-wasm/libbrotli{common,enc,dec}-static.a; do
378+
[ -f "$f" ] && [ -s "$f" ] || { echo "ERROR: $f missing or empty"; exit 1; }
379+
done
380+
for f in target/include/brotli/encode.h target/include/brotli/decode.h; do
381+
test -f "$f" || { echo "ERROR: $f missing"; exit 1; }
382+
done
383+
shell: bash
384+
385+
# cache-hit is 'true' only on exact key match. For Go/Rust
386+
# (restore-keys enabled), 'false' could mean a partial restore
387+
# or a full miss.
388+
- name: Cache status summary
389+
env:
390+
NODE_MODULES_HIT: ${{ steps.cache-node-modules.outputs.cache-hit }}
391+
GO_HIT: ${{ steps.cache-go.outputs.cache-hit }}
392+
SOLIDITY_HIT: ${{ steps.cache-solidity.outputs.cache-hit }}
393+
RUST_HIT: ${{ steps.cache-rust.outputs.cache-hit }}
394+
CBROTLI_HIT: ${{ steps.cache-cbrotli.outputs.cache-hit }}
395+
run: |
396+
set -euo pipefail
397+
echo "=== Cache Status ==="
398+
printf " %-14s %s\n" \
399+
"node_modules" "${NODE_MODULES_HIT:-MISS}" \
400+
"go" "${GO_HIT:-MISS}" \
401+
"solidity" "${SOLIDITY_HIT:-MISS}" \
402+
"rust" "${RUST_HIT:-MISS}" \
403+
"cbrotli" "${CBROTLI_HIT:-MISS}"
404+
shell: bash

0 commit comments

Comments
 (0)