Skip to content

Commit a7fba4f

Browse files
avrabeavrabe
committed
ci: migrate 11 of 13 ci.yml jobs to smithy + add mutants-weekly
Builds on the proven clippy migration (PR description, original commit on this branch). Two separate concerns: 1) ci.yml — broaden the migration Migrate every gating job that doesn't need infra we don't have on the smithy host. Two stay on ubuntu-latest with explicit comments explaining why; everything else now targets the matching smithy runner class: rust-cpu (12G MemoryHigh) clippy, test, bench-smoke, coverage, proptest, fuzz-smoke, rivet-validate lean-mem (24G MemoryHigh) miri, mutants light (4G MemoryHigh) fmt, audit, deny, supply-chain ubuntu-latest (kept) bazel-test (no Bazel on host), kani (kani-verifier bundles CBMC, ~100 MB install — not worth pre- provisioning until kani sees more use) The lean-mem class for miri / mutants is deliberate: both are RAM-aggressive (Miri's borrow tracker, mutants' parallel cargo invocations). The 24G MemoryHigh ceiling on smithy lean-mem runners is comfortably above the 12G rust-cpu cap. 2) mutants-weekly.yml — new heavy-quality workflow Counterpart to the gating `mutants:` job in ci.yml. Different operational pattern (smithy DD-pattern for "heavy quality"): - schedule: 02:00 UTC every Sunday + workflow_dispatch on demand - runs-on: lean-mem (24G), timeout-minutes: 720 - concurrency.cancel-in-progress: false (never cancel a quality run) - workflow_dispatch inputs: `shard` (default 0/8 for sanity, "all" for the full ~hours pass) + `packages` (space-separated -p list) - results land in GITHUB_STEP_SUMMARY (markdown table of missed/caught/timeout/unviable) plus an uploaded artefact with 90-day retention - no PR red lights; no auto-Issue filing yet (that's a follow-up once the report shape stabilises) This is the second-pattern pilot the smithy fleet was sized for — the lean-mem runners have been idle since registration; this puts them on the work they were labelled for.
1 parent fce03ff commit a7fba4f

2 files changed

Lines changed: 159 additions & 15 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 24 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ jobs:
2121
# ── Fast checks ───────────────────────────────────────────────────────
2222
fmt:
2323
name: Format
24-
runs-on: ubuntu-latest
24+
runs-on: [self-hosted, linux, x64, light]
2525
steps:
2626
- uses: actions/checkout@v4
2727
- uses: dtolnay/rust-toolchain@nightly
@@ -31,9 +31,6 @@ jobs:
3131

3232
clippy:
3333
name: Clippy
34-
# smithy: pilot-migrating this single job to the pulseengine self-hosted
35-
# fleet (hetzner-private runner group, rust-cpu class). Other jobs stay
36-
# on ubuntu-latest until this one is proven over a few runs.
3734
runs-on: [self-hosted, linux, x64, rust-cpu]
3835
steps:
3936
- uses: actions/checkout@v4
@@ -46,7 +43,7 @@ jobs:
4643
# ── Tests ─────────────────────────────────────────────────────────────
4744
test:
4845
name: Test
49-
runs-on: ubuntu-latest
46+
runs-on: [self-hosted, linux, x64, rust-cpu]
5047
steps:
5148
- uses: actions/checkout@v4
5249
- uses: dtolnay/rust-toolchain@nightly
@@ -68,7 +65,7 @@ jobs:
6865
# ── Bench compile smoke (fast regression gate) ──────────────────────
6966
bench-smoke:
7067
name: Bench compile smoke
71-
runs-on: ubuntu-latest
68+
runs-on: [self-hosted, linux, x64, rust-cpu]
7269
steps:
7370
- uses: actions/checkout@v4
7471
- uses: dtolnay/rust-toolchain@nightly
@@ -82,7 +79,7 @@ jobs:
8279
# ── Security audits ──────────────────────────────────────────────────
8380
audit:
8481
name: Security Audit (RustSec)
85-
runs-on: ubuntu-latest
82+
runs-on: [self-hosted, linux, x64, light]
8683
steps:
8784
- uses: actions/checkout@v4
8885
- uses: rustsec/audit-check@v2
@@ -91,7 +88,7 @@ jobs:
9188

9289
deny:
9390
name: Cargo Deny
94-
runs-on: ubuntu-latest
91+
runs-on: [self-hosted, linux, x64, light]
9592
steps:
9693
- uses: actions/checkout@v4
9794
- uses: EmbarkStudios/cargo-deny-action@v2
@@ -100,7 +97,7 @@ jobs:
10097
coverage:
10198
name: Code Coverage
10299
needs: [test]
103-
runs-on: ubuntu-latest
100+
runs-on: [self-hosted, linux, x64, rust-cpu]
104101
steps:
105102
- uses: actions/checkout@v4
106103
- uses: dtolnay/rust-toolchain@nightly
@@ -132,7 +129,9 @@ jobs:
132129
# ── Miri (undefined behavior, pointer provenance) ───────────────────
133130
miri:
134131
name: Miri
135-
runs-on: ubuntu-latest
132+
# lean-mem class — Miri allocates aggressively and benefits from the 24G
133+
# MemoryHigh ceiling on smithy lean-mem runners over the 12G rust-cpu cap.
134+
runs-on: [self-hosted, linux, x64, lean-mem]
136135
steps:
137136
- uses: actions/checkout@v4
138137
- uses: dtolnay/rust-toolchain@nightly
@@ -155,7 +154,7 @@ jobs:
155154
# parser/scheduler invariants get exercised on every change.
156155
proptest:
157156
name: Proptest (extended)
158-
runs-on: ubuntu-latest
157+
runs-on: [self-hosted, linux, x64, rust-cpu]
159158
steps:
160159
- uses: actions/checkout@v4
161160
- uses: dtolnay/rust-toolchain@nightly
@@ -169,7 +168,10 @@ jobs:
169168
mutants:
170169
name: Mutation Testing
171170
needs: [test]
172-
runs-on: ubuntu-latest
171+
# lean-mem — many parallel cargo invocations, RAM pressure under -j 4.
172+
# The full-workspace exhaustive run lives in mutants-weekly.yml; this
173+
# gating job stays narrow (spar-analysis) with a survivor ratchet.
174+
runs-on: [self-hosted, linux, x64, lean-mem]
173175
steps:
174176
- uses: actions/checkout@v4
175177
- uses: dtolnay/rust-toolchain@nightly
@@ -206,7 +208,7 @@ jobs:
206208
# ── Fuzz smoke (60s per target on PRs) ──────────────────────────────
207209
fuzz-smoke:
208210
name: Fuzz smoke (60s/target)
209-
runs-on: ubuntu-latest
211+
runs-on: [self-hosted, linux, x64, rust-cpu]
210212
# Only run on PRs — pushes to main hit the nightly workflow instead.
211213
if: github.event_name == 'pull_request'
212214
steps:
@@ -232,7 +234,7 @@ jobs:
232234
# ── Supply chain verification ───────────────────────────────────────
233235
supply-chain:
234236
name: Supply Chain (cargo-vet)
235-
runs-on: ubuntu-latest
237+
runs-on: [self-hosted, linux, x64, light]
236238
steps:
237239
- uses: actions/checkout@v4
238240
- uses: dtolnay/rust-toolchain@stable
@@ -249,7 +251,7 @@ jobs:
249251
# in artifacts/, safety/stpa/, and rivet.yaml.
250252
rivet-validate:
251253
name: Rivet validate (artifacts)
252-
runs-on: ubuntu-latest
254+
runs-on: [self-hosted, linux, x64, rust-cpu]
253255
steps:
254256
- uses: actions/checkout@v4
255257
- uses: dtolnay/rust-toolchain@stable
@@ -275,6 +277,9 @@ jobs:
275277
# Time budget: cold cache ≤30 min, warm ≤5 min (per #135).
276278
bazel-test:
277279
name: Bazel test (//...)
280+
# Stays on ubuntu-latest until Bazel is installed on the smithy host.
281+
# Tracked as a follow-up: smithy/group_vars/all.yml could add a
282+
# bazel apt-installable. Until then, hosted handles this.
278283
runs-on: ubuntu-latest
279284
continue-on-error: true
280285
timeout-minutes: 35
@@ -321,6 +326,10 @@ jobs:
321326
# 3. At that point, extend MAX_TASKS from 4 to 8 and re-tune unwinds.
322327
kani:
323328
name: Kani Bounded Model Checking
329+
# Stays on ubuntu-latest because kani-verifier bundles CBMC (~100 MB)
330+
# which we don't pre-install on smithy. Once smithy ships Kani as a
331+
# toolchain, switch to rust-cpu (the verification is RAM-modest but
332+
# CPU-bound; CBMC is single-threaded per harness).
324333
runs-on: ubuntu-latest
325334
continue-on-error: true
326335
steps:

‎.github/workflows/mutants-weekly.yml‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
name: Mutants Weekly
2+
3+
# Heavy-quality counterpart to the gating `mutants:` job in ci.yml.
4+
# That one runs on every PR with a survivor-count ratchet against
5+
# spar-analysis only. THIS one runs across the whole workspace on a
6+
# weekly cadence (and on demand) — no gating, just a long-form
7+
# quality signal you read async.
8+
#
9+
# Resource posture (DD: see smithy/artifacts/design-decisions.yaml):
10+
# - lean-mem runners (24 G MemoryHigh, 24 G usable per job)
11+
# - 12 h timeout cap
12+
# - never cancel an in-flight run; let it finish even when overlapping refs land
13+
# - results land in the run's GITHUB_STEP_SUMMARY + an uploaded artefact
14+
# (90-day retention) — no PR red lights, no Issue auto-filing yet
15+
# (that's a future iteration once the report shape stabilises)
16+
17+
on:
18+
schedule:
19+
- cron: "0 2 * * 0" # 02:00 UTC every Sunday
20+
workflow_dispatch:
21+
inputs:
22+
shard:
23+
description: "Mutant shard, e.g. '0/8' (default), '1/8', or 'all' for the full workspace pass (~hours)."
24+
required: false
25+
default: "0/8"
26+
packages:
27+
description: "Cargo packages to mutate (space-separated -p list, empty = whole workspace)."
28+
required: false
29+
default: ""
30+
31+
concurrency:
32+
group: mutants-weekly
33+
# Quality jobs don't cancel; an interrupted mutation report is worse
34+
# than a delayed one. Two overlapping runs share the lean-mem pool;
35+
# cgroup limits keep each within 24 G.
36+
cancel-in-progress: false
37+
38+
jobs:
39+
mutants:
40+
name: cargo-mutants ${{ github.event.inputs.shard || 'shard 0/8' }}
41+
runs-on: [self-hosted, linux, x64, lean-mem]
42+
timeout-minutes: 720
43+
44+
steps:
45+
- uses: actions/checkout@v4
46+
47+
- uses: dtolnay/rust-toolchain@nightly
48+
49+
- uses: Swatinem/rust-cache@v2
50+
with:
51+
# Distinct cache key from the gating mutants — different mutation
52+
# set, different sccache hit profile.
53+
shared-key: mutants-weekly
54+
55+
- name: Install cargo-mutants
56+
uses: taiki-e/install-action@v2
57+
with:
58+
tool: cargo-mutants
59+
60+
- name: Resolve inputs
61+
id: cfg
62+
run: |
63+
SHARD="${{ github.event.inputs.shard || '0/8' }}"
64+
PACKAGES="${{ github.event.inputs.packages }}"
65+
PKGS_FLAG=""
66+
if [ -n "$PACKAGES" ]; then
67+
for p in $PACKAGES; do
68+
PKGS_FLAG="$PKGS_FLAG -p $p"
69+
done
70+
fi
71+
SHARD_FLAG=""
72+
if [ "$SHARD" != "all" ]; then
73+
SHARD_FLAG="--shard $SHARD"
74+
fi
75+
echo "shard=$SHARD" >> $GITHUB_OUTPUT
76+
echo "shard_flag=$SHARD_FLAG" >> $GITHUB_OUTPUT
77+
echo "pkgs_flag=$PKGS_FLAG" >> $GITHUB_OUTPUT
78+
echo "Effective: cargo mutants ${PKGS_FLAG:-(workspace)} $SHARD_FLAG --timeout 180 --jobs 8 --output mutants-out -- --lib"
79+
80+
- name: Run cargo-mutants
81+
id: run
82+
# `|| true` so the report still uploads even when survivors exist;
83+
# the next step decides exit status from the missed.txt content.
84+
run: |
85+
set -o pipefail
86+
cargo mutants \
87+
${{ steps.cfg.outputs.pkgs_flag }} \
88+
${{ steps.cfg.outputs.shard_flag }} \
89+
--timeout 180 \
90+
--jobs 8 \
91+
--output mutants-out \
92+
--no-shuffle \
93+
-- --lib \
94+
|| echo "cargo-mutants exited non-zero (survivors expected; see report)"
95+
96+
- name: Summarise to job summary
97+
if: always()
98+
run: |
99+
MISSED=0
100+
[ -f mutants-out/missed.txt ] && MISSED=$(wc -l < mutants-out/missed.txt | tr -d ' ')
101+
CAUGHT=0
102+
[ -f mutants-out/caught.txt ] && CAUGHT=$(wc -l < mutants-out/caught.txt | tr -d ' ')
103+
UNVIABLE=0
104+
[ -f mutants-out/unviable.txt ] && UNVIABLE=$(wc -l < mutants-out/unviable.txt | tr -d ' ')
105+
TIMEOUT=0
106+
[ -f mutants-out/timeout.txt ] && TIMEOUT=$(wc -l < mutants-out/timeout.txt | tr -d ' ')
107+
{
108+
echo "## cargo-mutants weekly — ${{ steps.cfg.outputs.shard }}"
109+
echo
110+
echo "Runner: \`$(hostname)\` (${SMITHY_RUNNER_CLASS:-unknown class})"
111+
echo
112+
echo "| Outcome | Count |"
113+
echo "|---------|------:|"
114+
echo "| 🟥 Missed (test suite did not catch) | $MISSED |"
115+
echo "| 🟩 Caught (test suite caught) | $CAUGHT |"
116+
echo "| ⏱ Timeout | $TIMEOUT |"
117+
echo "| ⚪ Unviable (build failed) | $UNVIABLE |"
118+
echo
119+
if [ "$MISSED" -gt 0 ] && [ -f mutants-out/missed.txt ]; then
120+
echo "<details><summary>First 50 missed mutants</summary>"
121+
echo
122+
echo '```'
123+
head -50 mutants-out/missed.txt
124+
echo '```'
125+
echo "</details>"
126+
fi
127+
} >> "$GITHUB_STEP_SUMMARY"
128+
129+
- name: Upload mutants report
130+
if: always()
131+
uses: actions/upload-artifact@v4
132+
with:
133+
name: mutants-out-${{ github.run_id }}
134+
path: mutants-out/
135+
retention-days: 90

0 commit comments

Comments
 (0)