Commit 700a3c5
fix(ci): staleness gate fails on named defects, not on the calendar
MEASURED 2026-07-21: 294 of 350 consumers were red on the single pin
d7c2271 (2026-06-26) — every one of them green a fortnight earlier, with
nothing changed in any consumer. aspasia is the clean natural experiment:
same pin, 2026-07-09 run at 32 commits / 12d = passing notice; 2026-07-21 at
63 commits / 24d = hard error.
standards moves ~2.6 commits/day, so a consumer exhausts both window budgets
~14 days after any propagation. Holding the fleet green under that rule means
re-pinning ~300 repos every fortnight (order 7,800 PRs/year). A gate that
fails everyone on a timer is not a guard: it trains the estate to ignore it,
and it buries the findings that matter — the same sweep shows 148 real
Workflow security linter failures and 74 anti-pattern failures that read as
noise once the fleet is uniformly red.
1. Age outside the window is now a ::notice, not an error. The window is
still computed and still reported, so propagate-workflow-pins.sh and the
Hypatia sha_bump_propagation rule keep their signal.
2. A named deny-list (KNOWN_BAD_BEFORE) replaces age as the hard failure.
Each entry is <reusable>:<fix-sha>; a pin that is a strict ancestor of the
fix carries that defect and is rejected at any age. This is what the window
was only ever a proxy for, and it is strictly better in both directions: a
RECENT pin carrying the defect is now caught, and an old pin carrying none
is no longer punished by the calendar.
First entry is e9c8888 (#441). Before it, hypatia-scan-reusable and
the validate-hypatia-baseline job cached the built Hypatia scanner under a
keyless key while the build steps were guarded by `if [ ! -d ]`, so the
first scanner build ever cached was reused forever and scanner fixes never
took effect. A pin older than this reports a FALSE GREEN. All 294
consumers on d7c2271 predate it by one day and stay red — now for a true
and actionable reason.
3. Integrity no longer rests on the runner's clone. The UNKNOWN / "may be
forged" verdict was firing on legitimate pins: awesome-haskell pins
governance-reusable@5a93d9d5 and was accused on four consecutive runs over
17 days, while that commit verifies as a true ancestor of main locally in
both treeless and --depth 200 clones and via the compare API (behind=0,
ahead=90). The mechanism was never reproduced off-runner.
A hard FORGED verdict now requires either a COMPLETE local clone (not
shallow, not partial) or confirmation from
GET /repos/{nwo}/compare/{pin}...{branch}. Where neither is available the
gate warns and passes: "cannot verify" is not "compromised". Zero API calls
on the happy path — the server is consulted only when about to accuse.
The deny-list gets the same fallback, so a degraded runner cannot silently
skip it; "could not check" is reported as SKIPPED, never as passed.
Tests: the hermetic fixture suite goes 12 -> 16 cases. Out-of-window now
asserts exit 0 AND the advisory notice (a new run_case_out helper — an exit
code alone cannot distinguish "passed silently" from "passed with the notice
the propagation path depends on"). New coverage: deny-list rejects at any
age, rejects even when in-window, does not reject the fixing commit itself,
and is scoped per reusable. 16/16 pass, and the suite stays hermetic because
a complete fixture clone resolves the forged case without the network.
Also verified against live standards history and in degraded-clone mode; both
give identical verdicts for fresh / denied / old-but-clean / forged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>1 parent f9dca6d commit 700a3c5
3 files changed
Lines changed: 366 additions & 20 deletions
File tree
- docs/decisions
- scripts
- tests
Lines changed: 61 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
145 | 145 | | |
146 | 146 | | |
147 | 147 | | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
148 | 152 | | |
149 | 153 | | |
150 | 154 | | |
| |||
162 | 166 | | |
163 | 167 | | |
164 | 168 | | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
0 commit comments