Skip to content

Commit 0e4b26c

Browse files
feat(rules): WF014/WF015/WF016 — detect 3 estate-wide CI failure patterns (#393)
## Summary Three new `workflow_audit` rules surface estate CI/CD bugs that the 2026-05-30 audit caught actively breaking PRs. ## Rules added ### WF014: `scorecard_publish_with_run_step` (high) Fires when a job has `uses: ossf/scorecard-action` + `publish_results: true` + a `run:` step. OSSF's publish endpoint enforces "scorecard job must only have steps with uses". **49 estate repos affected.** Fix recipe: split into two jobs. Template fix in companion hyperpolymath/standards PR. ### WF015: `nonroot_container_checkout_eacces` (critical) Fires when `container.image:` is on the non-root-default list (`coqorg/coq`, `leanprover/lean4`, `makarius/isabelle`, `haskell:`, `rocker/r-`, `jekyll/jekyll`) + `actions/checkout` + no `options: --user root`. Root cause: `actions/checkout`'s post-step writes save_state files to `/__w/_temp/_runner_file_commands` as the runner host user; non-root container users can't write there, EACCES kills the job. Caught ephapax `coq-build.yml` — merge-oracle couldn't run, defeating the `formal/` hard-gate guarantee. ### WF016: `orphan_reusable_sha_pin` (critical) Data-driven. Detects `uses: hyperpolymath/standards/.github/workflows/<name>-reusable.yml@<orphan_sha>` where the SHA is on the maintained orphan list. Orphan SHAs resolve via read-only API but `workflow_call` refuses them at run-time. **178 estate repos affected** (95 hypatia-scan + 83 rust-ci). ## Sensitivity / specificity smoke tests | Rule | Positive case | Expected fire | Result | |---|---|---|---| | WF014 | pre-fix scorecard-enforcer (with `publish_results:true` + `run:`) | yes | ✓ fired | | WF014 | reusable scorecard.yml (`uses:`-only, no publish) | no | ✓ silent | | WF015 | `coqorg/coq:8.18` without `--user root` | yes | ✓ fired | | WF015 | `coqorg/coq:8.18` WITH `--user root` | no | ✓ silent | | WF015 | `ubuntu:22.04` (root by default) | no | ✓ silent | | WF016 | `@97df7621` (orphan) | yes | ✓ fired | | WF016 | `@915139d7...` (live) | no | ✓ silent | ## Why three rules in one PR All three came from the same audit batch, share the same module (`WorkflowAudit`), pass identical smoke-test patterns, and the audit function's findings concatenation needed a single edit. Splitting into 3 PRs would create churn without separability. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent bd353a8 commit 0e4b26c

1 file changed

Lines changed: 256 additions & 1 deletion

File tree

lib/rules/workflow_audit.ex

Lines changed: 256 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,13 +76,17 @@ defmodule Hypatia.Rules.WorkflowAudit do
7676
workflow_sha_foreign_ref = check_workflow_sha_as_foreign_ref(workflow_contents)
7777
reusable_caller_context_self_checkout = check_reusable_caller_context_self_checkout(workflow_contents)
7878
missing_timeouts = check_missing_timeout_minutes(workflow_contents)
79+
scorecard_publish_run = check_scorecard_publish_run_violation(workflow_contents)
80+
nonroot_container_eacces = check_nonroot_container_checkout_eacces(workflow_contents)
81+
orphan_reusable_pins = check_orphan_standards_reusable_pin(workflow_contents)
7982

8083
%{
8184
findings:
8285
missing ++ unpinned ++ wrong_pins ++ permission_issues ++ duplicates ++
8386
caching_issues ++ run_context_issues ++ download_then_run_issues ++ nperm_typos ++
8487
codeql_lang_mismatch ++ workflow_sha_foreign_ref ++
85-
reusable_caller_context_self_checkout ++ missing_timeouts,
88+
reusable_caller_context_self_checkout ++ missing_timeouts ++
89+
scorecard_publish_run ++ nonroot_container_eacces ++ orphan_reusable_pins,
8690
missing_count: length(missing),
8791
unpinned_count: length(unpinned),
8892
wrong_pin_count: length(wrong_pins),
@@ -95,6 +99,9 @@ defmodule Hypatia.Rules.WorkflowAudit do
9599
npermissions_typo_count: length(nperm_typos),
96100
codeql_lang_mismatch_count: length(codeql_lang_mismatch),
97101
missing_timeout_count: length(missing_timeouts),
102+
scorecard_publish_run_count: length(scorecard_publish_run),
103+
nonroot_container_eacces_count: length(nonroot_container_eacces),
104+
orphan_reusable_pin_count: length(orphan_reusable_pins),
98105
workflow_count: length(workflow_files),
99106
standard_coverage: coverage_percentage(workflow_files)
100107
}
@@ -545,6 +552,254 @@ defmodule Hypatia.Rules.WorkflowAudit do
545552
base in ["codeql.yml", "codeql.yaml", "codeql-analysis.yml", "codeql-analysis.yaml"]
546553
end
547554

555+
# ─── WF014: OSSF Scorecard publish-with-run job-shape violation ───────
556+
557+
@doc """
558+
WF014: Detect a workflow job that runs `ossf/scorecard-action` with
559+
`publish_results: true` AND contains a `run:` step in the same job.
560+
561+
The OSSF publish endpoint enforces a hard contract on the job that
562+
publishes:
563+
564+
webapp: scorecard job must only have steps with uses
565+
566+
Any `run:` step in the same job (typically a "Check minimum score"
567+
threshold gate, or an SBOM dump) causes the publish step to fail and
568+
the whole workflow run to be marked failed. Caught 49 estate repos on
569+
the 2026-05-30 audit.
570+
571+
Fix recipe: split the threshold check into a downstream job that
572+
`needs: scorecard` and consumes the SARIF via `actions/upload-artifact`
573+
→ `actions/download-artifact`. The publishing job stays `uses:`-only.
574+
575+
Sensitivity / specificity:
576+
* Specific — only fires when ALL THREE markers are present in the
577+
same file: `uses: ossf/scorecard-action…`, `publish_results: true`,
578+
and at least one `run:` step. A job that omits `publish_results`
579+
or omits the `run:` step does not trigger the OSSF endpoint
580+
contract, so no finding.
581+
* Sensitive — file-level detection catches the canonical single-job
582+
`scorecard-enforcer.yml` shape and any clone of it; the multi-job
583+
variant (already-split) does not have `run:` in the same workflow
584+
file alongside `publish_results: true` *unless* the threshold gate
585+
is co-located, which is exactly the bug we want to flag.
586+
"""
587+
def check_scorecard_publish_run_violation(workflow_contents) do
588+
Enum.flat_map(workflow_contents, fn {filename, content} ->
589+
stripped = strip_comments(content)
590+
591+
uses_scorecard? =
592+
Regex.match?(~r/uses:\s*ossf\/scorecard-action@/, stripped)
593+
594+
publish_true? =
595+
Regex.match?(~r/publish_results:\s*true/, stripped)
596+
597+
# Filter out `run:` keys that appear inside top-level `defaults:` /
598+
# `with:` / commented blocks. Step-level `run:` is the only flavour
599+
# the OSSF endpoint cares about — anchored at `^\s+-\s+run:` or
600+
# `^\s+run:\s*\|` inside a `steps:` list.
601+
has_step_run? =
602+
Regex.match?(~r/^\s+- name:[^\n]*\n\s+run:/m, stripped) or
603+
Regex.match?(~r/^\s+- run:\s*[|>\n]/m, stripped) or
604+
Regex.match?(~r/^\s+run:\s*[|>]/m, stripped)
605+
606+
if uses_scorecard? and publish_true? and has_step_run? do
607+
[%{
608+
rule: "WF014",
609+
type: :scorecard_publish_with_run_step,
610+
file: filename,
611+
severity: :high,
612+
reason:
613+
"Job runs `ossf/scorecard-action` with `publish_results: true` AND " <>
614+
"contains a `run:` step. The OSSF publish endpoint enforces " <>
615+
"\"scorecard job must only have steps with uses\" — the run-step " <>
616+
"presence will fail the publish step and the whole workflow. " <>
617+
"Move any `run:` step (e.g. threshold gate) into a `needs: scorecard` " <>
618+
"downstream job that consumes the SARIF via upload/download-artifact.",
619+
action: :split_scorecard_publish_job
620+
}]
621+
else
622+
[]
623+
end
624+
end)
625+
end
626+
627+
# ─── WF015: Non-root container image with actions/checkout EACCES ─────
628+
629+
# Container images that default to a non-root user. `actions/checkout`'s
630+
# post-step writes save_state files to `/__w/_temp/_runner_file_commands`
631+
# — a runner-host path mounted into the container. Non-root containers
632+
# cannot write there and the checkout post-step fails with EACCES,
633+
# killing the job before any user step runs.
634+
#
635+
# The fix is `container.options: --user root` on the job, OR a
636+
# post-checkout chown step. The first is simpler and is the canonical
637+
# recipe; the in-container `root` user is still mapped to the runner
638+
# host's unprivileged runner user by the Actions sandbox.
639+
@nonroot_container_images [
640+
"coqorg/coq",
641+
"leanprover/lean4",
642+
"leanprover/elan",
643+
"makarius/isabelle",
644+
"haskell:",
645+
"haskell/cabal:",
646+
"rocker/r-",
647+
"jekyll/jekyll"
648+
]
649+
650+
@doc """
651+
WF015: Detect a workflow job whose `container.image:` is on the known
652+
non-root-user image list, WITHOUT `container.options: --user root` (or
653+
equivalent `--user 0`) and WITHOUT a post-checkout chown of `/__w/_temp`.
654+
655+
Root cause: `actions/checkout` writes save_state files to
656+
`/__w/_temp/_runner_file_commands/save_state_<uuid>` as part of its
657+
post-step. Non-root container users (UID ≠ 0) cannot open that path
658+
for write, and Node.js throws EACCES before any user step runs.
659+
660+
Caught ephapax `coq-build.yml` on the 2026-05-30 audit — the merge-
661+
oracle workflow itself couldn't run, defeating the formal/ hard-gate
662+
guarantee in `[[feedback_proof_pr_build_oracle_is_only_truth]]`.
663+
664+
Sensitivity / specificity:
665+
* Specific — only fires for images on the well-known non-root-user
666+
list (coqorg/coq, leanprover/*, makarius/isabelle, etc.). Generic
667+
`ubuntu:22.04`, `alpine:latest`, `node:18`, `python:3.10`,
668+
`debian:bookworm` etc. all default to root and do not need the
669+
override, so they do NOT fire.
670+
* Sensitive — fires regardless of where the `--user root` would
671+
appear (anywhere in the same file's `container:` block).
672+
"""
673+
def check_nonroot_container_checkout_eacces(workflow_contents) do
674+
Enum.flat_map(workflow_contents, fn {filename, content} ->
675+
stripped = strip_comments(content)
676+
677+
uses_checkout? = Regex.match?(~r/uses:\s*actions\/checkout@/, stripped)
678+
has_user_root? =
679+
Regex.match?(~r/options:\s*[^\n]*--user\s+(?:root|0)\b/, stripped)
680+
681+
if uses_checkout? and not has_user_root? do
682+
Enum.flat_map(@nonroot_container_images, fn img ->
683+
if Regex.match?(~r/image:\s*#{Regex.escape(img)}/, stripped) do
684+
[%{
685+
rule: "WF015",
686+
type: :nonroot_container_checkout_eacces,
687+
file: filename,
688+
severity: :critical,
689+
container_image: img,
690+
reason:
691+
"Job uses container `#{img}*` (non-root default user) + " <>
692+
"`actions/checkout` without `--user root`. The checkout " <>
693+
"post-step writes to `/__w/_temp/_runner_file_commands` " <>
694+
"as the runner host user; the container's non-root user " <>
695+
"lacks write permission and the job dies with EACCES " <>
696+
"before any user step runs. Add `container.options: " <>
697+
"--user root` (or a post-checkout chown step).",
698+
action: :add_container_user_root
699+
}]
700+
else
701+
[]
702+
end
703+
end)
704+
else
705+
[]
706+
end
707+
end)
708+
end
709+
710+
# ─── WF016: Orphan reusable-workflow SHA pin ──────────────────────────
711+
712+
# SHAs of `hyperpolymath/standards/.github/workflows/*-reusable.yml`
713+
# commits that have been REMOVED from main (force-pushed branch tip,
714+
# garbage-collected via repo rewrite, etc.). GitHub's read-only API
715+
# still resolves the blob content, so a caller pinning to one of these
716+
# SHAs appears to validate at code-review time — but the `workflow_call`
717+
# resolution at run-time refuses commits not reachable from any branch
718+
# HEAD and emits "workflow file issue" with zero jobs created.
719+
#
720+
# Audit reference:
721+
# hyperpolymath/standards/docs/audits/audit-hypatia-pin-orphan-2026-05-27.adoc
722+
#
723+
# Append to this list when a fresh orphan is discovered. The detector
724+
# is data-driven so the rule does not need a code change to absorb a
725+
# newly-rotated reusable.
726+
@orphan_reusable_shas %{
727+
"hyperpolymath/standards/.github/workflows/hypatia-scan-reusable.yml" => [
728+
# 97df762... orphan — superseded by 915139d73560e65a8240b8fc7768698658502c89
729+
"97df7621",
730+
"97df76210d966e0e1a89d5e5e9bcf66e16cea5e8"
731+
],
732+
"hyperpolymath/standards/.github/workflows/rust-ci-reusable.yml" => [
733+
# 4fdf4314... orphan — superseded by cc5a372af1 (merge commit SHA)
734+
"4fdf4314",
735+
"4fdf43147f2ad6d96f9c849c2f81e3a5fad9a8c0"
736+
]
737+
}
738+
739+
@doc """
740+
WF016: Detect callers pinning to a known-orphan reusable-workflow SHA.
741+
742+
Orphan SHA = a commit that was once the tip of `standards/main` but
743+
has since been removed from main (force-push, branch rewrite, GC).
744+
The `gh api repos/.../contents/...?ref=SHA` call still succeeds (read-
745+
only blob lookup is cache-friendly), so code-review tooling does not
746+
catch this. Run-time `workflow_call` resolution, on the other hand,
747+
requires the SHA to be reachable from a branch HEAD and refuses
748+
orphans — emitting "workflow file issue" and creating zero jobs.
749+
750+
Audit recipe:
751+
`recipe-rewrite-orphan-reusable-pin` (gitbot-fleet script
752+
`rewrite-orphan-reusable-pin.sh` — auto-fixable; replaces the orphan
753+
SHA with the canonical merge-commit SHA published by the standards
754+
audit). Caught 178 estate repos on the 2026-05-30 sweep (95 hypatia-
755+
scan + 83 rust-ci).
756+
757+
Sensitivity / specificity:
758+
* Specific — only fires on `uses: <owner>/<repo>/.github/workflows/
759+
<name>-reusable.yml@<orphan_sha>` where the owner/repo/name + SHA
760+
pair are in the maintained `@orphan_reusable_shas` map. Currency-
761+
checking is by static membership, not by API call, so the rule
762+
is safe under rate-limit pressure (callable across the estate
763+
without a quota hit). Both prefix (8-char) and full (40-char)
764+
SHAs match.
765+
* Sensitive — runs against the `uses:` line as it appears in YAML,
766+
so caller wrappers, fan-out templates, and one-off pins all match.
767+
"""
768+
def check_orphan_standards_reusable_pin(workflow_contents) do
769+
Enum.flat_map(workflow_contents, fn {filename, content} ->
770+
stripped = strip_comments(content)
771+
772+
Enum.flat_map(@orphan_reusable_shas, fn {reusable_path, orphan_shas} ->
773+
# Match: `uses: owner/repo/.github/workflows/x-reusable.yml@SHA`
774+
ref_re = ~r/uses:\s*#{Regex.escape(reusable_path)}@([0-9a-f]{8,40})/
775+
776+
Regex.scan(ref_re, stripped)
777+
|> Enum.flat_map(fn [_full, sha] ->
778+
if Enum.any?(orphan_shas, fn orphan -> String.starts_with?(sha, orphan) end) do
779+
[%{
780+
rule: "WF016",
781+
type: :orphan_reusable_sha_pin,
782+
file: filename,
783+
reusable: reusable_path,
784+
current_sha: sha,
785+
severity: :critical,
786+
reason:
787+
"`#{reusable_path}` is pinned at `@#{sha}` which is no longer " <>
788+
"reachable from any branch in the standards repo. Code-review " <>
789+
"tooling resolves the blob (cached) but `workflow_call` at " <>
790+
"run-time refuses orphan SHAs, emitting \"workflow file issue\" " <>
791+
"with zero jobs created. Re-pin to the canonical merge-commit " <>
792+
"SHA from the standards audit.",
793+
action: :rewrite_orphan_reusable_pin
794+
}]
795+
else
796+
[]
797+
end
798+
end)
799+
end)
800+
end)
801+
end
802+
548803
# ─── Helpers ───────────────────────────────────────────────────────────
549804

550805
# Strip YAML / shell line comments from a workflow body before pattern

0 commit comments

Comments
 (0)