Skip to content

Commit 524f754

Browse files
feat(rules): BP008 phantom required context + AM010 admin-merge eligibility (#376)
## Summary - **BP008** (`lib/rules/branch_protection.ex`): walks the last N (default 5) commits on main, unions every observed `check_run.name`, and flags any required status-check context absent from that union. Severity `:high`, routes to `:sustainabot`. - **AM010** (`lib/rules/admin_merge_eligibility.ex`): orthogonal to AM001-AM009 (which are title-shape). Given `required_contexts`, the `phantom_contexts` from BP008, and a PR's `statusCheckRollup`, returns `{:eligible, "AM010"}` when every required context is either passing or phantom. ## Why The 2026-05-28 affinescript binding sweep tried to auto-merge 14 PRs across 9 repos. Every affinescript PR sat in `mergeStateStatus: BLOCKED` with every visible check green, because `spark-theatre-gate / SPARK Theatre Gate` is in the repo's `required_status_checks.contexts` but no workflow emits it. Auto-merge can never satisfy a context that never runs; admin-merge was the only path. This pattern is silent: nothing in the existing rules surfaced it. BP008 detects it from the API side; AM010 lets `sustainabot`'s budget-resume sweep clear phantom-blocked PRs without needing per-PR title patterns. ## Test plan - [x] `lib/rules/branch_protection.ex` compiles clean - [x] `lib/rules/admin_merge_eligibility.ex` compiles clean - [x] AM010 8/8 tests pass (`test/rules/admin_merge_eligibility_test.exs`) - [x] BP008 3/3 wiring tests pass (`test/rules/branch_protection_test.exs`) - [x] BP008 returns `[]` cleanly without a token (matches existing convention) - [x] BP008 wired into `scan/2` facade ## Follow-up A separate `hyperpolymath/affinescript` issue will be filed to either remove `spark-theatre-gate / SPARK Theatre Gate` from required contexts or wire up its emitting workflow — owner decision. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent c9fd28f commit 524f754

4 files changed

Lines changed: 506 additions & 5 deletions

File tree

lib/rules/admin_merge_eligibility.ex

Lines changed: 110 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,12 @@ defmodule Hypatia.Rules.AdminMergeEligibility do
1212
replace a heavy workflow with a thin wrapper, seed a doc, or apply a
1313
template-level chore, so blocking on CI provides no signal.
1414
15-
Rule IDs AM001-AM009.
15+
Rule IDs AM001-AM010.
16+
17+
AM010 is the only state-based recognizer in this module: a PR whose
18+
only failing/missing required check is a phantom (per
19+
`Hypatia.Rules.BranchProtection` BP008) is admin-eligible regardless
20+
of title shape. Caller supplies the phantom-context set.
1621
1722
Intended consumers:
1823
@@ -124,10 +129,113 @@ defmodule Hypatia.Rules.AdminMergeEligibility do
124129
"AM006" => "language-CI (rust/elixir) reusable-wrapper replacement",
125130
"AM007" => "CHANGELOG.md seed in Keep-a-Changelog format",
126131
"AM008" => "docs-template / README / tech-debt-audit doc",
127-
"AM009" => "linguist gitattributes / license-header migration"
132+
"AM009" => "linguist gitattributes / license-header migration",
133+
"AM010" => "phantom required check is the only blocker (paired with BP008)"
128134
}
129135
end
130136

137+
# ─── AM010: phantom-context-only blocker ──────────────────────────────
138+
139+
@doc """
140+
AM010: a PR is admin-eligible if every required status-check context
141+
passes EXCEPT one or more contexts that are known phantoms (per
142+
`Hypatia.Rules.BranchProtection` BP008). Auto-merge cannot satisfy a
143+
context that never emits a check; admin-merge is the only path.
144+
145+
This is a STATE check, not a title check — orthogonal to AM001-AM009.
146+
Call this BEFORE `classify/1` when you have the data, and fall
147+
through to title classification otherwise.
148+
149+
## Inputs
150+
151+
`pr_state` is a map with these keys:
152+
153+
* `:required_contexts` — list of context names from
154+
`required_status_checks.contexts` (or `.checks[*].context`).
155+
* `:phantom_contexts` — list of context names known to be phantom
156+
(BP008 findings, materialised as a set).
157+
* `:rollup` — list of `%{"name" => name, "conclusion" => conc}`
158+
maps, shaped like
159+
`gh pr view <n> --json statusCheckRollup`. Conclusions matching
160+
`["SUCCESS", "NEUTRAL", "SKIPPED"]` count as passing.
161+
162+
Returns `{:eligible, "AM010"}` when:
163+
164+
1. `phantom_contexts` (minus any context currently in-progress on
165+
THIS PR) is non-empty, AND
166+
2. every required context is either passing OR in the in-progress-
167+
subtracted `phantom_contexts`.
168+
169+
Returns `:not_phantom_only` otherwise.
170+
171+
Empty `required_contexts` returns `:not_phantom_only` — if nothing
172+
is required, auto-merge already works; AM010 doesn't apply.
173+
174+
## ALARP type-1 safety: in-progress subtraction
175+
176+
BP008 derives the phantom set from the LAST N MAIN COMMITS. A
177+
path-filtered workflow that happens to touch the path on THIS PR
178+
(but didn't on the recent main commits) would be classified as
179+
phantom even though it IS firing here. Admin-merging would bypass
180+
an actively-running check.
181+
182+
Defence: before deciding eligibility, subtract from the phantom set
183+
any context whose rollup entry is queued / pending / in_progress on
184+
this PR. A truly phantom context has NO rollup entry; a sometimes-
185+
firing context has a non-terminal entry. The subtraction collapses
186+
the latter back into the "wait for it" path.
187+
"""
188+
@spec am010_phantom_only_blocker?(map()) ::
189+
{:eligible, String.t()} | :not_phantom_only
190+
def am010_phantom_only_blocker?(pr_state) when is_map(pr_state) do
191+
required = Map.get(pr_state, :required_contexts, [])
192+
phantoms = MapSet.new(Map.get(pr_state, :phantom_contexts, []))
193+
rollup = Map.get(pr_state, :rollup, [])
194+
195+
passing =
196+
rollup
197+
|> Enum.filter(fn entry ->
198+
Map.get(entry, "conclusion") in ["SUCCESS", "NEUTRAL", "SKIPPED"]
199+
end)
200+
|> Enum.map(&Map.get(&1, "name"))
201+
|> Enum.reject(&is_nil/1)
202+
|> MapSet.new()
203+
204+
# ALARP Type 1 mitigation. A context flagged phantom by BP008 (sample-
205+
# based, main-branch) is only safe to treat as phantom on THIS PR when
206+
# it has NO rollup entry here. Any rollup entry — passing, failing,
207+
# pending, in-progress — means the context IS firing on this PR; the
208+
# BP008 main-branch sample was a false-negative for this commit. In
209+
# all such cases admin-merging would bypass an actual check (pass-by-
210+
# luck, fail-by-vote, or skip-an-in-progress run). Restrict the
211+
# phantom set to contexts truly absent from this PR's rollup.
212+
rollup_names =
213+
rollup
214+
|> Enum.map(&Map.get(&1, "name"))
215+
|> Enum.reject(&is_nil/1)
216+
|> MapSet.new()
217+
218+
phantoms_on_this_pr = MapSet.difference(phantoms, rollup_names)
219+
220+
cond do
221+
MapSet.size(phantoms_on_this_pr) == 0 ->
222+
:not_phantom_only
223+
224+
required == [] ->
225+
:not_phantom_only
226+
227+
Enum.all?(required, fn ctx ->
228+
MapSet.member?(passing, ctx) or MapSet.member?(phantoms_on_this_pr, ctx)
229+
end) ->
230+
{:eligible, "AM010"}
231+
232+
true ->
233+
:not_phantom_only
234+
end
235+
end
236+
237+
def am010_phantom_only_blocker?(_), do: :not_phantom_only
238+
131239
# ─── Stall detection (DBA001) ─────────────────────────────────────────
132240

133241
@doc """

lib/rules/branch_protection.ex

Lines changed: 142 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,16 +6,19 @@ defmodule Hypatia.Rules.BranchProtection do
66
Branch-protection hygiene rules drawn from the CIS GitHub Benchmark,
77
OSSF Scorecard, and NIST SP 800-218 (SSDF) PO/PS/PW practices.
88
9-
Rule IDs BP001-BP007.
9+
Rule IDs BP001-BP008.
1010
1111
These rules concern the **review and integrity controls** on the
1212
default branch — distinct from `BaselineHealth` (drift conditions),
1313
`WorkflowHardening` (workflow content) and `SupplyChain` (provenance).
1414
15-
All rules read from a single GitHub API endpoint:
15+
Most rules read from a single GitHub API endpoint:
1616
1717
gh api repos/{owner}/{repo}/branches/main/protection
1818
19+
BP008 additionally queries `/check-runs` against recent main-branch
20+
commits to detect required contexts that never emit a check.
21+
1922
## Provenance map
2023
2124
| Rule | Source | Upstream rule / check |
@@ -27,6 +30,7 @@ defmodule Hypatia.Rules.BranchProtection do
2730
| BP005 | CIS GH 1.6.x + NIST PO.3.2 | `require_code_owner_reviews: true` but CODEOWNERS missing/empty |
2831
| BP006 | CIS GH | `enforce_admins: false` |
2932
| BP007 | scorecard `Branch-Protection` tier 1 | default branch allows force-push or deletion |
33+
| BP008 | this estate (2026-05-28) | required status-check context never emits a check (phantom context blocks auto-merge) |
3034
3135
## Dispatch
3236
@@ -383,6 +387,81 @@ defmodule Hypatia.Rules.BranchProtection do
383387
end
384388
end
385389

390+
# ─── BP008: required context never emits a check (phantom) ──────────
391+
392+
@doc """
393+
BP008: `main` branch protection lists a required status-check context
394+
that has produced **zero** check_runs across the last N commits on
395+
main. The context is *phantom*: auto-merge will wait forever for it to
396+
pass, and admin-merge is the only path until it's removed (or the
397+
workflow that would emit it is wired up).
398+
399+
Severity: `:high` — every PR is silently blocked from auto-merge.
400+
Confidence: 0.85.
401+
402+
The empirical trigger was the 2026-05-28 estate sweep: every
403+
`hyperpolymath/affinescript` PR sat in `mergeStateStatus: BLOCKED`
404+
with all visible checks green, because `spark-theatre-gate / SPARK
405+
Theatre Gate` was a required context that no workflow emits. 14 PRs
406+
needed admin-merge to land.
407+
408+
Heuristic: walk the last `recent_commits` (default 5) on main, union
409+
every check_run name observed, and flag any required context that
410+
doesn't appear in that union. Five commits is enough signal because
411+
a real CI context emits one check per commit; a context that's been
412+
silent across five commits is either retired (worth removing) or
413+
never wired up (worth filing).
414+
415+
Returns one finding per phantom context. Returns `[]` cleanly when
416+
`GITHUB_TOKEN` / `HYPATIA_DISPATCH_PAT` is not set, when no contexts
417+
are required, or when the recent-commits lookup fails.
418+
419+
## Options
420+
421+
* `:recent_commits` — number of main-branch commits to sample
422+
(default: 5). Lower bound 1, upper bound 20.
423+
"""
424+
def bp008_phantom_required_context(owner, repo, opts \\ []) do
425+
n =
426+
opts
427+
|> Keyword.get(:recent_commits, 5)
428+
|> max(1)
429+
|> min(20)
430+
431+
with {:ok, protection} <- fetch_branch_protection(owner, repo, "main"),
432+
contexts when contexts != [] <- required_contexts(protection),
433+
{:ok, seen} <- fetch_recent_check_names(owner, repo, n) do
434+
phantoms = Enum.reject(contexts, &MapSet.member?(seen, &1))
435+
436+
Enum.map(phantoms, fn ctx ->
437+
%{
438+
rule: "BP008",
439+
file: "#{owner}/#{repo}",
440+
severity: :high,
441+
reason:
442+
"required status check `#{ctx}` has emitted zero check_runs " <>
443+
"across the last #{n} commits on main — auto-merge will " <>
444+
"stall on every PR until this context is removed or its " <>
445+
"workflow is wired up",
446+
action: :report,
447+
detail: %{
448+
branch: "main",
449+
phantom_context: ctx,
450+
sampled_commits: n,
451+
fix:
452+
"Either remove `#{ctx}` from required contexts " <>
453+
"(`gh api -X PATCH repos/#{owner}/#{repo}/branches/main/" <>
454+
"protection/required_status_checks -F 'contexts[]=...'` " <>
455+
"without it), or land a workflow whose job's name is " <>
456+
"exactly `#{ctx}`"
457+
}
458+
}
459+
end)
460+
else
461+
_ -> []
462+
end
463+
end
464+
386465
# ─── scan/2 facade ──────────────────────────────────────────────────
387466

388467
@doc """
@@ -408,7 +487,8 @@ defmodule Hypatia.Rules.BranchProtection do
408487
bp004_dismiss_stale_reviews_off(owner, repo) ++
409488
bp005_codeowners_required_but_missing(owner, repo) ++
410489
bp006_enforce_admins_off(owner, repo) ++
411-
bp007_force_push_or_delete_allowed(owner, repo)
490+
bp007_force_push_or_delete_allowed(owner, repo) ++
491+
bp008_phantom_required_context(owner, repo)
412492
else
413493
[]
414494
end
@@ -493,6 +573,30 @@ defmodule Hypatia.Rules.BranchProtection do
493573
defp maybe_add(list, true, item), do: list ++ [item]
494574
defp maybe_add(list, _, _item), do: list
495575

576+
# ─── Required-context extraction (BP008 helper) ─────────────────────
577+
#
578+
# `required_status_checks.contexts` is the legacy form; the newer
579+
# `checks` array carries the same names alongside per-app pin info
580+
# (`app_id`). Accept either; flatten to a string list. Returns `[]`
581+
# when no required-checks block is present (BP008 then no-ops).
582+
583+
defp required_contexts(%{"required_status_checks" => %{"contexts" => ctxs}})
584+
when is_list(ctxs) do
585+
Enum.filter(ctxs, &is_binary/1)
586+
end
587+
588+
defp required_contexts(%{"required_status_checks" => %{"checks" => checks}})
589+
when is_list(checks) do
590+
checks
591+
|> Enum.map(fn
592+
%{"context" => ctx} when is_binary(ctx) -> ctx
593+
_ -> nil
594+
end)
595+
|> Enum.reject(&is_nil/1)
596+
end
597+
598+
defp required_contexts(_), do: []
599+
496600
# ─── CODEOWNERS presence ────────────────────────────────────────────
497601

498602
# BP005 helper. GitHub looks for CODEOWNERS in three locations: root,
@@ -525,6 +629,41 @@ defmodule Hypatia.Rules.BranchProtection do
525629

526630
# ─── GitHub API ─────────────────────────────────────────────────────
527631

632+
# ─── Recent main-commit check-run sampling (BP008 helper) ───────────
633+
#
634+
# Walks the last `n` commits on main (default-branch HEAD), collects
635+
# every `check_runs[].name` observed, and returns the union as a
636+
# `MapSet`. A required context absent from this set after `n` commits
637+
# is a phantom. Returns `{:error, :no_token}` cleanly when no token.
638+
639+
defp fetch_recent_check_names(owner, repo, n) do
640+
with {:ok, commits} <- curl_github("repos/#{owner}/#{repo}/commits?per_page=#{n}") do
641+
shas =
642+
commits
643+
|> Enum.map(fn
644+
%{"sha" => sha} when is_binary(sha) -> sha
645+
_ -> nil
646+
end)
647+
|> Enum.reject(&is_nil/1)
648+
649+
names =
650+
Enum.reduce(shas, MapSet.new(), fn sha, acc ->
651+
case curl_github("repos/#{owner}/#{repo}/commits/#{sha}/check-runs?per_page=100") do
652+
{:ok, %{"check_runs" => runs}} when is_list(runs) ->
653+
Enum.reduce(runs, acc, fn
654+
%{"name" => name}, set when is_binary(name) -> MapSet.put(set, name)
655+
_, set -> set
656+
end)
657+
658+
_ ->
659+
acc
660+
end
661+
end)
662+
663+
{:ok, names}
664+
end
665+
end
666+
528667
defp fetch_branch_protection(owner, repo, branch) do
529668
case curl_github("repos/#{owner}/#{repo}/branches/#{branch}/protection") do
530669
{:ok, %{"message" => "Branch not protected"}} ->

0 commit comments

Comments
 (0)