feat(governance): promote the docs + package-policy gates from theatre to real (#505) (#513)

Promotes the two `governance-reusable.yml` jobs that **could not fail**
into real gates — tasks 1, 2 and 3 of #505.

## Blast radius was measured first, not guessed

Over the **412 real repo-root callers** of `governance-reusable` in the
estate checkout (1,106 raw grep hits reduce to 412 once nested monorepo
subdirs — whose `.github/workflows` GitHub never reads — are excluded):

| check | missing | decision |
|---|---|---|
| README | **0 / 412** | **blocking now** — arming reds nobody |
| LICENSE | **0 / 412** | **blocking now** — arming reds nobody |
| CONTRIBUTING | 54 / 412 (13%) | warn → **blocking from 2026-08-21** |
| no Guix/Nix packaging | 70 / 412 (17%) | warn → **blocking from
2026-08-21** |

**Verified against all 412 callers with the final scripts: 0 red today
on both gates.** The grace window buys time without manufacturing red
noise now — and it is a *real date that flips itself*, not a flag
defaulting to off (which would be the theatre this replaces).

**94 unique repos** must land a `CONTRIBUTING` and/or packaging file
before the cutoff. Full list attached as a comment below.

## What changed

**`scripts/check-docs-presence.sh`** (new) — replaces the
`::warning::`-only step. Split gate: README+LICENSE fail immediately,
CONTRIBUTING warns until the cutoff. Critically, it **never prints a
pass claim while a required file is absent** — grace mode prints `NOT
YET ENFORCED`, not a green tick.

**`scripts/check-package-policy.sh`** (new) — replaces the job that
ended in an unconditional `✅ Package policy check passed`. Two honesty
fixes beyond adding `exit 1`:
- Predicate tightened from *"any `*.scm` anywhere"* (a stray Guile
source file satisfied it) to a genuine packaging artefact
(`guix.scm`/`manifest.scm`/`channels.scm`/`.guix-channel`). Tightening
moved only **13** repos, so honesty was nearly free.
- Vendored trees (`node_modules/`, `deps/`, `.lake/`, `vendor/`,
`.git/`) are pruned, so a dependency can't satisfy the policy on the
caller's behalf.
- The `git diff HEAD~1` lockfile branch is **dropped, not stubbed**: it
never resolved under `ref: github.sha` without `fetch-depth`, so it was
inert. hypatia `cicd_rules/nodejs_detected` is the real rule. Broadening
it beyond `package-lock.json` is follow-up, noted below.

**Redundant trufflehog removed** (#505 task 3). It was
`continue-on-error: true` so it never gated, and it duplicated the real
gitleaks gate from #500. `secret-scanner-reusable.yml`'s header
*already* claimed "Trufflehog removed as redundant" — this reconciles
the estate with itself. Removing a never-failing step cannot newly-red
any caller.

**Both gates refuse to be disarmed.** A malformed cutoff would make the
date comparison pick the grace branch forever — silently restoring the
fake gate. Both scripts validate their own cutoff format and hard-fail
on garbage.

## A real bug, found only by running against real repos

`find … | head -1` under `set -o pipefail` is a **SIGPIPE race**: `head`
exits after line 1, `find` keeps writing, takes SIGPIPE, and `pipefail`
propagates it — aborting under `set -e`. It only bites on trees large
enough that `find` is still running, so **all 20 unit fixtures passed
while two real estate repos red intermittently for no policy reason**.
Fixed, with a regression test that was itself red-teamed: it *fails*
against the buggy form and passes against the fix. (The first version of
that test was a false-green — 3,000 non-matching files don't trigger it,
because `find` emits one line; the fixture now creates 4,000 *matching*
paths.)

## Verification

`just governance-gates-test` → **21 cases, all green**. Drives both
gates on **both sides of the cutoff** (`DOCS_TODAY` / `PKG_TODAY`
seams), plus alternate extensions, inclusive-cutoff boundary,
vendored-tree pruning, stray-`.scm` rejection, malformed-cutoff refusal,
and the SIGPIPE regression.

This matters because **#505 notes standards CI does not exercise the
reusable for SHA-pinned callers** — watching this repo go green proves
nothing about them. Note however that **396 callers pin `@main`**, so
they pick this up *immediately on merge*; the grace window is what makes
that safe.

## ⚠️ Reviewer note — the plumbing proof and the expected red

standards calls this reusable **on its own PRs** (`governance.yml` →
`./.github/workflows/governance-reusable.yml`), and the script checkout
is pinned `ref: main`. The new scripts are not on `main` until this
merges, so the final commit's governance run **will red for that reason
alone, not because the change is broken**.

To prove the plumbing (sparse-checkout → `cp` to `$RUNNER_TEMP` → `rm
-rf .standards-checkout` → run) actually works end-to-end, this branch
temporarily points `ref:` at itself; that run is the evidence, and the
`ref:` is reverted to `main` before merge. See the commit history.

## Rollback

Revert this PR, or re-pin callers to the previous `governance-reusable`
SHA. The two new scripts are additive — nothing else calls them.

## Follow-up (not in scope here)

- Broaden hypatia `cicd_rules/nodejs_detected` beyond
`package-lock.json` (yarn/Gemfile/Pipfile/poetry lockfiles), recovering
the coverage the inert branch nominally claimed.
- The 94-repo remediation list before 2026-08-21.

Refs #505, #500

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

Author: hyperpolymath

.github/workflows/governance-reusable.yml

@@ -535,22 +535,29 @@ jobs:
           # governance jobs on every PR estate-wide. github.sha resolves to the
           # same merge commit but is always fetchable.
           ref: ${{ github.sha }}
+
+      # A reusable workflow only auto-checks-out its own YAML, not sibling
+      # scripts. Sparse-check-out standards' scripts/ to get
+      # check-package-policy.sh, mirroring the baseline-filter job above.
+      - name: Check out standards for the package-policy gate
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        with:
+          repository: hyperpolymath/standards
+          ref: main
+          path: .standards-checkout
+          sparse-checkout: |
+            scripts
+          sparse-checkout-cone-mode: false
+
       - name: Enforce Guix primary / Nix fallback
         run: |
-          HAS_GUIX=$(find . -name "*.scm" -o -name ".guix-channel" -o -name "guix.scm" 2>/dev/null | head -1)
-          HAS_NIX=$(find . -name "*.nix" 2>/dev/null | head -1)
-          NEW_LOCKS=$(git diff --name-only --diff-filter=A HEAD~1 2>/dev/null | grep -E 'package-lock\.json|yarn\.lock|Gemfile\.lock|Pipfile\.lock|poetry\.lock' || true)
-          if [ -n "$NEW_LOCKS" ]; then
-            echo "⚠️ Lock files detected. Prefer Guix manifests for reproducibility."
-          fi
-          if [ -n "$HAS_GUIX" ]; then
-            echo "✅ Guix package management detected (primary)"
-          elif [ -n "$HAS_NIX" ]; then
-            echo "✅ Nix package management detected (fallback)"
-          else
-            echo "ℹ️ Consider adding guix.scm or flake.nix for reproducible builds"
-          fi
-          echo "✅ Package policy check passed"
+          # Move the checker OUT of the scanned tree and delete the standards
+          # checkout before scanning: the gate walks the whole caller tree, so
+          # a packaging file shipped inside .standards-checkout/ would satisfy
+          # the policy on the caller's behalf (same trap as the baseline job).
+          cp .standards-checkout/scripts/check-package-policy.sh "$RUNNER_TEMP/"
+          rm -rf .standards-checkout
+          bash "$RUNNER_TEMP/check-package-policy.sh" .
 
   security-policy:
     name: Security policy checks
@@ -813,17 +820,14 @@ jobs:
       - name: Check file permissions
         run: |
           find . -type f -perm /111 -name "*.sh" | head -10 || true
-      - name: Check for secrets
-        uses: trufflesecurity/trufflehog@27b0417c16317ca9a472a9a8092acce143b49c55 # v3.95.9
-        with:
-          path: ./
-          base: ${{ github.event.pull_request.base.sha || github.event.before }}
-          head: ${{ github.sha }}
-        # by-design: trufflehog is a best-effort advisory scan; a scanner
-        # diff/range hiccup must not fail the whole governance gate. The
-        # blocking secret check is the inline grep in the security job.
-        # (Tooling Version Integrity Rule 4 — documented soft-gate.)
-        continue-on-error: true
+      # NOTE: the advisory trufflehog scan that used to sit here was removed
+      # (standards#505). It was `continue-on-error: true`, so it never gated,
+      # and it duplicated the estate's real secret gate — gitleaks in
+      # secret-scanner-reusable.yml, which became genuinely blocking in #500
+      # (pinned + checksummed binary, `--exit-code 1`, no continue-on-error).
+      # secret-scanner-reusable's own header already claimed "Trufflehog removed
+      # as redundant"; keeping it here contradicted that across the same estate.
+      # Removing a never-failing step cannot newly-red any caller.
       - name: Check TODO/FIXME
         run: |
           echo "=== TODOs ==="
@@ -836,17 +840,27 @@ jobs:
         # advisory: formatting hygiene is reported from the reusable estate
         # bundle; repos opt into blocking formatter checks locally when ready.
         continue-on-error: true
+      # Sparse-check-out standards' scripts/ for the docs gate (a reusable
+      # workflow only auto-checks-out its own YAML, not sibling scripts).
+      - name: Check out standards for the docs gate
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        with:
+          repository: hyperpolymath/standards
+          ref: main
+          path: .standards-checkout
+          sparse-checkout: |
+            scripts
+          sparse-checkout-cone-mode: false
+
       - name: Check documentation
         run: |
-          MISSING=""
-          [ ! -f "README.md" ] && [ ! -f "README.adoc" ] && MISSING="$MISSING README"
-          [ ! -f "LICENSE" ] && [ ! -f "LICENSE.txt" ] && [ ! -f "LICENSE.md" ] && MISSING="$MISSING LICENSE"
-          [ ! -f "CONTRIBUTING.md" ] && [ ! -f "CONTRIBUTING.adoc" ] && MISSING="$MISSING CONTRIBUTING"
-          if [ -n "$MISSING" ]; then
-            echo "::warning::Missing docs:$MISSING"
-          else
-            echo "✅ Core documentation present"
-          fi
+          # Split gate (standards#505): README + LICENSE block immediately —
+          # measured 0/412 callers missing either, so arming them reds nobody.
+          # CONTRIBUTING (54/412 missing) warns until the cutoff baked into the
+          # script, then blocks. See scripts/check-docs-presence.sh.
+          cp .standards-checkout/scripts/check-docs-presence.sh "$RUNNER_TEMP/"
+          rm -rf .standards-checkout
+          bash "$RUNNER_TEMP/check-docs-presence.sh" .
 
   wellknown:
     name: Well-Known (RFC 9116 + RSR)

Justfile

@@ -53,6 +53,11 @@ automation-test:
 gates-test:
     @bash scripts/tests/wave8-gates-test.sh
 
+# standards#505 gate-promotion regression: docs + package-policy gates can fail,
+# and their grace cutoffs actually flip (tested from both sides of the date)
+governance-gates-test:
+    @bash scripts/tests/governance-gates-505-test.sh
+
 # Structural validation of the Mustfile contract (severity + run/verification per check)
 mustfile-check path=".machine_readable/contractiles/must/Mustfile.a2ml":
     @bash scripts/check-mustfile-structure.sh "{{path}}"

scripts/check-docs-presence.sh

@@ -0,0 +1,120 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: MPL-2.0
+# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell
+#
+# check-docs-presence.sh — gate on core repository documentation.
+#
+# Replaces the `::warning::`-only "Check documentation" step in
+# governance-reusable.yml, which could not fail over any input (standards#505).
+#
+# SPLIT GATE (wave8 doctrine: promote honestly, no manufactured red noise).
+# Blast radius measured 2026-07-21 over the 412 real repo-root callers of
+# governance-reusable in the local estate checkout:
+#
+#   README        0/412 missing  -> BLOCKING NOW. Arming reds nobody.
+#   LICENSE       0/412 missing  -> BLOCKING NOW. Arming reds nobody.
+#   CONTRIBUTING 54/412 missing  -> WARN until the cutoff, then BLOCKING.
+#
+# The CONTRIBUTING cutoff is a real date that flips itself. It is not a flag
+# defaulting to off: on the cutoff date this check starts failing with no
+# further edit to this file. That is the difference between a scheduled gate
+# and theatre.
+#
+# Accepted filenames follow the estate documentation policy (AsciiDoc is the
+# default; GitHub-required community-health files stay Markdown):
+#   README.adoc | README.md
+#   LICENSE | LICENSE.txt | LICENSE.md
+#   CONTRIBUTING.md | CONTRIBUTING.adoc
+#
+# Usage: check-docs-presence.sh [repo-root]
+#
+# Environment (test seams — the shipped policy is the default in each case):
+#   ENFORCE_CONTRIBUTING_FROM   YYYY-MM-DD; enforcement begins ON this date.
+#   DOCS_TODAY                  YYYY-MM-DD; overrides "now" so the pre-cutoff
+#                               and post-cutoff branches are both testable.
+#
+# Exit: 0 = pass (or in-grace warning), 1 = policy failure or bad configuration.
+
+set -euo pipefail
+
+ROOT="${1:-.}"
+
+# Enforcement begins ON this date (inclusive), estate-wide, for CONTRIBUTING.
+ENFORCE_CONTRIBUTING_FROM="${ENFORCE_CONTRIBUTING_FROM:-2026-08-21}"
+TODAY="${DOCS_TODAY:-$(date -u +%Y-%m-%d)}"
+
+# A malformed date would make the lexicographic comparison below silently
+# choose the grace branch forever — i.e. it would turn this gate back into the
+# fake gate it replaces. Refuse to run rather than run un-armed.
+valid_date() {
+  case "$1" in
+    [0-9][0-9][0-9][0-9]-[0-1][0-9]-[0-3][0-9]) return 0 ;;
+    *) return 1 ;;
+  esac
+}
+require_date() {
+  local name="$1" value="$2"
+  if ! valid_date "$value"; then
+    echo "::error::check-docs-presence: $name='$value' is not YYYY-MM-DD."
+    echo "Refusing to run: an unparseable cutoff would silently disarm this gate."
+    exit 1
+  fi
+}
+require_date ENFORCE_CONTRIBUTING_FROM "$ENFORCE_CONTRIBUTING_FROM"
+require_date DOCS_TODAY "$TODAY"
+
+if [ ! -d "$ROOT" ]; then
+  echo "::error::check-docs-presence: '$ROOT' is not a directory."
+  exit 1
+fi
+
+# have <name>... -> 0 if any of the candidate filenames exists at the root.
+have() {
+  local f
+  for f in "$@"; do
+    [ -f "$ROOT/$f" ] && return 0
+  done
+  return 1
+}
+
+blocking_missing=""
+grace_missing=""
+
+have README.adoc README.md      || blocking_missing="$blocking_missing README"
+have LICENSE LICENSE.txt LICENSE.md || blocking_missing="$blocking_missing LICENSE"
+
+if ! have CONTRIBUTING.md CONTRIBUTING.adoc; then
+  # String comparison is sound here: YYYY-MM-DD sorts chronologically, and both
+  # operands are format-validated above.
+  if [[ "$TODAY" < "$ENFORCE_CONTRIBUTING_FROM" ]]; then
+    grace_missing="CONTRIBUTING"
+  else
+    blocking_missing="$blocking_missing CONTRIBUTING"
+  fi
+fi
+
+if [ -n "$grace_missing" ]; then
+  echo "::warning::Missing docs (grace period):$grace_missing — this becomes a" \
+       "BLOCKING failure on $ENFORCE_CONTRIBUTING_FROM (today is $TODAY)."
+fi
+
+if [ -n "$blocking_missing" ]; then
+  echo "::error::Missing required documentation:$blocking_missing"
+  echo
+  echo "Required at the repository root (either extension where two are listed):"
+  echo "  README.adoc      (or README.md)"
+  echo "  LICENSE          (or LICENSE.txt / LICENSE.md)"
+  echo "  CONTRIBUTING.md  (or CONTRIBUTING.adoc)"
+  echo
+  echo "Estate policy: docs are AsciiDoc by default; see hyperpolymath/standards."
+  exit 1
+fi
+
+if [ -n "$grace_missing" ]; then
+  # Never claim a pass while a required file is absent. The job is green only
+  # because the cutoff has not arrived — say exactly that.
+  echo "NOT YET ENFORCED: CONTRIBUTING is missing but inside the grace window."
+  exit 0
+fi
+
+echo "✅ Core documentation present (README, LICENSE, CONTRIBUTING)"

scripts/check-package-policy.sh

@@ -0,0 +1,122 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: MPL-2.0
+# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell
+#
+# check-package-policy.sh — gate on the Guix-primary / Nix-fallback policy.
+#
+# Replaces the echo-only "Enforce Guix primary / Nix fallback" step in
+# governance-reusable.yml, whose every branch echoed and which terminated with
+# an unconditional `✅ Package policy check passed` (standards#505). It could
+# not detect a violation, and it claimed a pass over any input.
+#
+# POLICY (CLAUDE.md, "Package Management"): Guix primary (guix.scm), Nix
+# fallback (flake.nix). A repo satisfying neither is the violation.
+#
+# PREDICATE — deliberately tightened. The previous step accepted *any* `*.scm`
+# anywhere in the tree as proof of "Guix package management detected", which a
+# stray Guile source file satisfies without any packaging existing. This script
+# requires a genuine packaging artefact (guix.scm / manifest.scm / channels.scm
+# / .guix-channel) and prunes vendored trees so a dependency's file cannot
+# satisfy the policy on the repo's behalf. Measured 2026-07-21 over the 412 real
+# repo-root callers, tightening moved only 13 repos (57 -> 70 failing), so the
+# honest predicate is nearly free.
+#
+# GRACE WINDOW: 70/412 callers (17%) currently satisfy neither. They warn until
+# the cutoff, then fail. The cutoff is a real date that flips itself with no
+# further edit to this file.
+#
+# NOTE ON LOCKFILES: the replaced step also grepped `git diff HEAD~1` for
+# package-lock.json / yarn.lock / Gemfile.lock / Pipfile.lock / poetry.lock.
+# That branch was inert — governance checks out at `ref: github.sha` without
+# `fetch-depth: 0`, so `HEAD~1` does not resolve and the command was swallowed
+# by `2>/dev/null` / `|| true`. It is dropped here rather than kept as a stub:
+# the blocking rule for lockfiles is hypatia `cicd_rules/nodejs_detected`.
+# Broadening that rule beyond package-lock.json is tracked as follow-up.
+#
+# Usage: check-package-policy.sh [repo-root]
+#
+# Environment (test seams — the shipped policy is the default in each case):
+#   ENFORCE_PACKAGE_POLICY_FROM  YYYY-MM-DD; enforcement begins ON this date.
+#   PKG_TODAY                    YYYY-MM-DD; overrides "now" so the pre-cutoff
+#                                and post-cutoff branches are both testable.
+#
+# Exit: 0 = pass (or in-grace warning), 1 = policy failure or bad configuration.
+
+set -euo pipefail
+
+ROOT="${1:-.}"
+
+ENFORCE_PACKAGE_POLICY_FROM="${ENFORCE_PACKAGE_POLICY_FROM:-2026-08-21}"
+TODAY="${PKG_TODAY:-$(date -u +%Y-%m-%d)}"
+
+# An unparseable cutoff would make the comparison below pick the grace branch
+# forever, restoring the fake gate this script replaces. Refuse to run.
+valid_date() {
+  case "$1" in
+    [0-9][0-9][0-9][0-9]-[0-1][0-9]-[0-3][0-9]) return 0 ;;
+    *) return 1 ;;
+  esac
+}
+require_date() {
+  local name="$1" value="$2"
+  if ! valid_date "$value"; then
+    echo "::error::check-package-policy: $name='$value' is not YYYY-MM-DD."
+    echo "Refusing to run: an unparseable cutoff would silently disarm this gate."
+    exit 1
+  fi
+}
+require_date ENFORCE_PACKAGE_POLICY_FROM "$ENFORCE_PACKAGE_POLICY_FROM"
+require_date PKG_TODAY "$TODAY"
+
+if [ ! -d "$ROOT" ]; then
+  echo "::error::check-package-policy: '$ROOT' is not a directory."
+  exit 1
+fi
+
+# Vendored trees cannot satisfy the policy on the repo's behalf.
+PRUNE=( -name .git -o -name node_modules -o -name deps -o -name .lake -o -name vendor )
+
+# NB: `find … | head -1` under `set -o pipefail` is a SIGPIPE race — head exits
+# after the first line, find keeps writing, takes SIGPIPE, and pipefail
+# propagates the non-zero status, aborting the script under `set -e`. It only
+# manifests on trees large enough that find is still running when head exits, so
+# small fixtures never catch it (it red two real estate repos intermittently
+# while every unit fixture passed). `|| true` neutralises the pipeline status;
+# the guix/nix decision is made from the captured value, not the exit code.
+find_first() {
+  local out
+  out="$(find "$ROOT" \( "${PRUNE[@]}" \) -prune -o \( "$@" \) -print 2>/dev/null | head -1 || true)"
+  printf '%s' "$out"
+}
+
+GUIX="$(find_first -name guix.scm -o -name manifest.scm -o -name channels.scm -o -name .guix-channel)"
+NIX="$(find_first -name flake.nix -o -name default.nix -o -name shell.nix)"
+
+if [ -n "$GUIX" ]; then
+  echo "✅ Guix package management detected (primary): ${GUIX#"$ROOT"/}"
+  exit 0
+fi
+
+if [ -n "$NIX" ]; then
+  echo "✅ Nix package management detected (fallback): ${NIX#"$ROOT"/}"
+  echo "::notice::Guix is the estate primary; Nix is the accepted fallback."
+  exit 0
+fi
+
+# Violation: neither packaging system is present.
+if [[ "$TODAY" < "$ENFORCE_PACKAGE_POLICY_FROM" ]]; then
+  echo "::warning::No Guix or Nix packaging found — this becomes a BLOCKING" \
+       "failure on $ENFORCE_PACKAGE_POLICY_FROM (today is $TODAY)."
+  # Never claim a pass while the policy is unmet.
+  echo "NOT YET ENFORCED: package policy unmet but inside the grace window."
+  exit 0
+fi
+
+echo "::error::Package policy violation: no Guix or Nix packaging found."
+echo
+echo "Estate policy is Guix primary / Nix fallback. Add one of:"
+echo "  guix.scm | manifest.scm | channels.scm | .guix-channel   (primary)"
+echo "  flake.nix | default.nix | shell.nix                      (fallback)"
+echo
+echo "Files inside .git/ node_modules/ deps/ .lake/ vendor/ do not count."
+exit 1