Skip to content

Commit 3cf9591

Browse files
hyperpolymathhyperpolymath
committed
docs(audits): workflow convergence campaign 2026-05-26
5-candidate workflow-reusable extraction campaign. 4 reusables filed (#187 mirror, #190 secret-scanner, #192 codeql, #193 hypatia-scan) + classifier tooling (#194). Scorecard-reusable recommended as #195 to close the set. Documents the nested-path methodology gotcha (3-layer GitHub Code Search undercount + inert-workflow caveat) that corrects LOC-retired estimates ~3x upward and clarifies that nested copies are single-source-of-truth cleanup, not security-gap closure. Also reconciles divergence vs the parallel-session ranking on secret-scanner / codeql / scorecard — divergence root-caused to sample-drift vs full-pagination methodology.
1 parent 431adbb commit 3cf9591

1 file changed

Lines changed: 167 additions & 0 deletions

File tree

‎docs/audits/workflow-convergence-campaign-2026-05-26.md‎

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
# Workflow Convergence Campaign — 2026-05-26
2+
3+
> Generated by: estate-wide audit of drifting per-repo workflow templates
4+
> Scope: 5 candidate templates ranked by drift × deployments × feature-variance
5+
> Outcome: 5 reusable-workflow PRs filed in this repo; classifier tooling shipped; nested-path methodology gotcha documented for future campaigns
6+
7+
## Summary
8+
9+
This campaign extracted 4 (so far) drifting per-repo workflow templates into reusable workflows hosted in this repo, plus shipped the classifier tooling that planned the wrapper sweeps. A 5th template (`scorecard.yml`) is surveyed and recommended for a follow-up PR.
10+
11+
The campaign also surfaced a **methodology gotcha** that affects every future drift survey in this estate: `gh api /search/code` queries undercount monorepo-nested workflow files in two compounding ways, and the standard `path:.github/workflows` filter excludes them entirely. The corrected counts revise the LOC-retirement estimate upward by ~3× across affected templates.
12+
13+
## Filed PRs
14+
15+
| PR | Template | Estate deploys (top-level) | Reusable LOC | Top SHA share |
16+
|---|---|---|---|---|
17+
| [#187](https://github.com/hyperpolymath/standards/pull/187) | `mirror-reusable.yml` | 289 | 165 | 76% |
18+
| [#190](https://github.com/hyperpolymath/standards/pull/190) | `secret-scanner-reusable.yml` | 281 | 159 | 69% (across top 4 SHAs) |
19+
| [#192](https://github.com/hyperpolymath/standards/pull/192) | `codeql-reusable.yml` | 263 | 96 | 83% single-language |
20+
| [#193](https://github.com/hyperpolymath/standards/pull/193) | `hypatia-scan-reusable.yml` | 255 | 459 | 83.5% top-5 |
21+
| [#194](https://github.com/hyperpolymath/standards/pull/194) | sweep-classifier scripts | — | — | tooling |
22+
23+
All 5 PRs have auto-merge enabled (per the [auto-merge-on-all-PRs standing policy](https://github.com/hyperpolymath/.github)). The wrapper sweep does NOT fire automatically — each template is owner-gated post-merge.
24+
25+
## Recommended next PR
26+
27+
| PR | Template | Estate deploys | Why now |
28+
|---|---|---|---|
29+
| #195 (recommended) | `scorecard-reusable.yml` | 258 | Highest dominant-cluster (top SHA = 38.8%); ZERO feature variance; all drift mechanical (SPDX header + action-SHA-pin churn); smallest canonical (41 lines); no nested-path complication. |
30+
31+
## Ranking methodology
32+
33+
For each candidate, three signals were combined:
34+
35+
1. **Drift** = (unique blob SHAs / total deployments). Lower = more homogeneous.
36+
2. **Deployments** = number of repos carrying the template. Higher = more leverage.
37+
3. **Feature variance** = whether the SHAs differ in job-set / step-set / language matrix (real customization) or only in SPDX-header / action-pin / whitespace (mechanical lag).
38+
39+
A reusable is viable when (2) is high AND (3) is low. (1) tells you how much wrapper-sweep work the rollout needs.
40+
41+
| Template | Deploys | Drift | Feature variance | Reusable viable |
42+
|---|---|---|---|---|
43+
| `hypatia-scan.yml` | 255 | 11.8% | ZERO | ✅ (filed #193) |
44+
| `scorecard.yml` | 258 | 17.8% | ZERO | ✅ (recommended #195) |
45+
| `secret-scanner.yml` | 281 | 19% | LOW (job-set homogeneous; force-propagates `shell-secrets` guardrail) | ✅ (filed #190) |
46+
| `mirror.yml` | 289 | 24% | LOW (job-set homogeneous; same 7 forges across top SHAs) | ✅ (filed #187) |
47+
| `codeql.yml` | 263 | 26% | MEDIUM (language matrix variance — needed `language` + `build-mode` inputs) | ✅ (filed #192) |
48+
49+
## The nested-path methodology gotcha
50+
51+
### Layer 1: path-prefix filter excludes nested workflows
52+
53+
`gh api /search/code` with `path:.github/workflows` matches the path **PREFIX**, so:
54+
- `.github/workflows/codeql.yml` ✓ matches
55+
- `developer-ecosystem/asdf-augmenters/.github/workflows/codeql.yml` ✗ does NOT match
56+
57+
Removing the `path:` filter and running `filename:codeql.yml org:hyperpolymath` exposes nested copies inside monorepos like `developer-ecosystem`, `ssg-collection`, `ambientops`, `standards`, `julia-ecosystem`, `asdf-tool-plugins`, etc.
58+
59+
### Layer 2: even the broad query is org-scope-truncated
60+
61+
GitHub Code Search's org-scoped result set is capped well below the true file count. For `codeql.yml`:
62+
63+
| Monorepo | Broad query saw | Per-repo query (truth) | Missed |
64+
|---|---:|---:|---:|
65+
| developer-ecosystem | 41 | 170 | 129 |
66+
| asdf-tool-plugins | 11 | 111 | 100 |
67+
| ssg-collection | 39 | 68 | 29 |
68+
| standards | 26 | 39 | 13 |
69+
| ambientops | 12 | 29 | 17 |
70+
| julia-ecosystem | 11 | 20 | 9 |
71+
| **Top-6 total** | **140** | **437** | **297** |
72+
73+
Broad-query undercount factor: **~2.6×** inside heavily-nested monorepos. Per-repo queries (`gh api repos/<org>/<repo>/git/trees/HEAD?recursive=1 --jq '.tree[] | select(.path | endswith("/<file>.yml"))'`) are the only reliable source of truth for monorepo-nested workflow files.
74+
75+
### Layer 3: nested workflows are inert
76+
77+
**GitHub Actions only runs workflows from the repo-root `.github/workflows/` directory.** Workflows under any other path (e.g. `a2ml/bindings/deno/.github/workflows/secret-scanner.yml` inside the `standards` repo) are **never triggered**.
78+
79+
This means nested copies are one of:
80+
81+
1. **Vendored templates** — maintained as a snapshot of what a sub-package's own workflow would be if it were extracted into its own GitHub repo. Read-only intent; not load-bearing for the sub-package's CI.
82+
2. **Stale leftover** — surviving artifacts of a previous monorepo merger; pure dead code.
83+
84+
**Implication:** the security-gap argument that motivated the secret-scanner reusable (forcing `shell-secrets` propagation post-Cloudflare-leak) does NOT extend to nested copies. The ~282 nested `secret-scanner.yml` files do not represent missing-guardrail attack surface — they don't run. The wrapper sweep on nested copies is **single-source-of-truth cleanup**, not security-hardening.
85+
86+
### Corrected estate counts
87+
88+
| Template | Top-level | Nested (true ≥) | Total wrapper sites |
89+
|---|---:|---:|---:|
90+
| `hypatia-scan.yml` | 255 | 449 | 704 |
91+
| `mirror.yml` | 289 | 133 | 422 |
92+
| `secret-scanner.yml` | 281 | 282 | 563 |
93+
| `codeql.yml` | 263 | ~518 (per-repo verified for top-10 monorepos) | ~781 |
94+
| `scorecard.yml` | 258 | 0 | 258 |
95+
96+
`scorecard.yml` is the only template with no monorepo nesting — making it the cleanest sweep target if/when #195 lands.
97+
98+
## Classifier tooling pattern (`scripts/sweep-classifiers/`)
99+
100+
Each template has a `classify-<template>.sh` script (committed in [#194](https://github.com/hyperpolymath/standards/pull/194)) that:
101+
102+
1. Reads paginated `gh api /search/code` JSON into a `(repo, sha)` table.
103+
2. Fetches each unique blob ONCE (cached in `$BLOBS_DIR`) — bandwidth-efficient.
104+
3. Classifies each blob by job-set / line-count / language-matrix / SPDX-header lag.
105+
4. Emits a per-repo TSV with `<repo> <class> <evidence>` for sweep batching.
106+
107+
Pattern: the classifier output drives a 3-wave sweep (mechanical bulk → propagation-lag bulk → per-repo review). Wave-1 PRs are auto-generated wrappers pinned to the merged-reusable HEAD SHA.
108+
109+
**Outstanding tooling gap** (filed as follow-up): classifiers currently emit one row per repo, not per `(repo, path)` tuple. For monorepo-nested copies, the classifiers need to ingest the per-repo recursive-tree listing and emit one wrapper site per nested copy. This is a small change to each script's input loop.
110+
111+
## Per-template specifics
112+
113+
### `mirror.yml` (PR #187)
114+
115+
7 forge jobs (gitlab, bitbucket, codeberg, sourcehut, disroot, gitea, radicle), each gated on `vars.<FORGE>_MIRROR_ENABLED == 'true'`. Only input: `runs-on`. Caller MUST use `secrets: inherit` (jobs reference forge-specific secrets). Wrapper sweep is mostly mechanical; the 22 NEEDS_REVIEW cases in the classifier output are slim 2-3-forge variants (older repos pre-7-forge expansion) or NONE-stub placeholders.
116+
117+
### `secret-scanner.yml` (PR #190)
118+
119+
4 jobs: `trufflehog`, `gitleaks`, `rust-secrets`, `shell-secrets`. Self-conditioning — `rust-secrets` exits early on no `Cargo.toml`, `shell-secrets` no-ops without `.sh`/`.bash`. **Force-propagates** the `shell-secrets` job (post-Cloudflare-leak guardrail) to the estate, where it currently exists in only 1 of 281 top-level deployments (this repo itself). The 282 nested copies do NOT close additional security gaps (Layer 3 caveat) but DO consolidate the source of truth.
120+
121+
The full-pagination data corrected an initial 100-repo sample's "55% drift" reading to a true **19% drift** with 100% job-set homogeneity. The parallel-session ranking that said "skip secret-scanner" was based on the sample drift figure and missed the homogeneity finding.
122+
123+
### `codeql.yml` (PR #192)
124+
125+
Two inputs: `language` (default `javascript-typescript`) and `build-mode` (default `none`). Single-language single-job design — multi-language wrappers invoke the reusable per-language. Language matrix distribution across 263 top-level deployments:
126+
127+
- 84.8% `javascript-typescript`-only
128+
- 8.4% `actions`-only
129+
- ~3.4% multi-language
130+
- ~3.4% NONE-matrix (needs per-repo review)
131+
132+
The campaign's most complex reusable — but still ~93% mechanical sweep.
133+
134+
### `hypatia-scan.yml` (PR #193)
135+
136+
Byte-identical to the canonical 416-line `scan` job. Single `runs-on` input. Caller MUST grant `security-events: write` + `pull-requests: write` permissions AND use `secrets: inherit`. The 416-line canonical × 704 total estate sites = ~280k LOC retired (the largest leverage of the 5 campaigns). 30 unique blob SHAs / 11.8% drift / 100% single-`scan`-job homogeneity. All drift is pure propagation lag (older slimmer versions of the same body).
137+
138+
### `scorecard.yml` (recommended #195)
139+
140+
258 deployments, 17.8% drift, **100% mechanical drift** — every variant has one `analysis` job with `checkout → ossf/scorecard-action → upload-sarif`. Same single `scorecard-action` SHA pin across all variants; differences are SPDX header (`PMPL-1.0` / `PMPL-1.0-or-later` / `MPL-2.0`), `upload-sarif` SHA pin, `permissions: read-all` vs `contents: read`, and presence/absence of the estate concurrency-guardrail block. 41-line canonical → ~8k LOC retired top-level only. Zero nested copies, making it the cleanest sweep candidate.
141+
142+
## Reconciliation with the parallel-session ranking
143+
144+
A parallel session ([[project_foundational_workflow_survey_2026_05_26]] in agent memory) ranked the 5 candidates and concluded "pursue hypatia-scan + mirror; skip codeql / secret-scanner / scorecard." The recommendation set differs from this campaign's outcome on three templates:
145+
146+
| Template | Parallel call | This campaign | Reason for divergence |
147+
|---|---|---|---|
148+
| `secret-scanner.yml` | SKIP | PURSUE | Parallel based on 55% sample drift; full pagination → 19% drift, 100% homogeneity. The skip call missed the shell-secrets propagation gap. |
149+
| `codeql.yml` | SKIP | PURSUE | Real (but contained) language-matrix variance — 85% are single-language `javascript-typescript`, making the 2-input reusable dead-simple for the dominant case. |
150+
| `scorecard.yml` | SKIP | PURSUE | Same shape as secret-scanner mistake: 18% sample drift looks like real customization but is ALL mechanical (action-SHA churn + SPDX header drift). |
151+
152+
The agreement between sessions was strong on `hypatia-scan.yml` (both: pursue, HIGH homogeneity) and `mirror.yml` (both: pursue, ~76% top-SHA dominance). The divergence comes from the parallel session's reliance on sample-drift estimates without full pagination, where my campaign full-paginated and blob-fetched each candidate.
153+
154+
## Standing follow-ups
155+
156+
1. **Owner-review on the 5 PRs** (#187/#190/#192/#193/#194) — auto-merge enabled; CI-green will auto-land.
157+
2. **File #195** (scorecard-reusable) to close the 5-candidate convergence set.
158+
3. **Per-repo recursive-tree classifier inputs** — extend `scripts/sweep-classifiers/` to emit one row per `(repo, nested-path)` tuple, not just per repo. Required before any wrapper sweep that targets nested copies.
159+
4. **Wrapper sweep firing** is owner-gated per template. The 82-PR rust-ci sweep is the upper bound the user has signalled comfort with; mirror's 267-PR sweep would need explicit per-batch sign-off.
160+
5. **Nested-copy disposition decision** — for each monorepo (developer-ecosystem, ssg-collection, ambientops, etc.), are the nested workflows vendored templates (keep, sync to reusable) or stale leftover (delete)? This is a per-monorepo judgement call; the campaign defers it.
161+
162+
## Cross-references
163+
164+
- Prior pattern: PR [#168 language-policy convergence](https://github.com/hyperpolymath/standards/pull/168) — first instance of "subsume legacy workflow into governance-reusable" pattern; merged 2026-05-26.
165+
- Prior pattern: PR [#174 rust-ci + elixir-ci reusables](https://github.com/hyperpolymath/standards/pull/174) — extended the pattern; 82-PR wrapper sweep filed downstream (`project_rust_ci_wrapper_sweep_2026_05_26`).
166+
- Estate-wide language policy: [`.claude/CLAUDE.md`](../../.claude/CLAUDE.md) — Python/V-lang/ATS2/ReScript/TypeScript bans enforced by `governance-reusable.yml`.
167+
- Security context: the `shell-secrets` job force-propagation in #190 is direct mitigation for the [unrotated Cloudflare API token leak](#) (memory `project_cloudflare_token_leak_2026_05_21`).

0 commit comments

Comments
 (0)