|
| 1 | +// SPDX-License-Identifier: MPL-2.0 |
| 2 | +// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath) |
| 3 | += Batch-Fix Outcome Categorisation |
| 4 | +:toc: |
| 5 | +:status: ACTIVE |
| 6 | +:version: 1.0.0 |
| 7 | + |
| 8 | +== Purpose |
| 9 | + |
| 10 | +Every batch-fix script in the hyperpolymath estate — any tool that walks an inventory and applies a per-row transform across many targets — MUST classify each row's outcome into one of the canonical four terms defined here. Stable vocabulary means a `cut -f4 | sort | uniq -c` summary is readable across any script, any topic, any session. |
| 11 | + |
| 12 | +== The Four Terms |
| 13 | + |
| 14 | +[cols="1,2,2,1",options="header"] |
| 15 | +|=== |
| 16 | +| Term | Meaning | Typical cause | Followup needed? |
| 17 | + |
| 18 | +| `ok` |
| 19 | +| Transform applied successfully. Target now matches the corrected state. |
| 20 | +| Happy path — pre-flight passed, mutation landed. |
| 21 | +| No. |
| 22 | + |
| 23 | +| `corrected_not_emitted` |
| 24 | +| Pre-flight safety check failed: the desired post-fix state isn't observable on the target. Script deliberately skipped to avoid creating a *different* broken state. |
| 25 | +| Workflow not currently producing the corrected name; corrected file not on disk; target version not reachable; upstream dependency not yet landed. |
| 26 | +| Yes — usually unblocked by external change (budget reset, workflow re-run, dependency merge). Re-run script later. |
| 27 | + |
| 28 | +| `skip_long_tail` |
| 29 | +| Row's input isn't in the script's known transform map. Single-instance edge cases the script intentionally leaves alone. |
| 30 | +| Phantom/inventory entry that doesn't match the high-frequency patterns the script was designed to handle. |
| 31 | +| Per-target human investigation, not batched. |
| 32 | + |
| 33 | +| `error_patching` |
| 34 | +| The mutation itself (API PUT/PATCH, file write, etc.) returned non-zero. Pre-flight passed but the write didn't land. |
| 35 | +| Auth scope mismatch, body shape rejection, transient network, idempotency artefact (target already fixed by a prior pass). |
| 36 | +| Yes — investigate the single failure (or confirm it's an idempotency-rerun artefact, in which case no action). |
| 37 | +|=== |
| 38 | + |
| 39 | +== Why these four |
| 40 | + |
| 41 | +The vocabulary disambiguates the two skip modes that otherwise look identical: |
| 42 | + |
| 43 | +* `corrected_not_emitted` — "could not safely apply" (script's pre-flight defended against a deferred external state) |
| 44 | +* `skip_long_tail` — "would not apply" (script chose not to handle this row's transform) |
| 45 | + |
| 46 | +A user reading `73 corrected_not_emitted, 25 skip_long_tail` knows immediately: |
| 47 | + |
| 48 | +* the 73 are *temporary* and re-running later will pick them up |
| 49 | +* the 25 are *permanent* parked entries needing per-target review |
| 50 | + |
| 51 | +Conflating them under a single `skipped` would erase that distinction. |
| 52 | + |
| 53 | +Separately, `error_patching` keeps script/auth defects visible distinct from environmental skips. A high `error_patching` count means fix-the-script; a high `corrected_not_emitted` count means fix-the-environment or wait. |
| 54 | + |
| 55 | +== Required output format |
| 56 | + |
| 57 | +Batch-fix scripts MUST emit a TSV with at minimum these columns: |
| 58 | + |
| 59 | +* `target` (repo, file path, package name — whatever the batch operates over) |
| 60 | +* `input_key` (the inventory entry being processed) |
| 61 | +* `source` (where the transform is applied — `classic`, `ruleset:<id>`, `file`, `api`, `-` for skips) |
| 62 | +* `outcome` (exactly one of the four canonical strings above) |
| 63 | +* `corrected_value` (the post-fix value, or `-` for long-tail skips) |
| 64 | + |
| 65 | +The `outcome` column MUST be column 4 by convention so the canonical summary command works without flag tweaking: |
| 66 | + |
| 67 | +[source,bash] |
| 68 | +---- |
| 69 | +tail -n +2 results.tsv | cut -f4 | sort | uniq -c | sort -rn |
| 70 | +---- |
| 71 | + |
| 72 | +== Adding new outcome shapes |
| 73 | + |
| 74 | +If a new outcome category genuinely doesn't fit one of the four (e.g. `partial_apply`, `requires_consent`), add it *here* in a follow-up edit to this spec before forking the vocabulary per-script. Per-script synonyms (`success`, `failed`, `skipped`, `done`) are forbidden — they erode the cross-topic readability the spec exists to preserve. |
| 75 | + |
| 76 | +== Canonical example |
| 77 | + |
| 78 | +The 2026-05-28 BP008 phantom-required-context batch fix (`bp008_fix.sh` against `bp008_phantoms_2026-05-28.tsv`, 195 rows) produced: |
| 79 | + |
| 80 | +|=== |
| 81 | +| Outcome | Count | Interpretation |
| 82 | +| `ok` | 71 | Phantom renamed; auto-merge unblocked |
| 83 | +| `corrected_not_emitted` | 96 | Workflow not currently emitting corrected name (Actions budget exhausted); re-run after billing reset will recover most |
| 84 | +| `skip_long_tail` | 27 | Phantom not in 5-pattern rename map; per-repo investigation pending |
| 85 | +| `error_patching` | 1 | Spurious — same target was fixed by an earlier pilot pass (idempotency artefact) |
| 86 | +|=== |
| 87 | + |
| 88 | +Read aloud: "71 fixed, 96 parked pending budget reset, 27 long-tail, 0 real failures." The four-term split makes that decomposition immediate. |
| 89 | + |
| 90 | +== Related |
| 91 | + |
| 92 | +* `claude-watch` (estate dashboard tool) renders the canonical summary for any `*_results.tsv` it discovers under `/tmp/`. Scripts that emit the canonical format get free real-time monitoring. |
| 93 | +* `hypatia.Rules.AdminMergeEligibility` (AM010) and `hypatia.Rules.BranchProtection` (BP008) are the upstream detectors whose findings drive the BP008 canonical-example batch. |
0 commit comments