|
| 1 | +<!-- |
| 2 | +SPDX-License-Identifier: PMPL-1.0-or-later |
| 3 | +(MPL-2.0 is automatic legal fallback until PMPL is formally recognised) |
| 4 | +--> |
| 5 | + |
| 6 | +# Briefing Haiku (4.5 and later) |
| 7 | + |
| 8 | +Haiku is the cheapest, fastest tier. Its role in the estate is **bulk |
| 9 | +mechanical work whose output can be mechanically verified**. Haiku cost is |
| 10 | +roughly 1/15 of Opus per token, so delegating a large-context sweep to Haiku |
| 11 | +and having Opus read only the distilled result is typically **5–10× cheaper |
| 12 | +end-to-end** than letting Opus do the sweep directly. |
| 13 | + |
| 14 | +The catch: Haiku confabulates summaries. Any task that requires Haiku to |
| 15 | +characterise, synthesise, or judge is at high risk of being wrong in a way |
| 16 | +that looks plausible. Brief Haiku so that its output is a table of raw |
| 17 | +observations, not a verdict. |
| 18 | + |
| 19 | +## Fits |
| 20 | + |
| 21 | +- Markdown reformatting from a drafted source (not original prose) |
| 22 | +- Running existing scripts/tools and tabulating their output |
| 23 | +- Per-file data *transformation* requiring per-file reasoning (not just |
| 24 | + pattern match) — e.g. "read each README and extract the first paragraph + |
| 25 | + primary-language tag" |
| 26 | + |
| 27 | +## Does NOT fit (empirically, 2026-04-17 trial) |
| 28 | + |
| 29 | +The following task shapes look like good Haiku fits but are not — |
| 30 | +do them yourself with Bash or Grep: |
| 31 | + |
| 32 | +- **File-presence checks across the estate** ("which repos have X? which |
| 33 | + lack Y?"). A Bash loop is 1–2 orders of magnitude faster, correct by |
| 34 | + construction, and zero tokens. Haiku observed confabulating an aggregate |
| 35 | + summary ("vast majority have EXPLAINME") that was off by 40 percentage |
| 36 | + points. |
| 37 | +- **Pattern greps across the estate for a fixed literal or simple regex** |
| 38 | + (`believe_me`, `postulate`, `uses:`, SPDX headers). A few parallel Grep |
| 39 | + calls from the supervisor cover the same ground. Haiku observed missing |
| 40 | + 54+ `believe_me` occurrences it should have found, and listing rows from |
| 41 | + paths its own Scan notes claimed to have excluded. |
| 42 | +- **Checklist verification ("does repo Y contain Z")** — same reason. |
| 43 | + |
| 44 | +The failure pattern is: Haiku's glob/type filters are unreliable, and its |
| 45 | +aggregation step invents plausible-looking summaries that contradict the |
| 46 | +underlying data. Both failure modes require verification, and verification |
| 47 | +is itself the full Grep/Bash work, defeating the cost saving. |
| 48 | + |
| 49 | +## Does not fit |
| 50 | + |
| 51 | +- Proof work (Idris2, Agda, Lean, Coq, TLA+, L4, Iz, F\*) |
| 52 | +- Compiler / type-system / grammar decisions |
| 53 | +- Novel architectural choices or refactors with semantic depth |
| 54 | +- Prose review, paper review, or writing original narrative |
| 55 | +- Security judgement calls on dual-use tools |
| 56 | +- Any task where "is this a drift case?" requires reading context Haiku hasn't |
| 57 | + been given |
| 58 | +- Fuzzy tasks that require mid-flight replanning |
| 59 | + |
| 60 | +## Cost model |
| 61 | + |
| 62 | +Per-call spend splits between supervisor (Opus/Sonnet) and Haiku roughly as: |
| 63 | + |
| 64 | +- Supervisor: prompt (~500–2000 tokens) + returned report (~500–3000 tokens) |
| 65 | +- Haiku: prompt + all tool results + intermediate reasoning + final report |
| 66 | + |
| 67 | +The supervisor does not see Haiku's intermediate tool outputs, which is the |
| 68 | +point: the bulky context stays in the cheap tier. |
| 69 | + |
| 70 | +Not cheaper when: |
| 71 | + |
| 72 | +- The task is under ~5 file reads (overhead dominates) |
| 73 | +- You would need to intervene every few steps (chat loop wrecks the arithmetic) |
| 74 | +- Haiku's output is unreliable enough that you re-do the work (now paid twice) |
| 75 | + |
| 76 | +## Prompt scaffold |
| 77 | + |
| 78 | +```markdown |
| 79 | +You are doing a **read-only audit** across <scope> at <path>. **Do not modify |
| 80 | +any files. Use only Glob, Grep, Read.** |
| 81 | + |
| 82 | +# Critical output rule |
| 83 | + |
| 84 | +Your job is to produce **raw per-file findings**. Do NOT summarise, aggregate, |
| 85 | +or characterise the data. Do NOT say things like "homogeneous", "stable", |
| 86 | +"acceptable", "no critical drift". Just list findings. The supervising agent |
| 87 | +will aggregate. |
| 88 | + |
| 89 | +If you are tempted to write a paragraph of prose characterising the estate, |
| 90 | +delete it. Every row in your output must be a concrete observation tied to a |
| 91 | +specific file path. |
| 92 | + |
| 93 | +# Scope exclusions (mandatory) |
| 94 | + |
| 95 | +Skip any path containing any of these segments: |
| 96 | +- .git/ |
| 97 | +- node_modules/ |
| 98 | +- target/ |
| 99 | +- vendor/ |
| 100 | +- zig-cache/ / zig-out/ / .lake/ |
| 101 | +- _exploratory/ |
| 102 | +- hyperpolymath-archive/ |
| 103 | +- <task-specific archived repo names, e.g. boj-cartridges/polystack/> |
| 104 | +- <task-specific fixture/sample paths, e.g. .../linguist/samples/> |
| 105 | + |
| 106 | +# Task |
| 107 | + |
| 108 | +<One-paragraph statement of what to find. Include regexes, canonical values |
| 109 | +tables, or literal strings to match. No ambiguity about what counts as a hit.> |
| 110 | + |
| 111 | +# Output format — strict |
| 112 | + |
| 113 | +Return raw rows in the following exact markdown table schema: |
| 114 | + |
| 115 | +| file (relative to <path>) | <field 1> | <field 2> | ... | |
| 116 | + |
| 117 | +- One row per observation. No deduplication unless I say so. |
| 118 | +- If a file in scope has nothing to report, skip it (do not add an "OK" row). |
| 119 | +- If a file cannot be read, add a row: `ERROR: <reason>` for that file. |
| 120 | +- Hard cap: first N rows. If more, end with `M more rows truncated`. |
| 121 | +- If a category has zero files in scope: `(no files in scope)` under that heading. |
| 122 | + |
| 123 | +End with a `## Scan notes` section containing ONLY: |
| 124 | +- Total file counts per category |
| 125 | +- Paths you hit read errors on |
| 126 | +- Any exclusion you applied beyond the mandatory list |
| 127 | + |
| 128 | +# Hard rules |
| 129 | + |
| 130 | +- Read-only. No Edit, Write, Bash-that-mutates. |
| 131 | +- No find, cat, git. Glob + Grep + Read only. |
| 132 | +- **No synthesis, no summaries, no characterisations.** Raw per-file rows. |
| 133 | +- Do not write "no drift detected", "acceptable", "estate-wide", "excellent |
| 134 | + consistency", "no fragmentation". Just list data. |
| 135 | +- Do not propose fixes. Do not edit. Report only. |
| 136 | +- Cap total output at ~1200 lines. If a category would exceed its share of |
| 137 | + that budget, truncate within it and say so. |
| 138 | +``` |
| 139 | + |
| 140 | +## Hard rules to include verbatim |
| 141 | + |
| 142 | +Every Haiku brief must include these, because Haiku cannot read memory, global |
| 143 | +CLAUDE.md, or prior conversation: |
| 144 | + |
| 145 | +1. **Read-only.** No Edit, Write, or state-mutating Bash. |
| 146 | +2. **Tooling restriction.** "Use only Glob, Grep, Read." (Block `find`, `cat`, |
| 147 | + shell loops.) Haiku tends to reach for Bash when it should use structured |
| 148 | + tools. |
| 149 | +3. **Raw rows, no synthesis.** Named anti-patterns: "homogeneous", "acceptable", |
| 150 | + "no drift detected". Listing them explicitly suppresses the reflex. |
| 151 | +4. **Explicit exclusion list.** Archived repos (e.g. `polystack`), monorepo |
| 152 | + sub-paths that shouldn't be treated as separate repos, vendor/build trees, |
| 153 | + fixture/sample trees. Haiku has no access to the estate's archived-repos |
| 154 | + memory. |
| 155 | +5. **Output schema.** Exact column headers. Haiku will otherwise invent its own |
| 156 | + shape for each category. |
| 157 | +6. **Truncation protocol.** Hard cap + "N more rows truncated" footer, not |
| 158 | + prose like "many more observed". |
| 159 | +7. **`SAFE TO CLOSE` / session-close markers** are not required from Haiku — |
| 160 | + those are for Opus/Claude sessions. |
| 161 | + |
| 162 | +## Trust level & verification |
| 163 | + |
| 164 | +**Default trust: low.** Spot-check before acting. |
| 165 | + |
| 166 | +Before using Haiku's output: |
| 167 | + |
| 168 | +- Verify at least one finding per category by running the equivalent Grep or |
| 169 | + Read yourself. |
| 170 | +- If Haiku produced a prose summary ("X is homogeneous / stable / acceptable") |
| 171 | + despite the brief telling it not to, assume that summary is wrong and |
| 172 | + discard it. Use only the raw rows. |
| 173 | +- If row counts per category look suspiciously round or suspiciously low, |
| 174 | + verify the totals with an independent count. |
| 175 | + |
| 176 | +Empirically observed failure modes: |
| 177 | + |
| 178 | +- **Summary confabulation.** Haiku aggregated 14 Zig min-version values as "all |
| 179 | + 0.15.2" when the raw data contained five distinct versions. Mitigation: ask |
| 180 | + for raw rows only; aggregate yourself. |
| 181 | +- **Sampling without saying so.** Haiku will quietly "sample" 6 of 130 files |
| 182 | + and present them as the data. Mitigation: insist on one row per file, or |
| 183 | + ask for explicit `N of M scanned` counts in Scan notes. |
| 184 | +- **Prose truncation of a table.** Haiku will write "150+ more findings |
| 185 | + detected" rather than filling the table to its cap. Mitigation: make the |
| 186 | + cap high enough that truncation is rare, and require the literal footer |
| 187 | + `M more rows truncated`. |
| 188 | + |
| 189 | +## Anti-patterns in briefing |
| 190 | + |
| 191 | +- Asking "is there drift?" — that is a judgement. Ask "list every distinct |
| 192 | + value observed". |
| 193 | +- Leaving exclusions implicit — Haiku does not know that `polystack` is |
| 194 | + archived. |
| 195 | +- Asking for fixes — Haiku will attempt them, unevenly. Split report and fix |
| 196 | + into separate sessions. |
| 197 | +- Asking Haiku to decide which files are "relevant" — specify the glob. |
| 198 | + |
| 199 | +## Parallelism |
| 200 | + |
| 201 | +Hard cap in this estate: **3 parallel subagents**, **2 parallel Bash**. If the |
| 202 | +scope is large, prefer one wide Haiku pass over three narrow ones — the |
| 203 | +token-overhead of three briefs plus three summary reports usually exceeds the |
| 204 | +wall-clock savings. |
| 205 | + |
| 206 | +## License |
| 207 | + |
| 208 | +PMPL-1.0-or-later (MPL-2.0 automatic legal fallback). |
0 commit comments