|
| 1 | +# Directive: insertion-normalization — identity scheme, pin safety, persistence, and fixture strategy |
| 2 | + |
| 3 | +Requested by the reviewer on PR #272 (review round 1, 2026-07-31) before the |
| 4 | +next code-review pass. Scope: the four blockers' design questions, answered |
| 5 | +once here so #273/#276/#278/#281 inherit a settled scheme instead of |
| 6 | +re-litigating it per diff. Written against measurement, not argument; every |
| 7 | +load-bearing number cites its run. |
| 8 | + |
| 9 | +## Goal / Background |
| 10 | + |
| 11 | +CC re-serializes old `<system-reminder>` blocks mid-history (splice, merge, |
| 12 | +migration); each hit edits a settled message and re-bills everything after it |
| 13 | +(~40 kB per unmitigated hit; anthropics/claude-code#76606). The extension |
| 14 | +keeps a canonical per-conversation model keyed by content identity and pins |
| 15 | +volatile blocks to their first-seen serialization. The open design question |
| 16 | +(blocker 2): pin-mode identity *deliberately excludes* volatile blocks from |
| 17 | +the hash — that exclusion is the mechanism — so the extension cannot |
| 18 | +structurally distinguish CC re-serializing a reminder (pin: correct) from CC |
| 19 | +*changing its bytes* (pin would forward stale text: a fidelity risk the |
| 20 | +reviewer reproduced synthetically). |
| 21 | + |
| 22 | +## Non-Functional Requirements |
| 23 | + |
| 24 | +- **Size/complexity budget:** extension stays ~1,100 LOC (no new modules; |
| 25 | + `message-hash.mjs` remains the one shared primitive). Fixture budget is the |
| 26 | + real change: target is a ≥10× reduction of the 432 KB harvested fixture |
| 27 | + (see Fixture strategy); an implementation landing materially above these |
| 28 | + marks is a review flag. |
| 29 | +- **Threat model:** three surfaces. (1) Wire: the extension mutates request |
| 30 | + bodies; forwarding bytes CC did not send is the corruption class — bounded |
| 31 | + below by the allowlist + monitor design. (2) Disk: canonical state persists |
| 32 | + first-seen message bytes (`entry.m`, structurally required for re-serving); |
| 33 | + all conversation-derived writes land mode 0600 via a shared write-owner-only |
| 34 | + primitive (series-wide fix, 27 write sites). (3) Public repo: fixtures are |
| 35 | + conversation-derived; identifiers and content bytes must never be |
| 36 | + committable — enforced mechanically (absence scan at test time and at |
| 37 | + pre-push), not by care. |
| 38 | +- **Maintainability:** no new abstractions. Census/replay import the |
| 39 | + extension's own identity helpers rather than restating them (export |
| 40 | + `identityKey`, `unwrapVolatileText`, `pinnedForwardForm`, `hasCacheControl`; |
| 41 | + small proxy-side change riding the next deploy boundary). |
| 42 | +- **Performance/reliability:** unchanged; per-request work stays O(messages). |
| 43 | +- **Load-bearing?** **Yes** — mutates request bodies on the wire, persists |
| 44 | + conversation-derived state, changes forwarded history shape. Requires human |
| 45 | + (Chris) review before merge per repo policy. |
| 46 | + |
| 47 | +## Identity scheme (what it is, and what it cannot see) |
| 48 | + |
| 49 | +Identity = content hash + occurrence ordinal (`message-hash.mjs`), |
| 50 | +position-independent; repeated identical reminders stay distinct (a real |
| 51 | +history carried the same reminder 44×). In pin mode, volatile blocks are |
| 52 | +excluded from the hash and the first-seen serialization is stored and |
| 53 | +re-served on later matches. Consequence, stated plainly: a change *inside* a |
| 54 | +volatile block of a still-matching message is invisible to identity. Whether |
| 55 | +that is acceptable is an empirical question about CC's actual behavior. |
| 56 | + |
| 57 | +## The measurement the design rests on |
| 58 | + |
| 59 | +Full method and per-entry rows: `docs/code-reviews/` |
| 60 | +`blocker2-volatile-change-measurement.md` (census extension, commit 97867f3; |
| 61 | +red-first both directions; instrument imports the extension's own |
| 62 | +`computePinnedIdentities`/`isVolatileBlock`). Corpus: 36/36 captures, 7.1 GB, |
| 63 | +2026-07-28..08-01, 196 conversations, 11,074 same-conversation pairs. |
| 64 | +First-seen comparison per pinned identity — a strict superset of the |
| 65 | +comparisons the live pin performs (it can over-report change, never |
| 66 | +under-report). |
| 67 | + |
| 68 | +| CHANGED sub-kind (of 23,328 pin-rewritten comparisons) | occurrences | entries | |
| 69 | +|---|---|---| |
| 70 | +| VANISHED — every first-seen reminder gone | 11,478 | 80 | |
| 71 | +| REDUCED — a subset survives | 284 | 1 | |
| 72 | +| **IN-PLACE-TEXT — a reminder's text replaced** | **0** | **0** | |
| 73 | +| APPEARED / AUGMENTED | 0 | 0 | |
| 74 | + |
| 75 | +The zero is informative, not vacuous: the corpus carries 119 distinct |
| 76 | +reminder texts across 10 kinds (5 kinds with >1 text — one with 75), so |
| 77 | +in-place variation is possible by construction; it lives across messages, |
| 78 | +never inside a settled one. Removals (VANISHED/REDUCED) are the #76606 flip |
| 79 | +the pin exists to absorb: the model re-reads text it already consumed — |
| 80 | +information is repeated, never contradicted. Named boundaries: first-seen |
| 81 | +scope is per capture file (over-counts staleness across rotations — |
| 82 | +conservative in the right direction); the corpus is live (totals drifted |
| 83 | +across three runs; the zero held in all three); CC-version spread is not |
| 84 | +recoverable from captures. |
| 85 | + |
| 86 | +## Decision: evidenced allowlist, monitored — not fail-closed re-pin |
| 87 | + |
| 88 | +Pinning stays allowlist-scoped to blocks matching the volatile wrap contract, |
| 89 | +justified by the measured absence of the harmful class. The claim is kept |
| 90 | +**monitored, not assumed**: the census now computes IN-PLACE-TEXT on every |
| 91 | +daily gate sweep, so the first real occurrence surfaces mechanically. That |
| 92 | +occurrence is the build trigger for the fallback already designed: |
| 93 | +fail-closed re-pin — store the new bytes, honest reset of that boundary only. |
| 94 | + |
| 95 | +**Failure mode when the canon is wrong** (the reviewer's ask, answered |
| 96 | +per class): in-place change — zero observed; if one ever occurs it is served |
| 97 | +stale once per request until the next sweep flags it (bounded by the |
| 98 | +monitor's cadence), then the fallback ships. Removal — designed absorption, |
| 99 | +safety argument above. Structural mismatch (compaction, genuine rewrite) — |
| 100 | +honest reset, pins survive, order assumptions do not (unchanged behavior). |
| 101 | + |
| 102 | +## Persistence scope (answered once for the series) |
| 103 | + |
| 104 | +On disk: canonical entries including first-seen bytes (`entry.m` — raw bytes |
| 105 | +are structurally required; a hash cannot re-serve), event logs, and telemetry |
| 106 | +with stable session identifiers. All conversation-derived writes: owner-only |
| 107 | +0600 (mode at create + lazy chmod; Node's `mode` option is create-only). |
| 108 | +Retention: state lives under the state dir per conversation; an honest reset |
| 109 | +clears order state, pins expire with their conversation's state. Where bytes |
| 110 | +are not structurally required, hashes are stored instead. This answer carries |
| 111 | +to #275 (request capture) and #280 (prefix-diff snapshots) unchanged. |
| 112 | + |
| 113 | +## Fixture strategy (the 432 KB question) |
| 114 | + |
| 115 | +Principles first, then the concrete cut: |
| 116 | + |
| 117 | +- **Synthesized by default.** Constructing a fixture is additive — nothing |
| 118 | + identifying exists unless placed. Harvested-and-scrubbed is the exception, |
| 119 | + justified per fixture by real-pair evidence value (a class only real |
| 120 | + traffic teaches), and committable only with the absence scan green. |
| 121 | +- **Sanitization is checked, never claimed.** The scrubber one-way tokenizes |
| 122 | + content (`t_<sha12>_<len>` per paragraph preserving `"\n\n"` join |
| 123 | + relations; `data_<sha10>` for binary), replaces keys/sids with sha-derived |
| 124 | + tokens, rebases timestamps to a fixed epoch keeping intra-fixture deltas, |
| 125 | + and names files by token, not session UUID. A mechanical absence test |
| 126 | + (base64 runs, UUIDs, live timestamps, raw strings, signatures) went |
| 127 | + red-first 9/20 on the old fixtures and now guards the whole fixtures |
| 128 | + directory at test time and the push boundary via a pre-push scan. |
| 129 | +- **Minimum record set, answered honestly:** the current 54-record fixture is |
| 130 | + the harvester's range dump, not a measured minimum. The suppression pair |
| 131 | + needs its conversation prefix only to establish pin state; the cut is a |
| 132 | + pinned pair plus a minimal pin-establishing prefix, minified (single-line |
| 133 | + JSON — assertions read it, people don't). Acceptance for any cut: replayed |
| 134 | + classifier verdicts identical to the full-range fixture's, byte-relation |
| 135 | + assertions still exercised. Expected result ≥10× smaller; if the measured |
| 136 | + minimum genuinely needs more history, the fixture says so in a header |
| 137 | + comment with the number. |
| 138 | +- **body/headers retention:** full request bodies are not required by the |
| 139 | + suppression tests; tokenized structural shape preserves every asserted |
| 140 | + relation (join equality survives tokenization by construction). The boot |
| 141 | + record's gate-set dump is dropped from fixtures — same class as the #275 |
| 142 | + env-dump finding. |
| 143 | + |
| 144 | +## Public-repo hygiene class (proposed upstream adoption) |
| 145 | + |
| 146 | +The repo's hygiene section covers origin-server identifiers. This incident |
| 147 | +adds a second class with the same non-negotiable property (public git history |
| 148 | +is unscrubbable): **conversation/capture-derived data** — session/conversation |
| 149 | +keys, request sids, wall-clock timestamps, content or image bytes, and |
| 150 | +env/gate dumps from live systems. Same remedy shape: placeholders/tokens + |
| 151 | +mechanical scan before push. Proposed as an addition to CLAUDE.md's hygiene |
| 152 | +section; wording follows the fixture-strategy principles above. |
| 153 | + |
| 154 | +## Sequencing |
| 155 | + |
| 156 | +1. This directive reviewed (label per repo policy; implementation of the |
| 157 | + remaining deltas only after `plan-approved`). |
| 158 | +2. Branch rewrite of #272 per the agreed remediation (original fixture blobs |
| 159 | + never reachable from upstream `main`; sanitizer-hardened, minimized |
| 160 | + fixtures only) — already coordinated on-thread, awaiting reviewer |
| 161 | + confirmation; force-push follows that confirmation, nothing lands before. |
| 162 | +3. Stacked PRs rebase onto the rewritten base in order (#273 → #276 → #278, |
| 163 | + #281 last); the identity scheme lands once, here. |
| 164 | + |
| 165 | +## Verification |
| 166 | + |
| 167 | +- Census IN-PLACE-TEXT metric present in the daily gate output (standing |
| 168 | + monitor for the allowlist premise). |
| 169 | +- Absence scan green over every committed fixture; red-first history retained |
| 170 | + in the test file. |
| 171 | +- Replay verdict-identity check for every fixture cut (classifier verdicts |
| 172 | + byte-identical pre/post cut). |
| 173 | +- Full suite + the slice's targeted suites green at the rewritten tips. |
0 commit comments