Skip to content

Commit 40c11b2

Browse files
directive(cnighswonger#272): identity scheme, pin safety, persistence, fixture strategy — plus corpus-hygiene standing rules
The directive the PR cnighswonger#272 reviewer required before the next review pass: answers blocker 2 from measurement (IN-PLACE-TEXT = 0 corpus-wide, zero proven informative) with an evidenced allowlist kept monitored by the daily census sweep; settles persistence scope (0600, raw bytes only where structurally required) for the series; and sets the fixture strategy — synthesized by default, harvested-and-scrubbed only as a justified exception gated on the absence scan. dev-loop.md gains the standing corpus-hygiene rules minted from the fixture-leak incident (sanitization is subtractive; a sanitization claim counts only as its checker's output). BACKLOG: slice-port preflight widened to flag fixtures without absence coverage; blocker-2 entry records the directive's widened scope. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xwqzgc8FGzwAtoEJvzrZ79
1 parent 1ba9e24 commit 40c11b2

3 files changed

Lines changed: 202 additions & 0 deletions

File tree

BACKLOG.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -422,6 +422,13 @@ bullet, evidence pointer included.
422422
brief-prescribed discard paths (no-op resolution, proceed) from
423423
content (`UU`) conflicts (abort) — wave-2's executor had to
424424
deviate to deliver anything (report §d D1).
425+
WIDENED 2026-08-01 (fixture-leak post-incident, operator GO): the
426+
preflight also flags a FIXTURE mapped into a slice without its
427+
absence coverage — any test/fixtures/** file in a slice whose
428+
absence scan (tools/absence-scan.mjs, extracted from
429+
harvest-scrub-relations §6) is not runnable in that slice = red,
430+
named. The dotfiles pre-push guard is the boundary backstop either
431+
way, so this preflight arm is defense-in-depth, not the only line.
425432

426433
- **OPEN/HOT — MERGED-reminder standalone: the 587k's real mechanism
427434
(premise corrected by the builder's probe, dispatcher
@@ -1248,6 +1255,13 @@ bullet, evidence pointer included.
12481255
measured-real → fail-closed re-pin (store the NEW bytes, honest
12491256
reset of that boundary only). Deliverable: the directive the
12501257
reviewer asked for, with the measurement inside.
1258+
ALSO IN THE DIRECTIVE (2026-08-01 GO): the fixture-strategy section
1259+
the reviewer asked for — synthesized-by-default for public trees,
1260+
harvested-and-scrubbed as the justified exception gated on the
1261+
absence scan; minification; the body/headers-retention question
1262+
answered together with the persistence story; and the widened
1263+
public-repo hygiene class (conversation/capture data alongside
1264+
origin-server info) proposed for upstream's CLAUDE.md.
12511265

12521266
- STANDING GO (operator, 2026-07-31 late): the held execution items
12531267
below — blockers 3+4, census flap annotation, joined-standalone

docs/dev-loop.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -491,6 +491,21 @@ fingerprint known public texts. Ledgers are
491491
per-machine (`LEDGER-<host>.json`); novelty is judged against every sibling
492492
ledger, so N machines share one deduplicated corpus with no coordination.
493493

494+
Sanitization is subtractive — it removes the hazards someone enumerated, and
495+
the unanticipated field ships by default (measured 2026-07-31: conversation
496+
keys, sids, wall-clock timestamps, a session UUID in a filename, and nested
497+
`source.data` image bytes all rode through a scrubber whose header claimed
498+
completeness, into a public PR — caught by the upstream reviewer, not by us).
499+
Two standing rules from that incident. **Fixtures bound for a public tree are
500+
synthesized by default**; a harvested-and-scrubbed fixture is the exception,
501+
justified by real-pair evidence value (a class only real traffic teaches) and
502+
committable only with the absence scan green. **A sanitization claim counts
503+
only as its checker's output**: the absence classes live in
504+
`tools/absence-scan.mjs` — imported by `test/harvest-scrub-relations.test.mjs`
505+
at test time, run again by the dotfiles pre-push guard at the boundary where
506+
git history becomes unscrubbable. A new identifier class discovered later is
507+
added to the scanner first (red on the live instance), then scrubbed.
508+
494509
The gate reads captures **line by line**, so pointing it at a live
495510
multi-hundred-megabyte capture is the intended use, not an abuse. It slurped
496511
them until 2026-07-28, when a 955 MB capture produced `RangeError: Invalid
Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
# Directive: insertion-normalization — identity scheme, pin safety, persistence, and fixture strategy
2+
3+
Requested by the reviewer on PR #272 (review round 1, 2026-07-31) before the
4+
next code-review pass. Scope: the four blockers' design questions, answered
5+
once here so #273/#276/#278/#281 inherit a settled scheme instead of
6+
re-litigating it per diff. Written against measurement, not argument; every
7+
load-bearing number cites its run.
8+
9+
## Goal / Background
10+
11+
CC re-serializes old `<system-reminder>` blocks mid-history (splice, merge,
12+
migration); each hit edits a settled message and re-bills everything after it
13+
(~40 kB per unmitigated hit; anthropics/claude-code#76606). The extension
14+
keeps a canonical per-conversation model keyed by content identity and pins
15+
volatile blocks to their first-seen serialization. The open design question
16+
(blocker 2): pin-mode identity *deliberately excludes* volatile blocks from
17+
the hash — that exclusion is the mechanism — so the extension cannot
18+
structurally distinguish CC re-serializing a reminder (pin: correct) from CC
19+
*changing its bytes* (pin would forward stale text: a fidelity risk the
20+
reviewer reproduced synthetically).
21+
22+
## Non-Functional Requirements
23+
24+
- **Size/complexity budget:** extension stays ~1,100 LOC (no new modules;
25+
`message-hash.mjs` remains the one shared primitive). Fixture budget is the
26+
real change: target is a ≥10× reduction of the 432 KB harvested fixture
27+
(see Fixture strategy); an implementation landing materially above these
28+
marks is a review flag.
29+
- **Threat model:** three surfaces. (1) Wire: the extension mutates request
30+
bodies; forwarding bytes CC did not send is the corruption class — bounded
31+
below by the allowlist + monitor design. (2) Disk: canonical state persists
32+
first-seen message bytes (`entry.m`, structurally required for re-serving);
33+
all conversation-derived writes land mode 0600 via a shared write-owner-only
34+
primitive (series-wide fix, 27 write sites). (3) Public repo: fixtures are
35+
conversation-derived; identifiers and content bytes must never be
36+
committable — enforced mechanically (absence scan at test time and at
37+
pre-push), not by care.
38+
- **Maintainability:** no new abstractions. Census/replay import the
39+
extension's own identity helpers rather than restating them (export
40+
`identityKey`, `unwrapVolatileText`, `pinnedForwardForm`, `hasCacheControl`;
41+
small proxy-side change riding the next deploy boundary).
42+
- **Performance/reliability:** unchanged; per-request work stays O(messages).
43+
- **Load-bearing?** **Yes** — mutates request bodies on the wire, persists
44+
conversation-derived state, changes forwarded history shape. Requires human
45+
(Chris) review before merge per repo policy.
46+
47+
## Identity scheme (what it is, and what it cannot see)
48+
49+
Identity = content hash + occurrence ordinal (`message-hash.mjs`),
50+
position-independent; repeated identical reminders stay distinct (a real
51+
history carried the same reminder 44×). In pin mode, volatile blocks are
52+
excluded from the hash and the first-seen serialization is stored and
53+
re-served on later matches. Consequence, stated plainly: a change *inside* a
54+
volatile block of a still-matching message is invisible to identity. Whether
55+
that is acceptable is an empirical question about CC's actual behavior.
56+
57+
## The measurement the design rests on
58+
59+
Full method and per-entry rows: `docs/code-reviews/`
60+
`blocker2-volatile-change-measurement.md` (census extension, commit 97867f3;
61+
red-first both directions; instrument imports the extension's own
62+
`computePinnedIdentities`/`isVolatileBlock`). Corpus: 36/36 captures, 7.1 GB,
63+
2026-07-28..08-01, 196 conversations, 11,074 same-conversation pairs.
64+
First-seen comparison per pinned identity — a strict superset of the
65+
comparisons the live pin performs (it can over-report change, never
66+
under-report).
67+
68+
| CHANGED sub-kind (of 23,328 pin-rewritten comparisons) | occurrences | entries |
69+
|---|---|---|
70+
| VANISHED — every first-seen reminder gone | 11,478 | 80 |
71+
| REDUCED — a subset survives | 284 | 1 |
72+
| **IN-PLACE-TEXT — a reminder's text replaced** | **0** | **0** |
73+
| APPEARED / AUGMENTED | 0 | 0 |
74+
75+
The zero is informative, not vacuous: the corpus carries 119 distinct
76+
reminder texts across 10 kinds (5 kinds with >1 text — one with 75), so
77+
in-place variation is possible by construction; it lives across messages,
78+
never inside a settled one. Removals (VANISHED/REDUCED) are the #76606 flip
79+
the pin exists to absorb: the model re-reads text it already consumed —
80+
information is repeated, never contradicted. Named boundaries: first-seen
81+
scope is per capture file (over-counts staleness across rotations —
82+
conservative in the right direction); the corpus is live (totals drifted
83+
across three runs; the zero held in all three); CC-version spread is not
84+
recoverable from captures.
85+
86+
## Decision: evidenced allowlist, monitored — not fail-closed re-pin
87+
88+
Pinning stays allowlist-scoped to blocks matching the volatile wrap contract,
89+
justified by the measured absence of the harmful class. The claim is kept
90+
**monitored, not assumed**: the census now computes IN-PLACE-TEXT on every
91+
daily gate sweep, so the first real occurrence surfaces mechanically. That
92+
occurrence is the build trigger for the fallback already designed:
93+
fail-closed re-pin — store the new bytes, honest reset of that boundary only.
94+
95+
**Failure mode when the canon is wrong** (the reviewer's ask, answered
96+
per class): in-place change — zero observed; if one ever occurs it is served
97+
stale once per request until the next sweep flags it (bounded by the
98+
monitor's cadence), then the fallback ships. Removal — designed absorption,
99+
safety argument above. Structural mismatch (compaction, genuine rewrite) —
100+
honest reset, pins survive, order assumptions do not (unchanged behavior).
101+
102+
## Persistence scope (answered once for the series)
103+
104+
On disk: canonical entries including first-seen bytes (`entry.m` — raw bytes
105+
are structurally required; a hash cannot re-serve), event logs, and telemetry
106+
with stable session identifiers. All conversation-derived writes: owner-only
107+
0600 (mode at create + lazy chmod; Node's `mode` option is create-only).
108+
Retention: state lives under the state dir per conversation; an honest reset
109+
clears order state, pins expire with their conversation's state. Where bytes
110+
are not structurally required, hashes are stored instead. This answer carries
111+
to #275 (request capture) and #280 (prefix-diff snapshots) unchanged.
112+
113+
## Fixture strategy (the 432 KB question)
114+
115+
Principles first, then the concrete cut:
116+
117+
- **Synthesized by default.** Constructing a fixture is additive — nothing
118+
identifying exists unless placed. Harvested-and-scrubbed is the exception,
119+
justified per fixture by real-pair evidence value (a class only real
120+
traffic teaches), and committable only with the absence scan green.
121+
- **Sanitization is checked, never claimed.** The scrubber one-way tokenizes
122+
content (`t_<sha12>_<len>` per paragraph preserving `"\n\n"` join
123+
relations; `data_<sha10>` for binary), replaces keys/sids with sha-derived
124+
tokens, rebases timestamps to a fixed epoch keeping intra-fixture deltas,
125+
and names files by token, not session UUID. A mechanical absence test
126+
(base64 runs, UUIDs, live timestamps, raw strings, signatures) went
127+
red-first 9/20 on the old fixtures and now guards the whole fixtures
128+
directory at test time and the push boundary via a pre-push scan.
129+
- **Minimum record set, answered honestly:** the current 54-record fixture is
130+
the harvester's range dump, not a measured minimum. The suppression pair
131+
needs its conversation prefix only to establish pin state; the cut is a
132+
pinned pair plus a minimal pin-establishing prefix, minified (single-line
133+
JSON — assertions read it, people don't). Acceptance for any cut: replayed
134+
classifier verdicts identical to the full-range fixture's, byte-relation
135+
assertions still exercised. Expected result ≥10× smaller; if the measured
136+
minimum genuinely needs more history, the fixture says so in a header
137+
comment with the number.
138+
- **body/headers retention:** full request bodies are not required by the
139+
suppression tests; tokenized structural shape preserves every asserted
140+
relation (join equality survives tokenization by construction). The boot
141+
record's gate-set dump is dropped from fixtures — same class as the #275
142+
env-dump finding.
143+
144+
## Public-repo hygiene class (proposed upstream adoption)
145+
146+
The repo's hygiene section covers origin-server identifiers. This incident
147+
adds a second class with the same non-negotiable property (public git history
148+
is unscrubbable): **conversation/capture-derived data** — session/conversation
149+
keys, request sids, wall-clock timestamps, content or image bytes, and
150+
env/gate dumps from live systems. Same remedy shape: placeholders/tokens +
151+
mechanical scan before push. Proposed as an addition to CLAUDE.md's hygiene
152+
section; wording follows the fixture-strategy principles above.
153+
154+
## Sequencing
155+
156+
1. This directive reviewed (label per repo policy; implementation of the
157+
remaining deltas only after `plan-approved`).
158+
2. Branch rewrite of #272 per the agreed remediation (original fixture blobs
159+
never reachable from upstream `main`; sanitizer-hardened, minimized
160+
fixtures only) — already coordinated on-thread, awaiting reviewer
161+
confirmation; force-push follows that confirmation, nothing lands before.
162+
3. Stacked PRs rebase onto the rewritten base in order (#273#276#278,
163+
#281 last); the identity scheme lands once, here.
164+
165+
## Verification
166+
167+
- Census IN-PLACE-TEXT metric present in the daily gate output (standing
168+
monitor for the allowlist premise).
169+
- Absence scan green over every committed fixture; red-first history retained
170+
in the test file.
171+
- Replay verdict-identity check for every fixture cut (classifier verdicts
172+
byte-identical pre/post cut).
173+
- Full suite + the slice's targeted suites green at the rewritten tips.

0 commit comments

Comments
 (0)