Skip to content

Commit 9702540

Browse files
harvest: scrub wrapped reminders with their own token, not a fixed constant
Deviation found while validating the first real --pin (capture s-633915a8, n=26->28): scrubText collapsed every <system-reminder>-wrapped block to the literal constant "REDACTED", regardless of its real content. That breaks insertion-normalization's wrap/unwrap cross-identity check (findSuppressibleDuplicate/unwrapVolatileText, proxy/extensions/ insertion-normalization.mjs) -- CC sometimes migrates a reminder OUT of its wrapper into a standalone duplicate message, and suppression fires only when the wrapped original's stripped bytes hash-match the standalone duplicate's bytes. Under the fixed constant, the wrapped original always hashed to "REDACTED" while the unwrapped duplicate hashed its real text independently, so the two never matched post-scrub. Measured directly: replaying the sanitized n=26->28 fixture through the real pipeline gave suppressed=0 / outputForm="splice@31" (the pre-fix defect shape) instead of the live capture's suppressed=1 / outputForm="append". The committed pinned fixture would have silently reproduced stale, wrong behaviour once the live capture rotated away -- worse than the SKIP it was meant to replace. Fix: a wrapped reminder now re-wraps scrubText's own recursive token for its inner text instead of a fixed placeholder. The wrapper tags still survive verbatim (any check that only tests for wrapper PRESENCE is unaffected -- verified against test/harvest.test.mjs's existing "wrappers survive" test, unchanged and still green), and two reminders with equal real bytes -- wrapped or not -- now hash equal after scrubbing, matching what the sanitizer already guarantees for ordinary text. Verified: test/harvest.test.mjs (14/14, no regression), and the fixed scrubber's output re-run through the actual pipeline (findMitigationGaps, findSafetyViolations) reproduces the live capture's real values exactly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit fda83cc)
1 parent 7e38919 commit 9702540

1 file changed

Lines changed: 29 additions & 4 deletions

File tree

tools/harvest.mjs

Lines changed: 29 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -49,8 +49,14 @@
4949
// only the arrangement matters.
5050
//
5151
// Two things survive verbatim, because for them the content IS the class:
52-
// - <system-reminder> wrappers (the volatile-block detector matches on
53-
// the wrapper, so replacing it would erase the very property under test)
52+
// - <system-reminder> WRAPPER TAGS (the volatile-block detector matches on
53+
// the wrapper, so replacing it would erase the very property under
54+
// test); the text they wrap is still tokenized like any other text
55+
// (scrubText), not replaced by a fixed placeholder — a fixed
56+
// placeholder made every reminder hash identically regardless of real
57+
// content, which breaks the separate class where a reminder migrates
58+
// OUT of its wrapper into a standalone duplicate message and must still
59+
// hash-match its wrapped original post-scrub (see scrubText's comment)
5460
// - structural ids: tool_use_id / id pairs, which must stay consistent or
5561
// the tool-adjacency invariant breaks
5662
//
@@ -94,14 +100,33 @@ const sha = (s) => createHash("sha256").update(s).digest("hex");
94100

95101
// --- Sanitization ---
96102

97-
const VOLATILE_WRAP = /^<system-reminder>\n[\s\S]*\n<\/system-reminder>\s*$/;
103+
const VOLATILE_WRAP = /^<system-reminder>\n([\s\S]*)\n<\/system-reminder>\s*$/;
98104

99105
// Deterministic placeholder: same input text always yields the same token, so
100106
// a message that repeats across requests still compares equal — which is the
101107
// whole point, since identity matching is what we are testing.
108+
//
109+
// A wrapped reminder re-wraps its OWN deterministic token instead of a fixed
110+
// constant. A fixed constant ("REDACTED" for every reminder regardless of
111+
// content) was tried first and is wrong: CC sometimes migrates a reminder
112+
// OUT of its wrapper into a standalone duplicate message
113+
// (insertion-normalization.mjs's findSuppressibleDuplicate/
114+
// unwrapVolatileText compares the wrapped original's stripped bytes against
115+
// the standalone copy's bytes to suppress the duplicate). A fixed constant
116+
// made the wrapped original hash to "REDACTED" while the unwrapped
117+
// duplicate — never matching VOLATILE_WRAP — hashed its real text
118+
// independently, so the two never matched post-scrub and the suppression
119+
// class became unobservable in any fixture built from it (measured
120+
// empirically while building the harvest --pin fixture for capture
121+
// s-633915a8 n=26->28: suppressed count 1->0, outputForm "append"->
122+
// "splice@31" under the fixed-constant scrub). Recursing scrubText on the
123+
// captured inner text keeps both sides deterministic and equal when their
124+
// real bytes were equal, wrapped or not — the wrapper tags still survive
125+
// verbatim, so a check that only tests for wrapper PRESENCE is unaffected.
102126
function scrubText(text) {
103127
if (typeof text !== "string") return text;
104-
if (VOLATILE_WRAP.test(text)) return "<system-reminder>\nREDACTED\n</system-reminder>";
128+
const wrapped = VOLATILE_WRAP.exec(text);
129+
if (wrapped) return `<system-reminder>\n${scrubText(wrapped[1])}\n</system-reminder>`;
105130
if (text === "") return "";
106131
return `t_${sha(text).slice(0, 12)}_${text.length}`;
107132
}

0 commit comments

Comments
 (0)