feat(deferred-tool-rewrite): hold tools[] byte-stable; announce additions via the mid-conversation beta - #273
Open
Gunther-Schulz wants to merge 3 commits into
Conversation
…story rewrites stop busting the cache Claude Code re-serializes <system-reminder> hook blocks inside otherwise-stable user messages later in the session — moving one into its own message or merging it into a neighbour — which edits history mid-prefix and re-bills everything after the edit (reported independently as anthropics/claude-code#76606; measured here as the splice/insert-mid class, ~40 kB re-billed per unmitigated hit on a real session). The extension keeps a per-conversation canonical model of the message history keyed by content identity (message-hash.mjs: content hash + occurrence ordinal — position-independent, so repeated identical reminders stay distinct). Incoming volatile blocks are pinned to their first-seen serialization: when CC re-shapes an old reminder, the forwarded bytes keep the canonical form and the prefix survives. A history that stops matching the model (compaction, true rewrites) resets honestly rather than forcing a stale canon — pins survive the reset, order assumptions do not. Gated off by default: CACHE_FIX_INSERTION_NORMALIZE=1 enables normalization, CACHE_FIX_VOLATILE_PIN=1 the pinning. State persists under the state dir and survives proxy restarts. Measured on live traffic (513-request session, 2026-07-28): every observed splice/insert-mid pair forwarded with 0 re-billed bytes; the canonical-order invariant, cross-request stability and sequence gates all report 0 violations over 2.5 GB of captures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ions via the mid-conversation beta Every ToolSearch/deferred-tool load makes Claude Code re-send a different tools[] array. tools[] heads the cache prefix, so each load re-bills the entire context — the class reported in anthropics/claude-code#81967 (there triggered by LSP add/remove; the deferred-tool path hits it far more often). Anthropic's API already has the fix — the documented mid-conversation-tool-changes-2026-07-01 beta — and Claude Code 2.1.220 ships that beta's documentation in its own binary without using it on the wire. The extension freezes tools[] at its first-seen form per session (keyed session + system-prompt + conversation, so subagents and sidecars never share a baseline). A mid-session addition keeps the frozen bytes and instead announces the new tool with a tool_addition system message anchored at the tail, re-injected at a stable position on every subsequent request, with the beta header added. Removals and reorders are held byte-stable outright; a schema change resets honestly. Announcements are opt-in per MODEL, with evidence required: the beta is rolled out per model family and an unsupported one rejects the whole request with a 400 (measured on claude-sonnet-5 and claude-haiku-4-5; the haiku error names the gating capability — mid-conversation system content). An unknown model degrades to forwarding the changed tools[] — the status-quo bust, never a lost request — and the first suppressed announcement per model warns with the way out. tools/probe-tool-addition.mjs measures a candidate model in one real request; CACHE_FIX_TOOL_ADDITION_EXTRA admits a candidate on a throwaway proxy for the live probe. Allowlisted with wire evidence: claude-opus-5, claude-fable-5. Gated off by default: CACHE_FIX_TOOL_REWRITE=1. Stacked on pr/insertion-normalization (message-hash identity + system-prompt sub-key). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
Production soak status, for merge timing:
We'll post a week-of-soak summary here as well. If you'd rather land the extension with an empty allowlist and let users probe their own models, that's a one-line change — say the word. 🤖 Generated with Claude Code |
This was referenced Jul 29, 2026
Open
…red anchor no longer reorders the forwarded prefix During a session-boot MCP discovery cascade (tools[] growing 0->11->428 while the conversation stays static), every addition spliced at its anchor's idx+1 — newest first, pushing all earlier additions back: a LIFO stack that reordered the already-forwarded prefix on each new tool. Measured live: 25 self-inflicted cache busts in one session's burst. Additions now resolve their landing position against the original array and splice grouped in discovery order, so the forwarded prefix stays byte-stable and only the run's tail grows. Red-before- green in units and on the originating capture (25 violations -> 0, all other replay invariants unchanged). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
Gunther-Schulz
added a commit
to Gunther-Schulz/claude-code-cache-fix
that referenced
this pull request
Jul 30, 2026
…tput metric; extensions synced Three replay improvements from operating the gate: an unmissable stderr warning when a gated capture replays under default gates (the instrument error that booked a wrong verdict three times in one day — and whose first live fire caught the operator's own gateless replay); a --gates-from-capture flag applying the all-boot-records union so nobody hand-extracts gates; and outputForm now strips cache_control before comparing (a moved cache marker is not a content splice — five pairs totalling ~0.6 MB of phantom "re-billed splice" were CC's own benign marker relocation). Extensions synced to the cnighswonger#272/cnighswonger#273 tips so the slice's real-capture tests exercise the actual pipeline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
Contributor
Author
|
Pushed 6636aa9: a real bug fix found by the replay gate in live traffic. During a session-boot MCP tool-discovery cascade (tools[] growing 0→11→428 while the conversation stays one message), injectAdditions spliced every announcement at its anchor's idx+1 — newest first, reordering the already-forwarded prefix on every new tool. One session paid 25 self-inflicted cache busts in a single burst. Additions now land grouped in discovery order against the original array, so the forwarded prefix stays byte-stable. Red-before-green in units and on the originating capture: 25 stability violations → 0, all other replay invariants unchanged (safety/sequence/canonical-order 0, fidelity identical). 40/40 in the slice. 🤖 Generated with Claude Code |
Gunther-Schulz
added a commit
to Gunther-Schulz/claude-code-cache-fix
that referenced
this pull request
Aug 1, 2026
…tput metric; extensions synced Three replay improvements from operating the gate: an unmissable stderr warning when a gated capture replays under default gates (the instrument error that booked a wrong verdict three times in one day — and whose first live fire caught the operator's own gateless replay); a --gates-from-capture flag applying the all-boot-records union so nobody hand-extracts gates; and outputForm now strips cache_control before comparing (a moved cache marker is not a content splice — five pairs totalling ~0.6 MB of phantom "re-billed splice" were CC's own benign marker relocation). Extensions synced to the cnighswonger#272/cnighswonger#273 tips so the slice's real-capture tests exercise the actual pipeline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bust class
Every ToolSearch/deferred-tool load makes Claude Code re-send a different
tools[]array.tools[]heads the cache prefix, so each load re-bills the whole context. This is the class in anthropics/claude-code#81967 (there triggered by LSP add/remove — the deferred-tool path hits it far more often; at 700k context one load re-bills ~700k tokens).The remarkable part: the API already ships the fix. The documented
mid-conversation-tool-changes-2026-07-01beta lets a request announce an added tool in atool_additionblock at the tail of the conversation, leaving the prefix untouched — and Claude Code 2.1.220 carries that beta's documentation inside its own binary without using it on the wire.The mitigation
tools[]is frozen at its first-seen form per session (keyed session + system-prompt + conversation, so subagents/sidecars never share a baseline — the keying collision was a measured bug before it was a design note).tool_additionsystem message anchored at the tail, re-injected at a stable anchor on every subsequent request, beta header added. Removals/reorders are held byte-stable; a schema change resets honestly.claude-sonnet-5,claude-haiku-4-5— haiku's error names the gating capability, "requires a model that supports mid-conversation system content"). Unknown models degrade to the status-quo bust, never a lost request, and the first suppressed announcement per model logs a warning naming the way out.tools/probe-tool-addition.mjsmeasures a candidate model in one real request;CACHE_FIX_TOOL_ADDITION_EXTRAadmits a candidate on a throwaway proxy for a live probe. Allowlisted with wire evidence:claude-opus-5,claude-fable-5(capture holds the injected block; replaying the pipeline reproduces the forwarded body hash byte-for-byte; outcome record shows the API streamed a 200).Off by default:
CACHE_FIX_TOOL_REWRITE=1.Evidence
In production on this fork across multi-hundred-request sessions: additions announced with 0 re-billed prefix bytes, removals held stable (
14->13incoming forwarded as14->14), 0 stability/sequence/order violations under a daily replay gate over ~2.5 GB of captures. 41 tests including bite tests for the model gate (unsupported model: no block, no beta header, no defer_loading markers — demonstrated red before the gate existed, when it killed a day of sonnet subagent dispatches).🤖 Generated with Claude Code