[BUG] Prompt cache invalidated by rewrites of messages in long sessions

[oakif]

What's Wrong?

Claude code sessions sometimes reprocess the entire conversation instead of just a new message that a user sends. This is not due to any reason visible in the chat itself.

I found two causes by diffing /v1/messages requests around a cost spike.

1. Claude Code rewrites an old hook reminder's shape later in the session. It does this by either moving it into its own message, or merging it into a neighboring one. Becausr this edits a message from earlier in the session's history, the entire session cache gets invalidated. I hit this multiple times in the same day.
2. Long sessions only get one cache checkpoint, at the very end. This means that if this checkpoint is missed, you're SOL and the entire convo hits a cache write.

What Should Happen?

Old messages should not be modified, since this invalidates the entire cache, which can cost up to $20 (or even more...)

Steps to Reproduce

1. Run a long VS Code session with PreToolUse hooks that add extra context to tool calls
2. Diff consecutive raw request bodies around a big cache_creation_input_tokens spike which should hit eventually
3. You'll see an old <system-reminder> block change shape, either split into its own message or merged into a neighboring one, breaking the cache for everything after it

Claude Code Version

2.1.197 (Claude Code)

Is this a regression?

Not sure, probably not

Platform

Anthropic API

Operating System

macOS

Terminal/Shell

VS Code extension (CLI wrapper)

[hksbindra]

Reproduced deterministically, with a controlled A/B. The trigger is the VS Code extension's claudeCode.autosave setting — and I caught the injected payload being born, one edit at a time.

Setup: Claude Code 2.1.207, VS Code extension, Windows 11, Opus 4.8, 1h TTL.
Corpus: 155 sessions, 4,639 deduplicated requests, 868 user-turn boundaries, 49 real messages_changed events totalling 7,555,163 missed tokens.

The A/B

Same machine, same workspace, same 40 files, same operator, same edit pattern. One variable.

claudeCode.autosave	requests	edits	distinct files	messages_changed
OFF	147	~60	40	0
ON	61	41	40	1

With autosave off I actively tried to break it for six hours — 45 sequential read-only calls, 19 edits to an open file with live ruff errors, 40 sequential writes to 40 distinct new files, long user pauses. cache_read grew monotonically to 255,933. Zero drops. Not one IDE payload was ever injected.

With autosave on, in a fresh session, it died on the 41st edit:

01:34:00   cache_read = 125,948   cache_creation =     761   --
01:36:03   cache_read =  20,658   cache_creation = 106,250   messages_changed (91,773 missed)

cache_read collapsed from 125,948 to the system residual. The entire conversation was re-written at the 1h-write multiplier. ~$2.75.

Watching the injection happen

This is the part I haven't seen reported. In the autosave-on session, 39 of 40 sequential Edit calls came back carrying an <ide_diagnostics> payload, delivered as a PostToolUse:Edit hook injection — a <system-reminder> wrapping the diagnostics. It fired on the very first edit:

probe_01.py:8  "Undefined name `_witness_01`"           Error     (Ruff)
probe_01.py:5  "`datetime.datetime` imported but unused" Warning   (Ruff)
probe_01.py:8  "\"_witness_01\" is not defined"          Error     (Pylance)
probe_01.py:5  "\"datetime\" is not accessed"            Hint      (Pylance)

Two things follow immediately:

1. It is PostToolUse, so it is structurally late. The diagnostic caused by an edit is not in the request that made the edit. It lands in the next build of the message array. That is the late-arrival model, observed directly rather than inferred.

2. Reads are free; writes are the trigger. In the same session, with all 40 files open in the editor and the language server live, I made 40 reads and got zero payloads. The payload appears the instant a write happens. This deconfounds tool volume from the actual cause.

The race, caught in the act

Edit 18 published nothing. Every other edit in the chain — 1–17 and 19–40 — produced a payload. probe_18 produced none, and probe_19's payload named only probe_19.py, so probe_18's diagnostic was not late-folded into a later sample either. It was simply still outstanding when the hook took its snapshot.

That single gap is the whole bug in miniature. LSP textDocument/publishDiagnostics is a server-initiated push — the language server sends it when it finishes analysing, and there is no synchronous "give me diagnostics now" path in a save handler. The producer (language server) and the consumer (the PostToolUse hook) are not synchronised. The hook takes a point-in-time snapshot of an asynchronously-published stream, and then embeds that snapshot into an append-only, content-addressed, prefix-cached structure whose entire contract is that a message, once sent, never changes.

A diagnostic that misses its snapshot has nowhere to go but a later rebuild — of a message that has already been serialized, sent, and cached. That is the "old reminder changes shape" rewrite OP describes.

To be precise about what is mine and what is OP's: I observed the payloads being injected, the missed sample, and the resulting messages_changed drop. I did not capture the byte-level rewrite of an already-sent block — OP did, from raw request bodies. The two reports are looking at the same object from opposite ends: OP sees it mutate; I watched it get born and timed the miss.

This is an async producer writing into an immutable, already-transmitted, prefix-cached structure. It is a race against the cache, and the cache always loses.

Deterministic reproduction

1. VS Code extension, single window, claudeCode.autosave enabled (it is the default).
2. Start a session and grow the conversation past ~50k tokens. Across 49 recorded events this bug has never fired below 31,238 tokens (median 183,602). A small session comes back clean and proves nothing — I voided three experiments learning this.
3. Have the agent make ~40 sequential Edit calls to files the language server actively flags, each edit introducing a fresh hard error (an undefined symbol — a red Pylance error is a stronger signal than a yellow lint warning).
4. Watch the tool results: <ide_diagnostics> payloads will start appearing immediately.
5. Send any user message. This matters — see below.
6. Read message.diagnostics.cache_miss_reason from the session JSONL (dedupe assistant records by requestId). It will read messages_changed.

Workaround available today: disable claudeCode.autosave. After disabling it, this corpus recorded ~350 requests and ~118 edits across four sessions with zero invalidations — sessions that would previously have been near-certain to drop.

Two structural findings

1. messages_changed fires only at user-turn boundaries. 49 of 49. 0 of 3,774 mid-chain requests, ever. The rewrite happens when the message array is rebuilt, never while a tool chain is streaming — which is exactly the window in which a late async payload lands.

2. There is exactly ONE cache_control breakpoint in the messages array (confirming OP's second point, from captured request bodies):

system[1]         -> ephemeral 1h
system[2]         -> ephemeral 1h
msg[18] block[4]  -> ephemeral 1h    <- the only one in the entire messages array

So there is nothing to fall back to. A mutation at message 3 and a mutation at message 93 are identically catastrophic. 37 of my 49 drops land on the system residual (13.1k–20.4k) regardless of how large the conversation was.

Correction for #63930

The "≥12 parallel tool calls" trigger does not replicate. Grouping tool_use blocks by requestId (Claude Code writes each parallel call as its own JSONL record, so naive per-record counting silently reads 1):

max parallel fan-out	n	drops	rate
0–1	597	19	3.2%
2–3	202	22	10.9%
4–7	54	8	14.8%
8–11	7	0	0.0%
12+	5	0	0.0%

My widest turns never dropped once. Holding volume fixed, low fan-out drops more. Parallelism is not the trigger — IDE-mediated file saves are.

Asks

1. Never back-fill an already-sent message. Late-arriving IDE context must append to the newest message, never rewrite one that has been serialized and cached. If the diagnostic missed its snapshot, it belongs in the next turn or nowhere.
2. More than one breakpoint in the messages array. A single checkpoint at the tail makes every invalidation total, no matter how deep the mutation.
3. Document that claudeCode.autosave carries this cost, until (1) lands. It is on by default, and it is silently expensive on exactly the workload Claude Code is for: long sessions that edit a lot of files.

[Gunther-Schulz]

Independent corroboration of cause 1, with different instrumentation and numbers.

We capture every /v1/messages body through a local proxy (CC 2.1.220, linux, ~2.5 GB/day). A census over consecutive same-conversation request pairs shows exactly the mechanism you describe — an old <system-reminder> hook block re-serialized later in the session, moved into its own message or merged into a neighbour:

• The splice/insert-mid class fires several times per evening session (measured pairs at request 92, 100, 316, 466, 473, 497… of one 513-request session), ~40 kB re-billed per hit at our depth.
• Position census over replace/edit mutations: 15 of 20 were mid-history, not tail — so these are full-prefix invalidations, the expensive kind. The worst single candidate sat at index 768 of 783, 15 s before a 484k-token cache rewrite.

We mitigate it in a proxy by pinning each volatile block to its first-seen serialization, keyed by content identity (so the forwarded bytes never change even when CC re-shapes the block): cnighswonger/claude-code-cache-fix#272. That works as an external patch, but the native fix is simpler: serialize reminder blocks in their final shape at creation time and never re-shape a message that has already been sent — history behind the last message should be append-only bytes.

🤖 Generated with Claude Code

Update (same day): the full verification toolchain behind these numbers is now also PR'd — pre-pipeline request capture (cnighswonger/claude-code-cache-fix#275) and the replay/census/harvest gate (cnighswonger/claude-code-cache-fix#276) — so the measurements above are reproducible against anyone's own traffic, not just ours.

Method note, since it generalizes: the census in that toolchain classifies every consecutive same-conversation request pair by divergence shape (append-only / splice / replace-edit / tools-delta / …) and prices each class in re-billed bytes — so it surfaces and ranks invalidation causes you have not named yet, rather than confirming known ones. That is how the mid-history classes in this thread were found.