Skip to content

fix(agent): stop resume hydration from overfilling the context window#3331

Draft
tatoalo wants to merge 3 commits into
mainfrom
posthog-code/fix-resume-hydration-token-budget
Draft

fix(agent): stop resume hydration from overfilling the context window#3331
tatoalo wants to merge 3 commits into
mainfrom
posthog-code/fix-resume-hydration-token-budget

Conversation

@tatoalo

@tatoalo tatoalo commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Problem

Resuming a long-lived cloud task can fail hard with Internal error: Prompt is too long, and once it does, every subsequent resume of that task fails the same way so the task is permanently stuck

image

Loosely:

  1. Directory resume snapshots only cover the workspace, not ~/.claude, so the native session transcript is never on disk after a resume (warm: false). Every resume goes through hydrateSessionJsonl, which rebuilds a transcript from the task's ACP notification logs.
  2. The logs endpoint concatenates the entire resume chain (oldest ancestor first), and in-session compactions are invisible in that log, so hydration resurrects history the session had already compacted away.
  3. selectRecentTurns budgets 800k estimated tokens for 1M-context models at 4 chars/token. JSON-heavy tool payloads actually tokenize at ~2.5–3 chars/token, so the "800k" selection can be well over 1M real tokens. In the failing run, hydration kept 66 turns (~2.9M chars, estimated 732k tokens) that the API counted as 1,155,179 tokens against a 1,000,000 window. The 200k-window budget (150k estimated) has the same overflow, just less headroom to hide it.
  4. The rebuilt transcript writes usage: 0 on every message, so the SDK's context accounting starts near zero: it can't auto-compact before the first API call (the run's compact_boundary reported preTokens: 16250 while getContextUsage() said 1.15M). The first call 400s, the SDK's recovery compaction can't help, and the turn fails as a non-recoverable agent_error.

Changes

  • Estimate turn cost at 3 chars/token instead of 4, so JSON-heavy payloads are no longer undercounted
  • Lower the hydration budgets to target roughly half the real window: 400k estimated for 1M-context models (worst case ~500k real), 80k for 200k-window models. This leaves room for system prompt, tools, skills, and the resumed run's own work, and un-bricks existing tasks (the next resume re-hydrates within budget)

Cold-resume JSONL hydration selected up to 800k estimated tokens of
history at 4 chars/token, but JSON-heavy tool payloads tokenize at
~2.5-3 chars/token, so the rebuilt transcript could exceed the model's
real context window before the run even started. Because the hydrated
messages carry zero usage metadata, the SDK cannot auto-compact ahead
of the first API call, and every request (including compaction itself)
fails with "prompt is too long", permanently bricking the task on
every subsequent resume.

Estimate at 3 chars/token and target roughly half the window
(400k estimated for 1M-context models, 80k for 200k), leaving room
for the system prompt, tools, skills, and the resumed run's own work.

Generated-By: PostHog Code
Task-Id: c6065620-5694-4478-a20a-fec4ccde8f1a
@trunk-io

trunk-io Bot commented Jul 10, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown

React Doctor found no issues in the changed files. 🎉

Reviewed by React Doctor for commit ec17957.

@tatoalo tatoalo self-assigned this Jul 10, 2026
@tatoalo tatoalo added the Stamphog This will request an autostamp by stamphog on small changes label Jul 10, 2026
Generated-By: PostHog Code
Task-Id: c6065620-5694-4478-a20a-fec4ccde8f1a
Generated-By: PostHog Code
Task-Id: c6065620-5694-4478-a20a-fec4ccde8f1a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Stamphog This will request an autostamp by stamphog on small changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant