Skip to content

perf(codex): incremental turn-end token calc to fix O(N²) reparse#1840

Draft
gtrrz-victor wants to merge 1 commit into
mainfrom
perf/codex-incremental-token-calc
Draft

perf(codex): incremental turn-end token calc to fix O(N²) reparse#1840
gtrrz-victor wants to merge 1 commit into
mainfrom
perf/codex-incremental-token-calc

Conversation

@gtrrz-victor

@gtrrz-victor gtrrz-victor commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

https://entire.io/gh/entireio/cli/trails/929

Fixes #1836.

Problem

Codex reports cumulative token totals (event_msgtoken_counttotal_token_usage), so the turn-end token calc re-parsed the entire rollout from line 0 on every hook — O(N) per hook, O(N²) per session — amplified by large encrypted_content reasoning blobs. Long sessions visibly stalled at each turn end. Claude Code avoids this by slicing (its counts are per-message, not cumulative).

Fix

  • agent.IncrementalTokenCalculator (new, built-in-only capability): scan only the transcript lines added since the last checkpoint, baselining off a cumulative snapshot persisted in SessionState.CumulativeTokenBaseline.
    • Codex rollouts are append-only, use a per-rollout session id, and fire no compaction hook, so the snapshot always indexes stable content.
    • Cold start (first checkpoint / resumed / imported) or a shrunk transcript (snapshot.LineCount > fromOffset) falls back to a one-time full scan.
    • The delta is provably identical to the full-scan CalculateTokenUsage — only the amount of parsing changes.
  • Codex now implements SubagentAwareExtractor (no subagents) so turn-end modified-file extraction runs on the in-memory transcript bytes instead of a second full-file disk read.
  • CalculateTokenUsage (full-scan) kept byte-for-byte identical for back-compat and the fallback path.

Measured win (per-hook, warm baseline)

turns before (full scan) after (incremental) allocs before→after
100 1251µs 29µs 2617 → 32
1000 12.3ms 0.5ms 26020 → 32
10000 119ms 9ms 260027 → 32

Allocs flat at 32/op regardless of session length — the definitive O(delta) proof (residual ns/op is SliceFromLine's cheap byte-walk, not JSON parsing).

Safety (no regressions)

  • File-change identification: bytes extractor uses the same fromOffset as the old disk path and produces identical results for JSONL (no blank lines); the skip-checkpoint decision also relies on git-status new/deleted + merged modified, independent of transcript parsing. No missed changes → no spurious carry-over branches.
  • Attribution: token deltas unchanged (verified against fixtures: multi-turn, cached-input, mid-offset); the guard falls back to a full scan on any misalignment, so numbers are never wrong. Line attribution (agent vs human) is computed from git commits and is untouched.

Test plan

  • New unit tests (incremental_test.go): incremental == full scan across turns/cached/mid-offset; cold-start reusable baseline; resumed/stale-offset fallback; DoesNotParseHistory (corrupted pre-offset lines ignored — proves bounded parsing); bytes extractor == disk extractor.
  • Validation scenarios (perf_validation_test.go) documenting the root cause + benchmarks.
  • mise run fmt / lint (v2.11.3): clean · unit (codex/agent/session/cli): pass · integration: 437 pass · Vogon canary: 4/4.

🤖 Generated with Claude Code


Note

Cursor Bugbot is generating a summary for commit 406904c. Configure here.

Codex reports cumulative token totals, so the old turn-end token calc
re-parsed the entire rollout from line 0 every hook — O(N) per hook,
O(N²) per session, amplified by large encrypted_content blobs. Long
sessions visibly stalled at turn end.

Add agent.IncrementalTokenCalculator: scan only the lines added since
the last checkpoint, baselining off a cumulative snapshot persisted in
SessionState.CumulativeTokenBaseline. Codex is append-only with a
per-rollout session id and fires no compaction hook, so the snapshot
always indexes stable content; cold start or a shrunk transcript falls
back to a one-time full scan. Deltas are provably identical to the full
scan. Also implement SubagentAwareExtractor so turn-end file extraction
runs on the in-memory transcript bytes instead of a second disk read.

Per-hook cost with a warm baseline drops from O(N) to O(delta): allocs
flat at 32/op across 100→10000 turns (was 260k). Fixes #1836.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 01KY7AMC577ZGBN3J6QYQHK2P0
@gtrrz-victor
gtrrz-victor requested a review from a team as a code owner July 23, 2026 11:09
Copilot AI review requested due to automatic review settings July 23, 2026 11:09
@gtrrz-victor
gtrrz-victor marked this pull request as draft July 23, 2026 11:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves turn-end performance for the Codex agent by introducing an incremental token-usage calculation path that avoids re-parsing the full cumulative rollout transcript on every checkpoint, and by enabling in-memory transcript parsing for modified-file extraction.

Changes:

  • Added a persisted CumulativeTokenBaseline snapshot in session state and a new agent.IncrementalTokenCalculator interface to support incremental token deltas for cumulative-count transcripts (Codex).
  • Updated lifecycle turn-end handling to prefer incremental token calculation when available and persist the updated baseline after SaveStep.
  • Updated Codex to implement IncrementalTokenCalculator and SubagentAwareExtractor, and added/updated unit & perf validation tests to lock in correctness and measure the win.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
cmd/entire/cli/session/state.go Persists a cumulative token baseline snapshot in session state for incremental calc.
cmd/entire/cli/lifecycle.go Uses incremental token calc when supported and persists the next baseline post-SaveStep.
cmd/entire/cli/agent/token_snapshot.go Introduces CumulativeTokenSnapshot and the IncrementalTokenCalculator interface.
cmd/entire/cli/agent/codex/transcript.go Implements incremental token calculation and a bytes-based modified-file extractor for Codex.
cmd/entire/cli/agent/codex/perf_validation_test.go Adds perf/scenario tests documenting the original O(N²) behavior and the post-fix benchmarks/guards.
cmd/entire/cli/agent/codex/incremental_test.go Adds unit tests asserting incremental == full scan and bounded parsing behavior.
cmd/entire/cli/agent/capabilities.go Adds AsIncrementalTokenCalculator helper for built-in-only capability detection.
CLAUDE.md Documents the new incremental token calc behavior in the architecture notes.

Comment on lines +927 to +935
if inc, ok := agent.AsIncrementalTokenCalculator(ag); ok {
// Cumulative-count agents (Codex): parse only lines added since the
// last checkpoint, baselining off the persisted cumulative snapshot.
// nextTokenBaseline is persisted AFTER SaveStep (below), since SaveStep
// may reinitialize session state.
tokenUsage, nextTokenBaseline = computeIncrementalTokenUsage(ctx, logCtx, inc, sessionID, transcriptData, transcriptLinesAtStart)
} else {
tokenUsage = agent.CalculateTokenUsage(ctx, ag, transcriptData, transcriptLinesAtStart, subagentsDir)
}
Comment on lines 275 to +278
func (c *CodexAgent) CalculateTokenUsage(transcriptData []byte, fromOffset int) (*agent.TokenUsage, error) {
var baselineUsage *tokenUsageData // last token_count at or before offset
var lastUsage *tokenUsageData // last token_count after offset
usage, _ := scanTokenDelta(transcriptData, 0, fromOffset, nil)
return usage, nil
}
Comment on lines +388 to +405
func (c *CodexAgent) ExtractAllModifiedFiles(transcriptData []byte, fromOffset int, _ string) ([]string, error) {
seen := make(map[string]struct{})
var files []string
lineNum := 0
for _, lineData := range splitJSONL(transcriptData) {
lineNum++
if lineNum <= fromOffset {
continue
}
for _, f := range extractFilesFromLine(lineData) {
if _, ok := seen[f]; !ok {
seen[f] = struct{}{}
files = append(files, f)
}
}
}
return files, nil
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Codex checkpoint hooks slow down over a session (O(N²) transcript reparse in token-usage calc)

2 participants