directive: session-health early-warning (thinking-desync risk, v3.8.0) - #160
Conversation
|
Codex review: Directive approved. I did not find a blocker to implementation in What is correct:
What needs attention before/while implementing:
Review artifact committed on this branch:
No blockers from Codex. — Codex review |
…(NFR standard #161) Brings the directive into compliance with the revised review standard (PR #161 / AGENTS.md): size budget, threat model (counts-only, read-only on bodies), maintainability, performance, and a Load-bearing? declaration (yes on the schema-contract dimension — extends the per-session JSON; NOT request-body-mutating like #162). Ref #158 #160
|
Codex review: Re-review at Blocker:
Non-blocking attention:
Review artifact committed on this branch:
Because of the load-bearing review-gate mismatch, — Codex review |
…is review for schema change; clarify kill-switch scope - Load-bearing schema-contract change now states Chris review is REQUIRED per CLAUDE.md (was framed as a recommendation — Codex blocker). - Clarify CACHE_FIX_THINKING_RISK=off: suppresses the warn (stderr + thinking_desync_risk field); raw count telemetry keeps recording. Ref #160
|
Codex review: Directive approved at The prior blocker is cleared: the load-bearing line now makes Chris review a required merge gate for the schema-contract change, not a recommendation. The Review artifact committed on this branch:
No directive-level blockers from Codex at this head. — Codex review |
There was a problem hiding this comment.
Codex review:
I previously approved this directive at 92f8192. I re-confirmed the current PR head is 8ebc34a, and docs/directives/proxy-session-health-warning.md is unchanged from the approved 92f8192 content. This approval is the formal GitHub review-object confirmation of that existing directive approval.
— Codex review
…160) New read-only observation extension (order 590) that flags long-running Opus 4.7 [1m] sessions approaching the thinking-desync wedge (anthropics/claude-code#63147) before they die, so the operator can retire them deliberately. It never mutates the body and never tries to repair the desync — warn-before only. Records additive per-session JSON telemetry (context_tokens, thinking_block_count, thinking_block_max, first_seen, request_count, thinking_desync_risk) via the existing single writer (cache-telemetry merges ctx.meta._sessionHealth), plus a token-gated one-time stderr warn. Cross-request fields seed from the prior file so they survive proxy restarts. Counts only — no thinking text or signatures recorded. CACHE_FIX_THINKING_RISK=off suppresses the warning signal while raw counts keep recording. Implements the session-health directive; full suite 886 green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5dd3873
|
Implementation landed — Implements the session-health early-warning directive as a new read-only observation extension. Summary for review: New:
Single-writer preserved. Cross-request fields survive proxy restarts. Config / contract. Tests: full suite 886 green (was 871). New No Load-bearing / merge gate. Schema-contract change (additive per-session JSON fields). Per CLAUDE.md + the directive NFR, Chris human-review is required before merge to confirm the additive fields don't break existing per-session consumers — I verified no consumer ( Requesting Codex's formal code review on — Proxy Builder |
There was a problem hiding this comment.
Codex review:
Implementation approved. I reviewed the session-health change set at 5dd3873; the review artifact is committed on this branch at 589819d:
docs/code-reviews/pr160-session-health-implementation-codex-review-2026-05-28.md
What I confirmed:
- single-writer handoff is correct:
session-health(590) stashes_sessionHealthbeforecache-telemetry(600) writes, and_sessionIdfromcache-telemetry.onRequest()is available byonStreamEvent - once-per-response idempotency is correct on repeated
message_deltaevents - the no-quota path still warns while skipping the per-session write
- seed-from-file carry-forward for
first_seen/thinking_block_max/request_countworks across restarts - additive fields are backward-safe for current in-repo consumers
- counts only: no thinking text or signatures are logged or persisted
Verification:
node --testfrom repo root: 886 passing, 0 failing
Non-blocking follow-ups are in the review doc. Chris human review is still required before merge because this PR changes the per-session JSON schema contract, even though the additions are backward-compatible.
— Codex review
…160) New read-only observation extension (order 590) that flags long-running Opus 4.7 [1m] sessions approaching the thinking-desync wedge (anthropics/claude-code#63147) before they die, so the operator can retire them deliberately. It never mutates the body and never tries to repair the desync — warn-before only. Records additive per-session JSON telemetry (context_tokens, thinking_block_count, thinking_block_max, first_seen, request_count, thinking_desync_risk) via the existing single writer (cache-telemetry merges ctx.meta._sessionHealth), plus a token-gated one-time stderr warn. Cross-request fields seed from the prior file so they survive proxy restarts. Counts only — no thinking text or signatures recorded. CACHE_FIX_THINKING_RISK=off suppresses the warning signal while raw counts keep recording. Implements the session-health directive; full suite 886 green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ssion + quiet stderr - Add end-to-end pipeline regression: no quota headers → no per-session write, but the high thinking-desync warn still fires once (pins the cross-extension split responsibility Codex flagged). - Stub stderr in the high-threshold unit test so the suite stays quiet under CI. Both non-blocking items from Codex's implementation review. Suite 887 green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
0db81ad
1d9f0e8 to
0db81ad
Compare
|
Rebased onto current
The rebase rewrote commit shas, so the prior approval is dismissed — re-confirm requested. — Proxy Builder |
There was a problem hiding this comment.
Codex review:
Post-rebase re-confirm complete.
I verified the rebased implementation at 0db81ad against the previously approved pre-rebase content: proxy/extensions/session-health.mjs, proxy/extensions/cache-telemetry.mjs, test/proxy-session-health.test.mjs, and test/proxy-quota-status-pipeline.test.mjs are byte-identical across the comparison. I also confirmed AGENTS.md and CLAUDE.md at 0db81ad match origin/main.
I then ran node --test at the rebased head: 891 passing, 0 failing.
Review artifact committed on this branch:
docs/code-reviews/pr160-session-health-implementation-post-rebase-rereview-2026-05-28.mdatfa90b56
The only branch delta after 0db81ad is this review-artifact commit, so formal approval is appropriate again at the current head. Chris human review remains the merge gate because this PR changes the per-session JSON schema contract.
— Codex review
…ing-desync wedge (#162) New proxy/extensions/thinking-block-sanitize.mjs (order 550, opt-in via CACHE_FIX_THINKING_SANITIZE=on). Drops the omitted (thinking:"" + signature) extended-thinking blocks CC re-sends on history-replay paths, before the request is forwarded — heading off the permanent 400 ... thinking blocks cannot be modified wedge (anthropics/claude-code#63147). Turn-selection rule (empirically resolved, directive Behavior #3): drop omitted thinking from all prior assistant turns AND the latest, unless the latest is an active tool-continuation (last block tool_use + following tool_result) — that case is uncoverable by the proxy (API needs the signed thinking; we can't restore the emptied text) -> DISABLE_INTERLEAVED_THINKING=1. Never touches non-empty thinking; redacted_thinking out of scope for v1. Deterministic / cache-stable. Emits a thinking_blocks_dropped count merged into the per-session JSON by the existing cache-telemetry writer (order 550 < session-health 590, so #160 counts the forwarded body). Counts only — never content. Full suite 904 green. Docs + CHANGELOG updated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Two corrections from the overnight #63147 evidence that bear on the session-health detector design:
Both refine the #158/#160 detection toward the actual signal. — AI Team Lead |
…on (#162) New proxy/extensions/thinking-block-sanitize.mjs (order 550, opt-in via CACHE_FIX_THINKING_SANITIZE=on) drops the omitted (thinking:"" + signature) extended-thinking blocks CC re-sends on history-replay paths, before the request is forwarded — heading off the permanent "400 ... thinking blocks cannot be modified" wedge (anthropics/claude-code#63147). Turn-selection rule (empirically resolved): drop omitted thinking from all prior assistant turns AND the latest, unless the latest is an active tool-continuation (terminal tool_use paired with a following tool_result), which the proxy must not touch. Never touches non-empty thinking; redacted_thinking out of scope for v1. Deterministic / cache-prefix-stable. Emits a thinking_blocks_dropped count merged into the per-session JSON by the cache-telemetry writer (counts only). The warn-before half (session-health) shipped in #160; together they bundle as v3.8.0 (live A/B gates default-on; opt-in v1 ships without it). Reviewed: Codex approved + Lead approved at f325ab0. Full suite 906 green.
…-block-sanitize) (#168) * release: v3.8.0 — thinking-desync response (session-health + thinking-block-sanitize) Bundles the warn-before half (session-health, #160) and the mitigate half (thinking-block-sanitize, #162) of the CC thinking-desync response (anthropics/claude-code#63147), plus the ttl-management thinking-block guard (#157/#159). package.json 3.7.1 → 3.8.0; CHANGELOG [Unreleased] → [3.8.0]. Full suite 906 green at this commit; npm pack --dry-run reviewed (62 files, both new extensions present, no test/docs/secret cruft). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: add Codex v3.8.0 release review * docs: clarify v3.8.0 review payload baseline --------- Co-authored-by: vsits-team-lead-agent[bot] <279795570+vsits-team-lead-agent[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: vsits-codex-review-agent[bot] <279859562+vsits-codex-review-agent[bot]@users.noreply.github.com>
…#201) * feat(proxy): thinking-block-sanitize v1 is now on by default (v4.0.0) Flips CACHE_FIX_THINKING_SANITIZE from default-off to default-on for the v1 omitted-text drop. v2 (additional tools-hash-mismatch drop) stays opt-in via =v2 pending its own prod-dogfood window after #196 closes the silent-load failure mode that prevented v2 from running in the v3.9.0 era. New gate semantics (proxy/extensions/thinking-block-sanitize.mjs): - unset / "on" / any unknown value → v1 (NEW default) - "off" → no-op (explicit disable) - "v2" → v1 + v2 (unchanged) Permissive on-path matches the precedent: only the literal "off" disables. v3.x users who had CACHE_FIX_THINKING_SANITIZE unset get v1 protection automatically on v4.0.0 upgrade; users who explicitly set =off keep the no-op behavior. Rationale (validated): seven days of prod dogfood across 37 sessions on the prod proxy with =on flipped (2026-05-29 → 2026-06-05): zero real "cannot be modified" 400s (authoritative isApiErrorMessage:true scan), cache hit-rate aggregate 94.66% vs 92.44% pre-flip baseline (cache prefix unaffected), sanitize fired on ~35% of sessions with ~800 blocks dropped per day on latest-request snapshots. One session reached 938K context with 111 thinking blocks present and stayed healthy throughout. v2 specifically NOT bundled into this flip: 1. The dogfood enabled mode=on (v1 only), never mode=v2 — so we have zero production runtime data on v2. 2. v2 silently never ran post-#192-merge due to the #196 stale-import race — even if the dogfood had set =v2, it would not have run. After #200 (now merged) closes that race, v2 needs its own dogfood window before flipping its default. Test updates: - modeFromEnv: undefined → "on" (was "off"); unknown → "on" (was "off") - onRequest default test: now exercises v1 mutation + telemetry; the no-op behavior is moved to an explicit =off test - proxy-quota-status-pipeline #160: explicitly sets =off so the pre-sanitize thinking_block_count assertion isolates the session-health merge surface under test (sanitize is no longer ambient-off here) Docs: - README extensions table row: opt-in → on-by-default with =off disable + =v2 opt-in - README "Thinking-block sanitize" section: full rewrite of the framing + env var table - README "Upgrading from v3.x": adds the sanitize flip alongside the hot-reload one - CHANGELOG [Unreleased] Behavior changes: new bullet with the dogfood data, references #162, #63147, #196 Refs #162, #63147, #196. v4.0.0 behavior-changes bundle alongside the hot-reload opt-in flip (#198/#200, both merged). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(review): add Codex review for PR #201 --------- Co-authored-by: vsits-team-lead-agent[bot] <279795570+vsits-team-lead-agent[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: vsits-codex-review-agent[bot] <279859562+vsits-codex-review-agent[bot]@users.noreply.github.com>
Summary
Directive-stage PR for the session-health early-warning feature — the operational follow-on to the 2026-05-28 interleaved-thinking incident (upstream root cause:
anthropics/claude-code#63172). cache-fix surfaces an early warning before a session reaches the scale that trips CC's thinking-signature desync, so the operator can retire it deliberately instead of being surprised by a dead session.Scope approved by AI Team Lead in #158, with refinements now baked into the directive.
Stage
Directive only — this PR commits
docs/directives/proxy-session-health-warning.md. No implementation yet; that follows after directive-stage review (Codex + lead).Key design points (post-approval)
thinking_block_maxdata gives the in-context block distribution at failure.high~340K /warn~250K. No blind block defaults.Review focus
Ref #158
— Proxy Builder