Cadence-aware prompt-cache TTL: 1h caching for sparse sessions

[pufit]

Problem

Anthropic prompt caching defaults to a 5-minute write TTL. A large class of Nerve sessions has a turn cadence beyond 5 minutes by design — persistent crons (30–60 min apart), ScheduleWakeup monitoring loops (20–30 min re-arms), and spaced web conversations. Every such turn finds a cold cache and re-writes the entire accumulated prefix at 1.25× base input instead of reading it at 0.1×. Last week: 119M cache-write tokens, cache_creation_1h = 0 always — despite the schema/usage pipeline already tracking the 1h split end-to-end.

Backtest (28 days of real session_usage traffic, real pricing table)

source	5m $/wk	auto $/wk	Δ/wk
web	836.80	789.78	+47.01
cron	492.47	255.39	+237.08
telegram	16.33	16.05	+0.28
total	1345.60	1061.22	+284.37

Last-7-days (current model mix): +$432/wk. Blanket-1h would lose $5.79/wk on web — hence a cadence-aware policy, not a global flip. Reproduce with scripts/backtest_cache_ttl.py.

What this does

• agent.cache_ttl config: "5m" (default — zero behavior change), "1h", or "auto". Per-cron-job override via cache_ttl in jobs.yaml. Model exclusion list mirrors context_1m_excluded_models.
• nerve/agent/cache_policy.py — resolution at client-build time: observed cadence wins (median of recent turn gaps in (5min, 1h] → 1h); no-history priors: wakeup turns and persistent-mode cron sessions → 1h, everything else 5m. Recomputed on every client (re)build, so the idle-sweep recycle naturally tracks cadence changes.
• Wiring — the Claude Code CLI supports the 1h TTL natively: ENABLE_PROMPT_CACHING_1H=1 (+ ENABLE_PROMPT_CACHING_1H_BEDROCK on Bedrock); it adds the extended-cache-ttl-2025-04-11 beta and cache_control ttl itself. Verified empirically: probe turn returned ephemeral_1h_input_tokens > 0, the exact shape extract_cache_ttl_split() already parses. Resolved TTL is logged at INFO and stamped into session metadata.
• Observability & guardrail — /api/diagnostics usage block gains cache_ttl: 1h write share + estimated savings vs a 5m baseline per source; a source where 1h traffic costs more than the baseline is flagged in regressions and logged at WARNING. Verified against live data: all-5m traffic → exactly zero delta, no false positives.

Prompt byte-stability (required for the cron win to materialize)

Persistent crons discard their SDK client after each run; the next run rebuilds the system prompt. Live data shows each run cold-rewrites its full ~170k prefix. A 1h TTL only converts that if the rebuilt prompt is byte-identical, so:

• System prompt now carries Current date (day resolution) instead of a minute-level timestamp; precise wall-clock time moves to a trailing <system-reminder> on each user message — fresher than the old build-time stamp (per turn, not per client build), invisible in the UI (not persisted to the DB message), and free cache-wise.
• Pre-recalled memories are frozen per session (stored in session metadata on first successful recall, reused on rebuilds). A resumed session keeps its original priors — live recall stays available via the memory_recall tool.
• Skill summaries sorted for deterministic ordering across restarts.

Rollout

1. This PR lands with cache_ttl: "5m" — zero behavior change.
2. Live canaries (already staged in jobs.yaml on the box, activate on restart): inbox-processor + bee-ambient get cache_ttl: "1h" (both */30 persistent — backtest ~$220/wk and ~$40/wk respectively). Deviation from the original plan: self-improvement was named as a canary but runs daily — 24h gaps expire a 1h cache anyway, so flipping it would only pay the 2× write premium.
3. Watch cache_creation_1h go nonzero and the diagnostics cache_ttl.savings for a week; if it matches the backtest, flip the global default to auto. Revert = one config line.

Tests

tests/test_cache_policy.py (33 tests): resolution matrix (mode × source × exclusion × cadence × no-data fallback), env wiring iff 1h, hand-computed simulator costs (dense/sparse/mixed/model-switch/compaction cap), live-counterfactual estimator + regression guardrail. Full suite: 1433 passed.

Generated by Nerve