You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(scheduler): add cache_ttl "never" sentinel for always-warm lanes
Lanes kept warm by external keepwarm mechanisms (prewarm proxies,
dedicated cache-keep tools) re-warm the provider prompt cache
out-of-band, so MC's idle>TTL heuristic false-positives: both TTL
consumers (scheduler idle-execute and the ttl_idle m[0] fold) initiate
a rebuild believed free that is actually a full paid cache-write
(measured 450-560K tokens on large sessions).
cache_ttl: "never" (string or per-model value, case-insensitive)
disables both consumers: parseCacheTtl returns Infinity, so
elapsed>ttl / elapsed>=ttl never fire. Rust scheduler mirrors the
sentinel with u64::MAX (predicates already use saturating_sub).
Status surfaces render "never expires (always-warm lane)" instead of
a bogus countdown: /ctx-status, the TUI sidebar (JSON-safe
cacheNeverExpires flag on StatusDetail), and Pi's status dialog
(which previously parsed the TTL with a private fallback parser).
hardCacheExpired extracted to a pure computeHardCacheExpired helper
with direct coverage of the never-chain.
Tradeoff documented in CONFIGURATION.md: on a genuinely cold start
the free-fold window is not detected on such lanes; mutations then
apply at the execute threshold.
Copy file name to clipboardExpand all lines: CONFIGURATION.md
+7Lines changed: 7 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -88,6 +88,13 @@ Per-model overrides for mixed-model workflows:
88
88
89
89
Supported formats: `"30s"`, `"5m"`, `"1h"`.
90
90
91
+
**`"never"` sentinel:** set `cache_ttl` to `"never"` to disable the idle-TTL heuristic entirely. Both consumers —
92
+
the scheduler (which converts defer passes to execute after TTL expiry) and the HARD-fold trigger (which folds
93
+
m[1] into m[0] on a "free" prefix rebuild) — no longer act on idle time. Use this on lanes kept warm by an
94
+
external keepwarm mechanism (prewarm proxies, dedicated cache-keep tools) where the idle heuristic false-positives
95
+
and causes paid full-prefix cache-writes. After a genuine cold start (e.g. the keepwarm process died), MC
96
+
won't detect the free-fold window on that lane — mutations then apply only at the execute threshold.
97
+
91
98
Higher-tier models with longer cache windows benefit from a longer TTL. Setting it too low wastes cache hits. Setting it too high delays reduction on long sessions.
"description": "Cache TTL: string (e.g. \"5m\", \"1h\", \"30s\") or per-model object ({ default: \"5m\", \"model-id\": \"10m\" }). Set to \"never\" for lanes kept warm by an external keepwarm proxy — disables the idle-TTL heuristic so MC never initiates a rebuild based on elapsed time.",
|`cache_ttl`| string \\| map<string, string> |`"5m"`| Cache TTL: string (e.g. "5m", "1h", "30s") or per-model object ({ default: "5m", "model-id": "10m" }). Set to "never" for lanes kept warm by an external keepwarm proxy — disables the idle-TTL heuristic so MC never initiates a rebuild based on elapsed time.|
50
50
|`execute_threshold_percentage`| number (20–80) \\| map<string, number (20–80)> |`65`| Context percentage that forces queued operations to execute. Number or per-model object ({ default: 65, "provider/model": 45 }). Values above 80 are rejected because the runtime caps at 80% for cache safety (MAX_EXECUTE_THRESHOLD). Default: DEFAULT_EXECUTE_THRESHOLD_PERCENTAGE |
51
51
|`execute_threshold_tokens`| object | — | Absolute token thresholds per model. When matched, overrides execute_threshold_percentage for that model. Accepts `default` for all models or per-model keys. Values above 80% × context_limit are clamped with a warning log. Min 5_000, max 2_000_000. |
52
52
|`execute_threshold_tokens.default`| number (5000–2000000) | — ||
'Cache TTL: string (e.g. "5m", "1h", "30s") or per-model object ({ default: "5m", "model-id": "10m" }). Set to "never" for lanes kept warm by an external keepwarm proxy — disables the idle-TTL heuristic so MC never initiates a rebuild based on elapsed time.',
0 commit comments