Commit bc50b59
fix: [AI-6622] enable cache_control trigger for altimate-backend provider (#850)
Adds `model.providerID === "altimate-backend"` to the cache-control gating
condition in `ProviderTransform.message()` so messages routed through the
Altimate Gateway get `cache_control: ephemeral` markers stamped on the last
content block. Required for the gateway's Anthropic-fallback path: when
Azure GPT-5.5 errors, litellm falls back to Anthropic Sonnet 4.6, which
honors these markers; without them, the fallback calls would not cache.
For the Azure-primary path: Azure ignores `cache_control` markers (it uses
automatic prefix-based caching), so stamping them is a no-op there. The
companion backend change extracts `prompt_tokens_details.cached_tokens` from
the response and surfaces it back to clients so cache rates and per-session
cost become visible.
Companion backend change on `altimate-backend` branch
`fix/AI-6622-llm-cache-plumbing`. Both repos pair together.
Full RCA + investigation journey: `~/Documents/altimate-deliverables/AI-6622-*.md`.
Co-authored-by: Haider <haider@altimate.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 56259a3 commit bc50b59
1 file changed
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
284 | 284 | | |
285 | 285 | | |
286 | 286 | | |
| 287 | + | |
287 | 288 | | |
288 | 289 | | |
289 | 290 | | |
| |||
0 commit comments