Commit c97ab6a
feat(anthropic_sdk): default-on automatic prompt caching
Adds 'cache_control: {type: ephemeral}' to messages.stream() calls. The
API auto-places a cache breakpoint at the longest cacheable prefix
(tools + system + accumulated messages) and moves it forward on each
turn -- multi-turn agent loops get cache reads on every turn after the
first.
Default-on because all current Claude models support cache_control and
CAPI accepts it (validated end-to-end against claude-mythos-5 via CAPI
on 2026-06-12). Callers pointed at proxies that strip / reject
cache_control can opt out with 'prompt_caching: false' in model_settings.
A string value (e.g. 'prompt_caching: 1h') sets a custom TTL.
Local validation against anticomputer/vulnerable-test-app on the same
audit pipeline, same model config, only changing prompt_caching:
metric | off | on | delta
--------------------+-----------+-----------+------------
requests | 60 | 62 | +2 (noise)
input tokens fresh | 909,806 | 124 | -99.99%
cache read tokens | 0 | 728,079 | new
cache write tokens | 0 | 210,261 | new
output tokens | 42,300 | 44,933 | similar
vulnerabilities | 4 | 5 | +1
est. mythos cost | $11.21 | $5.60 | -50%
Same or better audit quality, half the token cost. Real audits with
larger system prompts + more tool definitions amortize the cache writes
over more reads, so production savings are typically larger than 50%.
Tests added:
- prompt_caching default-on emits cache_control
- prompt_caching=False suppresses cache_control (opt-out)
- prompt_caching='1h' includes the ttl field
23 tests pass total, hatch fmt clean.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 4eea127 commit c97ab6a
2 files changed
Lines changed: 170 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
202 | 202 | | |
203 | 203 | | |
204 | 204 | | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
205 | 221 | | |
206 | 222 | | |
207 | 223 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
227 | 227 | | |
228 | 228 | | |
229 | 229 | | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
0 commit comments