Make RotatingKVCache trimmable so prefix cache reuse works for sliding-window models

[amirarsalan90]

Summary

mlx-lm's automatic prompt-cache prefix reuse works for full-attention models but silently fails for sliding-window-attention models (Gemma 2/3/3n, GPT-OSS) once the prompt exceeds the attention window — every request reprocesses the whole prompt.

Root cause: RotatingKVCache.is_trimmable() returned False once its ring buffer wrapped. Since sliding-window models interleave layer types (e.g. Gemma 3 mixes RotatingKVCache with a full KVCache every 6th layer), the all-or-nothing can_trim_prompt_cache() gate then returned False for the entire model, so the server discarded the prompt cache and recomputed from scratch.

Fix

• Split RotatingKVCache's overloaded offset into two fields: offset (absolute chronological position, drives RoPE) and _offset (physical tokens resident in the ring buffer, capped at max_size). They diverge only after a trim; pre-wrap behavior is byte-identical. This mirrors the design BatchRotatingKVCache already uses.
• Implement a correct trim() for the wrapped case by linearizing the ring buffer into temporal order, bounded by max_size (older tokens are physically overwritten on wrap, so reuse is bounded by the window).
• Make the server reuse/eviction paths window-aware: fetch_nearest_cache only reuses a trimmed entry if the trim fully succeeded; insert_cache keeps prefixes older than the window as fallbacks; trim_prompt_cache returns the min trimmed across layers.

Scope: RotatingKVCache + BatchRotatingKVCache. SSM/Mamba (ArraysCache) is out of scope — recurrent state can't be split at a token boundary.

Results

gemma-3-1b-it-4bit, 2802-token prompt ([long fixed prefix] + [short changing tail]), sliding window W=512:

	Before	After
Warm cached_tokens	0	2788
Cold latency	1.24 s	1.24 s
Warm latency (median)	~0.84 s	0.15 s
Speedup	1.1× (none)	~8×

Greedy (temperature=0) warm output is identical to cold.

Tests

• RotatingKVCache / BatchRotatingKVCache now report trimmable after wrap and trim correctly back within the window.
• Added unit tests that a wrapped trim matches a fresh prefix build (state-level and through-model argmax identity), plus _offset save/load round-trip and backward-compat loading of the old 4-value meta_state.

python -m unittest tests.test_prompt_cache tests.test_server tests.test_generate passes.

Refs #980.