|
5 | 5 | - Fixed DFlash on target tensor-split / Meta placement. Auto-detected DFlash draft loading now keeps tensor-split placement when shared target output tensors live in a Meta buffer, instead of reloading the drafter onto one GPU and crashing during draft warmup. Explicit incompatible `--spec-draft-device` placement now fails closed with a direct diagnostic. |
6 | 6 | - Added the Meta backend support needed by DFlash tensor-split runs: TurboQuant WHT split-state propagation, public Meta device introspection for capture placement, Meta-backed hidden/prefill capture allocation, Meta-safe GPU ring readback fallback, Meta-safe recurrent backup row copies, and a full-logits drafter fallback when compact argmax/top-k cannot run on Meta output. |
7 | 7 | - Verified the single-RTX-3090 tensor-split crash proxy with DFlash enabled. Forced `-sm tensor` now completes and generates drafts; normal single-GPU DFlash remains on the fast CUDA path. The forced single-GPU tensor-split path is still a correctness fallback and is slower because recurrent backup and hidden-ring transfer avoid unsafe raw Meta pointers. |
| 8 | +- Added experimental KVarN KV-cache compression as cache-type names: `--cache-type-k kvarn2|kvarn3|kvarn4` and `--cache-type-v kvarn2|kvarn3|kvarn4`. KVarN is kept to the target context, validates unsupported placements at runtime, uses non-unified CUDA streams, preserves stream-scoped backend state, and is wired for the current Qwen3.6 and Gemma4 paths. |
| 9 | +- Hardened KVarN memory ownership and server rollback behavior. Full prompt-cache sequence clears are now allowed, arbitrary old suffix trims are rejected before mutation, composite memory preflights child removals before changing either side, and DFlash prompt-checkpoint reuse now skips or fully reprocesses prompts when the active memory cannot safely roll back. |
| 10 | +- Fixed the Gemma4 and Qwen3.6 DFlash + KVarN crash class seen after prompt-cache reuse. The server no longer asks KVarN-backed target memory to delete compressed historical groups during checkpoint restore, so repeated chat requests fall back to safe prefill instead of aborting in `seq_rm`. |
| 11 | +- Reduced KVarN side effects outside KVarN runs. KVarN is disabled for draft and auxiliary contexts, iSWA now keeps the SWA side on the compact upstream cache instead of allocating a full-size KVarN SWA cache, and DFlash backup streams are gated by target architecture so Gemma recurrent paths do not allocate unused backup state. |
| 12 | +- Simplified the public KVarN surface while it is still experimental. The extra tuning flags were removed; the only user-facing selection is the cache type itself. Internal defaults such as sink tokens and Sinkhorn iterations now come from the selected `kvarnN` type. |
| 13 | +- Improved KVarN CUDA behavior and validation. Cross-stream state copies are explicitly ordered, CUDA materialization group counts are precomputed, and regression coverage now checks stream synchronization, sequence-removal capability, prompt-cache rollback safety, iSWA sizing policy, unsupported runtime policy, and argument parsing. |
| 14 | +- Updated preview/release maintenance for v0.3.2, including preview workflow routing, asset download handling, and avoiding cached release toolchains. |
8 | 15 |
|
9 | 16 | ## v0.3.1 |
10 | 17 |
|
|
0 commit comments