Skip to content

Commit 329dffb

Browse files
committed
docs: update v0.3.2 changelog
1 parent 645cfb7 commit 329dffb

1 file changed

Lines changed: 7 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,13 @@
55
- Fixed DFlash on target tensor-split / Meta placement. Auto-detected DFlash draft loading now keeps tensor-split placement when shared target output tensors live in a Meta buffer, instead of reloading the drafter onto one GPU and crashing during draft warmup. Explicit incompatible `--spec-draft-device` placement now fails closed with a direct diagnostic.
66
- Added the Meta backend support needed by DFlash tensor-split runs: TurboQuant WHT split-state propagation, public Meta device introspection for capture placement, Meta-backed hidden/prefill capture allocation, Meta-safe GPU ring readback fallback, Meta-safe recurrent backup row copies, and a full-logits drafter fallback when compact argmax/top-k cannot run on Meta output.
77
- Verified the single-RTX-3090 tensor-split crash proxy with DFlash enabled. Forced `-sm tensor` now completes and generates drafts; normal single-GPU DFlash remains on the fast CUDA path. The forced single-GPU tensor-split path is still a correctness fallback and is slower because recurrent backup and hidden-ring transfer avoid unsafe raw Meta pointers.
8+
- Added experimental KVarN KV-cache compression as cache-type names: `--cache-type-k kvarn2|kvarn3|kvarn4` and `--cache-type-v kvarn2|kvarn3|kvarn4`. KVarN is kept to the target context, validates unsupported placements at runtime, uses non-unified CUDA streams, preserves stream-scoped backend state, and is wired for the current Qwen3.6 and Gemma4 paths.
9+
- Hardened KVarN memory ownership and server rollback behavior. Full prompt-cache sequence clears are now allowed, arbitrary old suffix trims are rejected before mutation, composite memory preflights child removals before changing either side, and DFlash prompt-checkpoint reuse now skips or fully reprocesses prompts when the active memory cannot safely roll back.
10+
- Fixed the Gemma4 and Qwen3.6 DFlash + KVarN crash class seen after prompt-cache reuse. The server no longer asks KVarN-backed target memory to delete compressed historical groups during checkpoint restore, so repeated chat requests fall back to safe prefill instead of aborting in `seq_rm`.
11+
- Reduced KVarN side effects outside KVarN runs. KVarN is disabled for draft and auxiliary contexts, iSWA now keeps the SWA side on the compact upstream cache instead of allocating a full-size KVarN SWA cache, and DFlash backup streams are gated by target architecture so Gemma recurrent paths do not allocate unused backup state.
12+
- Simplified the public KVarN surface while it is still experimental. The extra tuning flags were removed; the only user-facing selection is the cache type itself. Internal defaults such as sink tokens and Sinkhorn iterations now come from the selected `kvarnN` type.
13+
- Improved KVarN CUDA behavior and validation. Cross-stream state copies are explicitly ordered, CUDA materialization group counts are precomputed, and regression coverage now checks stream synchronization, sequence-removal capability, prompt-cache rollback safety, iSWA sizing policy, unsupported runtime policy, and argument parsing.
14+
- Updated preview/release maintenance for v0.3.2, including preview workflow routing, asset download handling, and avoiding cached release toolchains.
815

916
## v0.3.1
1017

0 commit comments

Comments
 (0)