You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fork: reconcile MTP lineage with TurboQuant+ KV cache
Integration glue so the upstream MTP lineage (ggml-org#23198..ggml-org#23398) builds on
this fork without disturbing TurboQuant+ or the custom kernels:
- llama_kv_cache ctor: thread the new `hparams` param and `layer_share_cb`
through all call sites (iswa, memory-hybrid, dsa, model.cpp); keep the
fork's turbo auto-asymmetric K upgrade, n_layer_kv() sizing (+3 rotation
tensors), and per-side LLAMA_ATTN_ROT_* policy (default OFF) — now nested
under the new `if (other) { share } else { ... }` KV-sharing branch.
- hparams: carry n_layer_all/n_layer_nextn + n_layer()/n_layer_kv() from the
refactor while keeping the fork's n_layer_kv_from_start; restore the
swa_layers->is_swa_impl / recurrent_layer_arr->is_recr_impl /
nextn_predict_layers->n_layer_nextn renames across fork models.
- add n_outputs_max to cparams / common_params / llama_context_params and
wire it through; restore deepstack_mapping_arr.
- server: keep the ggml-org#23398 ctx_other (MTP draft KV-sharing) wiring; drop the
ggml-org#23988 --fit VRAM pre-estimation block (depends on upstream helpers not on
this fork; MTP does not need it).
- drop upstream-only models pulled in by the refactor (deepseek32, mellum,
talkie); keep non-MTP fork models on their own source + mechanical refactor.
Builds clean on Metal; turbo quant unit test passes (turbo2/3/4 round-trip).
Kernels (ggml-cuda / ggml-metal) untouched.
0 commit comments