fix(dflash): drafter rope_theta align + ring=max_ctx long-ctx uncripple#4
Open
dusterbloom wants to merge 3 commits into
Open
fix(dflash): drafter rope_theta align + ring=max_ctx long-ctx uncripple#4dusterbloom wants to merge 3 commits into
dusterbloom wants to merge 3 commits into
Conversation
… full ctx - drafter GGUF baked rope.freq_base=1M but trains/serves at the target's 10M (converter bug); the unpark guard only corrected the 8-layer drafter, so the 6-layer drafter ran at 1M vs target 10M. Align dw_.rope_theta to the target at both load sites (initial + unpark). - DFLASH_FEAT_RING_CAP default 4096 wrapped the target-feature ring above 4K ctx, feeding the drafter stale features and collapsing accept to 0.1% at 27K. Default to max_ctx so the ring covers the full reserved context; env lowers it for VRAM. - both restore dFlash spec-decode acceptance on long-context MoE (0.1% -> ~16% on 27K agentic; content-dependent ceiling otherwise). - harness: repo_inspection path dflash/->server/ (repo renamed in 39fe251); run_claude_code flags fixed to --allowedTools/--dangerously-skip-permissions (the old --tools/--permission-mode dontAsk are invalid on claude-code 2.x); session_inject_proxy gains --force-temperature, thinking injection and body dump for bench control; add qwen35moe dflash gate harness.
…code; bound MoE prefill sync - root cause: the long-context accept cliff was the target-feature ring cap (FEAT_RING_CAP), NOT a drafter 2048 self-attention limit. When prompt_tokens > ring_cap the ring wraps, the drafter cross-attends stale features, and the commit-EMA gate floors to AR. Verified by a fully-crossed draft_ctx x ring_cap 2x2 (momus-reviewed). - ring_cap must be >= max_ctx (mandatory for correctness); the shipped RECIPE's FEAT_RING_CAP=4096 reintroduced the cliff. - DRAFT_CTX_MAX=2048 was an amputation, not a fix: needle-in-middle control shows it craters distant recall ~46pp (marker 2.6K from end: 76.8% -> 30.9%). draft_ctx=8192 is the VRAM-max uncrippled window (~4x recall reach), with DFLASH_FEATURE_DTYPE=f16 freeing the mirror headroom. - KV quant does not move the draft_ctx ceiling (q4/q8/tq3 all cap at 8192) -- the limit is the draft compute graph's decode scratch, not KV reservation; draft_ctx>8192 is self-defeating (gate floors to AR even when f16 makes it fit). - bound the MoE prefill feature-sync at qwen35moe_backend.cpp:1205 to min(committed, cap) (was raw committed, silently no-ops when committed>cap -> stale features), matching the restore-path and base-path patterns. - add ctxsweep harness + prompt fixtures + analysis docs documenting the 2x2, KV grid, and needle controls.
…uard removal core 71371d8 removed the !layout_known_ short-circuit; cold_prefix_boundary now returns the last eligible boundary. Updates the stale ==0 expectation. CI: test_server_unit.cpp.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave-2, stacks on the spec-gate PR. Diff = the two cliff-fix commits only.
Note: the rope_theta runtime-align here is a load-time defense for drafter GGUFs that were already baked with the wrong 1M freq_base. The durable cure for NEW conversions is the convert_modal_dflash_to_gguf.py fix in the converter PR (Luce-Org#447) — read rope_parameters + fail-loud instead of silently defaulting 1M. Both are kept: defend existing artifacts, prevent new ones.