Commit 3f26661
committed
fix(mtp): pin capture mode to FULL_SEQ when MTP attached — eliminates pre_norm null
Pre-norm hidden capture warning ("hidden_at_pos_pre_norm returned null at
base_pos=N") was firing every run and silently halving accept_rate. Root
cause: orchestrator switched capture_mode_ to LAST_ROW_ONLY after prefill,
which captures only the last candidate row. On partial-accept chain iters
the runner asks for the hidden at the COMMITTED position (earlier than the
last candidate) — out of range, returns null, MTP falls back to post-norm
and crushes D>=2 accept.
Correct-by-construction fix:
- enable_hidden_seq_capture(true) now pins capture_mode_ = FULL_SEQ and
sets capture_pinned_ flag
- set_hidden_capture_mode / set_hidden_capture_scope become no-ops while
pinned (MTP-bound targets cannot have capture demoted)
- Orchestrator's post-prefill LAST_ROW_ONLY toggle removed (would be a
no-op anyway, but cleaner intent)
Measured (Claude Code 24K Heron, same hardware):
accept_rate 0.41 -> 0.78 (model card ceiling 0.83 at gamma<=2)
MTP decode 34.7 -> 49.7 tok/s (+43%)
Zero "pre_norm returned null" warnings in log
vs blog max 29.6 tok/s: +68%. vs DFlash on same config: +141%.1 parent 7e30f8b commit 3f26661
2 files changed
Lines changed: 31 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
54 | | - | |
55 | | - | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
56 | 58 | | |
57 | 59 | | |
58 | 60 | | |
| |||
81 | 83 | | |
82 | 84 | | |
83 | 85 | | |
84 | | - | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
85 | 89 | | |
86 | 90 | | |
87 | 91 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
48 | | - | |
49 | | - | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
50 | 60 | | |
51 | 61 | | |
52 | 62 | | |
| |||
62 | 72 | | |
63 | 73 | | |
64 | 74 | | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
65 | 78 | | |
| 79 | + | |
66 | 80 | | |
67 | 81 | | |
68 | 82 | | |
69 | 83 | | |
70 | | - | |
71 | 84 | | |
| 85 | + | |
72 | 86 | | |
73 | 87 | | |
74 | 88 | | |
| |||
216 | 230 | | |
217 | 231 | | |
218 | 232 | | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
219 | 239 | | |
220 | 240 | | |
221 | 241 | | |
| |||
0 commit comments