Skip to content

Commit aa0a8b5

Browse files
kmbandyclaude
andcommitted
docs(wp): correct plan Task 2 (two-filter) + Task 5 (verify-only) after code trace
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014ZRfPpL8XFzk1hep9MMg9P
1 parent 07874db commit aa0a8b5

1 file changed

Lines changed: 31 additions & 2 deletions

File tree

docs/superpowers/plans/2026-07-07-moe-resident-split-multidevice.md

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,25 @@ git add src/llama-model-loader.h src/llama-model.cpp
7575
git commit -m "feat(wp): tag routed-expert tensors with is_expert at load"
7676
```
7777

78-
### Task 2: Gate the paging filter on `WP_RESIDENT_DENSE`
78+
### Task 2: Gate BOTH filters on `WP_RESIDENT_DENSE` (revised after code trace)
79+
80+
> **Correction (implemented as `07874dba3`):** The original single-filter plan
81+
> below was WRONG and would have produced garbage. Weight-paging has **two
82+
> independent filters** that must agree: `is_paged_weight()` (`llama-model.cpp:1608-1616`,
83+
> the load-bearing one — it drives the manual per-tensor resident allocator at
84+
> `:1624-1663`) and the catalog population (`:1800`). Touching only the catalog
85+
> leaves dense tensors `is_paged_weight==true` ⇒ no buffer ⇒ `load_all_data`
86+
> skips them (`llama-model-loader.cpp:1607` skips `data==NULL`) ⇒
87+
> never-allocated-never-loaded. The stale comment at `:1822-1833` ("we page
88+
> everything") is false — the manual allocator already keeps token_embd/output/
89+
> norms/tiny resident; this feature just widens that set. **Actual change:** a
90+
> shared `wp_is_routed_expert(name)` predicate + `wp_resident_dense_enabled()`,
91+
> added to BOTH `is_paged_weight()` (dense → return false → resident) AND the
92+
> catalog push (`page_this = !gate || info.is_expert`). `init_weight_pager` runs
93+
> AFTER `load_tensors` (`llama.cpp:604-613`), so the pager pool auto-sizes to the
94+
> VRAM remaining after dense is resident.
95+
96+
**Original (superseded) single-filter plan:**
7997

8098
**Files:**
8199
- Modify: `src/llama-model.cpp:1799-1803` (the push into `weight_page_infos` + `weight_tensor_ptrs`)
@@ -224,7 +242,18 @@ git add tests/test-weight-pager.cpp
224242
git commit -m "test(wp): catalog is_expert classification (dense vs routed)"
225243
```
226244
227-
### Task 5: VRAM budget — reserve dense-resident before sizing the expert pool
245+
### Task 5: VRAM budget — VERIFY ordering (likely no code, per Task-2 trace)
246+
247+
> **Simplified after the Task-2 code trace.** `init_weight_pager` (which sizes
248+
> the pager pool from free VRAM) runs AFTER `load_tensors` allocates the manual
249+
> resident buffer that now includes all dense tensors. So `hipMemGetInfo` at
250+
> pool-sizing time already reports VRAM net of dense-resident — the pool
251+
> auto-sizes to what's left. This task reduces to: confirm via the Phase-1 gate
252+
> run that free_vram at pool sizing excludes dense (log line), and that
253+
> dense+pool+KV fit. Add the subtraction below ONLY if the gate shows the pool
254+
> over-allocating. Do not add speculative code.
255+
256+
**Contingency (only if the gate shows over-allocation):**
228257
229258
**Files:**
230259
- Modify: `src/llama.cpp:184-211` (the auto slot-budget block)

0 commit comments

Comments
 (0)