Skip to content

Commit 00c7fa2

Browse files
apstenku123apstenku123
committed
docs(v8-r11): full-loop cookbook + UAT script + integration smoke
Closes cppmega-mlx-yz6n.11. Three deliverables: 1. docs/raschka_full_loop.md — definitive 10-section cookbook taking any of the 62 Raschka-gallery presets from a click in the preset launcher all the way to a finished N-step train run with the extras.train v8 schema populated. Steps map 1-to-1 to R01-R09. 2. docs/uat/v8_raschka_full_loop.md — 10-step manual UAT journey with expected-state checkpoints at every step. The human-side counterpart to the R12 Playwright spec. 3. tests/v4/test_raschka_full_loop_integration.py — pure-Python integration smoke covering steps 1-3-9 across the 5 R11 target presets (llama3_8b, qwen3_dense_4b, kimi_linear, gemma3_27b, gpt_oss_20b). Verifies build_preset_specs → defaults block → auto_fit (scale_down + suggest_sharding) → memory.matrix is a well-formed chain. 6/6 green. No new RPCs; this ticket is integration + docs only.
1 parent b9b043e commit 00c7fa2

3 files changed

Lines changed: 411 additions & 0 deletions

File tree

‎docs/raschka_full_loop.md‎

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
# Raschka Full-Loop Cookbook (V8)
2+
3+
> Companion to `VisualBuilderPlan-v8.md` / `VisualBuilderSpec-v8.md`.
4+
> This document is the **definitive end-to-end recipe** for taking any
5+
> preset in the Raschka gallery (currently 62 presets) and running an
6+
> N-step training job, with paper-anchored defaults, automatic
7+
> scaling, memory-fit verification, mid-canvas feature injection, and
8+
> streaming data, all from the Visual Builder UI.
9+
10+
## 0. Prerequisites
11+
12+
- A built `vbgui` (`pnpm -C vbgui install && pnpm -C vbgui dev`).
13+
- The cppmega backend running on `127.0.0.1:8767` (`uvicorn
14+
cppmega_v4.jsonrpc.server:app --port 8767`).
15+
- A working tokenizer (the default bundled `cppmega_v3` ships at
16+
`cppmega_mlx/tokenizer/tokenizer.json`).
17+
- For the HF-quickstart path: outbound HTTPS to `huggingface.co`.
18+
Set `VBGUI_DISABLE_NETWORK=1` to refuse the RPC in offline runs.
19+
20+
## 1. Pick a preset (R01)
21+
22+
Open the Visual Builder, click the **Preset launcher** dropdown,
23+
choose e.g. `llama3_8b`.
24+
25+
- `build_preset_specs` runs → canvas populates with the preset's
26+
brick chain.
27+
- The `defaults` block in the same response auto-fills the OptimTab
28+
(`adamw`, lr=`3e-4`, mixed_precision on, grad-clip 1.0) and the
29+
first group's ScheduleEditor (`wsd`, warmup 2000).
30+
- Paper anchor: <https://arxiv.org/abs/2407.21783>.
31+
32+
## 2. Scale down to fit (R02)
33+
34+
Switch to the **Gallery** tab.
35+
36+
- A `GalleryScaleDownSlider` lives above the per-preset table.
37+
- Pick the target preset (same dropdown), drag the slider to a budget
38+
(default 1 GiB).
39+
- The `architectures.scale_down` RPC fires (debounced 250 ms) and
40+
surfaces the chosen `(hidden_size, num_layers)` + estimated bytes
41+
+ scaled-down-from canonical shape.
42+
- Click **Apply scaled preset** → canvas swaps to the scaled chain
43+
and `dim_env.H` updates.
44+
45+
## 3. Verify it fits everywhere (R03)
46+
47+
Open the **Memory** sidebar tab.
48+
49+
- `memory.matrix` returns a 4×5 grid of (topology × precision) cells.
50+
- Default axes:
51+
`[h100_8x, m3_ultra_solo, gb10_quarter, tpu_v6e_8]` ×
52+
`[fp32, bf16, fp16, fp8, mxfp4]`.
53+
- Each cell shows estimated bytes + a `fits` chip (green/red) +
54+
full per-component breakdown on hover (weights / grads / optimizer
55+
/ activations / kv-cache / edge handoff).
56+
- Use this matrix to validate the scale-down decision matches the
57+
intended deployment target.
58+
59+
## 4. Auto-fit to your devbox (R04)
60+
61+
Click **Auto-fit to my devbox** in the GalleryScaleDownSlider.
62+
63+
- `architectures.auto_fit` chains `platform.get_info` → `scale_down`
64+
→ `suggest_sharding` into a single result.
65+
- The banner that appears reports the chosen topology, the largest
66+
(H, L) that still fits at 90 % headroom, and a sharding-proposal
67+
axis summary (`dp×N, tp×M, ...`).
68+
- Apply slot still works the same; auto-fit just picks the slider
69+
value for you based on the detected hardware.
70+
71+
## 5. Inject a feature mid-canvas (R08)
72+
73+
In the **Canvas** tab the `FeatureInjectionBar` floats above the
74+
flow chart.
75+
76+
- The dropdown is populated by
77+
`catalog.list_options('feature_injectors')` — 5 options today:
78+
- `mtp_weighted` → `rewriter:MTPRewriter` (K=2 head + weighted loss)
79+
- `ifim_shaped` → `rewriter:IFIMRewriter` (span-aware reshape)
80+
- `mhc_attn_bias` → `rewriter:MHCRewriter` (co-occurrence bias)
81+
- `engram` → `brick:engram` (standalone n-gram branch)
82+
- `ngram_2_3_4` → `brick:engram` with 2,3,4-gram defaults
83+
- Apply dispatches either `rewriters.add` (mutates `spec.rewriters`,
84+
flows into `pipeline.run`) or inserts a new brick node into the
85+
canvas with a fresh edge from the tail.
86+
- Subsequent training picks up the rewriter; e.g. MTP populates
87+
`extras.train.mtp` automatically.
88+
89+
## 6. Stream training data from HF Hub (R09)
90+
91+
In the **Data** tab → **HF quickstart** button.
92+
93+
- Pick a dataset (`HuggingFaceFW/fineweb-edu` is the default), set
94+
the `n_tokens` target, click **Run**.
95+
- `data.hf_quickstart` streams the dataset, tokenizes each document
96+
with `CppMegaTokenizer`, and writes a parquet shard with the
97+
canonical 4-column schema
98+
(`token_ids / doc_ids / byte_offsets / byte_lengths`).
99+
- Live progress streams on `/ws/data/{job_id}` (start / progress /
100+
done frames). The result parquet path lands in the inspector's
101+
path field — click **Use for training** to point the next train
102+
run at it.
103+
104+
## 7. Run N steps (R11)
105+
106+
Back on the **Canvas** tab:
107+
108+
- Click **Run pipeline** (or the split-button "Train" mode).
109+
- The runner threads everything wired in steps 1–6 into a
110+
`pipeline.run` request: scaled spec + rewriters from the injection
111+
bar + parquet from the quickstart + topology from the matrix view +
112+
ScheduleEditor schedule kind.
113+
- The `stage_train` stage publishes per-step
114+
`{step, loss, lr, overflow, grad_norms, mem_mb, ...}` events onto
115+
`train_event_bus`; the `LiveTrainPanel` subscribes via
116+
`/ws/train/{run_id}` and renders the sparkline + step counter.
117+
- A `finish: "ok"` frame fires the toast; the `extras.train` block
118+
now contains:
119+
- `preset_origin` — the originally-picked preset name (R01).
120+
- `scale_down_factor` — `H_scaled / H_canonical` (R02).
121+
- `memory_matrix_cell_used` — the cell the run actually used (R03).
122+
- `feature_injections` — list of applied injections (R08).
123+
- `data_source` — `{kind: "hf_quickstart", dataset_id, n_tokens}`
124+
(R09 / R10).
125+
- All the existing V7 extras (schedule_kind, optimizer kind, ...).
126+
127+
## 8. Where this fits in the larger picture
128+
129+
This loop covers the **"preset → trainable model"** path. The
130+
companions are:
131+
132+
- The Visual Builder distributed-semantics layer (V7) handles
133+
multi-rank training once you've finished step 7 and want to fan
134+
out across multiple devboxes.
135+
- The Plasticity Toolkit (FIRE / DASH / ReDo) hooks happen inside
136+
`stage_train` — they don't surface in this cookbook but are
137+
active on every step.
138+
- MXFP4 + compile-trace + sync-checker (R05 / R06 / R07) are the
139+
remaining V8 P2 items — they extend step 3 (memory matrix) and
140+
step 7 (extras.train) with deeper diagnostics but aren't gating
141+
for a successful loop.
142+
143+
## 9. Reference: extras.train v8 schema
144+
145+
```jsonc
146+
{
147+
"preset_origin": "llama3_8b", // R01
148+
"scale_down_factor": 0.125, // R02
149+
"memory_matrix_cell_used":
150+
{"topology": "m3_ultra_solo", "precision": "bf16"}, // R03
151+
"compile_trace": { // R06
152+
"fused_groups_count": 3, "dlpack_crossings": 1,
153+
"materialised_ops_count": 2
154+
},
155+
"sync_check": {"redundant_syncs_count": 0}, // R07
156+
"feature_injections":
157+
["mtp_weighted", "engram"], // R08
158+
"data_source": { // R09 / R10
159+
"kind": "hf_quickstart",
160+
"dataset_id": "HuggingFaceFW/fineweb-edu",
161+
"n_tokens": 100032
162+
}
163+
}
164+
```
165+
166+
## 10. Smoke test
167+
168+
The pure-Python integration test
169+
`tests/v4/test_raschka_full_loop_integration.py` covers steps 1-3-9
170+
without any UI:
171+
172+
```bash
173+
.venv/bin/python -m pytest tests/v4/test_raschka_full_loop_integration.py -v
174+
```
175+
176+
This proves the RPC plumbing end-to-end. The full UI loop is in
177+
`vbgui/e2e/scenarios/v8_raschka_full_loop.spec.ts` (R12).

‎docs/uat/v8_raschka_full_loop.md‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# UAT — V8 Raschka Full-Loop (10-step user journey)
2+
3+
> Manual acceptance script. Walks a real user through the full
4+
> preset → train loop. Take a screenshot at every checkpoint. The
5+
> automated counterpart is `vbgui/e2e/scenarios/v8_raschka_full_loop.spec.ts`.
6+
7+
## Pre-flight
8+
9+
- Start the backend: `uvicorn cppmega_v4.jsonrpc.server:app --port 8767`.
10+
- Start the frontend: `pnpm -C vbgui dev`.
11+
- Open `http://127.0.0.1:5176/`.
12+
13+
## Step 1 — Pick a preset (R01)
14+
15+
1. Click the **Preset launcher** dropdown in the top bar.
16+
2. Select `llama3_8b`.
17+
18+
**Expected**:
19+
20+
- Canvas populates with the preset's brick chain (≥ 2 nodes).
21+
- Switch to the **Optim** sidebar tab — `kind=adamw`,
22+
`optim-clip` field shows `1.0`, `optim-mp` checked, the first
23+
group's lr is `0.0003`.
24+
- Click the group's clock icon → ScheduleEditor opens with `kind=wsd`.
25+
26+
## Step 2 — Scale down (R02)
27+
28+
1. Click the **Gallery** tab.
29+
2. The `GalleryScaleDownSlider` panel appears above the per-preset
30+
table. The picker says `llama3_8b`.
31+
3. Drag the slider down until the target reads `1.00 GB`.
32+
33+
**Expected**:
34+
35+
- `gallery-scaledown-est-bytes` shows a value < 1 GB after a 250 ms
36+
pause.
37+
- `gallery-scaledown-shape` reads `H=512 L=32 (from H=4096 L=32)`.
38+
- `gallery-scaledown-fits` chip is green and says "fits budget".
39+
40+
4. Click **Apply scaled preset**.
41+
42+
**Expected**: canvas swaps to the scaled chain (64 nodes for 32
43+
layers of `attn+mlp`).
44+
45+
## Step 3 — Verify memory (R03)
46+
47+
1. Click the **Memory** sidebar tab.
48+
49+
**Expected**:
50+
51+
- A 4×5 grid renders within ~200 ms.
52+
- Most cells under `bf16/fp8/mxfp4` are green (`fits`).
53+
- Hover a cell — a tooltip lists every component byte count.
54+
55+
## Step 4 — Auto-fit (R04)
56+
57+
1. Back to the **Gallery** tab.
58+
2. Click **Auto-fit to my devbox**.
59+
60+
**Expected**:
61+
62+
- A blue chip appears below the slider:
63+
`gb10_quarter · hidden=4096, layers=32, axis=dp×1, peak=… GB / … GB`
64+
(or your local topology).
65+
66+
## Step 5 — Inject MTP (R08)
67+
68+
1. Click the **Canvas** tab.
69+
2. In the yellow `FeatureInjectionBar`, select `mtp_weighted`.
70+
3. Click **Apply**.
71+
72+
**Expected**: the applied-list at the right of the bar shows
73+
`mtp_weighted`. Open the **Rewriters** sidebar tab — `MTPRewriter`
74+
appears with `K=2, weight=0.5`.
75+
76+
## Step 6 — Data quickstart (R09)
77+
78+
1. Click the **Data** tab.
79+
2. Click **HF quickstart** in the header.
80+
81+
**Expected**: modal opens; dataset field shows
82+
`HuggingFaceFW/fineweb-edu`; n_tokens shows `8192`.
83+
84+
3. Click **Run**.
85+
86+
**Expected**: within ~30 s the modal shows a green result block with
87+
the parquet path and `n tokens, n docs, … ms` counters.
88+
89+
4. Click **Close**. The DataInspector path field is now the new
90+
parquet path.
91+
92+
## Step 7 — Use for training
93+
94+
1. Click **Use for training** next to the path. The button turns
95+
green (`✓ Training`).
96+
97+
## Step 8 — Run 4 steps
98+
99+
1. Click **Run pipeline** in the top bar → choose **Train**.
100+
2. Set `num_steps` to `4`.
101+
3. Click **Train**.
102+
103+
**Expected**:
104+
105+
- `LiveTrainPanel` appears with a sparkline that fills in 4 dots.
106+
- The toast fires when training completes.
107+
- The `extras.train` overlay on the run-result modal shows
108+
`preset_origin: llama3_8b`, `scale_down_factor < 1.0`,
109+
`feature_injections: [mtp_weighted]`, `data_source.kind:
110+
hf_quickstart`.
111+
112+
## Step 9 — Inspect outputs
113+
114+
Open the run-history menu (top-right of the canvas) and confirm the
115+
run lands with green status, with the v8 extras visible in the
116+
overlay.
117+
118+
## Step 10 — Reset
119+
120+
Refresh the page. Cycle through Steps 1-2 with a different preset
121+
(e.g. `qwen3_dense_4b`) to confirm defaults switch and the loop
122+
remains stable.
123+
124+
## Pass / fail
125+
126+
This UAT passes if every "Expected" block above is satisfied without
127+
any browser-console error in `chrome://devtools/console` or the
128+
backend uvicorn log.

0 commit comments

Comments
 (0)