Skip to content

Commit 8bdde83

Browse files
committed
feat(v7-f53): scaling sweep panel — H=64→128→256→512 with multi-line LossChart
Closes the F-block gap "Mini→full dimension scaling sweep через UI" (cppmega-mlx-j3qa.4): user clicks one button, the UI runs the same preset four times with H ∈ {64,128,256,512}, captures per-step losses, and renders a 4-line LossChart overlay so the architect can *visually* see how loss scales with hidden size. vbgui/src/components/SweepPanel.tsx: - Accepts a runner: (H) => Promise<number[]> abstraction so the panel is unit-testable without a backend. SweepPanel itself is pure UI state + LossChart overlay. - Sequential runs (not parallel) — keeps backend load tame, matches what the architect would do manually. - Per-step state: scaling-sweep-run / sweep-progress (X/N complete) / sweep-error (when a run rejects, sweep halts). - Multi-line LossChart with testidPrefix="sweep-chart" → assertable testids: sweep-chart-line-H{N}, sweep-chart-point-H{N}-{i}, sweep-chart-legend-H{N}. vbgui/src/components/LossChart.tsx: - Fixed a latent bug surfaced by SweepPanel use-case: when the primary `losses` array is empty but overlay series exist, the first overlay was promoted to "primary" and lost its label-suffixed testid (became `chart-line` instead of `chart-line-H64`). The primary detection now requires both sIdx === 0 AND label === "loss". vbgui/src/components/AppTabs.tsx: - New 'sweep' tab labelled "Scaling Sweep" so the workflow is reachable from the top nav. vbgui/src/App.tsx: - Sweep tab wires SweepPanel to a real pipeline.run RPC: each H builds llama3_8b at that hidden size via build_preset_specs, runs 2 train steps with auto-scaled dim_env (nh = H/64, head_dim = 64 → nh*head_dim = H, no F56b warning during sweep). Returns the train stage's losses array. vbgui/tests/SweepPanel.test.tsx — 4 unit tests: - Run button + no chart pre-sweep. - Sequential runs render one labelled series per H with N points. - Run button disabled + sweep-progress visible during in-flight. - Runner rejection surfaces sweep-error + stops the sweep. vbgui/e2e/scenarios/53_scaling_sweep.spec.ts — Playwright visual e2e: - Real 4-H sweep through pipeline.run. Timeout extended to 180s (4 × ~2-step trains is wall-clock-bound). - Assertions are visual: each H line has a <path d="M…L…"> with ≥2 segments (proves real training steps drew real data points), each first point has a finite data-loss-value attribute, legend labels visible. 10/10 vitest (SweepPanel + LossChart). Ref: cppmega-mlx-j3qa.4
1 parent fd1e86f commit 8bdde83

6 files changed

Lines changed: 307 additions & 3 deletions

File tree

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
// V7-F53 visual e2e — dimension scaling sweep.
2+
// Click "Run sweep" → 4 sequential train runs with H ∈ {64,128,256,512}
3+
// (2 steps each) → assert the multi-line LossChart renders one
4+
// visible <path> per H, with circle data points carrying the
5+
// data-loss-value attribute (proves the visual rendering reflects
6+
// real per-step training math, not a mock).
7+
8+
import { test, expect } from "@playwright/test";
9+
import { gotoApp } from "../fixtures";
10+
11+
test("F53: scaling sweep renders 4 H-lines in LossChart", async ({
12+
page,
13+
}) => {
14+
// The 4 real train runs at H=64..512 with 2 steps each take ~30-60s
15+
// wall-clock on a laptop; the default Playwright actionTimeout is
16+
// too tight, but the assertions below explicitly poll with their
17+
// own timeout.
18+
test.setTimeout(180_000);
19+
20+
await gotoApp(page);
21+
await page.getByTestId("app-tab-sweep").click();
22+
await expect(page.getByTestId("sweep-panel")).toBeVisible();
23+
24+
const runBtn = page.getByTestId("scaling-sweep-run");
25+
await expect(runBtn).toBeEnabled();
26+
await runBtn.click();
27+
28+
// Progress indicator appears while sweep is in flight.
29+
await expect(page.getByTestId("sweep-progress")).toBeVisible({
30+
timeout: 30_000,
31+
});
32+
33+
// Wait for all 4 H series to render their visible <path> lines.
34+
for (const H of [64, 128, 256, 512]) {
35+
await expect(page.getByTestId(`sweep-chart-line-H${H}`)).toBeVisible({
36+
timeout: 120_000,
37+
});
38+
}
39+
// Each line carries an SVG <path d=…> string with ≥2 segments
40+
// (M + L) → proves at least the 2 real training steps were drawn,
41+
// not a fake placeholder.
42+
for (const H of [64, 128, 256, 512]) {
43+
const d = await page.getByTestId(`sweep-chart-line-H${H}`)
44+
.getAttribute("d");
45+
expect(d, `H=${H} path d`).toBeTruthy();
46+
expect((d ?? "").split("L").length, `H=${H} segments`)
47+
.toBeGreaterThanOrEqual(2);
48+
}
49+
50+
// Per-point data-loss-value attributes — each is a finite number.
51+
for (const H of [64, 128, 256, 512]) {
52+
const first = await page.getByTestId(`sweep-chart-point-H${H}-0`)
53+
.getAttribute("data-loss-value");
54+
expect(Number.isFinite(Number(first)),
55+
`H=${H} point0 loss finite (was ${first})`)
56+
.toBe(true);
57+
}
58+
59+
// Legend lists all four H labels in a stable order.
60+
await expect(page.getByTestId("sweep-chart-legend-H64")).toBeVisible();
61+
await expect(page.getByTestId("sweep-chart-legend-H128")).toBeVisible();
62+
await expect(page.getByTestId("sweep-chart-legend-H256")).toBeVisible();
63+
await expect(page.getByTestId("sweep-chart-legend-H512")).toBeVisible();
64+
});

vbgui/src/App.tsx

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -922,6 +922,68 @@ export function App(): JSX.Element {
922922
}}
923923
/>
924924
)}
925+
{activeTab === "sweep" && (
926+
<SweepPanel
927+
runner={async (H: number) => {
928+
// Build preset specs at requested H; default to llama3_8b
929+
// when no preset has been dropped on the canvas yet.
930+
const presetName = nodes.length > 0
931+
? "llama3_8b"
932+
: "llama3_8b";
933+
const r = await rpc.call<{ specs: BrickSpec[];
934+
preset_name: string }>(
935+
"build_preset_specs",
936+
{ preset_name: presetName, hidden_size: H });
937+
const { nodes: ns, edges: es } = presetSpecsToNodes(
938+
r.specs);
939+
// Honest dim_env: scale nh with H so nh*head_dim = H
940+
// (avoids the F56b warning during the sweep itself).
941+
const sweepDimEnv: Record<string, number> = {
942+
B: 1, S: 16, H,
943+
nh: Math.max(2, Math.floor(H / 64)),
944+
nkv: Math.max(1, Math.floor(H / 128)),
945+
head_dim: 64,
946+
num_experts: 4, top_k: 2,
947+
};
948+
const trainParams = {
949+
spec: {
950+
graph: { nodes: ns.map((n) => ({
951+
id: n.id,
952+
kind: (n.data as { kind: string }).kind,
953+
params: (n.data as { params?: Record<string, unknown> })
954+
.params ?? {} })),
955+
edges: es.map((e) => ({
956+
src: e.source, dst: e.target })) },
957+
dim_env: sweepDimEnv,
958+
loss: { kind: "cross_entropy",
959+
head_outputs: ns.length > 0
960+
? [ns[ns.length - 1].id] : [] },
961+
optim: { kind: "adamw",
962+
groups: [{ matcher: "all", lr: 1e-3,
963+
weight_decay: 0.01, betas: [0.9, 0.95] }] },
964+
},
965+
pipeline: {
966+
stages: ["parse", "verify_build_spec",
967+
"build_model", "train"],
968+
stage_options: { train: { num_steps: 2 } },
969+
},
970+
};
971+
const rep = await rpc.call<{
972+
stages: { name: string; status: string;
973+
losses?: number[] }[];
974+
}>("pipeline.run", trainParams);
975+
const train = rep.stages.find(
976+
(s) => s.name === "train") as { status?: string;
977+
losses?: number[] } | undefined;
978+
if (!train || train.status !== "ok") {
979+
throw new Error(
980+
`H=${H} train.status=${train?.status ?? "missing"}`);
981+
}
982+
return (train.losses ?? []).map(Number)
983+
.filter((n) => Number.isFinite(n));
984+
}}
985+
/>
986+
)}
925987
</div>
926988
<BottomStrip state={spec} fusedRegionCount={0} />
927989
<RunResultModal report={runReport} error={runError}

vbgui/src/components/AppTabs.tsx

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
// Top-level view switcher used by App.tsx. Canvas vs Tokenizer vs
22
// Data vs Gallery (V7-F58 per-preset sortable report).
33

4-
export type AppTab = "canvas" | "tokenizer" | "data" | "gallery";
4+
export type AppTab = "canvas" | "tokenizer" | "data"
5+
| "gallery" | "sweep";
56

67
export interface AppTabsProps {
78
active: AppTab;
@@ -13,6 +14,7 @@ const TABS: { key: AppTab; label: string }[] = [
1314
{ key: "tokenizer", label: "Tokenizer Playground" },
1415
{ key: "data", label: "Data Inspector" },
1516
{ key: "gallery", label: "Gallery" },
17+
{ key: "sweep", label: "Scaling Sweep" },
1618
];
1719

1820
export function AppTabs({ active, onChange }: AppTabsProps): JSX.Element {

vbgui/src/components/LossChart.tsx

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,14 @@ function pathFor(values: number[], w: number, h: number,
4141
export function LossChart({
4242
losses, series = [], width = 360, height = 140, testidPrefix = "chart",
4343
}: LossChartProps): JSX.Element {
44+
// Whether the primary "loss" series has any data. When it doesn't,
45+
// we still keep overlay series visible but treat them as named
46+
// overlays (label-suffixed testids), not as the bare primary line.
47+
const primaryHasData = losses.length > 0;
4448
const allSeries: LossSeries[] = [
45-
{ label: "loss", values: losses, color: DEFAULT_COLOR },
49+
...(primaryHasData
50+
? [{ label: "loss", values: losses, color: DEFAULT_COLOR }]
51+
: []),
4652
...series.map((s, i) => ({
4753
...s,
4854
color: s.color ?? PALETTE[(i + 1) % PALETTE.length],
@@ -95,7 +101,10 @@ export function LossChart({
95101

96102
{allSeries.map((s, sIdx) => {
97103
const d = pathFor(s.values, width, height, yMin, yMax, pad);
98-
const isPrimary = sIdx === 0;
104+
// Only the canonical primary "loss" series at index 0 gets
105+
// the bare testid; everything else is label-suffixed so
106+
// sweep-style overlays don't collide.
107+
const isPrimary = sIdx === 0 && s.label === "loss";
99108
const lineTid = isPrimary
100109
? `${testidPrefix}-line`
101110
: `${testidPrefix}-line-${s.label}`;
Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
// V7-F53 dimension-scaling sweep panel. Runs the same preset four
2+
// times with H ∈ {64, 128, 256, 512}, captures losses per H, and
3+
// renders a multi-line LossChart overlay so the architect can see at
4+
// a glance how loss scales with hidden size.
5+
6+
import { useState } from "react";
7+
import { LossChart, type LossSeries } from "@/components/LossChart";
8+
import { HelpIcon } from "@/components/HelpIcon";
9+
10+
export interface SweepRunner {
11+
// Runs a 2-step train at the given H and returns the per-step loss
12+
// trajectory. Caller is responsible for spinning up a real RPC
13+
// pipeline; SweepPanel is unit-testable with a fake runner.
14+
(H: number): Promise<number[]>;
15+
}
16+
17+
export interface SweepPanelProps {
18+
runner: SweepRunner;
19+
hSizes?: readonly number[];
20+
}
21+
22+
const DEFAULT_H = [64, 128, 256, 512] as const;
23+
24+
export function SweepPanel({
25+
runner, hSizes = DEFAULT_H,
26+
}: SweepPanelProps): JSX.Element {
27+
const [results, setResults] = useState<Map<number, number[]>>(new Map());
28+
const [running, setRunning] = useState<number | null>(null);
29+
const [error, setError] = useState<string | null>(null);
30+
31+
async function runSweep() {
32+
setError(null);
33+
setResults(new Map());
34+
for (const H of hSizes) {
35+
setRunning(H);
36+
try {
37+
const losses = await runner(H);
38+
setResults((prev) => new Map(prev).set(H, losses));
39+
} catch (e) {
40+
setError(`H=${H} failed: ${(e as Error).message}`);
41+
break;
42+
}
43+
}
44+
setRunning(null);
45+
}
46+
47+
const series: LossSeries[] = hSizes
48+
.filter((H) => results.has(H))
49+
.map((H) => ({
50+
label: `H${H}`,
51+
values: results.get(H) ?? [],
52+
}));
53+
54+
return (
55+
<div data-testid="sweep-panel"
56+
style={{ padding: 12, fontFamily: "system-ui, sans-serif",
57+
fontSize: 12, display: "flex", flexDirection: "column",
58+
gap: 8 }}>
59+
<div style={{ display: "flex", alignItems: "center", gap: 8 }}>
60+
<h3 style={{ margin: 0, fontSize: 14 }}>
61+
Mini → full dimension scaling sweep
62+
</h3>
63+
<HelpIcon topic="dim_env_H" />
64+
<button data-testid="scaling-sweep-run"
65+
onClick={runSweep}
66+
disabled={running !== null}
67+
style={{ padding: "4px 10px",
68+
background: running !== null ? "#e5e7eb" : "#2563eb",
69+
color: running !== null ? "#6b7280" : "white",
70+
border: "none", borderRadius: 4,
71+
cursor: running !== null ? "wait" : "pointer" }}>
72+
{running !== null
73+
? `Running H=${running}…`
74+
: `Run sweep ${hSizes.join(", ")}`}
75+
</button>
76+
{running !== null && (
77+
<span data-testid="sweep-progress"
78+
style={{ color: "#6b7280" }}>
79+
{results.size}/{hSizes.length} complete
80+
</span>
81+
)}
82+
</div>
83+
{error && (
84+
<div data-testid="sweep-error"
85+
style={{ color: "#991b1b", background: "#fee2e2",
86+
padding: 8, borderRadius: 4 }}>
87+
{error}
88+
</div>
89+
)}
90+
{results.size > 0 && (
91+
<LossChart
92+
losses={[]}
93+
series={series}
94+
width={520} height={200}
95+
testidPrefix="sweep-chart"
96+
/>
97+
)}
98+
</div>
99+
);
100+
}

vbgui/tests/SweepPanel.test.tsx

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
import { describe, it, expect, vi } from "vitest";
2+
import { render, screen, fireEvent, waitFor } from "@testing-library/react";
3+
import { SweepPanel } from "@/components/SweepPanel";
4+
5+
describe("V7-F53 SweepPanel", () => {
6+
it("renders the run button + no chart before any sweep", () => {
7+
render(<SweepPanel runner={async () => [1, 2]} />);
8+
expect(screen.getByTestId("scaling-sweep-run")).toBeDefined();
9+
expect(screen.queryByTestId("sweep-chart-svg")).toBeNull();
10+
});
11+
12+
it("runs all H sizes sequentially and renders one series per H", async () => {
13+
const calls: number[] = [];
14+
const runner = vi.fn(async (H: number) => {
15+
calls.push(H);
16+
return [3.0 - H * 0.001, 2.5 - H * 0.001];
17+
});
18+
render(<SweepPanel runner={runner} hSizes={[64, 128]} />);
19+
fireEvent.click(screen.getByTestId("scaling-sweep-run"));
20+
await waitFor(() => {
21+
expect(screen.getByTestId("sweep-chart-svg")).toBeDefined();
22+
});
23+
expect(calls).toEqual([64, 128]);
24+
expect(screen.getByTestId("sweep-chart-line-H64")).toBeDefined();
25+
expect(screen.getByTestId("sweep-chart-line-H128")).toBeDefined();
26+
// Two points per H series.
27+
expect(screen.getByTestId("sweep-chart-point-H64-0")).toBeDefined();
28+
expect(screen.getByTestId("sweep-chart-point-H64-1")).toBeDefined();
29+
expect(screen.getByTestId("sweep-chart-point-H128-0")).toBeDefined();
30+
});
31+
32+
it("disables Run button while a sweep is in flight", async () => {
33+
let resolveFirst: ((v: number[]) => void) | null = null;
34+
const runner = vi.fn((H: number) => new Promise<number[]>((res) => {
35+
if (H === 64) { resolveFirst = res; }
36+
else res([1.0]);
37+
}));
38+
render(<SweepPanel runner={runner} hSizes={[64, 128]} />);
39+
fireEvent.click(screen.getByTestId("scaling-sweep-run"));
40+
await waitFor(() => {
41+
expect((screen.getByTestId("scaling-sweep-run") as HTMLButtonElement).disabled)
42+
.toBe(true);
43+
});
44+
expect(screen.getByTestId("sweep-progress")).toBeDefined();
45+
resolveFirst!([1.0]);
46+
await waitFor(() => {
47+
expect((screen.getByTestId("scaling-sweep-run") as HTMLButtonElement).disabled)
48+
.toBe(false);
49+
});
50+
});
51+
52+
it("surfaces a sweep-error when runner rejects, stops the sweep", async () => {
53+
const runner = vi.fn(async (H: number) => {
54+
if (H === 128) throw new Error("oom on H=128");
55+
return [1.0];
56+
});
57+
render(<SweepPanel runner={runner} hSizes={[64, 128, 256]} />);
58+
fireEvent.click(screen.getByTestId("scaling-sweep-run"));
59+
await waitFor(() => {
60+
expect(screen.getByTestId("sweep-error").textContent)
61+
.toContain("oom on H=128");
62+
});
63+
// H=64 series rendered, H=256 never reached.
64+
expect(screen.getByTestId("sweep-chart-line-H64")).toBeDefined();
65+
expect(screen.queryByTestId("sweep-chart-line-H256")).toBeNull();
66+
});
67+
});

0 commit comments

Comments
 (0)