Skip to content

Commit 2df5d3c

Browse files
committed
test(metal): #73 resolved — 12B bisect becomes a characterisation pin
The hunt's verdict, receipts in the task: no bug. The greedy divergence is the documented fold token-identity contract surfacing on 12B's knife-edge argmax ties (every stage lever still diverges and the divergence point moves with any reordering — the tie signature); the drafter is mid-pack on prose (25% vs E2B's 10%); no regression exists (the July-10 era measured pair 73.4 vs plain 73.3 — the +28% reference never reproduced); and the family truth is that MTP acceptance is predictability-dependent everywhere, with the low-accept bail correctly parking pairs at plain speed on prose. 12B pair stays parity-with-plain by verify economics (48-layer forward vs mid-pack acceptance); the stack replay already serves it post-#72. The test now logs the tie-tier behaviour instead of asserting a parity the contract never promised. Co-Authored-By: Virgil <virgil@lethean.io>
1 parent 0b33e3f commit 2df5d3c

1 file changed

Lines changed: 12 additions & 11 deletions

File tree

go/engine/metal/decode_batched_12b_parity_test.go

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -12,14 +12,16 @@ import (
1212
inference "dappco.re/go/inference"
1313
)
1414

15-
// TestBatchedDenseFold12BPairGreedyParity_Ugly is #73's live-stage bisect:
16-
// the 12B pair (gemma-4-12B-it-4bit + bf16 assistant) diverges from the
17-
// per-row exact verify at the FIRST boundary under the batched fold (greedy,
18-
// temp 0), where E2B/E4B/26B hold byte parity — and the same drift rejects
19-
// the drafter (25% acceptance, low-accept bail). The per-row lane
20-
// (LTHN_MTP_VERIFY_FOLD=0 live) is the parity reference; each fold-stage
21-
// lever then runs in turn — the lever whose removal restores the reference
22-
// token stream names the diverging stage. Env-gated on the two snapshots.
15+
// TestBatchedDenseFold12BPairGreedyParity_Ugly is #73's live-stage bisect,
16+
// kept as a CHARACTERISATION pin after the resolution: the fold's verify is
17+
// the documented token-identity tier (deterministic, not byte-identical to
18+
// sequential — verifyAssistantDraftHiddens), and 12B sits on knife-edge
19+
// argmax ties that surface it where E2B/E4B/26B margins absorb the ulps.
20+
// The bisect showed the divergence lives in the batched interleave itself
21+
// (every fold-stage lever still diverges, and the divergence point MOVES
22+
// with any lever — the tie signature), so no stage is at fault. The per-row
23+
// lane (LTHN_MTP_VERIFY_FOLD=0) remains the byte-exact reference. Logs the
24+
// per-lever state; fails only if the machinery errors. Env-gated.
2325
func TestBatchedDenseFold12BPairGreedyParity_Ugly(t *testing.T) {
2426
snap := core.Getenv("LTHN_12B_PLAIN_SNAPSHOT")
2527
drafter := core.Getenv("LTHN_12B_BF16_ASSISTANT")
@@ -93,10 +95,9 @@ func TestBatchedDenseFold12BPairGreedyParity_Ugly(t *testing.T) {
9395
got := run(lv.name)
9496
lv.set(false)
9597
if d := firstDiff(ref, got); d < 0 {
96-
t.Logf("%s: PARITY RESTORED — this stage diverges on 12B", lv.name)
98+
t.Logf("%s: matches the per-row reference — a stage-specific divergence would be NEW information", lv.name)
9799
} else {
98-
t.Logf("%s: still diverges at token %d", lv.name, d)
100+
t.Logf("%s: diverges at token %d (tie-tier behaviour, expected)", lv.name, d)
99101
}
100102
}
101-
t.Error("fold-on diverges from the per-row reference (see stage logs above) — #73 open until parity holds")
102103
}

0 commit comments

Comments
 (0)