You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/mlx_port_master_plan.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -235,7 +235,7 @@ Before fanning out across all 10 streams, prove the smallest viable path on the
235
235
**Gate set** (Mac-local gates must be green before scaling effort; external reference closure is tracked fail-closed but does not block Mac-local M0.4 work):
236
236
- M0.1 — **CLOSED**. Tokenizer: vendor the deployed GB10 tokenizer artifact with vocab=65536 and the reserved ID contract (id 2=BOS, 3=EOT/EOS, 4=FIM_PREFIX, 5=FIM_MIDDLE, 6=FIM_SUFFIX, 7=CODE_START, 45=FIM_INSTRUCTION, 46=SPACE, 47=NL). The wrapper normalizes whitespace runs at encode ([\r\n]+-><NL>, [ \t]+-><SPACE>) and decode is plain token concat with sentinel substitution; this fixes the BPE-split decode bug (e.g., sum->s,u,m->s u m) and gives byte-exact round-trip for inputs without multi-char whitespace runs. The explicit-token approach matches the CUDA-side nanochat/cpp_tokenizer.py decode behavior. Do not reopen this gate over the deployed HF decoder=null artifact's non-reversible decode(encode(text)) behavior.
237
237
- M0.2 — Model factory entry for local_gb10_quarter in cppmega_mlx/recipes/model_factory.py. With M0.1 closed, the default profile is unblocked; acceptance remains the profile contract plus build → forward closure on shape (B=1, T=512) returning finite logits with config schema validation rejecting invalid combos.
238
-
- M0.3 — **External reference closure, not a Mac-local blocker**. Random-init seed-matched forward parity remains useful for comparing MLX against a CUDA/GB10 reference, but it does not gate local MLX/Metal M0.4 training progress. The fail-closed scaffold constructs the same local_gb10_quarter config and fixed B=1,T=512,seed=3003 input batch (tokens_sha256=c645ca4053e5206dcbe58c13aa26f4a9e56c5aa2aee90a4d4778bbc9d9c33549). No real CUDA logits artifact exists at bench/parity/cuda/m03_local_gb10_quarter_seed3003_logits.json, so scripts/m03_forward_parity_manifest.py must refuse the default missing artifact and keep m0_3_closed=false. A metadata-valid CUDA artifact only reaches artifact_preflight_status=valid_not_evaluated; it is not external reference acceptance until a separate numerical harness compares the full logits tensor (shape=[1,512,65536], numel=33554432) against MLX and records pass/fail parity.
238
+
- M0.3 — **External reference closure, not a Mac-local blocker**. Random-init seed-matched forward parity remains useful for comparing MLX against a CUDA/GB10 reference, but it does not gate local MLX/Metal M0.4 training progress. The fail-closed scaffold constructs the same local_gb10_quarter config and fixed B=1,T=512,seed=3003 input batch (tokens_sha256=c645ca4053e5206dcbe58c13aa26f4a9e56c5aa2aee90a4d4778bbc9d9c33549). No real CUDA logits artifact exists at bench/parity/cuda/m03_local_gb10_quarter_seed3003_logits.json, so scripts/m03_forward_parity_manifest.py must refuse the default missing artifact and keep m0_3_closed=false. A metadata-valid CUDA artifact only reaches artifact_preflight_status=valid_not_evaluated; it is not external reference acceptance until a separate numerical harness compares the full logits tensor (shape=[1,512,65536], numel=33554432) against MLX and records pass/fail parity. The non-blocking bd follow-up is cppmega-mlx-uwhj; the old Mac-local blocker task cppmega-mlx-t8f.3 is superseded by that external-reference task, not closed as completed parity evidence.
239
239
- M0.4 — One training step in bf16 (loss + backward + optimizer.update). No NaNs. Loss decrease over 100 steps on the target local parquet sample. **Data source**: local ignored data/parquet_samples/gb10/clang_semantic_4k_v10/val_00000.parquet (~492 MB total of GB10 validation shards when present locally; not committed to git). Full local_gb10_quarter acceptance is captured in bench/baselines/m04_train_step.json with --grad-checkpoint on the target parquet: 100/100 steps, loss 11.2004585 -> 6.9402304, finite loss/grad path, AdamW with FP32 master moments, mean step time 10.72670 s, 16.28 target tokens/s, and peak process memory 23,917,647,383 bytes. No GB10 scp or full-corpus prep needed for M0.
240
240
- M0.5 — MTP K=2 head wired with β=0.6 / λ=0.3; per-depth losses tracked. MTP-disabled inference path also returns sane logits. Current local coverage is bounded to MLX training-side loss semantics plus fail-closed manifest hardening, including nested CUDA/MLX overclaim rejection. External CUDA/GB10 FastMTP loss+grad-norm closure remains open until a fixed-seed artifact exists and an MLX-vs-CUDA numerical harness evaluates it; that external reference closure does not block Mac-local M0.4 training acceptance.
241
241
- M0.6 — Memory math validated on the actual dev box (Mac Studio M4 Max 128 GB): peak unified-memory < 75% of installed RAM with --grad-checkpoint and AdamW.
0 commit comments