Skip to content

Commit 5fc3cf4

Browse files
committed
tooling(probe): report torch-CUDA peak + mamba3 chunk flag in fwd/bwd mem probe
The MLX get_peak_memory metric is blind to the torch-CUDA allocator that owns the Path-B mamba3 bwd scratch (the dominant seq=4096 term). Add torch.cuda. max_memory_allocated and surface the CPPMEGA_MAMBA3_BWD_SEQ_CHUNK setting so the probe shows the real unified-memory breakdown.
1 parent 7f32db4 commit 5fc3cf4

1 file changed

Lines changed: 14 additions & 1 deletion

File tree

scripts/probe_moe_forward_mem.py

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,16 +108,29 @@ def loss_fn(m, x):
108108
loss_val = float(logits.astype(mx.float32).mean())
109109

110110
peak_gb = _peak_gb()
111+
torch_peak_gb = None
112+
try:
113+
import torch
114+
115+
if torch.cuda.is_available():
116+
torch_peak_gb = round(
117+
float(torch.cuda.max_memory_allocated()) / (1024**3), 3
118+
)
119+
except Exception:
120+
torch_peak_gb = None
121+
mamba_chunk = os.environ.get("CPPMEGA_MAMBA3_BWD_SEQ_CHUNK", "").strip() or None
111122
result = {
112123
"efficient_moe": efficient,
124+
"mamba3_bwd_seq_chunk": mamba_chunk,
113125
"batch": args.batch,
114126
"seq": args.seq,
115127
"backward": bool(args.backward),
116128
"grad_checkpoint": bool(grad_ckpt),
117129
"build_s": round(build_s, 2),
118130
"run_s": round(run_s, 2),
119131
"after_params_peak_gb": round(after_params_gb, 3),
120-
"peak_gb": round(peak_gb, 3),
132+
"mlx_peak_gb": round(peak_gb, 3),
133+
"torch_cuda_peak_gb": torch_peak_gb,
121134
"loss_or_mean": loss_val,
122135
}
123136
print("PROBE_RESULT " + json.dumps(result), flush=True)

0 commit comments

Comments
 (0)