Commit f687b4f
committed
docs(autosplit): hardware-aware automatic kernel-splitting design (three-tier: live-query + preset + calibration)
Replaces every hardcoded device-limit constant in the Path-C direct-chain splitter
(METAL_FORWARD/BACKWARD_MAX_SEGMENT_NODES, _METAL_SHARED_SCRATCH_TRIGGER/DEMOTE,
DESCRIPTOR_DEFAULT_MAX_ROWS_PER_LAUNCH=64, 32KiB/31-arg, the recurrent/independent
op frozensets) with hardware-derived values, per David's directive that the
splitting must be automatic + hardware-aware.
Three-tier strategy (FA2/cuDNN-style): TIER1 live-query the directly-queryable
limits (threadgroup-mem via ctypes-objc maxThreadgroupMemoryLength=32768 on M4 Max;
CUDA shared_memory_per_block(_optin) via torch.cuda props); TIER2 a committed
preset table of NON-queryable limits (watchdog window, MTLCompilerService MSL
ceiling, per-op time/row) keyed by device identity, seeded from THIS session's
measurements (M4 Max watchdog 5s / ceiling 140K between 116K-ok and 176K-crash;
GB10 no watchdog / sm_121); TIER3 immediate one-time auto-calibration (probe +
bisect + persistent cache) when the device is unknown OR a preset is proven wrong
in practice, logging PRESET MISS / NO PRESET loudly (RULE #1 self-correction, not
a silent fallback). Static per-segment estimator (shared bytes, buffer-arg count,
MSL size, GPU-time) drives the split decisions; recurrent-vs-independent becomes a
structural test, deleting the frozensets.
Includes constant-replacement table, 7 RULE #1 fail-loud points (kill the
except:pass+0x18C00 floor at schedules.py:10420; promote codegen_metal.cc:1248
WARNING->ICHECK; PathCSplitInfeasible raise), byte-for-byte Metal/CUDA parity
tests, and an 8-step incremental implementation plan gated by those parity tests.1 parent b330bdb commit f687b4f
1 file changed
Lines changed: 768 additions & 0 deletions
0 commit comments