opt=2: fail closed when a variant's accuracy cannot be computed - #157
Conversation
The variant gate read `if relerr > tol: reject`, and every comparison with nan is False, so a variant whose error could not be computed was accepted rather than rejected. `not (relerr <= tol)` rejects nan and inf while leaving the inclusive boundary at tol unchanged. Not a fix for sbryngelson#153: that repro goes through opt=1, which never reaches this gate. Sent separately at the maintainer's request because nan slipping through a `>` test is a hazard for future lossy variants on the opt=2 path. Five build-level tests, CI-runnable without an ANE. One pins the form itself, so an edit back to `relerr > tol` fails rather than silently reopening the hole. Corpus GATE GREEN 90/90.
|
Merged -- thanks. This closes a real hole: a One follow-up worth a look, since your #153 root-cause pointed at a different function. The nan you originally traced is in |
I built this branch's _optimize.py from the sbryngelson#156 branch, which predates sbryngelson#157, so copying it reverted the fail-closed relerr check. sbryngelson#157's source-form test caught it in CI, which is what that test exists for. Verified the rest of the copy carried nothing else stale: the diff against main is now only the ceiling gate and the rename.
…vably safe (#161) * compile: declare input activation ranges so opt=1 keeps int8 when provably safe Implements #155, and supersedes #156 by carrying its gate plus the bound machinery that makes the gate an optimization instead of a near-blanket decline. af.input(shape, max_abs=) records a promise about |value| at runtime. Undeclared inputs stay unbounded, so the fail-closed default from #153 is unchanged. _propagate_max_abs walks the graph conservatively. Anything not modelled returns None rather than a guess, since over-estimating only declines a lossy variant more often while under-estimating would wrongly keep one. Covers non-expanding elementwise ops, scalar mul/add, add/sub/mul/min/max over both operands, the matmul and linear weight row-sum bound, bmm, clip by its limits, and the saturating activations that are bounded whatever feeds them. That last group matters more than the declaration itself: sigmoid, tanh, softmax, l2_norm and relu6 bound an undeclared input, so anything downstream of a softmax or a sigmoid recovers int8 at opt=1 with no annotation at all. _act_max_abs now bounds the activation feeding each weight-consuming node rather than the graph output, which is the quantity a weight-quantizing variant actually has to encode. Verified on M2 Pro: the repro goes from 179/496 non-finite to 0/496, and a graph declaring max_abs=8.0 with real activations peaking at 7.1 keeps int8 and stays finite at relerr 4.6e-3, against 2.5e-4 for the lossless path. 21 new build-level tests, the 10 from #156 still pass, corpus GATE GREEN 90/90. Also commits bench/opt1_int8_saturation_repro.py, which the review on #156 asked for, and fixes the test docstring that referenced it while it was absent. * Restore the merged fail-closed gate in _optimize I built this branch's _optimize.py from the #156 branch, which predates #157, so copying it reverted the fail-closed relerr check. #157's source-form test caught it in CI, which is what that test exists for. Verified the rest of the copy carried nothing else stale: the diff against main is now only the ceiling gate and the rename.
The small independent PR you asked for in #153. Not the fix for that issue (that is #156, on the
opt=1path); this is theopt=2hardening you called merge-worthy on its own.The hole
_optimize.measuregated variants with:Every comparison with
nanisFalse, so a variant whose error could not be computed was accepted rather than rejected.not (relerr <= tol)rejectsnanandinfwhile leaving the inclusive boundary attolexactly as it was.One line, plus a comment recording why the form matters.
Why it is worth landing anyway
It does not change the #153 repro, and I want to be clear about that:
opt=1returns from_compile_optbefore reaching this gate, so this line was never on that path. What it does protect is theopt=2/tunepath against future lossy variants. Any variant that saturates, produces a shape mismatch upstream of the compare, or otherwise yields an uncomputable error currently rides through on anan. int4-LUT and sparse are exactly the sort of additions that would trip it.Verification
tests/test_variant_gate_failclosed.pytests/test_tune_guards.pytests/run_corpus.pyruff/ pre-commitThe tests cover
nanrejected,infrejected (computed from a real saturated-vs-finite pair, so theinfis produced rather than asserted), within-tolerance still accepted including the boundary attol, and above-tolerance still rejected.One of them pins the source form:
That is deliberate. The failure mode here is a plausible-looking edit back to
relerr > tol, which no value-based test can catch once the gate stops being reachable with ananin a normal run. Verified it fails onmainand passes here. Happy to drop it if pinning source text is not to your taste; the other four stand on their own.