Skip to content

opt=2: fail closed when a variant's accuracy cannot be computed - #157

Merged
sbryngelson merged 1 commit into
sbryngelson:mainfrom
axiom-of-choice:fix-relerr-gate-failclosed
Aug 2, 2026
Merged

opt=2: fail closed when a variant's accuracy cannot be computed#157
sbryngelson merged 1 commit into
sbryngelson:mainfrom
axiom-of-choice:fix-relerr-gate-failclosed

Conversation

@axiom-of-choice

Copy link
Copy Markdown
Contributor

The small independent PR you asked for in #153. Not the fix for that issue (that is #156, on the opt=1 path); this is the opt=2 hardening you called merge-worthy on its own.

The hole

_optimize.measure gated variants with:

if relerr > tol: return float("inf"), None

Every comparison with nan is False, so a variant whose error could not be computed was accepted rather than rejected. not (relerr <= tol) rejects nan and inf while leaving the inclusive boundary at tol exactly as it was.

One line, plus a comment recording why the form matters.

Why it is worth landing anyway

It does not change the #153 repro, and I want to be clear about that: opt=1 returns from _compile_opt before reaching this gate, so this line was never on that path. What it does protect is the opt=2/tune path against future lossy variants. Any variant that saturates, produces a shape mismatch upstream of the compare, or otherwise yields an uncomputable error currently rides through on a nan. int4-LUT and sparse are exactly the sort of additions that would trip it.

Verification

Check Result
tests/test_variant_gate_failclosed.py 5 passed, build-level so they run in CI without an ANE
tests/test_tune_guards.py 9 passed
tests/run_corpus.py GATE: GREEN, 90/90
ruff / pre-commit clean

The tests cover nan rejected, inf rejected (computed from a real saturated-vs-finite pair, so the inf is produced rather than asserted), within-tolerance still accepted including the boundary at tol, and above-tolerance still rejected.

One of them pins the source form:

assert "not (relerr <= tol)" in src, "the accuracy gate must reject nan, not just large errors"

That is deliberate. The failure mode here is a plausible-looking edit back to relerr > tol, which no value-based test can catch once the gate stops being reachable with a nan in a normal run. Verified it fails on main and passes here. Happy to drop it if pinning source text is not to your taste; the other four stand on their own.

The variant gate read `if relerr > tol: reject`, and every comparison with nan is
False, so a variant whose error could not be computed was accepted rather than
rejected. `not (relerr <= tol)` rejects nan and inf while leaving the inclusive
boundary at tol unchanged.

Not a fix for sbryngelson#153: that repro goes through opt=1, which never reaches this gate.
Sent separately at the maintainer's request because nan slipping through a `>`
test is a hazard for future lossy variants on the opt=2 path.

Five build-level tests, CI-runnable without an ANE. One pins the form itself, so
an edit back to `relerr > tol` fails rather than silently reopening the hole.
Corpus GATE GREEN 90/90.
@sbryngelson
sbryngelson merged commit 5ede142 into sbryngelson:main Aug 2, 2026
14 checks passed
@sbryngelson

Copy link
Copy Markdown
Owner

Merged -- thanks. This closes a real hole: a nan relerr in measure() (_optimize.py:336) was silently accepted (nan > tol is False), and not (relerr <= tol) correctly rejects it. The source-form pin test is a nice touch.

One follow-up worth a look, since your #153 root-cause pointed at a different function. The nan you originally traced is in _measure_with_ref (_optimize.py:628 sets relerr = float("nan") when there's no usable reference), and that value flows into tune_precision, not measure(). There it hits r["relerr"] <= target_error (:697, which nan correctly fails) but then the fallbacks min(usable, key=lambda r: r["relerr"]) (:703/709/714) -- and min with a nan key sorts unpredictably, so an unusable variant could still get chosen there. Worth confirming whether that path also needs a fail-closed guard (drop rows with non-finite relerr before the min), or whether measure() is the gate that actually governs the opt=2/tune() route. Not blocking anything -- just flagging that #157 may be one of two, since it patches measure() and the analysis named _measure_with_ref.

axiom-of-choice added a commit to axiom-of-choice/ANEForge that referenced this pull request Aug 3, 2026
I built this branch's _optimize.py from the sbryngelson#156 branch, which predates sbryngelson#157, so
copying it reverted the fail-closed relerr check. sbryngelson#157's source-form test caught
it in CI, which is what that test exists for.

Verified the rest of the copy carried nothing else stale: the diff against main is
now only the ceiling gate and the rename.
sbryngelson pushed a commit that referenced this pull request Aug 3, 2026
…vably safe (#161)

* compile: declare input activation ranges so opt=1 keeps int8 when provably safe

Implements #155, and supersedes #156 by carrying its gate plus the bound machinery
that makes the gate an optimization instead of a near-blanket decline.

af.input(shape, max_abs=) records a promise about |value| at runtime. Undeclared
inputs stay unbounded, so the fail-closed default from #153 is unchanged.

_propagate_max_abs walks the graph conservatively. Anything not modelled returns
None rather than a guess, since over-estimating only declines a lossy variant more
often while under-estimating would wrongly keep one. Covers non-expanding
elementwise ops, scalar mul/add, add/sub/mul/min/max over both operands, the
matmul and linear weight row-sum bound, bmm, clip by its limits, and the
saturating activations that are bounded whatever feeds them.

That last group matters more than the declaration itself: sigmoid, tanh, softmax,
l2_norm and relu6 bound an undeclared input, so anything downstream of a softmax
or a sigmoid recovers int8 at opt=1 with no annotation at all.

_act_max_abs now bounds the activation feeding each weight-consuming node rather
than the graph output, which is the quantity a weight-quantizing variant actually
has to encode.

Verified on M2 Pro: the repro goes from 179/496 non-finite to 0/496, and a graph
declaring max_abs=8.0 with real activations peaking at 7.1 keeps int8 and stays
finite at relerr 4.6e-3, against 2.5e-4 for the lossless path. 21 new build-level
tests, the 10 from #156 still pass, corpus GATE GREEN 90/90.

Also commits bench/opt1_int8_saturation_repro.py, which the review on #156 asked
for, and fixes the test docstring that referenced it while it was absent.

* Restore the merged fail-closed gate in _optimize

I built this branch's _optimize.py from the #156 branch, which predates #157, so
copying it reverted the fail-closed relerr check. #157's source-form test caught
it in CI, which is what that test exists for.

Verified the rest of the copy carried nothing else stale: the diff against main is
now only the ceiling gate and the rename.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants