Commit 8f88d0f
committed
fix(kpm): use combined absolute+relative tolerance in M6-2 dual-mode tests
The fixed 1e-5 absolute tolerance in solve_linear_system_2x2_matches_cpp
and solve_symmetric_linear_system_3x3_matches_cpp is too tight for f32
values with magnitude > ~10. Apple Silicon ARM64 uses FMA differently
from x86_64, producing tiny precision differences that exceed 1e-5 for
larger results.
Example failure on macOS ARM64:
rust=-40.826614, cpp=-40.8266, diff=1.5e-5
The diff of 1.5e-5 represents a relative error of ~3.7e-7, which is
well within f32 precision but exceeds the absolute tolerance.
Fix: use combined tolerance:
tol = max(1e-5, |value| * 1e-6)
This gives:
- |x| <= 10: tolerance unchanged at 1e-5 (preserves existing strictness)
- |x| = 100: tolerance 1e-4 (~1e-6 relative)
- |x| = 1000: tolerance 1e-3 (~1e-6 relative)
These bounds are consistent with f32's ~7-digit precision and
accommodate platform-specific FMA rounding without losing test
sensitivity for typical values.
Surfaced now because PR #117's new dual-mode CI step exposed this
pre-existing issue on macOS runners. The math itself is correct;
only the test tolerance was too strict.
Refs #1161 parent 76c3552 commit 8f88d0f
1 file changed
Lines changed: 16 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2112 | 2112 | | |
2113 | 2113 | | |
2114 | 2114 | | |
| 2115 | + | |
| 2116 | + | |
| 2117 | + | |
| 2118 | + | |
| 2119 | + | |
| 2120 | + | |
2115 | 2121 | | |
2116 | | - | |
2117 | | - | |
| 2122 | + | |
| 2123 | + | |
2118 | 2124 | | |
2119 | 2125 | | |
2120 | 2126 | | |
2121 | | - | |
| 2127 | + | |
| 2128 | + | |
2122 | 2129 | | |
2123 | 2130 | | |
2124 | 2131 | | |
| |||
2183 | 2190 | | |
2184 | 2191 | | |
2185 | 2192 | | |
| 2193 | + | |
| 2194 | + | |
2186 | 2195 | | |
2187 | | - | |
2188 | | - | |
| 2196 | + | |
| 2197 | + | |
2189 | 2198 | | |
2190 | 2199 | | |
2191 | 2200 | | |
2192 | | - | |
| 2201 | + | |
| 2202 | + | |
2193 | 2203 | | |
2194 | 2204 | | |
2195 | 2205 | | |
| |||
0 commit comments