Skip to content

Commit 400186a

Browse files
author
gHashTag
committed
feat(conformance): add WP-18 corpus-integrity gate + fix 6 mislabeled gf16 rows
Adds a stdlib-only conformance-corpus integrity gate (sibling to the catalog count-gate). Six independent checks: pack-set == SSOT (83), INDEX recount, per-pack SHA freshness, finite-row abs_error re-derivation (TestFloat-style independent oracle), special-value same-class round-trip (nan/inf), and an auditable honesty allow-list for disclosed nonzero errors. Self-test (15/15, every check FAIL-reachable) gates the gate before it runs. F-1 correction: the re-derivation oracle exposed 6 gf16 rows storing abs_error=0.0 while decoding off-grid (phi, inv_phi, phi_squared, inv_phi_sq, e, pi). Round-trip was correct; only the error label was wrong. Each abs_error is set to the measured value and allow-listed as not_on_grid. The anchor identity phi^2 + 1/phi^2 = 3 remains exact in f64. Verdict now CLEAN. - tools/wp18_conformance_gate.py, tools/wp18_selftest_gate.py - .github/workflows/conformance-integrity-gate.yml (self-test then live gate) - conformance/vectors/abs_error_allowlist.json (8 disclosed entries) - conformance/integrity/fixtures/ (frozen evidence rows) Closes #1079
1 parent c6664e6 commit 400186a

8 files changed

Lines changed: 785 additions & 7 deletions
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# WP-18 conformance-corpus integrity gate (CI).
2+
#
3+
# Sibling to catalog-count-gate.yml: that gate locks the SSOT catalog COUNT (==83);
4+
# this gate locks the conformance CORPUS against silent drift and undisclosed error.
5+
# Runs on every push / PR touching the SSOT, the vector corpus, or the gate itself.
6+
#
7+
# stdlib-only Python, no third-party deps, no network. The self-test runs FIRST and
8+
# must prove the gate is still falsifiable (every check FAIL-reachable) before the gate
9+
# is trusted against the live corpus.
10+
name: conformance-integrity-gate
11+
12+
on:
13+
push:
14+
paths:
15+
- "specs/numeric/formats_catalog.t27"
16+
- "conformance/vectors/**"
17+
- "tools/wp18_conformance_gate.py"
18+
- "tools/wp18_selftest_gate.py"
19+
- ".github/workflows/conformance-integrity-gate.yml"
20+
pull_request:
21+
paths:
22+
- "specs/numeric/formats_catalog.t27"
23+
- "conformance/vectors/**"
24+
- "tools/wp18_conformance_gate.py"
25+
- "tools/wp18_selftest_gate.py"
26+
- ".github/workflows/conformance-integrity-gate.yml"
27+
workflow_dispatch: {}
28+
29+
permissions:
30+
contents: read
31+
32+
jobs:
33+
integrity-gate:
34+
runs-on: ubuntu-latest
35+
steps:
36+
- name: Checkout
37+
uses: actions/checkout@v4
38+
39+
- name: Set up Python
40+
uses: actions/setup-python@v5
41+
with:
42+
python-version: "3.11"
43+
44+
- name: Self-test (gate must be falsifiable)
45+
# If any planted mutant is NOT killed, the gate is not trustworthy -> fail here.
46+
working-directory: tools
47+
run: python3 wp18_selftest_gate.py
48+
49+
- name: Run integrity gate against live corpus
50+
run: |
51+
python3 tools/wp18_conformance_gate.py \
52+
--ssot specs/numeric/formats_catalog.t27 \
53+
--vectors conformance/vectors \
54+
--allowlist conformance/vectors/abs_error_allowlist.json \
55+
--json wp18_report.json
56+
57+
- name: Upload integrity report
58+
if: always()
59+
uses: actions/upload-artifact@v4
60+
with:
61+
name: wp18-conformance-report
62+
path: wp18_report.json
63+
if-no-files-found: warn
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{
2+
"fixture": "fp8_e5m2_pos_inf_special_roundtrip",
3+
"kind": "special_value_row",
4+
"expectation": "Check D2 MUST pass: +inf encodes and decodes to +inf; abs_error is a placeholder, not an arithmetic error; exempt from D and E.",
5+
"pack_id": "fp8_e5m2",
6+
"row": {
7+
"name": "pos_inf",
8+
"input_f64": "Inf",
9+
"input_f64_hex": "0x7FF0000000000000",
10+
"fp8_bits_hex": "0x7C",
11+
"fp8_bits_int": 124,
12+
"decoded_f64": "Inf",
13+
"decoded_f64_hex": "0x7FF0000000000000",
14+
"abs_error": 0.0,
15+
"category": "inf"
16+
},
17+
"severity": "none",
18+
"note": "Same-class round-trip inf->inf. The v1 gate wrongly flagged this as a nan-mismatch (gate handling gap); v2 D2 validates it correctly."
19+
}
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
"fixture": "gf16_phi_mislabeled_abs_error",
3+
"kind": "finite_value_row",
4+
"expectation": "Check D MUST flag: stored abs_error 0.0 but abs(decoded-input) is nonzero.",
5+
"pack_id": "gf16",
6+
"row": {
7+
"name": "phi",
8+
"input_f64": 1.618033988749895,
9+
"input_f64_hex": "0x3FF9E3779B97F4A8",
10+
"gf16_bits_hex": "0x3F3C",
11+
"gf16_bits_int": 16188,
12+
"decoded_f64": 1.6171875,
13+
"decoded_f64_hex": "0x3FF9E00000000000",
14+
"abs_error": 0.0,
15+
"category": "phi_anchor"
16+
},
17+
"rederived_abs_error": 0.0008464887498949025,
18+
"severity": "Risk",
19+
"note": "gf16 decode of phi rounds to 1.6171875; the 8.46e-04 error was stored as 0.0. Anchor identity phi^2+1/phi^2=3 still holds in f64; only the per-row abs_error label is wrong."
20+
}

conformance/vectors/INDEX_all_formats.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -321,7 +321,7 @@
321321
"file": "gf16_conformance_v0.json",
322322
"kind": "bitexact",
323323
"source": "hand-curated (pre-existing)",
324-
"sha256": "7aea5b9e86ea71a54ae0c1601cea13e2d90d95fecaf2ae969eac1349cf7a2b42"
324+
"sha256": "d1c0eb5bd66247b3c5db9a00a95e29cf4359653aec56f2f9e6827f96898d1509"
325325
},
326326
{
327327
"id": "gf20",
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
{
2+
"schema": "wp18-abs-error-allowlist/v1",
3+
"note": "Explicit, auditable allow-list of bit-exact-pack rows that carry a NONZERO abs_error for an honest, documented reason. Every entry is a row where the round-trip is CORRECT but the input is not exactly representable (not_on_grid) or saturates (overflow_to_inf). The gate fails on any nonzero abs_error NOT listed here. Never add an entry to silence a real error; each must be a genuinely-disclosed case with a fixture.",
4+
"allow": [
5+
{
6+
"pack_id": "bfloat16",
7+
"row_name": "pos_0p1",
8+
"reason": "not_on_grid",
9+
"detail": "0.1 is not exactly representable in bf16; decoded_f64=0.10009765625, abs_error~9.77e-05 is the true nearest-representable error. Honest, expected."
10+
},
11+
{
12+
"pack_id": "gf16",
13+
"row_name": "overflow_to_inf",
14+
"reason": "overflow_to_inf",
15+
"detail": "input 1e40 exceeds the gf16 dynamic range and saturates to +inf; abs_error=inf is the honest record of an out-of-range input, not an encode/decode bug."
16+
},
17+
{
18+
"pack_id": "gf16",
19+
"row_name": "phi",
20+
"reason": "not_on_grid",
21+
"detail": "phi is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000846489 is the true nearest-representable error (WP-18 F-1 correction)."
22+
},
23+
{
24+
"pack_id": "gf16",
25+
"row_name": "inv_phi",
26+
"reason": "not_on_grid",
27+
"detail": "inv_phi is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000130074 is the true nearest-representable error (WP-18 F-1 correction)."
28+
},
29+
{
30+
"pack_id": "gf16",
31+
"row_name": "phi_squared",
32+
"reason": "not_on_grid",
33+
"detail": "phi_squared is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000846489 is the true nearest-representable error (WP-18 F-1 correction)."
34+
},
35+
{
36+
"pack_id": "gf16",
37+
"row_name": "inv_phi_sq",
38+
"reason": "not_on_grid",
39+
"detail": "inv_phi_sq is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000130074 is the true nearest-representable error (WP-18 F-1 correction)."
40+
},
41+
{
42+
"pack_id": "gf16",
43+
"row_name": "e",
44+
"reason": "not_on_grid",
45+
"detail": "e is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000468172 is the true nearest-representable error (WP-18 F-1 correction)."
46+
},
47+
{
48+
"pack_id": "gf16",
49+
"row_name": "pi",
50+
"reason": "not_on_grid",
51+
"detail": "pi is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000967654 is the true nearest-representable error (WP-18 F-1 correction)."
52+
}
53+
]
54+
}

conformance/vectors/gf16_conformance_v0.json

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@
8787
"gf16_bits_int": 16188,
8888
"decoded_f64": 1.6171875,
8989
"decoded_f64_hex": "0x3FF9E00000000000",
90-
"abs_error": 0.0,
90+
"abs_error": 0.0008464887498949025,
9191
"category": "phi_anchor"
9292
},
9393
{
@@ -98,7 +98,7 @@
9898
"gf16_bits_int": 15481,
9999
"decoded_f64": 0.6181640625,
100100
"decoded_f64_hex": "0x3FE3C80000000000",
101-
"abs_error": 0.0,
101+
"abs_error": 0.0001300737501052085,
102102
"category": "phi_anchor"
103103
},
104104
{
@@ -109,7 +109,7 @@
109109
"gf16_bits_int": 16542,
110110
"decoded_f64": 2.6171875,
111111
"decoded_f64_hex": "0x4004F00000000000",
112-
"abs_error": 0.0,
112+
"abs_error": 0.0008464887498949025,
113113
"category": "phi_anchor"
114114
},
115115
{
@@ -120,7 +120,7 @@
120120
"gf16_bits_int": 15118,
121121
"decoded_f64": 0.3818359375,
122122
"decoded_f64_hex": "0x3FD8700000000000",
123-
"abs_error": 0.0,
123+
"abs_error": 0.00013007375010509747,
124124
"category": "phi_anchor"
125125
},
126126
{
@@ -230,7 +230,7 @@
230230
"gf16_bits_int": 16568,
231231
"decoded_f64": 2.71875,
232232
"decoded_f64_hex": "0x4005C00000000000",
233-
"abs_error": 0.0,
233+
"abs_error": 0.0004681715409549092,
234234
"category": "normal"
235235
},
236236
{
@@ -241,7 +241,7 @@
241241
"gf16_bits_int": 16676,
242242
"decoded_f64": 3.140625,
243243
"decoded_f64_hex": "0x4009200000000000",
244-
"abs_error": 0.0,
244+
"abs_error": 0.000967653589793116,
245245
"category": "normal"
246246
}
247247
]

0 commit comments

Comments
 (0)