Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/catalog-count-gate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Catalog Count Invariant (CI-01)

# Kills the count-drift bug class. Enforces SSOT == fresh regenerated codegen
# on every PR touching the numeric catalog or its codegen path. The canonical
# number is the count of `// CATALOG:` lines in the SSOT; codegen run fresh
# against it must reproduce that number exactly (catches the parser-bug class
# that silently dropped formula-bias rows). Paper-count divergence
# (arXiv:2606.09686 says 84; SSOT canonical is 83) is surfaced as a WARN that
# requires an erratum, not a silent edit.
#
# NOTE: per the repo constitution (L2 GENERATION), gen/ is DERIVED and is never
# hand-committed in a PR. This gate therefore regenerates into a temp dir and
# checks that against the SSOT; it does NOT diff committed gen/.

on:
pull_request:
branches: [master]
paths:
- "specs/numeric/formats_catalog.t27"
- "tools/gen_formats_catalog.py"
- "tools/check_catalog_count.py"
- ".github/workflows/catalog-count-gate.yml"
workflow_dispatch:

jobs:
count-invariant:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Enforce SSOT == fresh regen (canonical count)
run: |
python3 tools/check_catalog_count.py
63 changes: 63 additions & 0 deletions .github/workflows/conformance-integrity-gate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# WP-18 conformance-corpus integrity gate (CI).
#
# Sibling to catalog-count-gate.yml: that gate locks the SSOT catalog COUNT (==83);
# this gate locks the conformance CORPUS against silent drift and undisclosed error.
# Runs on every push / PR touching the SSOT, the vector corpus, or the gate itself.
#
# stdlib-only Python, no third-party deps, no network. The self-test runs FIRST and
# must prove the gate is still falsifiable (every check FAIL-reachable) before the gate
# is trusted against the live corpus.
name: conformance-integrity-gate

on:
push:
paths:
- "specs/numeric/formats_catalog.t27"
- "conformance/vectors/**"
- "tools/wp18_conformance_gate.py"
- "tools/wp18_selftest_gate.py"
- ".github/workflows/conformance-integrity-gate.yml"
pull_request:
paths:
- "specs/numeric/formats_catalog.t27"
- "conformance/vectors/**"
- "tools/wp18_conformance_gate.py"
- "tools/wp18_selftest_gate.py"
- ".github/workflows/conformance-integrity-gate.yml"
workflow_dispatch: {}

permissions:
contents: read

jobs:
integrity-gate:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Self-test (gate must be falsifiable)
# If any planted mutant is NOT killed, the gate is not trustworthy -> fail here.
working-directory: tools
run: python3 wp18_selftest_gate.py

- name: Run integrity gate against live corpus
run: |
python3 tools/wp18_conformance_gate.py \
--ssot specs/numeric/formats_catalog.t27 \
--vectors conformance/vectors \
--allowlist conformance/vectors/abs_error_allowlist.json \
--json wp18_report.json

- name: Upload integrity report
if: always()
uses: actions/upload-artifact@v4
with:
name: wp18-conformance-report
path: wp18_report.json
if-no-files-found: warn
104 changes: 104 additions & 0 deletions ERRATA_2026-06-14.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Erratum to arXiv:2606.09686 (Catalog Count Reconciliation)

**Paper:** "An 84-Format Numeric Catalog with Bit-Exact Conformance Vectors:
A Vendor-Neutral Reference for FP8, BF16, MXFP4, and Microscaling Formats"
(arXiv:2606.09686v1, 8 Jun 2026).

**Date of erratum:** 2026-06-14
**Canonical source of truth:** `specs/numeric/formats_catalog.t27` in
`github.com/gHashTag/t27` (verified live by fresh clone, HEAD `16042f4`).

---

## Summary

The paper title and Table 1 state the catalog contains **exactly 84 formats**
in 13 clusters. The live single-source-of-truth (SSOT) catalog contains
**83 formats** in 13 clusters. This erratum reconciles the divergence honestly,
states the canonical number, and records the CI invariant now added to prevent
recurrence. No conformance pack, SHA-256, or bit-exact vector is affected; the
divergence is purely in the catalog row count, not in any shipped vector.

**Canonical count going forward: 83.** The number 84 in the paper is superseded.

## What diverged, and why (four numbers, one cause)

At the time the paper was finalized, four artifacts carried different counts:

| Source | Count | Status |
|---|---|---|
| arXiv:2606.09686 Table 1 / title | 84 | aspirational; written against an earlier SSOT snapshot |
| SSOT `formats_catalog.t27` (HEAD `16042f4`) | **83** | **canonical, live** |
| codegen fresh re-run (this date) | 83 | now matches SSOT (parser fix verified) |
| committed `gen/numeric/*` (pre-fix) | 77 | stale; codegen had not been re-run after PR #1051 |

Root causes, now all resolved:

1. **Paper snapshot lag.** The paper's Table 1 was built against a catalog
state that split the Microscaling cluster into 5 rows (MXFP6 as two rows
E2M3 + E3M2, plus a standalone E8M0 scale) and listed 16 GoldenFloat rungs.
The live SSOT collapses MXFP6 to a single `mxfp6` row (3 Microscaling rows:
mxfp8, mxfp6, mxfp4) and has since expanded GoldenFloat to 22 rungs
(PR #1051 added gf10/gf14/gf48/gf96/gf512/gf1024 and others).

2. **Stale committed codegen.** After PR #1051 expanded the SSOT, the generated
artifacts in `gen/numeric/` were not regenerated, so they remained at the
pre-expansion count of 77. (An earlier parser limitation that dropped the
formula-biased rungs gf512/gf1024 -- whose bias is `2^194-1` / `2^390-1`,
exceeding int64 -- has been fixed: `parse_bias` now carries these as a
formula string with an int64-overflow sentinel, so codegen no longer drops
them. Fresh codegen yields 83.)

## Canonical cluster breakdown (SSOT, 83 formats, 13 clusters)

| Cluster | Count |
|---|---|
| GoldenFloat | 22 |
| HistoricalVendor | 10 |
| PositUnumIII | 8 |
| IntegerFixed | 8 |
| MlLowPrecision | 7 |
| Ieee754Binary | 5 |
| Theoretical | 4 |
| Lns | 4 |
| CompressionTrick | 4 |
| Microscaling | 3 |
| Ieee754Decimal | 3 |
| ExtendedFloat | 3 |
| QuantTuned | 2 |
| **TOTAL** | **83** |

All 83 IDs are unique (verified). No duplicate or malformed rows.

## Delta vs the paper's Table 1 (the -1 net)

- **GoldenFloat:** paper 16 -> SSOT 22 (+6: gf10, gf14, gf48, gf96, gf512,
gf1024 added by PR #1051).
- **Microscaling/OCP MX:** paper 5 -> SSOT 3 (-2: MXFP6 is one row, not split
E2M3/E3M2; E8M0 block scale is represented in the conformance packs, not as a
standalone catalog row).
- Net effect against the paper's other clusters yields the canonical 83.

The paper's 84 is therefore an over-count of 1 relative to the current SSOT,
arising from a different Microscaling row-split convention combined with a
later GoldenFloat expansion. Neither choice is "wrong"; the SSOT convention is
the one we can ship end-to-end with bit-exact provenance, so it is canonical.

## Corrective actions taken

1. Regenerated all 16 `gen/numeric/*` artifacts from the SSOT -> all now 83.
2. Added `tools/check_catalog_count.py`: a CI invariant enforcing
`SSOT count == regenerated gen count == committed gen count`, and surfacing
any paper-count divergence as an explicit erratum reminder (not a silent
edit).
3. Added `.github/workflows/catalog-count-gate.yml` (CI-01): runs the invariant
on every PR touching the catalog, codegen, or gen artifacts; also fails if
committed gen artifacts are stale relative to a fresh codegen.

## Recommended paper-side action

Update arXiv:2606.09686 with a v2 or a published erratum changing all "84"
references to "83", citing this reconciliation. Until v2 is posted, the
abstract's "84" should be read as superseded by the canonical SSOT count of 83.
The conformance packs, SHA-256 anchors, and the phi^2 + 1/phi^2 = 3 anchor
vector are unaffected.
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"fixture": "fp8_e5m2_pos_inf_special_roundtrip",
"kind": "special_value_row",
"expectation": "Check D2 MUST pass: +inf encodes and decodes to +inf; abs_error is a placeholder, not an arithmetic error; exempt from D and E.",
"pack_id": "fp8_e5m2",
"row": {
"name": "pos_inf",
"input_f64": "Inf",
"input_f64_hex": "0x7FF0000000000000",
"fp8_bits_hex": "0x7C",
"fp8_bits_int": 124,
"decoded_f64": "Inf",
"decoded_f64_hex": "0x7FF0000000000000",
"abs_error": 0.0,
"category": "inf"
},
"severity": "none",
"note": "Same-class round-trip inf->inf. The v1 gate wrongly flagged this as a nan-mismatch (gate handling gap); v2 D2 validates it correctly."
}
20 changes: 20 additions & 0 deletions conformance/integrity/fixtures/fixture_gf16_phi_mislabeled.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"fixture": "gf16_phi_mislabeled_abs_error",
"kind": "finite_value_row",
"expectation": "Check D MUST flag: stored abs_error 0.0 but abs(decoded-input) is nonzero.",
"pack_id": "gf16",
"row": {
"name": "phi",
"input_f64": 1.618033988749895,
"input_f64_hex": "0x3FF9E3779B97F4A8",
"gf16_bits_hex": "0x3F3C",
"gf16_bits_int": 16188,
"decoded_f64": 1.6171875,
"decoded_f64_hex": "0x3FF9E00000000000",
"abs_error": 0.0,
"category": "phi_anchor"
},
"rederived_abs_error": 0.0008464887498949025,
"severity": "Risk",
"note": "gf16 decode of phi rounds to 1.6171875; the 8.46e-04 error was stored as 0.0. Anchor identity phi^2+1/phi^2=3 still holds in f64; only the per-row abs_error label is wrong."
}
2 changes: 1 addition & 1 deletion conformance/vectors/INDEX_all_formats.json
Original file line number Diff line number Diff line change
Expand Up @@ -321,7 +321,7 @@
"file": "gf16_conformance_v0.json",
"kind": "bitexact",
"source": "hand-curated (pre-existing)",
"sha256": "7aea5b9e86ea71a54ae0c1601cea13e2d90d95fecaf2ae969eac1349cf7a2b42"
"sha256": "d1c0eb5bd66247b3c5db9a00a95e29cf4359653aec56f2f9e6827f96898d1509"
},
{
"id": "gf20",
Expand Down
54 changes: 54 additions & 0 deletions conformance/vectors/abs_error_allowlist.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{
"schema": "wp18-abs-error-allowlist/v1",
"note": "Explicit, auditable allow-list of bit-exact-pack rows that carry a NONZERO abs_error for an honest, documented reason. Every entry is a row where the round-trip is CORRECT but the input is not exactly representable (not_on_grid) or saturates (overflow_to_inf). The gate fails on any nonzero abs_error NOT listed here. Never add an entry to silence a real error; each must be a genuinely-disclosed case with a fixture.",
"allow": [
{
"pack_id": "bfloat16",
"row_name": "pos_0p1",
"reason": "not_on_grid",
"detail": "0.1 is not exactly representable in bf16; decoded_f64=0.10009765625, abs_error~9.77e-05 is the true nearest-representable error. Honest, expected."
},
{
"pack_id": "gf16",
"row_name": "overflow_to_inf",
"reason": "overflow_to_inf",
"detail": "input 1e40 exceeds the gf16 dynamic range and saturates to +inf; abs_error=inf is the honest record of an out-of-range input, not an encode/decode bug."
},
{
"pack_id": "gf16",
"row_name": "phi",
"reason": "not_on_grid",
"detail": "phi is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000846489 is the true nearest-representable error (WP-18 F-1 correction)."
},
{
"pack_id": "gf16",
"row_name": "inv_phi",
"reason": "not_on_grid",
"detail": "inv_phi is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000130074 is the true nearest-representable error (WP-18 F-1 correction)."
},
{
"pack_id": "gf16",
"row_name": "phi_squared",
"reason": "not_on_grid",
"detail": "phi_squared is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000846489 is the true nearest-representable error (WP-18 F-1 correction)."
},
{
"pack_id": "gf16",
"row_name": "inv_phi_sq",
"reason": "not_on_grid",
"detail": "inv_phi_sq is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000130074 is the true nearest-representable error (WP-18 F-1 correction)."
},
{
"pack_id": "gf16",
"row_name": "e",
"reason": "not_on_grid",
"detail": "e is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000468172 is the true nearest-representable error (WP-18 F-1 correction)."
},
{
"pack_id": "gf16",
"row_name": "pi",
"reason": "not_on_grid",
"detail": "pi is not exactly representable on the gf16 grid; round-trip is correct, abs_error=0.000967654 is the true nearest-representable error (WP-18 F-1 correction)."
}
]
}
12 changes: 6 additions & 6 deletions conformance/vectors/gf16_conformance_v0.json
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@
"gf16_bits_int": 16188,
"decoded_f64": 1.6171875,
"decoded_f64_hex": "0x3FF9E00000000000",
"abs_error": 0.0,
"abs_error": 0.0008464887498949025,
"category": "phi_anchor"
},
{
Expand All @@ -98,7 +98,7 @@
"gf16_bits_int": 15481,
"decoded_f64": 0.6181640625,
"decoded_f64_hex": "0x3FE3C80000000000",
"abs_error": 0.0,
"abs_error": 0.0001300737501052085,
"category": "phi_anchor"
},
{
Expand All @@ -109,7 +109,7 @@
"gf16_bits_int": 16542,
"decoded_f64": 2.6171875,
"decoded_f64_hex": "0x4004F00000000000",
"abs_error": 0.0,
"abs_error": 0.0008464887498949025,
"category": "phi_anchor"
},
{
Expand All @@ -120,7 +120,7 @@
"gf16_bits_int": 15118,
"decoded_f64": 0.3818359375,
"decoded_f64_hex": "0x3FD8700000000000",
"abs_error": 0.0,
"abs_error": 0.00013007375010509747,
"category": "phi_anchor"
},
{
Expand Down Expand Up @@ -230,7 +230,7 @@
"gf16_bits_int": 16568,
"decoded_f64": 2.71875,
"decoded_f64_hex": "0x4005C00000000000",
"abs_error": 0.0,
"abs_error": 0.0004681715409549092,
"category": "normal"
},
{
Expand All @@ -241,7 +241,7 @@
"gf16_bits_int": 16676,
"decoded_f64": 3.140625,
"decoded_f64_hex": "0x4009200000000000",
"abs_error": 0.0,
"abs_error": 0.000967653589793116,
"category": "normal"
}
]
Expand Down
Loading
Loading