gHashTag · gHashTag · Jun 14, 2026 · Jun 14, 2026 · Jun 14, 2026
@@ -2,11 +2,22 @@
 
 Last updated: 2026-06-14
 
+## repro-numerics-certifying-manifest -- schema-conforming certifying NMSE manifest + seal-hash honesty (Closes #1074)
+
+- **WHERE** (`repro/numerics/` + `repro/Makefile` + `docs/NUMERICS_VALIDATION.md`): adds a certifying-manifest mode to `repro/numerics/nmse_gf16.py` and a standalone validator `repro/numerics/validate_manifest.py`. The script now emits a SECOND manifest `repro/numerics/nmse_manifest_protocol_v1.json` that STRICTLY conforms to `schemas/nmse-protocol-v1.json` (`additionalProperties:false`; results restricted to the five schema distributions `D_NORM/D_LOG/D_RELU/D_PHI/D_DEEP`; each result is `nmse_gf16`/`nmse_bf16`/`ratio`; `schema_version`/`protocol_version`/`rng{family:pcg64,seed:0x...}`/`samples_per_distribution`/`bf16_subnormal_policy`/`identity_witness`/`runner`/`timestamp`/`x_extension`). The rich `nmse_manifest.json` (with `D_WIDE`, ULP, overflow) is kept and stays NOT schema-bound. A `--seal` flag re-hashes `bootstrap/src/compiler.rs` and sets `seal_hash` to the `bootstrap/stage0/FROZEN_HASH` digest ONLY on an exact match; otherwise `seal_hash` stays the literal `unsealed`. New `make -C repro repro-numerics-certify` target runs the validator, which checks the schema AND enforces the honesty rule (a non-`unsealed` hash must equal FROZEN_HASH).
+- **Why**: `docs/NUMERICS_VALIDATION.md` section 11 named the remaining step after the host oracle (#1072) as a "pinned-toolchain sealed run for a certifying manifest". This delivers the schema-conforming certifying manifest and its validation gate. R5-HONEST: the live compiler source does not match FROZEN_HASH on this unpinned host, so the committed `seal_hash` is honestly `unsealed` (informational, not a silicon certifying claim, protocol section 8); the script never fabricates a seal. D_WIDE intentionally excluded from the certifying manifest (not a schema result key); phi identity_witness is a sanity check, not a superiority claim. Builds on #1072. Context preprint: arXiv:2606.05017. Closes #1074.
+- **Anchor**: phi^2 + phi^-2 = 3
+
+## repro-numerics-l4-l5-oracle -- measured GF16 vs bf16/fp16 round-trip NMSE/ULP (Closes #1072)
+
+- **WHERE** (`repro/numerics/` + `docs/NUMERICS_VALIDATION.md`): adds `repro/numerics/nmse_gf16.py`, `repro/numerics/nmse_manifest.json`, and `repro/numerics/README.md`. The script round-trips real -> format -> real through the GF16 codec SSOT (`conformance/gf16_ref.py`, E6M9) and through `ml_dtypes.bfloat16` and `numpy.float16`, computing NMSE, NMSE ratios, max-abs-err, a ULP-like (kappa-approx) metric, and an overflow/saturation rate over the protocol's reference distributions (`D_NORM`, `D_LOG`, `D_RELU`, `D_PHI`, `D_DEEP`) plus a documented wide-range extension `D_WIDE` (log2|x| ~ U(-28,28)). It enforces the L5 identity witness (phi^2=phi+1, phi^2+phi^-2=3) before any number and aborts non-zero on witness failure or negative NMSE. Replaces the `TBD` cells in `docs/NUMERICS_VALIDATION.md` sections 4 (ladder L4/L5), 5 (differential oracle table), 6 (IEEE/bf16 baseline) with measured values; updates section 11 reproduction and the closing line.
+- **Why**: `docs/NUMERICS_VALIDATION.md` named the missing differential/comparative oracle as the project's predictable-skepticism gap (L4/L5 = TBD/P1). This closes it at host level with reproducible numbers (seed 2718281, 2,000,000 samples/distribution): GF16 NMSE is ~16x lower than bf16 (ratio ~0.063, 9 vs 7 mantissa bits) and ~4x higher than fp16 (10 vs 9 mantissa bits) near 1.0, while on D_WIDE fp16 overflows 21.4% of samples and GF16 (max ~4.29e9) and bf16 lose none -- the honest exponent/mantissa tradeoff predicted by protocol section 3.3 and matching the E6M9 motivation of IBM DLFloat (DOI 10.1109/ARITH.2019.00023) and Popescu et al. (arXiv:2103.15940). R5-HONEST: host-measured, unsealed => informational, not a silicon certifying claim; D_PHI is an identity sanity check, not a superiority claim. ASCII-only. L6 SSOT untouched. Context preprint: arXiv:2606.05017. Closes #1072.
+- **Anchor**: phi^2 + phi^-2 = 3
+
 ## conformance-vectors-all-83-formats -- catalog-wide conformance packs, 49 bit-exact + 34 structural (Closes #1071)
 
 - **WHERE** (conformance vectors): extends the v0 packs (#1055, five Tier-1 packs) to full catalog coverage under `conformance/vectors/`. Adds 78 new `*_conformance_v0.json` packs so that all 83 formats in the numeric catalog SSOT have exactly one pack each, no gaps. 49 packs are bit-exact (fixed radix-2 S:E:M layout, full encode->decode->f64 round-trip with honest abs_error) and 34 are structural (formats without a fixed radix-2 S:E:M layout -- parametric, table-based, radix-16, tapered-log, extended-precision, or open-R&D-parameter -- marked `bitexact: false` with a `structural_reason` field and no round-trip rows). New `INDEX_all_formats.json` (`total_packs=83, bitexact_packs=49, structural_packs=34`) and new master generator `gen_all_formats.py` reading the catalog SSOT. `README.md` rewritten to document all 83 packs: coverage policy, 49-pack bit-exact index, 34-pack structural index with reasons, full SHA-256 block, schema, provenance. The five pre-existing hand-curated generators are kept.
 - **Why**: gives the t27 numeric SSOT a citable conformance reference for every catalog format, not just the FP8/MXFP4/bf16 Tier-1 subset. Anchor identity phi^2 + 1/phi^2 = 3 lands exactly on grid for 43 of 49 bit-exact packs; the six exceptions (LNS-8/16/32/64, GF4, MXGF4) honestly record the nearest representable value and true abs_error -- log2(3) is not representable in LNS and the 4-bit grids are too coarse, so these are expected, not bugs. The six ml_dtypes-0.5.4-cross-checked packs (GF16, MXFP4 element, BF16, FP8 E4M3, FP8 E5M2, E8M0) remain a deeply-validated Tier-1 reference subset; absence of a pack is never a quality claim. Coverage policy: formats <=8-bit get exhaustive enumeration, >8-bit get curated named vectors, non-S:E:M get structural packs. ASCII-only. These are conformance data + reproduction scripts, not a quality claim; no [Verified] silicon claim is asserted beyond the existing GF16 rung. L6 gf16 SSOT untouched; L4 not applicable (data files, not specs); L2 (`gen/`) untouched. Context preprint: arXiv:2606.05017. Closes #1071.
-- **Anchor**: phi^2 + phi^-2 = 3
 
 ## ops-unify-email -- consolidate contact emails to admin@t27.ai (Closes #1067)
 

@@ -44,35 +44,39 @@ Until filled, treat numeric behavior as **implementation-defined** outside confo
 | L1 | **Exhaustive** encode/decode + op table | GF4 (and GF8 if feasible) | TBD |
 | L2 | **Conformance JSON** — existing `conformance/gf*_vectors.json` | GF4–GF32 as covered | partial |
 | L3 | **Property-based / randomized** boundaries | GF16+ | TBD |
-| L4 | **Differential** vs reference (Python `decimal`, or MPFR) | GF16 primary | TBD — P1 |
-| L5 | **Comparative** vs IEEE fp16 / fp32 / bfloat16 on same corpus | GF16 vs fp16/bf16 | TBD |
+| L4 | **Differential** vs reference (round-trip oracle) | GF16 primary | **measured (host, unsealed)** — `repro/numerics/` |
+| L5 | **Comparative** vs IEEE fp16 / bfloat16 on same corpus | GF16 vs fp16/bf16 | **measured (host, unsealed)** — `repro/numerics/nmse_manifest.json` |
 | L6 | **Optional** posit reference (where tooling exists) | TBD | TBD |
 
 ---
 
 ## 5. Differential oracle — skeleton results table
 
-*Replace `TBD` with versioned runs; one row per (format, operation, corpus slice).*
+*Measured runs (host, unsealed). Reference oracle = f64 round-trip `real -> format -> real`. Seed 2718281, 2,000,000 samples/distribution. Reproduce: `python repro/numerics/nmse_gf16.py`.*
 
 | Run ID | Format | Operation | Corpus | Reference oracle | Max abs err | ULP-like metric | Pass? | Artifact |
 |--------|--------|-----------|--------|------------------|-------------|-----------------|-------|----------|
-| TBD | GF16 | add | conformance subset | Python `decimal` | TBD | TBD | TBD | `repro/numerics/` (future) |
-| TBD | GF16 | mul | … | … | TBD | TBD | TBD | … |
-| TBD | GF32 | add | … | … | TBD | TBD | TBD | … |
+| nmse-2718281-D_NORM | GF16 | round-trip | D_NORM (N(0,1)) | f64 | 3.90e-03 | 3.53e-04 | yes | `repro/numerics/nmse_manifest.json` |
+| nmse-2718281-D_LOG | GF16 | round-trip | D_LOG (log2\|x\|~U(-10,10)) | f64 | (see manifest) | (see manifest) | yes | `repro/numerics/nmse_manifest.json` |
+| nmse-2718281-D_WIDE | GF16 | round-trip | D_WIDE (log2\|x\|~U(-28,28)) | f64 | (see manifest) | (see manifest) | yes | `repro/numerics/nmse_manifest.json` |
 
-**Falsification:** any cell exceeds stated envelope once §2 is normative → **fail CI** or **downgrade claim** in `RESEARCH_CLAIMS.md`.
+**Falsification:** any cell exceeds stated envelope once §2 is normative → **fail CI** or **downgrade claim** in `RESEARCH_CLAIMS.md`. The runner already aborts non-zero if the L5 identity witness fails or any NMSE < 0.
 
 ---
 
 ## 6. IEEE / bfloat16 baseline — skeleton comparison
 
 Same inputs as §5 where bit patterns map sensibly; document **non-comparable** cases explicitly.
 
-| Metric | GF16 | IEEE fp16 | bfloat16 | IEEE fp32 | Notes |
-|--------|------|-----------|----------|-----------|-------|
-| Dynamic range (stated) | TBD | TBD | TBD | TBD | From spec / measured |
-| MSE on N(0,1) sample | TBD | TBD | TBD | TBD | Trinity Phase-1 style table may be ported |
-| Add latency (soft impl) | TBD | TBD | — | TBD | Host-only; not FPGA |
+*Measured, host, unsealed (`repro/numerics/nmse_manifest.json`). Non-comparable cases noted.*
+
+| Metric | GF16 | IEEE fp16 | bfloat16 | Notes |
+|--------|------|-----------|----------|-------|
+| Mantissa / exponent bits | 9 / 6 | 10 / 5 | 7 / 8 | bit split |
+| Max finite magnitude | ~4.29e9 | ~6.55e4 | ~3.39e38 | dynamic range |
+| NMSE on N(0,1) (D_NORM) | 1.73e-07 | 4.30e-08 | 2.76e-06 | GF16 ~16x better than bf16; fp16 ~4x better than GF16 |
+| Overflow rate on D_WIDE (log2\|x\|~U(-28,28)) | 0.0000 | 0.2144 | 0.0000 | fp16 saturates; GF16/bf16 do not |
+| Add latency (soft impl) | n/a | n/a | n/a | host-only round-trip study; latency out of scope (see protocol §1) |
 
 ---
 
@@ -110,8 +114,10 @@ Constant comparisons (if any) must cite **year and revision** and uncertainty; d
 ## 11. Reproduction
 
 - **Smoke:** `make -C repro repro-numerics` (JSON validity).  
-- **Future:** `make repro-numerics-diff` (pinned Python + lockfile) — add in `repro/Makefile` when L4 exists.
+- **L4/L5 oracle (measured):** `python repro/numerics/nmse_gf16.py` — round-trip NMSE/ULP of GF16 vs bf16/fp16 over the protocol distributions; writes `repro/numerics/nmse_manifest.json` (rich) and `repro/numerics/nmse_manifest_protocol_v1.json` (certifying). Host, unsealed.
+- **Certifying manifest:** `make -C repro repro-numerics-certify` (or `python repro/numerics/validate_manifest.py`) validates `repro/numerics/nmse_manifest_protocol_v1.json` against `schemas/nmse-protocol-v1.json` and enforces the seal-hash honesty rule.
+- **Sealed run:** `python repro/numerics/nmse_gf16.py --seal` sets `seal_hash` to the `bootstrap/stage0/FROZEN_HASH` digest **only** if the live seal source matches it under a pinned toolchain; otherwise it stays `unsealed`. As committed the manifest is `unsealed` (host, unpinned) — informational, not a silicon certifying claim (protocol section 8).
 
 ---
 
-*Without differential oracles, GoldenFloat will face predictable skepticism — this file is the contract to close that gap.*
+*The L4/L5 differential/comparative oracle is now MEASURED (host, unsealed) in `repro/numerics/` — the predictable-skepticism gap is closed at the host level; the remaining step is a sealed-toolchain certifying run.*
@@ -1,5 +1,5 @@
 # Reproducibility targets — run: make -C repro <target>
-.PHONY: repro-smoke repro-language repro-numerics repro-ar repro-paper-figures
+.PHONY: repro-smoke repro-language repro-numerics repro-numerics-certify repro-ar repro-paper-figures
 
 ROOT := $(abspath $(dir $(lastword $(MAKEFILE_LIST)))/..)
 T27C := $(ROOT)/bootstrap/target/release/t27c
@@ -17,6 +17,12 @@ repro-numerics:
 	cd "$(ROOT)" && bash tests/validate_conformance.sh
 	@echo "Numeric vectors: conformance/gf*_vectors.json, sacred_physics*.json (see module field in each JSON)."
 
+# Validate the committed certifying manifest against schemas/nmse-protocol-v1.json.
+# Re-validation only (no measurement); the measured run is the L4/L5 oracle below.
+repro-numerics-certify:
+	cd "$(ROOT)" && python3 repro/numerics/validate_manifest.py
+	@echo "Certifying manifest conforms to schemas/nmse-protocol-v1.json (seal_hash honesty enforced)."
+
 repro-ar:
 	cd "$(ROOT)" && bash tests/validate_conformance.sh
 	@echo "AR vectors: conformance/ar_*.json"

@@ -0,0 +1,106 @@
+# repro/numerics -- L4/L5 differential accuracy oracle for GF16
+
+This directory closes the L4 (differential) and L5 (comparative) rungs of the
+numeric validation ladder for GoldenFloat **GF16** (E6M9), per
+`docs/GF16_BFLOAT16_NMSE_PROTOCOL.md` and `docs/NUMERICS_VALIDATION.md`.
+
+It produces **measured** round-trip accuracy numbers for GF16 versus
+**bfloat16** and **IEEE float16** -- the two 16-bit alternatives of the same
+memory footprint -- over the protocol's reference distributions.
+
+> **R5-HONEST.** These are **host-measured, unsealed** numbers. Per protocol
+> section 8 they are **informational, not a silicon certifying claim**. No
+> result here asserts a measured silicon NMSE for any product. `D_PHI` is an
+> identity-anchored sanity check (L5), **not** a superiority claim.
+
+## How to reproduce
+
+```
+python repro/numerics/nmse_gf16.py            # 2,000,000 samples/distribution
+python repro/numerics/nmse_gf16.py --samples 10000000 --seed 2718281
+```
+
+- GF16 codec under test: `conformance/gf16_ref.py` (BIAS=31, EXP_BITS=6, MANT_BITS=9).
+- bf16: `ml_dtypes.bfloat16`; fp16: `numpy.float16`.
+- The run aborts non-zero if the L5 identity witness
+  (`|phi^2-(phi+1)|<1e-15`, `|phi^2+phi^-2-3|<1e-15`) fails or any NMSE < 0.
+- Output manifests:
+  - `nmse_manifest.json` -- the **rich** manifest (fields per protocol section 6:
+    all six distributions including `D_WIDE`, ULP-like metric, overflow rates,
+    representable-max table). Intentionally NOT schema-bound.
+  - `nmse_manifest_protocol_v1.json` -- the **certifying** manifest, strictly
+    conforming to `schemas/nmse-protocol-v1.json` (`additionalProperties:false`;
+    only the five schema distributions `D_NORM/D_LOG/D_RELU/D_PHI/D_DEEP`; each
+    result is `nmse_gf16`/`nmse_bf16`/`ratio`). `D_WIDE` is deliberately absent.
+
+## Sealing and the certifying manifest (R5-HONEST)
+
+The certifying manifest carries a `seal_hash` field. It is set to the 64-hex
+SHA-256 in `bootstrap/stage0/FROZEN_HASH` **only** when `--seal` is passed AND
+the live seal source (`bootstrap/src/compiler.rs`) hashes to exactly that
+digest under a pinned toolchain. In every other case it stays the literal
+`"unsealed"`. The script never fabricates a seal: a host run on an unpinned
+tree is honestly `unsealed` and informational only.
+
+```
+python repro/numerics/nmse_gf16.py --seal          # seals iff source matches FROZEN_HASH
+python repro/numerics/validate_manifest.py         # re-validate the committed manifest
+make -C repro repro-numerics-certify               # CI-facing validation step
+```
+
+`validate_manifest.py` validates the manifest against the schema AND enforces
+the honesty rule: a non-`unsealed` `seal_hash` must equal the FROZEN_HASH
+digest, or it exits non-zero. As committed, `seal_hash = "unsealed"` because
+this manifest was produced on an unpinned host (the live compiler source does
+not match the frozen digest) -- exactly the state protocol section 8 calls
+informational, not a silicon certifying claim.
+
+## Measured results (seed 2718281, 2,000,000 samples/distribution, unsealed host)
+
+`NMSE(F) = E[(x - Q_F(x))^2] / E[x^2]`. Headline = `NMSE_GF16 / NMSE_BF16`
+(< 1 means GF16 is closer to the reference).
+
+| Distribution | NMSE GF16 | NMSE BF16 | NMSE FP16 | GF16/BF16 | GF16/FP16 |
+|--------------|-----------|-----------|-----------|-----------|-----------|
+| D_NORM  | 1.73e-07 | 2.76e-06 | 4.30e-08 | **0.063** | 4.02 |
+| D_LOG   | 1.48e-07 | 2.35e-06 | 3.68e-08 | **0.063** | 4.02 |
+| D_RELU  | 1.73e-07 | 2.76e-06 | 4.32e-08 | **0.063** | 4.01 |
+| D_PHI   | 1.78e-07 | 2.85e-06 | 4.45e-08 | **0.063** | 4.00 |
+| D_DEEP  | 1.48e-07 | 2.37e-06 | 3.65e-08 | **0.062** | 4.04 |
+| D_WIDE  | 1.47e-07 | 2.36e-06 | 3.67e-08 | **0.063** | 4.02 |
+
+Saturation / overflow rate (fraction of non-zero samples beyond a format's
+finite max: GF16 ~4.29e9, BF16 ~3.39e38, FP16 ~6.55e4):
+
+| Distribution | GF16 | BF16 | FP16 |
+|--------------|------|------|------|
+| D_WIDE (log2|x| ~ U(-28,28)) | 0.0000 | 0.0000 | **0.2144** |
+
+(all other distributions: 0.0000 for every format)
+
+## Honest interpretation (facts, not claims)
+
+1. **GF16 vs bf16 (mantissa).** GF16 has 9 mantissa bits, bf16 has 7. Across
+   every distribution GF16's round-trip NMSE is ~16x lower (ratio ~0.063).
+   This is the expected consequence of the bit split, not a surprise.
+2. **GF16 vs fp16 (mantissa).** fp16 has 10 mantissa bits, GF16 has 9, so fp16
+   is ~4x more accurate near 1.0 (ratio ~4.0). GF16 does **not** beat fp16 on
+   near-1.0 precision -- stated plainly.
+3. **Dynamic range (exponent).** This is where the tradeoff flips. fp16's
+   exponent saturates at ~65504, so on a wide-range distribution **21.4% of
+   fp16 samples overflow**, while GF16 (max ~4.29e9, 6-bit exponent) and bf16
+   (8-bit exponent) lose none. GF16's wider range is the price fp16 pays for
+   its extra mantissa bit.
+4. **Net.** GF16 sits between bf16 and fp16: more precise than bf16, wider
+   range than fp16. This matches the E6M9 motivation of IBM DLFloat
+   (ARITH 2019, DOI 10.1109/ARITH.2019.00023) and Popescu et al.
+   (arXiv:2103.15940), which independently pick a 1/6/9 split.
+
+## Cross-links
+
+- Protocol: `docs/GF16_BFLOAT16_NMSE_PROTOCOL.md`
+- Validation ladder: `docs/NUMERICS_VALIDATION.md` (this fills L4/L5)
+- Codec SSOT: `conformance/gf16_ref.py`, `conformance/FORMAT-SPEC-001.json`
+- Preprint context: arXiv:2606.05017
+
+phi^2 + phi^-2 = 3  |  TRINITY