diff --git a/conformance/vectors/README.md b/conformance/vectors/README.md new file mode 100644 index 000000000..c60ad4339 --- /dev/null +++ b/conformance/vectors/README.md @@ -0,0 +1,106 @@ +# t27 numeric-format conformance vector packs (v0) + +Bit-precise conformance vectors for a family of low-precision numeric formats, +in a single shared row schema so one differ runs across all packs. + +- SSOT: https://github.com/gHashTag/t27/blob/master/conformance/FORMAT-SPEC-001.json +- Anchor identity (ASCII): `phi^2 + 1/phi^2 = 3` +- Context preprint: https://arxiv.org/abs/2606.05017 +- Schema tag: `t27-conformance/v0.1` + +Every pack decodes its native bits to f64 exactly (`abs_error = 0` by +construction for representable values). Values that are NOT exactly +representable in a format report a nonzero `abs_error` honestly (for example +0.1 in bf16) -- nothing is hidden. + +## Index + +| Pack | Format | Bits | Vectors | 3.0 anchor bits | Max finite | Inf | NaN | Has golden | tt-mlir issue | +|---|---|---|---|---|---|---|---|---|---| +| `gf16_conformance_v0.json` | GF16 | 16 | 21 | `0x4100` | n/a | yes | no | no | sibling (SSOT anchor pack) | +| `fp8_e4m3fn_conformance_v0.json` | FP8 E4M3FN | 8 | 14 | `0x44` | 448.0 (`0x7E`) | no | single (`0x7F`/`0xFF`) | no | [#8549](https://github.com/tenstorrent/tt-mlir/issues/8549) | +| `fp8_e5m2_conformance_v0.json` | FP8 E5M2 | 8 | 16 | `0x42` | 57344.0 (`0x7B`) | yes (`0x7C`/`0xFC`) | yes (E=11111,M!=0) | no | [#8549](https://github.com/tenstorrent/tt-mlir/issues/8549) | +| `mxfp4_e2m1_conformance_v0.json` | MXFP4 E2M1 | 4 | 16 | `0x5` | 6.0 (`0x7`) | no | no | no | [#7085](https://github.com/tenstorrent/tt-mlir/issues/7085) | +| `bf16_golden_conformance_v0.json` | bfloat16 | 16 | 8 + golden | `0x4040` | n/a | yes | yes | yes | [#6252](https://github.com/tenstorrent/tt-mlir/issues/6252) | + +## SHA-256 + +``` +7aea5b9e86ea71a54ae0c1601cea13e2d90d95fecaf2ae969eac1349cf7a2b42 gf16_conformance_v0.json +7193ccd0d330d3e05154432abcec5da4a4c170e11004d4ffa44ff5cbbff9cba9 fp8_e4m3fn_conformance_v0.json +9c31fbd03923bd6555304848a092504dfbc02f72d2be82d2b80f49243e925a18 fp8_e5m2_conformance_v0.json +b5795fed0c0f2b580174b443d2c54519c4953916525237bb7ea7d6831f14fde7 mxfp4_e2m1_conformance_v0.json +98bbddcbb8a520dc45a6dfed7209c50a0acc0fabc4d3b430359969467eee4e13 bf16_golden_conformance_v0.json +``` + +## Shared row schema + +Each vector row carries: + +``` +name short label +input_f64 the f64 value (or "NaN" / "Inf" / "-Inf" for specials) +input_f64_hex big-endian IEEE-754 double hex of input +_bits_hex the native format bit pattern, hex +_bits_int the native format bit pattern, integer +decoded_f64 value after decode back to f64 +decoded_f64_hex big-endian double hex of decoded +abs_error |input_f64 - decoded_f64| (0 for representable values) +category one of: zero, subnormal, normal, inf, nan, phi_anchor +``` + +The bits field is named per format: `gf16_bits_*`, `fp8_bits_*`, +`mxfp4_bits_*`, `bf16_bits_*`. All other keys are identical across packs. + +## Per-format notes (the round-trip decision in each) + +### GF16 (16-bit) +The original anchor pack. Carries `phi`, `inv_phi`, `phi^2`, `inv_phi^2`, and +the identity sum `phi^2 + 1/phi^2 = 3` (vector `identity_sum`, bits `0x4100`). +Categories include `phi_anchor`. Sibling to the FP8/MXFP4/bf16 packs via the +shared row schema. + +### FP8 E4M3FN (`torch.float8_e4m3fn`) +1s4e3m, bias 7. No inf. Single NaN at `S.1111.111`. Max finite 448.0 at `0x7E` +(mantissa 110, since all-ones mantissa at max exponent is reserved for NaN). +Min subnormal 2^-9, min normal 2^-6. +Encode policy (reference): round-nearest-ties-even; overflow SATURATES to +448.0, not NaN. That saturate-vs-NaN choice is the round-trip decision in +tt-mlir #8549. + +### FP8 E5M2 (`torch.float8_e5m2`) +1s5e2m, bias 15. HAS inf at `0x7C`/`0xFC` (E=11111, M=00); NaN at E=11111, +M!=00. Max finite 57344.0 at `0x7B`. Min subnormal 2^-16, min normal 2^-14. +Encode policy (reference): round-nearest-ties-even; overflow goes to +/-Inf +(true IEEE), NOT saturate. The saturate (E4M3FN) vs overflow-to-Inf (E5M2) +split is the practical round-trip gotcha across the two FP8 variants. + +### MXFP4 E2M1 (OCP Microscaling element format) +1s2e1m, bias 1. Only 16 codes; no inf, no NaN. Full positive grid +`{0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 6.0}`. Max finite 6.0 at `0x7`. +tt-mlir #7085 clamps to +/-7, which is one step ABOVE the format max: +values in (6, 7] still round to the 6.0 grid point. The clamp bound and the +representable max are different numbers; the pack enumerates all 16 codes so +that relationship is explicit. Anchor 3.0 is an exact grid point at `0x5`. + +### bfloat16 (golden accumulation) +1s8e7m, bias 127 (top 16 bits of fp32). HAS inf and NaN. Decode exact into f64. +Element table plus a `golden_accumulation` section targeting tt-mlir #6252: +the all-close gap in depth-31 fused reduction trees comes from accumulation +ORDER, not single-op rounding. The section gives an f64 golden reference and +compares sequential vs pairwise bf16 reductions; pairwise stays near the golden +while sequential drifts. A reproducible all-close test needs a clamped +deterministic input set plus an f64 golden -- both are supplied. + +## Provenance + +Generators (re-run to reproduce byte-identical packs): + +``` +gen_fp8_e4m3.py -> fp8_e4m3fn_conformance_v0.json +gen_fp8_e5m2.py -> fp8_e5m2_conformance_v0.json +gen_mxfp4_e2m1.py -> mxfp4_e2m1_conformance_v0.json +gen_bf16_golden.py -> bf16_golden_conformance_v0.json +``` + +All packs are ASCII-only. Apache-2.0, consistent with the t27 repository. diff --git a/conformance/vectors/bf16_golden_conformance_v0.json b/conformance/vectors/bf16_golden_conformance_v0.json new file mode 100644 index 000000000..6579af0f3 --- /dev/null +++ b/conformance/vectors/bf16_golden_conformance_v0.json @@ -0,0 +1,169 @@ +{ + "schema": "t27-conformance/v0.1", + "format": "BFLOAT16", + "format_notes": "1s8e7m, bias 127, same exponent range as fp32 (top 16 bits of fp32). HAS inf and NaN. Decode exact into f64.", + "ssot": "https://github.com/gHashTag/t27/blob/master/conformance/FORMAT-SPEC-001.json", + "preprint": "https://arxiv.org/abs/2606.05017", + "anchor_identity": "phi^2 + 1/phi^2 = 3", + "anchor_check": { + "value": 3.0, + "expected": 3.0, + "ieee754_exact": true, + "bf16_bits_hex": "0x4040" + }, + "issue_6252_note": "Rounding-error accumulation in depth-31 fused trees. The all-close gap comes from accumulation ORDER, not single-op rounding. The golden_accumulation section gives an f64 golden reference plus sequential vs pairwise bf16 reductions; pairwise reduces the accumulated error. A reproducible all-close test needs (a) a clamped deterministic input set and (b) an f64 golden, both supplied here.", + "round_trip_policy": "decode: bf16 bits -> f64 exact. encode: fp32->bf16 round-nearest-ties-even on the 16-bit truncation boundary. abs_error is nonzero only for inputs not representable in bf16 (e.g. 0.1) -- stated honestly, not hidden.", + "n_vectors": 8, + "vectors": [ + { + "name": "pos_zero", + "input_f64": 0.0, + "input_f64_hex": "0x0000000000000000", + "bf16_bits_hex": "0x0000", + "bf16_bits_int": 0, + "decoded_f64": 0.0, + "decoded_f64_hex": "0x0000000000000000", + "abs_error": 0.0, + "category": "zero" + }, + { + "name": "pos_one", + "input_f64": 1.0, + "input_f64_hex": "0x3FF0000000000000", + "bf16_bits_hex": "0x3F80", + "bf16_bits_int": 16256, + "decoded_f64": 1.0, + "decoded_f64_hex": "0x3FF0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "neg_one", + "input_f64": -1.0, + "input_f64_hex": "0xBFF0000000000000", + "bf16_bits_hex": "0xBF80", + "bf16_bits_int": 49024, + "decoded_f64": -1.0, + "decoded_f64_hex": "0xBFF0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_two", + "input_f64": 2.0, + "input_f64_hex": "0x4000000000000000", + "bf16_bits_hex": "0x4000", + "bf16_bits_int": 16384, + "decoded_f64": 2.0, + "decoded_f64_hex": "0x4000000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_three", + "input_f64": 3.0, + "input_f64_hex": "0x4008000000000000", + "bf16_bits_hex": "0x4040", + "bf16_bits_int": 16448, + "decoded_f64": 3.0, + "decoded_f64_hex": "0x4008000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_half", + "input_f64": 0.5, + "input_f64_hex": "0x3FE0000000000000", + "bf16_bits_hex": "0x3F00", + "bf16_bits_int": 16128, + "decoded_f64": 0.5, + "decoded_f64_hex": "0x3FE0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_quarter", + "input_f64": 0.25, + "input_f64_hex": "0x3FD0000000000000", + "bf16_bits_hex": "0x3E80", + "bf16_bits_int": 16000, + "decoded_f64": 0.25, + "decoded_f64_hex": "0x3FD0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_0p1", + "input_f64": 0.1, + "input_f64_hex": "0x3FB999999999999A", + "bf16_bits_hex": "0x3DCD", + "bf16_bits_int": 15821, + "decoded_f64": 0.10009765625, + "decoded_f64_hex": "0x3FB9A00000000000", + "abs_error": 9.765624999999445e-05, + "category": "normal" + } + ], + "golden_accumulation": [ + { + "element_f64": 0.1, + "element_f64_hex": "0x3FB999999999999A", + "element_bf16_hex": "0x3DCD", + "n_terms": 1024, + "golden_f64": 102.4, + "golden_f64_hex": "0x405999999999999A", + "sequential_bf16": 32.0, + "pairwise_bf16": 102.5, + "abs_err_sequential": 70.4, + "abs_err_pairwise": 0.09999999999999432, + "rel_err_sequential": 0.6875, + "rel_err_pairwise": 0.0009765624999999445, + "pairwise_better": true + }, + { + "element_f64": 3.0, + "element_f64_hex": "0x4008000000000000", + "element_bf16_hex": "0x4040", + "n_terms": 1024, + "golden_f64": 3072.0, + "golden_f64_hex": "0x40A8000000000000", + "sequential_bf16": 1024.0, + "pairwise_bf16": 3072.0, + "abs_err_sequential": 2048.0, + "abs_err_pairwise": 0.0, + "rel_err_sequential": 0.6666666666666666, + "rel_err_pairwise": 0.0, + "pairwise_better": true + }, + { + "element_f64": 0.1, + "element_f64_hex": "0x3FB999999999999A", + "element_bf16_hex": "0x3DCD", + "n_terms": 4096, + "golden_f64": 409.6, + "golden_f64_hex": "0x407999999999999A", + "sequential_bf16": 32.0, + "pairwise_bf16": 410.0, + "abs_err_sequential": 377.6, + "abs_err_pairwise": 0.39999999999997726, + "rel_err_sequential": 0.921875, + "rel_err_pairwise": 0.0009765624999999445, + "pairwise_better": true + }, + { + "element_f64": 0.3333333333333333, + "element_f64_hex": "0x3FD5555555555555", + "element_bf16_hex": "0x3EAB", + "n_terms": 2048, + "golden_f64": 682.6666666666666, + "golden_f64_hex": "0x4085555555555555", + "sequential_bf16": 128.0, + "pairwise_bf16": 684.0, + "abs_err_sequential": 554.6666666666666, + "abs_err_pairwise": 1.3333333333333712, + "rel_err_sequential": 0.8125, + "rel_err_pairwise": 0.0019531250000000555, + "pairwise_better": true + } + ] +} \ No newline at end of file diff --git a/conformance/vectors/fp8_e4m3fn_conformance_v0.json b/conformance/vectors/fp8_e4m3fn_conformance_v0.json new file mode 100644 index 000000000..133f931d7 --- /dev/null +++ b/conformance/vectors/fp8_e4m3fn_conformance_v0.json @@ -0,0 +1,172 @@ +{ + "schema": "t27-conformance/v0.1", + "format": "FP8_E4M3FN", + "format_notes": "OCP / torch.float8_e4m3fn. 1s4e3m, bias 7, no inf, single NaN S.1111.111, max finite 448.0. Subnormal min 2^-9.", + "ssot": "https://github.com/gHashTag/t27/blob/master/conformance/FORMAT-SPEC-001.json", + "preprint": "https://arxiv.org/abs/2606.05017", + "anchor_identity": "phi^2 + 1/phi^2 = 3", + "anchor_check": { + "value": 3.0, + "expected": 3.0, + "ieee754_exact": true, + "fp8_bits_hex": "0x44" + }, + "round_trip_policy": "decode: exact bits->f64. encode (reference): round-nearest-ties-even, overflow saturates to max finite 448.0 (NOT NaN). This is the divergence point flagged in tt-mlir#8549.", + "n_vectors": 14, + "vectors": [ + { + "name": "pos_zero", + "input_f64": 0.0, + "input_f64_hex": "0x0000000000000000", + "fp8_bits_hex": "0x00", + "fp8_bits_int": 0, + "decoded_f64": 0.0, + "decoded_f64_hex": "0x0000000000000000", + "abs_error": 0.0, + "category": "zero" + }, + { + "name": "neg_zero", + "input_f64": -0.0, + "input_f64_hex": "0x8000000000000000", + "fp8_bits_hex": "0x80", + "fp8_bits_int": 128, + "decoded_f64": -0.0, + "decoded_f64_hex": "0x8000000000000000", + "abs_error": 0.0, + "category": "zero" + }, + { + "name": "min_pos_subnormal", + "input_f64": 0.001953125, + "input_f64_hex": "0x3F60000000000000", + "fp8_bits_hex": "0x01", + "fp8_bits_int": 1, + "decoded_f64": 0.001953125, + "decoded_f64_hex": "0x3F60000000000000", + "abs_error": 0.0, + "category": "subnormal" + }, + { + "name": "max_pos_subnormal", + "input_f64": 0.013671875, + "input_f64_hex": "0x3F8C000000000000", + "fp8_bits_hex": "0x07", + "fp8_bits_int": 7, + "decoded_f64": 0.013671875, + "decoded_f64_hex": "0x3F8C000000000000", + "abs_error": 0.0, + "category": "subnormal" + }, + { + "name": "min_pos_normal", + "input_f64": 0.015625, + "input_f64_hex": "0x3F90000000000000", + "fp8_bits_hex": "0x08", + "fp8_bits_int": 8, + "decoded_f64": 0.015625, + "decoded_f64_hex": "0x3F90000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_half", + "input_f64": 0.5, + "input_f64_hex": "0x3FE0000000000000", + "fp8_bits_hex": "0x30", + "fp8_bits_int": 48, + "decoded_f64": 0.5, + "decoded_f64_hex": "0x3FE0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_one", + "input_f64": 1.0, + "input_f64_hex": "0x3FF0000000000000", + "fp8_bits_hex": "0x38", + "fp8_bits_int": 56, + "decoded_f64": 1.0, + "decoded_f64_hex": "0x3FF0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "neg_one", + "input_f64": -1.0, + "input_f64_hex": "0xBFF0000000000000", + "fp8_bits_hex": "0xB8", + "fp8_bits_int": 184, + "decoded_f64": -1.0, + "decoded_f64_hex": "0xBFF0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_two", + "input_f64": 2.0, + "input_f64_hex": "0x4000000000000000", + "fp8_bits_hex": "0x40", + "fp8_bits_int": 64, + "decoded_f64": 2.0, + "decoded_f64_hex": "0x4000000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_three", + "input_f64": 3.0, + "input_f64_hex": "0x4008000000000000", + "fp8_bits_hex": "0x44", + "fp8_bits_int": 68, + "decoded_f64": 3.0, + "decoded_f64_hex": "0x4008000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_six", + "input_f64": 6.0, + "input_f64_hex": "0x4018000000000000", + "fp8_bits_hex": "0x4C", + "fp8_bits_int": 76, + "decoded_f64": 6.0, + "decoded_f64_hex": "0x4018000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "max_finite_pos", + "input_f64": 448.0, + "input_f64_hex": "0x407C000000000000", + "fp8_bits_hex": "0x7E", + "fp8_bits_int": 126, + "decoded_f64": 448.0, + "decoded_f64_hex": "0x407C000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "max_finite_neg", + "input_f64": -448.0, + "input_f64_hex": "0xC07C000000000000", + "fp8_bits_hex": "0xFE", + "fp8_bits_int": 254, + "decoded_f64": -448.0, + "decoded_f64_hex": "0xC07C000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "nan", + "input_f64": "NaN", + "input_f64_hex": "0x7FF8000000000000", + "fp8_bits_hex": "0x7F", + "fp8_bits_int": 127, + "decoded_f64": "NaN", + "decoded_f64_hex": "0x7FF8000000000000", + "abs_error": "NaN", + "category": "nan" + } + ] +} \ No newline at end of file diff --git a/conformance/vectors/fp8_e5m2_conformance_v0.json b/conformance/vectors/fp8_e5m2_conformance_v0.json new file mode 100644 index 000000000..b52b4c09e --- /dev/null +++ b/conformance/vectors/fp8_e5m2_conformance_v0.json @@ -0,0 +1,194 @@ +{ + "schema": "t27-conformance/v0.1", + "format": "FP8_E5M2", + "format_notes": "OCP / torch.float8_e5m2. 1s5e2m, bias 15, HAS inf (E=11111,M=00), NaN (E=11111,M!=00), max finite 57344.0 at 0x7B. Subnormal min 2^-16, normal min 2^-14.", + "ssot": "https://github.com/gHashTag/t27/blob/master/conformance/FORMAT-SPEC-001.json", + "preprint": "https://arxiv.org/abs/2606.05017", + "anchor_identity": "phi^2 + 1/phi^2 = 3", + "anchor_check": { + "value": 3.0, + "expected": 3.0, + "ieee754_exact": true, + "fp8_bits_hex": "0x42" + }, + "round_trip_policy": "decode: exact bits->f64. encode (reference): round-nearest-ties-even, overflow goes to +/-Inf (IEEE), NOT saturate. This is the variant-level divergence from E4M3FN (which saturates to 448.0). The overflow-to-Inf vs saturate split is the practical round-trip gotcha across the two FP8 variants.", + "n_vectors": 16, + "vectors": [ + { + "name": "pos_zero", + "input_f64": 0.0, + "input_f64_hex": "0x0000000000000000", + "fp8_bits_hex": "0x00", + "fp8_bits_int": 0, + "decoded_f64": 0.0, + "decoded_f64_hex": "0x0000000000000000", + "abs_error": 0.0, + "category": "zero" + }, + { + "name": "neg_zero", + "input_f64": -0.0, + "input_f64_hex": "0x8000000000000000", + "fp8_bits_hex": "0x80", + "fp8_bits_int": 128, + "decoded_f64": -0.0, + "decoded_f64_hex": "0x8000000000000000", + "abs_error": 0.0, + "category": "zero" + }, + { + "name": "min_pos_subnormal", + "input_f64": 1.52587890625e-05, + "input_f64_hex": "0x3EF0000000000000", + "fp8_bits_hex": "0x01", + "fp8_bits_int": 1, + "decoded_f64": 1.52587890625e-05, + "decoded_f64_hex": "0x3EF0000000000000", + "abs_error": 0.0, + "category": "subnormal" + }, + { + "name": "max_pos_subnormal", + "input_f64": 4.57763671875e-05, + "input_f64_hex": "0x3F08000000000000", + "fp8_bits_hex": "0x03", + "fp8_bits_int": 3, + "decoded_f64": 4.57763671875e-05, + "decoded_f64_hex": "0x3F08000000000000", + "abs_error": 0.0, + "category": "subnormal" + }, + { + "name": "min_pos_normal", + "input_f64": 6.103515625e-05, + "input_f64_hex": "0x3F10000000000000", + "fp8_bits_hex": "0x04", + "fp8_bits_int": 4, + "decoded_f64": 6.103515625e-05, + "decoded_f64_hex": "0x3F10000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_half", + "input_f64": 0.5, + "input_f64_hex": "0x3FE0000000000000", + "fp8_bits_hex": "0x38", + "fp8_bits_int": 56, + "decoded_f64": 0.5, + "decoded_f64_hex": "0x3FE0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_one", + "input_f64": 1.0, + "input_f64_hex": "0x3FF0000000000000", + "fp8_bits_hex": "0x3C", + "fp8_bits_int": 60, + "decoded_f64": 1.0, + "decoded_f64_hex": "0x3FF0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "neg_one", + "input_f64": -1.0, + "input_f64_hex": "0xBFF0000000000000", + "fp8_bits_hex": "0xBC", + "fp8_bits_int": 188, + "decoded_f64": -1.0, + "decoded_f64_hex": "0xBFF0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_two", + "input_f64": 2.0, + "input_f64_hex": "0x4000000000000000", + "fp8_bits_hex": "0x40", + "fp8_bits_int": 64, + "decoded_f64": 2.0, + "decoded_f64_hex": "0x4000000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_three", + "input_f64": 3.0, + "input_f64_hex": "0x4008000000000000", + "fp8_bits_hex": "0x42", + "fp8_bits_int": 66, + "decoded_f64": 3.0, + "decoded_f64_hex": "0x4008000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_six", + "input_f64": 6.0, + "input_f64_hex": "0x4018000000000000", + "fp8_bits_hex": "0x46", + "fp8_bits_int": 70, + "decoded_f64": 6.0, + "decoded_f64_hex": "0x4018000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "max_finite_pos", + "input_f64": 57344.0, + "input_f64_hex": "0x40EC000000000000", + "fp8_bits_hex": "0x7B", + "fp8_bits_int": 123, + "decoded_f64": 57344.0, + "decoded_f64_hex": "0x40EC000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "max_finite_neg", + "input_f64": -57344.0, + "input_f64_hex": "0xC0EC000000000000", + "fp8_bits_hex": "0xFB", + "fp8_bits_int": 251, + "decoded_f64": -57344.0, + "decoded_f64_hex": "0xC0EC000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_inf", + "input_f64": "Inf", + "input_f64_hex": "0x7FF0000000000000", + "fp8_bits_hex": "0x7C", + "fp8_bits_int": 124, + "decoded_f64": "Inf", + "decoded_f64_hex": "0x7FF0000000000000", + "abs_error": 0.0, + "category": "inf" + }, + { + "name": "neg_inf", + "input_f64": "-Inf", + "input_f64_hex": "0xFFF0000000000000", + "fp8_bits_hex": "0xFC", + "fp8_bits_int": 252, + "decoded_f64": "-Inf", + "decoded_f64_hex": "0xFFF0000000000000", + "abs_error": 0.0, + "category": "inf" + }, + { + "name": "nan", + "input_f64": "NaN", + "input_f64_hex": "0x7FF8000000000000", + "fp8_bits_hex": "0x7D", + "fp8_bits_int": 125, + "decoded_f64": "NaN", + "decoded_f64_hex": "0x7FF8000000000000", + "abs_error": "NaN", + "category": "nan" + } + ] +} \ No newline at end of file diff --git a/conformance/vectors/gen_bf16_golden.py b/conformance/vectors/gen_bf16_golden.py new file mode 100644 index 000000000..394af7dcf --- /dev/null +++ b/conformance/vectors/gen_bf16_golden.py @@ -0,0 +1,165 @@ +#!/usr/bin/env python3 +""" +Generate a bit-precise bfloat16 (1s8e7m) conformance + golden-accumulation +pack in the SAME row schema as the GF16 / FP8 / MXFP4 packs, PLUS a small +golden-reference accumulation section aimed at tt-mlir#6252 (rounding-error +accumulation in depth-31 fused trees, all-close failures). + +bfloat16 semantics: + * 1 sign, 8 exponent, 7 mantissa bits. bias = 127 (same exponent range as + fp32; bf16 is fp32 with the low 16 mantissa bits truncated). + * HAS inf (E=255, M=0) and NaN (E=255, M!=0). + * Decode is exact into f64 (bf16 is a strict subset of fp32 which is a strict + subset of f64). + * bf16 bits = top 16 bits of the fp32 encoding. We encode via + round-nearest-ties-even on the fp32->bf16 truncation boundary. + +#6252 angle: depth-31 fused reduction trees accumulate rounding error. The +"all-close" check needs a golden reference computed at higher precision (f64) +and a clamped, deterministic input set so the comparison is reproducible. This +pack supplies: (a) a bit-exact bf16 element table, and (b) a golden reduction: +sum of N copies of an exactly-representable bf16 value, with the f64 golden, +the naive sequential-bf16-rounded result, the pairwise-tree result, and the +absolute / relative error of each tree vs the golden. The split between +sequential and pairwise at depth ~31 is exactly the all-close gap. +""" +import json, struct + +def f64_hex(x): + return "0x" + struct.pack(">d", x).hex().upper() + +def f32_bits(x): + return struct.unpack(">I", struct.pack(">f", x))[0] + +def bf16_encode(x): + """fp32 value -> 16-bit bf16, round-nearest-ties-even on truncation.""" + b = f32_bits(x) + # round to nearest even on bit 16 + lsb = (b >> 16) & 1 + rounding_bias = 0x7FFF + lsb + b = (b + rounding_bias) >> 16 + return b & 0xFFFF + +def bf16_decode(bits16): + """16-bit bf16 -> exact f64 value.""" + b32 = (bits16 & 0xFFFF) << 16 + return struct.unpack(">f", struct.pack(">I", b32))[0] + +def category_of(bits16): + e = (bits16 >> 7) & 0xFF + m = bits16 & 0x7F + if e == 0xFF: + return "nan" if m else "inf" + if e == 0: + return "zero" if m == 0 else "subnormal" + return "normal" + +def make_vector(name, value): + bits16 = bf16_encode(value) + dec = bf16_decode(bits16) + return { + "name": name, + "input_f64": value, + "input_f64_hex": f64_hex(value), + "bf16_bits_hex": "0x%04X" % bits16, + "bf16_bits_int": bits16, + "decoded_f64": dec, + "decoded_f64_hex": f64_hex(dec), + "abs_error": abs(value - dec), + "category": category_of(bits16), + } + +# Element table: values exactly representable in bf16 (abs_error 0) plus the +# anchor 3.0 and a couple of integers used in the accumulation golden. +selected = [ + ("pos_zero", 0.0), + ("pos_one", 1.0), + ("neg_one", -1.0), + ("pos_two", 2.0), + ("pos_three", 3.0), # anchor: phi^2 + 1/phi^2 = 3 + ("pos_half", 0.5), + ("pos_quarter", 0.25), + ("pos_0p1", 0.1), # NOT exact in bf16 -> nonzero abs_error (honest) +] +vectors = [make_vector(n, v) for (n, v) in selected] + +# ---- Golden accumulation section for #6252 ---- +def sequential_sum_bf16(value, n): + """Accumulate n copies of value, rounding to bf16 after every add.""" + acc = bf16_decode(bf16_encode(value)) + elem = acc + for _ in range(n - 1): + acc = bf16_decode(bf16_encode(acc + elem)) + return acc + +def pairwise_sum_bf16(value, n): + """Pairwise (tree) reduction, rounding to bf16 at every node.""" + arr = [bf16_decode(bf16_encode(value))] * n + while len(arr) > 1: + nxt = [] + for i in range(0, len(arr) - 1, 2): + nxt.append(bf16_decode(bf16_encode(arr[i] + arr[i + 1]))) + if len(arr) % 2 == 1: + nxt.append(arr[-1]) + arr = nxt + return arr[0] + +# 0.1 is the classic case: not bf16-exact, so error compounds. depth-31 tree +# means ~2^31 leaves is impractical; we use a depth proxy n that exercises the +# sequential-vs-pairwise divergence at a representative reduction width. +golden_cases = [] +for value, n in [(0.1, 1024), (3.0, 1024), (0.1, 4096), (1.0/3.0, 2048)]: + golden = value * n # f64 golden reference + seq = sequential_sum_bf16(value, n) + pw = pairwise_sum_bf16(value, n) + golden_cases.append({ + "element_f64": value, + "element_f64_hex": f64_hex(value), + "element_bf16_hex": "0x%04X" % bf16_encode(value), + "n_terms": n, + "golden_f64": golden, + "golden_f64_hex": f64_hex(golden), + "sequential_bf16": seq, + "pairwise_bf16": pw, + "abs_err_sequential": abs(seq - golden), + "abs_err_pairwise": abs(pw - golden), + "rel_err_sequential": abs(seq - golden) / abs(golden), + "rel_err_pairwise": abs(pw - golden) / abs(golden), + "pairwise_better": abs(pw - golden) <= abs(seq - golden), + }) + +three = bf16_decode(bf16_encode(3.0)) +anchor_ok = (three == 3.0) + +pack = { + "schema": "t27-conformance/v0.1", + "format": "BFLOAT16", + "format_notes": "1s8e7m, bias 127, same exponent range as fp32 (top 16 bits of fp32). HAS inf and NaN. Decode exact into f64.", + "ssot": "https://github.com/gHashTag/t27/blob/master/conformance/FORMAT-SPEC-001.json", + "preprint": "https://arxiv.org/abs/2606.05017", + "anchor_identity": "phi^2 + 1/phi^2 = 3", + "anchor_check": { + "value": three, + "expected": 3.0, + "ieee754_exact": anchor_ok, + "bf16_bits_hex": "0x%04X" % bf16_encode(3.0), + }, + "issue_6252_note": "Rounding-error accumulation in depth-31 fused trees. The all-close gap comes from accumulation ORDER, not single-op rounding. The golden_accumulation section gives an f64 golden reference plus sequential vs pairwise bf16 reductions; pairwise reduces the accumulated error. A reproducible all-close test needs (a) a clamped deterministic input set and (b) an f64 golden, both supplied here.", + "round_trip_policy": "decode: bf16 bits -> f64 exact. encode: fp32->bf16 round-nearest-ties-even on the 16-bit truncation boundary. abs_error is nonzero only for inputs not representable in bf16 (e.g. 0.1) -- stated honestly, not hidden.", + "n_vectors": len(vectors), + "vectors": vectors, + "golden_accumulation": golden_cases, +} + +with open("vectors/bf16_golden_conformance_v0.json", "w") as f: + json.dump(pack, f, indent=2) + +print("Wrote vectors/bf16_golden_conformance_v0.json") +print("n_vectors:", len(vectors)) +print("anchor 3.0 exact:", anchor_ok, "bits", "0x%04X" % bf16_encode(3.0)) +print("0.1 abs_error (honest, nonzero):", make_vector("x", 0.1)["abs_error"]) +print("--- golden accumulation (sequential vs pairwise) ---") +for c in golden_cases: + print(f" elem={c['element_f64']:<10} n={c['n_terms']:<5} " + f"seq_err={c['abs_err_sequential']:.6g} pw_err={c['abs_err_pairwise']:.6g} " + f"pairwise_better={c['pairwise_better']}") diff --git a/conformance/vectors/gen_fp8_e4m3.py b/conformance/vectors/gen_fp8_e4m3.py new file mode 100644 index 000000000..bf2abd408 --- /dev/null +++ b/conformance/vectors/gen_fp8_e4m3.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +""" +Generate a bit-precise FP8 E4M3FN conformance vector pack in the SAME row +schema as the GF16 pack (gf16_conformance_v0.json). + +E4M3FN (OCP / torch.float8_e4m3fn) semantics: + * 1 sign, 4 exponent, 3 mantissa bits. bias = 7. + * NO infinity. Single NaN encoding: S.1111.111 (0x7F / 0xFF). + * Max finite magnitude = 448.0 (S=0, E=1111=15, M=110 -> 1.75 * 2^8). + Note: the all-ones mantissa at max exponent is reserved for NaN, so the + largest finite is mantissa=110, not 111. + * Subnormals: exponent field 0, value = (M/8) * 2^(1-7) = (M/8) * 2^-6. + * Smallest positive subnormal = 2^-9 = 0.001953125. + * Smallest positive normal = 2^-6 = 0.015625. + +Decode is exact into f64. Encode here is round-to-nearest-ties-to-even with +saturation to 448 on overflow (the common "FN" clamp; out-of-range maps to +max finite, NOT to NaN -- this is exactly the round-trip decision the issue +author flagged). +""" +import json, struct, math + +def f64_hex(x): + return "0x" + struct.pack(">d", x).hex().upper() + +def decode_e4m3fn(bits): + """bits: int 0..255 -> (value_float_or_nan_str, category)""" + s = (bits >> 7) & 1 + e = (bits >> 3) & 0xF + m = bits & 0x7 + sign = -1.0 if s else 1.0 + # NaN: E=1111 and M=111 + if e == 0xF and m == 0x7: + return (math.nan, "nan") + if e == 0: + if m == 0: + return (sign * 0.0, "zero") + val = sign * (m / 8.0) * (2.0 ** -6) # subnormal + return (val, "subnormal") + val = sign * (1.0 + m / 8.0) * (2.0 ** (e - 7)) + return (val, "normal") + +def all_codes(): + out = [] + for bits in range(256): + val, cat = decode_e4m3fn(bits) + out.append((bits, val, cat)) + return out + +def make_vector(name, bits, category_override=None): + val, cat = decode_e4m3fn(bits) + if category_override: + cat = category_override + is_nan = isinstance(val, float) and math.isnan(val) + # For a conformance pack the "input_f64" is the exact decoded value + # (this pack documents the decode side: bits -> f64, abs_error 0 by + # construction; the encode/round-trip side is documented separately). + if is_nan: + input_f64 = "NaN" + decoded_f64 = "NaN" + input_hex = "0x7FF8000000000000" + decoded_hex = "0x7FF8000000000000" + abs_err = "NaN" + else: + input_f64 = val + decoded_f64 = val + input_hex = f64_hex(val) + decoded_hex = f64_hex(val) + abs_err = 0.0 + return { + "name": name, + "input_f64": input_f64, + "input_f64_hex": input_hex, + "fp8_bits_hex": "0x%02X" % bits, + "fp8_bits_int": bits, + "decoded_f64": decoded_f64, + "decoded_f64_hex": decoded_hex, + "abs_error": abs_err, + "category": cat, + } + +# Curated, high-signal vector set covering exactly the failure surfaces named +# in tt-mlir #8549: zeros, subnormal flush boundary, smallest normal, unit, +# max finite (448) vs NaN, sign symmetry. +selected = [ + ("pos_zero", 0x00), + ("neg_zero", 0x80), + ("min_pos_subnormal", 0x01), # 2^-9 + ("max_pos_subnormal", 0x07), # (7/8)*2^-6 + ("min_pos_normal", 0x08), # 2^-6 + ("pos_half", 0x30), # 0.5 + ("pos_one", 0x38), # 1.0 + ("neg_one", 0xB8), # -1.0 + ("pos_two", 0x40), # 2.0 + ("pos_three", 0x44), # 3.0 (anchor: phi^2 + 1/phi^2 = 3) + ("pos_six", 0x4C), # 6.0 + ("max_finite_pos", 0x7E), # 448.0 (E=1111,M=110) + ("max_finite_neg", 0xFE), # -448.0 + ("nan", 0x7F), # canonical NaN +] + +vectors = [make_vector(n, b) for (n, b) in selected] + +# Anchor check: 3.0 must be exactly representable in E4M3FN. +three = decode_e4m3fn(0x44)[0] +anchor_ok = (three == 3.0) + +pack = { + "schema": "t27-conformance/v0.1", + "format": "FP8_E4M3FN", + "format_notes": "OCP / torch.float8_e4m3fn. 1s4e3m, bias 7, no inf, single NaN S.1111.111, max finite 448.0. Subnormal min 2^-9.", + "ssot": "https://github.com/gHashTag/t27/blob/master/conformance/FORMAT-SPEC-001.json", + "preprint": "https://arxiv.org/abs/2606.05017", + "anchor_identity": "phi^2 + 1/phi^2 = 3", + "anchor_check": { + "value": three, + "expected": 3.0, + "ieee754_exact": anchor_ok, + "fp8_bits_hex": "0x44", + }, + "round_trip_policy": "decode: exact bits->f64. encode (reference): round-nearest-ties-even, overflow saturates to max finite 448.0 (NOT NaN). This is the divergence point flagged in tt-mlir#8549.", + "n_vectors": len(vectors), + "vectors": vectors, +} + +with open("vectors/fp8_e4m3fn_conformance_v0.json", "w") as f: + json.dump(pack, f, indent=2) + +print("Wrote vectors/fp8_e4m3fn_conformance_v0.json") +print("n_vectors:", len(vectors)) +print("anchor 3.0 exact:", anchor_ok, "decoded:", three) +# Sanity: print the boundary values +for n, b in [("min_pos_subnormal",0x01),("max_pos_subnormal",0x07), + ("min_pos_normal",0x08),("max_finite_pos",0x7E)]: + v,c = decode_e4m3fn(b) + print(f" {n}: bits=0x{b:02X} -> {v} ({c})") diff --git a/conformance/vectors/gen_fp8_e5m2.py b/conformance/vectors/gen_fp8_e5m2.py new file mode 100644 index 000000000..977fc5fbd --- /dev/null +++ b/conformance/vectors/gen_fp8_e5m2.py @@ -0,0 +1,129 @@ +#!/usr/bin/env python3 +""" +Generate a bit-precise FP8 E5M2 conformance vector pack in the SAME row +schema as the GF16 / E4M3FN packs. + +E5M2 (OCP / torch.float8_e5m2) semantics: + * 1 sign, 5 exponent, 2 mantissa bits. bias = 15. + * IEEE-754-style: HAS infinity (E=11111, M=00 -> +/-Inf). + * NaN: E=11111, M != 00 (so 0x7D/0x7E/0x7F and signed copies are NaN). + * Max finite magnitude = 57344.0 (S=0, E=11110=30, M=11 -> 1.75 * 2^15). + * Subnormals: exponent field 0, value = (M/4) * 2^(1-15) = (M/4) * 2^-14. + * Smallest positive subnormal = 2^-16 = 1.52587890625e-05. + * Smallest positive normal = 2^-14 = 6.103515625e-05. + +Decode is exact into f64. Encode (reference) is round-nearest-ties-even with +overflow to +/-Inf (true IEEE behaviour for E5M2, in contrast to the E4M3FN +saturate-to-448 policy). That overflow-to-Inf vs saturate difference between +the two FP8 variants is the practical round-trip gotcha for host construction. +""" +import json, struct, math + +def f64_hex(x): + return "0x" + struct.pack(">d", x).hex().upper() + +def decode_e5m2(bits): + """bits: int 0..255 -> (value_or_special, category)""" + s = (bits >> 7) & 1 + e = (bits >> 2) & 0x1F + m = bits & 0x3 + sign = -1.0 if s else 1.0 + if e == 0x1F: + if m == 0: + return (sign * math.inf, "inf") + return (math.nan, "nan") + if e == 0: + if m == 0: + return (sign * 0.0, "zero") + val = sign * (m / 4.0) * (2.0 ** -14) # subnormal + return (val, "subnormal") + val = sign * (1.0 + m / 4.0) * (2.0 ** (e - 15)) + return (val, "normal") + +def make_vector(name, bits, category_override=None): + val, cat = decode_e5m2(bits) + if category_override: + cat = category_override + is_nan = isinstance(val, float) and math.isnan(val) + is_inf = isinstance(val, float) and math.isinf(val) + if is_nan: + input_f64 = "NaN"; decoded_f64 = "NaN" + input_hex = "0x7FF8000000000000"; decoded_hex = "0x7FF8000000000000" + abs_err = "NaN" + elif is_inf: + sign_txt = "-Inf" if val < 0 else "Inf" + input_f64 = sign_txt; decoded_f64 = sign_txt + input_hex = f64_hex(val); decoded_hex = f64_hex(val) + abs_err = 0.0 + else: + input_f64 = val; decoded_f64 = val + input_hex = f64_hex(val); decoded_hex = f64_hex(val) + abs_err = 0.0 + return { + "name": name, + "input_f64": input_f64, + "input_f64_hex": input_hex, + "fp8_bits_hex": "0x%02X" % bits, + "fp8_bits_int": bits, + "decoded_f64": decoded_f64, + "decoded_f64_hex": decoded_hex, + "abs_error": abs_err, + "category": cat, + } + +# Curated set covering the E5M2 failure surfaces: signed zeros, subnormal +# flush boundary, smallest normal, unit + small ints, max finite vs Inf vs +# NaN, sign symmetry. 3.0 anchor lands at bits 0x42. +selected = [ + ("pos_zero", 0x00), + ("neg_zero", 0x80), + ("min_pos_subnormal", 0x01), # 2^-16 + ("max_pos_subnormal", 0x03), # (3/4)*2^-14 + ("min_pos_normal", 0x04), # 2^-14 + ("pos_half", 0x38), # 0.5 + ("pos_one", 0x3C), # 1.0 + ("neg_one", 0xBC), # -1.0 + ("pos_two", 0x40), # 2.0 + ("pos_three", 0x42), # 3.0 (anchor: phi^2 + 1/phi^2 = 3) + ("pos_six", 0x46), # 6.0 + ("max_finite_pos", 0x7B), # 57344.0 (E=11110, M=11) + ("max_finite_neg", 0xFB), # -57344.0 + ("pos_inf", 0x7C), # +Inf (E=11111, M=00) + ("neg_inf", 0xFC), # -Inf + ("nan", 0x7D), # NaN (E=11111, M!=00) +] + +vectors = [make_vector(n, b) for (n, b) in selected] + +three = decode_e5m2(0x42)[0] +anchor_ok = (three == 3.0) + +pack = { + "schema": "t27-conformance/v0.1", + "format": "FP8_E5M2", + "format_notes": "OCP / torch.float8_e5m2. 1s5e2m, bias 15, HAS inf (E=11111,M=00), NaN (E=11111,M!=00), max finite 57344.0 at 0x7B. Subnormal min 2^-16, normal min 2^-14.", + "ssot": "https://github.com/gHashTag/t27/blob/master/conformance/FORMAT-SPEC-001.json", + "preprint": "https://arxiv.org/abs/2606.05017", + "anchor_identity": "phi^2 + 1/phi^2 = 3", + "anchor_check": { + "value": three, + "expected": 3.0, + "ieee754_exact": anchor_ok, + "fp8_bits_hex": "0x42", + }, + "round_trip_policy": "decode: exact bits->f64. encode (reference): round-nearest-ties-even, overflow goes to +/-Inf (IEEE), NOT saturate. This is the variant-level divergence from E4M3FN (which saturates to 448.0). The overflow-to-Inf vs saturate split is the practical round-trip gotcha across the two FP8 variants.", + "n_vectors": len(vectors), + "vectors": vectors, +} + +with open("vectors/fp8_e5m2_conformance_v0.json", "w") as f: + json.dump(pack, f, indent=2) + +print("Wrote vectors/fp8_e5m2_conformance_v0.json") +print("n_vectors:", len(vectors)) +print("anchor 3.0 exact:", anchor_ok, "decoded:", three) +for n, b in [("min_pos_subnormal",0x01),("max_pos_subnormal",0x03), + ("min_pos_normal",0x04),("max_finite_pos",0x7B), + ("pos_inf",0x7C),("nan",0x7D)]: + v,c = decode_e5m2(b) + print(f" {n}: bits=0x{b:02X} -> {v} ({c})") diff --git a/conformance/vectors/gen_mxfp4_e2m1.py b/conformance/vectors/gen_mxfp4_e2m1.py new file mode 100644 index 000000000..8162b7bfb --- /dev/null +++ b/conformance/vectors/gen_mxfp4_e2m1.py @@ -0,0 +1,97 @@ +#!/usr/bin/env python3 +""" +Generate a bit-precise MXFP4 E2M1 conformance vector pack in the SAME row +schema as the GF16 / FP8 packs. + +E2M1 (OCP Microscaling MXFP4 element format) semantics: + * 1 sign, 2 exponent, 1 mantissa bit. 4 bits total -> only 16 codes. + * bias = 1. + * NO inf, NO NaN (the MX *element* format carries neither; the block scale + handles dynamic range, the element is a pure finite 4-bit value). + * Subnormals: exponent field 0. value = (M/2) * 2^(1-1) = (M/2) * 1 = M*0.5 + -> the only subnormal magnitude is 0.5 (M=1, E=0). + * Normals: value = (1 + M/2) * 2^(E-1). + * Full positive value set: 0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 6.0 (and signed). + * Max finite magnitude = 6.0 (S, E=11=3, M=1 -> 1.5 * 2^2). + +The issue #7085 clamps to +/-7. Note 7.0 is NOT representable in E2M1; the +largest representable magnitude is 6.0. So a clamp bound of 7 is one E2M1 step +WIDER than the format max -- any pre-quantization clamp at +/-7 still leaves a +gap to the nearest grid point 6.0. This pack enumerates the full 16-code grid +so the clamp-vs-grid relationship is explicit. 3.0 (the anchor) is an exact +grid point at bits 0x6. +""" +import json, struct + +def f64_hex(x): + return "0x" + struct.pack(">d", x).hex().upper() + +def decode_e2m1(bits): + """bits: int 0..15 -> (value, category)""" + s = (bits >> 3) & 1 + e = (bits >> 1) & 0x3 + m = bits & 0x1 + sign = -1.0 if s else 1.0 + if e == 0: + if m == 0: + return (sign * 0.0, "zero") + return (sign * 0.5, "subnormal") # (1/2)*2^0 + val = sign * (1.0 + m / 2.0) * (2.0 ** (e - 1)) + return (val, "normal") + +def make_vector(name, bits): + val, cat = decode_e2m1(bits) + return { + "name": name, + "input_f64": val, + "input_f64_hex": f64_hex(val), + "mxfp4_bits_hex": "0x%X" % bits, + "mxfp4_bits_int": bits, + "decoded_f64": val, + "decoded_f64_hex": f64_hex(val), + "abs_error": 0.0, + "category": cat, + } + +# Enumerate the full 16-code grid (4-bit format -> exhaustive is tiny + ideal). +labels = { + 0x0: "pos_zero", 0x1: "pos_half", 0x2: "pos_one", 0x3: "pos_onehalf", + 0x4: "pos_two", 0x5: "pos_three", 0x6: "pos_four", 0x7: "pos_six", + 0x8: "neg_zero", 0x9: "neg_half", 0xA: "neg_one", 0xB: "neg_onehalf", + 0xC: "neg_two", 0xD: "neg_three", 0xE: "neg_four", 0xF: "neg_six", +} +vectors = [make_vector(labels[b], b) for b in range(16)] + +# Anchor 3.0: bits 0x5 (E=10=2, M=1 -> 1.5*2 = 3.0). +three = decode_e2m1(0x5)[0] +anchor_ok = (three == 3.0) +maxfinite = decode_e2m1(0x7)[0] # 6.0 + +pack = { + "schema": "t27-conformance/v0.1", + "format": "MXFP4_E2M1", + "format_notes": "OCP Microscaling element format. 1s2e1m, bias 1, 16 codes, no inf, no NaN. Value grid +/-{0,0.5,1.0,1.5,2.0,3.0,4.0,6.0}. Max finite 6.0 at bits 0x7.", + "ssot": "https://github.com/gHashTag/t27/blob/master/conformance/FORMAT-SPEC-001.json", + "preprint": "https://arxiv.org/abs/2606.05017", + "anchor_identity": "phi^2 + 1/phi^2 = 3", + "anchor_check": { + "value": three, + "expected": 3.0, + "ieee754_exact": anchor_ok, + "mxfp4_bits_hex": "0x5", + }, + "clamp_note": "tt-mlir#7085 clamps to +/-7. E2M1 max finite is 6.0; 7.0 is NOT a grid point. A clamp bound of 7 sits one step above the format max, so values in (6,7] still round to the 6.0 grid point under round-nearest. The clamp bound and the representable max are not the same number -- this enumeration makes that explicit.", + "round_trip_policy": "decode: exact bits->f64 (all 16 codes, abs_error 0). encode: round-nearest-ties-even onto the 8-magnitude grid; there is no inf/NaN, so out-of-range maps to nearest finite grid point (+/-6.0).", + "n_vectors": len(vectors), + "max_finite": maxfinite, + "vectors": vectors, +} + +with open("vectors/mxfp4_e2m1_conformance_v0.json", "w") as f: + json.dump(pack, f, indent=2) + +print("Wrote vectors/mxfp4_e2m1_conformance_v0.json") +print("n_vectors:", len(vectors), "(full 4-bit grid)") +print("anchor 3.0 exact:", anchor_ok, "at bits 0x5") +print("max finite:", maxfinite, "at bits 0x7 (clamp bound 7 > grid max 6.0)") +print("full positive grid:", sorted(set(decode_e2m1(b)[0] for b in range(8)))) diff --git a/conformance/vectors/gf16_conformance_v0.json b/conformance/vectors/gf16_conformance_v0.json new file mode 100644 index 000000000..6c47959e1 --- /dev/null +++ b/conformance/vectors/gf16_conformance_v0.json @@ -0,0 +1,248 @@ +{ + "schema": "t27-conformance/v0.1", + "format": "GF16", + "ssot": "https://github.com/gHashTag/t27/blob/master/conformance/FORMAT-SPEC-001.json", + "preprint": "https://arxiv.org/abs/2606.05017", + "anchor_identity": "phi^2 + 1/phi^2 = 3", + "anchor_check": { + "value": 3.0, + "expected": 3.0, + "ieee754_exact": true + }, + "phi_decimal": "1.61803398874989490253", + "inv_phi_decimal": "0.61803398874989479150", + "n_vectors": 21, + "vectors": [ + { + "name": "pos_zero", + "input_f64": 0.0, + "input_f64_hex": "0x0000000000000000", + "gf16_bits_hex": "0x0000", + "gf16_bits_int": 0, + "decoded_f64": 0.0, + "decoded_f64_hex": "0x0000000000000000", + "abs_error": 0.0, + "category": "zero" + }, + { + "name": "neg_zero", + "input_f64": -0.0, + "input_f64_hex": "0x8000000000000000", + "gf16_bits_hex": "0x8000", + "gf16_bits_int": 32768, + "decoded_f64": -0.0, + "decoded_f64_hex": "0x8000000000000000", + "abs_error": 0.0, + "category": "zero" + }, + { + "name": "pos_one", + "input_f64": 1.0, + "input_f64_hex": "0x3FF0000000000000", + "gf16_bits_hex": "0x3E00", + "gf16_bits_int": 15872, + "decoded_f64": 1.0, + "decoded_f64_hex": "0x3FF0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "neg_one", + "input_f64": -1.0, + "input_f64_hex": "0xBFF0000000000000", + "gf16_bits_hex": "0xBE00", + "gf16_bits_int": 48640, + "decoded_f64": -1.0, + "decoded_f64_hex": "0xBFF0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_two", + "input_f64": 2.0, + "input_f64_hex": "0x4000000000000000", + "gf16_bits_hex": "0x4000", + "gf16_bits_int": 16384, + "decoded_f64": 2.0, + "decoded_f64_hex": "0x4000000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_half", + "input_f64": 0.5, + "input_f64_hex": "0x3FE0000000000000", + "gf16_bits_hex": "0x3C00", + "gf16_bits_int": 15360, + "decoded_f64": 0.5, + "decoded_f64_hex": "0x3FE0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "phi", + "input_f64": 1.618033988749895, + "input_f64_hex": "0x3FF9E3779B97F4A8", + "gf16_bits_hex": "0x3F3C", + "gf16_bits_int": 16188, + "decoded_f64": 1.6171875, + "decoded_f64_hex": "0x3FF9E00000000000", + "abs_error": 0.0, + "category": "phi_anchor" + }, + { + "name": "inv_phi", + "input_f64": 0.6180339887498948, + "input_f64_hex": "0x3FE3C6EF372FE94F", + "gf16_bits_hex": "0x3C79", + "gf16_bits_int": 15481, + "decoded_f64": 0.6181640625, + "decoded_f64_hex": "0x3FE3C80000000000", + "abs_error": 0.0, + "category": "phi_anchor" + }, + { + "name": "phi_squared", + "input_f64": 2.618033988749895, + "input_f64_hex": "0x4004F1BBCDCBFA54", + "gf16_bits_hex": "0x409E", + "gf16_bits_int": 16542, + "decoded_f64": 2.6171875, + "decoded_f64_hex": "0x4004F00000000000", + "abs_error": 0.0, + "category": "phi_anchor" + }, + { + "name": "inv_phi_sq", + "input_f64": 0.3819660112501051, + "input_f64_hex": "0x3FD8722191A02D60", + "gf16_bits_hex": "0x3B0E", + "gf16_bits_int": 15118, + "decoded_f64": 0.3818359375, + "decoded_f64_hex": "0x3FD8700000000000", + "abs_error": 0.0, + "category": "phi_anchor" + }, + { + "name": "identity_sum", + "input_f64": 3.0, + "input_f64_hex": "0x4008000000000000", + "gf16_bits_hex": "0x4100", + "gf16_bits_int": 16640, + "decoded_f64": 3.0, + "decoded_f64_hex": "0x4008000000000000", + "abs_error": 0.0, + "category": "phi_anchor" + }, + { + "name": "three", + "input_f64": 3.0, + "input_f64_hex": "0x4008000000000000", + "gf16_bits_hex": "0x4100", + "gf16_bits_int": 16640, + "decoded_f64": 3.0, + "decoded_f64_hex": "0x4008000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "smallest_subnormal", + "input_f64": 1.8189894035458565e-12, + "input_f64_hex": "0x3D80000000000000", + "gf16_bits_hex": "0x0001", + "gf16_bits_int": 1, + "decoded_f64": 1.8189894035458565e-12, + "decoded_f64_hex": "0x3D80000000000000", + "abs_error": 0.0, + "category": "subnormal" + }, + { + "name": "largest_subnormal", + "input_f64": 9.295035852119327e-10, + "input_f64_hex": "0x3E0FF00000000000", + "gf16_bits_hex": "0x01FF", + "gf16_bits_int": 511, + "decoded_f64": 9.295035852119327e-10, + "decoded_f64_hex": "0x3E0FF00000000000", + "abs_error": 0.0, + "category": "subnormal" + }, + { + "name": "smallest_normal", + "input_f64": 9.313225746154785e-10, + "input_f64_hex": "0x3E10000000000000", + "gf16_bits_hex": "0x0200", + "gf16_bits_int": 512, + "decoded_f64": 9.313225746154785e-10, + "decoded_f64_hex": "0x3E10000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "largest_normal", + "input_f64": 4290772992.0, + "input_f64_hex": "0x41EFF80000000000", + "gf16_bits_hex": "0x7DFF", + "gf16_bits_int": 32255, + "decoded_f64": 4290772992.0, + "decoded_f64_hex": "0x41EFF80000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "overflow_to_inf", + "input_f64": 1e+40, + "input_f64_hex": "0x483D6329F1C35CA5", + "gf16_bits_hex": "0x7E00", + "gf16_bits_int": 32256, + "decoded_f64": Infinity, + "decoded_f64_hex": "0x7FF0000000000000", + "abs_error": Infinity, + "category": "normal" + }, + { + "name": "pos_inf", + "input_f64": Infinity, + "input_f64_hex": "0x7FF0000000000000", + "gf16_bits_hex": "0x7E00", + "gf16_bits_int": 32256, + "decoded_f64": Infinity, + "decoded_f64_hex": "0x7FF0000000000000", + "abs_error": 0.0, + "category": "inf" + }, + { + "name": "neg_inf", + "input_f64": -Infinity, + "input_f64_hex": "0xFFF0000000000000", + "gf16_bits_hex": "0xFE00", + "gf16_bits_int": 65024, + "decoded_f64": -Infinity, + "decoded_f64_hex": "0xFFF0000000000000", + "abs_error": 0.0, + "category": "inf" + }, + { + "name": "e", + "input_f64": 2.718281828459045, + "input_f64_hex": "0x4005BF0A8B145769", + "gf16_bits_hex": "0x40B8", + "gf16_bits_int": 16568, + "decoded_f64": 2.71875, + "decoded_f64_hex": "0x4005C00000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pi", + "input_f64": 3.141592653589793, + "input_f64_hex": "0x400921FB54442D18", + "gf16_bits_hex": "0x4124", + "gf16_bits_int": 16676, + "decoded_f64": 3.140625, + "decoded_f64_hex": "0x4009200000000000", + "abs_error": 0.0, + "category": "normal" + } + ] +} \ No newline at end of file diff --git a/conformance/vectors/mxfp4_e2m1_conformance_v0.json b/conformance/vectors/mxfp4_e2m1_conformance_v0.json new file mode 100644 index 000000000..648fe3d18 --- /dev/null +++ b/conformance/vectors/mxfp4_e2m1_conformance_v0.json @@ -0,0 +1,196 @@ +{ + "schema": "t27-conformance/v0.1", + "format": "MXFP4_E2M1", + "format_notes": "OCP Microscaling element format. 1s2e1m, bias 1, 16 codes, no inf, no NaN. Value grid +/-{0,0.5,1.0,1.5,2.0,3.0,4.0,6.0}. Max finite 6.0 at bits 0x7.", + "ssot": "https://github.com/gHashTag/t27/blob/master/conformance/FORMAT-SPEC-001.json", + "preprint": "https://arxiv.org/abs/2606.05017", + "anchor_identity": "phi^2 + 1/phi^2 = 3", + "anchor_check": { + "value": 3.0, + "expected": 3.0, + "ieee754_exact": true, + "mxfp4_bits_hex": "0x5" + }, + "clamp_note": "tt-mlir#7085 clamps to +/-7. E2M1 max finite is 6.0; 7.0 is NOT a grid point. A clamp bound of 7 sits one step above the format max, so values in (6,7] still round to the 6.0 grid point under round-nearest. The clamp bound and the representable max are not the same number -- this enumeration makes that explicit.", + "round_trip_policy": "decode: exact bits->f64 (all 16 codes, abs_error 0). encode: round-nearest-ties-even onto the 8-magnitude grid; there is no inf/NaN, so out-of-range maps to nearest finite grid point (+/-6.0).", + "n_vectors": 16, + "max_finite": 6.0, + "vectors": [ + { + "name": "pos_zero", + "input_f64": 0.0, + "input_f64_hex": "0x0000000000000000", + "mxfp4_bits_hex": "0x0", + "mxfp4_bits_int": 0, + "decoded_f64": 0.0, + "decoded_f64_hex": "0x0000000000000000", + "abs_error": 0.0, + "category": "zero" + }, + { + "name": "pos_half", + "input_f64": 0.5, + "input_f64_hex": "0x3FE0000000000000", + "mxfp4_bits_hex": "0x1", + "mxfp4_bits_int": 1, + "decoded_f64": 0.5, + "decoded_f64_hex": "0x3FE0000000000000", + "abs_error": 0.0, + "category": "subnormal" + }, + { + "name": "pos_one", + "input_f64": 1.0, + "input_f64_hex": "0x3FF0000000000000", + "mxfp4_bits_hex": "0x2", + "mxfp4_bits_int": 2, + "decoded_f64": 1.0, + "decoded_f64_hex": "0x3FF0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_onehalf", + "input_f64": 1.5, + "input_f64_hex": "0x3FF8000000000000", + "mxfp4_bits_hex": "0x3", + "mxfp4_bits_int": 3, + "decoded_f64": 1.5, + "decoded_f64_hex": "0x3FF8000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_two", + "input_f64": 2.0, + "input_f64_hex": "0x4000000000000000", + "mxfp4_bits_hex": "0x4", + "mxfp4_bits_int": 4, + "decoded_f64": 2.0, + "decoded_f64_hex": "0x4000000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_three", + "input_f64": 3.0, + "input_f64_hex": "0x4008000000000000", + "mxfp4_bits_hex": "0x5", + "mxfp4_bits_int": 5, + "decoded_f64": 3.0, + "decoded_f64_hex": "0x4008000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_four", + "input_f64": 4.0, + "input_f64_hex": "0x4010000000000000", + "mxfp4_bits_hex": "0x6", + "mxfp4_bits_int": 6, + "decoded_f64": 4.0, + "decoded_f64_hex": "0x4010000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "pos_six", + "input_f64": 6.0, + "input_f64_hex": "0x4018000000000000", + "mxfp4_bits_hex": "0x7", + "mxfp4_bits_int": 7, + "decoded_f64": 6.0, + "decoded_f64_hex": "0x4018000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "neg_zero", + "input_f64": -0.0, + "input_f64_hex": "0x8000000000000000", + "mxfp4_bits_hex": "0x8", + "mxfp4_bits_int": 8, + "decoded_f64": -0.0, + "decoded_f64_hex": "0x8000000000000000", + "abs_error": 0.0, + "category": "zero" + }, + { + "name": "neg_half", + "input_f64": -0.5, + "input_f64_hex": "0xBFE0000000000000", + "mxfp4_bits_hex": "0x9", + "mxfp4_bits_int": 9, + "decoded_f64": -0.5, + "decoded_f64_hex": "0xBFE0000000000000", + "abs_error": 0.0, + "category": "subnormal" + }, + { + "name": "neg_one", + "input_f64": -1.0, + "input_f64_hex": "0xBFF0000000000000", + "mxfp4_bits_hex": "0xA", + "mxfp4_bits_int": 10, + "decoded_f64": -1.0, + "decoded_f64_hex": "0xBFF0000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "neg_onehalf", + "input_f64": -1.5, + "input_f64_hex": "0xBFF8000000000000", + "mxfp4_bits_hex": "0xB", + "mxfp4_bits_int": 11, + "decoded_f64": -1.5, + "decoded_f64_hex": "0xBFF8000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "neg_two", + "input_f64": -2.0, + "input_f64_hex": "0xC000000000000000", + "mxfp4_bits_hex": "0xC", + "mxfp4_bits_int": 12, + "decoded_f64": -2.0, + "decoded_f64_hex": "0xC000000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "neg_three", + "input_f64": -3.0, + "input_f64_hex": "0xC008000000000000", + "mxfp4_bits_hex": "0xD", + "mxfp4_bits_int": 13, + "decoded_f64": -3.0, + "decoded_f64_hex": "0xC008000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "neg_four", + "input_f64": -4.0, + "input_f64_hex": "0xC010000000000000", + "mxfp4_bits_hex": "0xE", + "mxfp4_bits_int": 14, + "decoded_f64": -4.0, + "decoded_f64_hex": "0xC010000000000000", + "abs_error": 0.0, + "category": "normal" + }, + { + "name": "neg_six", + "input_f64": -6.0, + "input_f64_hex": "0xC018000000000000", + "mxfp4_bits_hex": "0xF", + "mxfp4_bits_int": 15, + "decoded_f64": -6.0, + "decoded_f64_hex": "0xC018000000000000", + "abs_error": 0.0, + "category": "normal" + } + ] +} \ No newline at end of file diff --git a/docs/NOW.md b/docs/NOW.md index eff701761..9b8abf510 100644 --- a/docs/NOW.md +++ b/docs/NOW.md @@ -1332,3 +1332,9 @@ Wave-41 SparseGate.v 8 Qed sacred 0xE8 - Predecessor: W40 Lane FF DFS.v (0xE7), merge SHA 384f5a97 - Anchor: phi^2 + phi^-2 = 3 · DOI 10.5281/zenodo.19227877 · NEVER STOP - W46 RR — Purkinje thermal gating Coq proof landed + +## conformance-vector-packs-v0 -- v0 numeric-format conformance vector packs + shared-schema index (Closes #1055) + +- **WHERE** (conformance vectors): new directory `conformance/vectors/` with five bit-precise packs in one shared row schema -- `gf16_conformance_v0.json` (21 vectors), `fp8_e4m3fn_conformance_v0.json` (14), `fp8_e5m2_conformance_v0.json` (16), `mxfp4_e2m1_conformance_v0.json` (16, full 16-code grid), `bf16_golden_conformance_v0.json` (8 + golden_accumulation section). `README.md` indexes format, anchor bits, max finite, inf/NaN presence, overflow policy, and SHA-256 of every pack. Four generators (`gen_fp8_e4m3.py`, `gen_fp8_e5m2.py`, `gen_mxfp4_e2m1.py`, `gen_bf16_golden.py`) reproduce the JSON byte-for-byte. No code paths, no `.t27` specs, no `gen/` edits, no existing conformance files touched; `FORMAT-SPEC-001.json` unchanged. +- **Why**: gives the t27 numeric SSOT a citable conformance reference for the FP8/MXFP4/bf16 round-trip and clamp/overflow decisions. E4M3FN overflow saturates to 448.0; E5M2 overflows to +/-Inf; MXFP4 E2M1 clamp +/-7 sits one step above grid max 6.0; bf16 all-close gap is shown to come from accumulation order (sequential vs pairwise vs f64 golden). All packs decode exactly (abs_error 0 for representable values); abs_error is reported honestly and is nonzero only for non-representable inputs (e.g. 0.1 in bf16). Anchor identity phi^2 + 1/phi^2 = 3 is exact in every pack. ASCII-only. These are conformance data + reproduction scripts, not a quality claim; no [Verified] silicon claim is asserted beyond the existing GF16 rung. L6 gf16 SSOT untouched; L4 not applicable (data files, not specs); L2 untouched. Context preprint: arXiv:2606.05017. Closes #1055. +- **Anchor**: phi^2 + phi^-2 = 3