ci(conformance): gate INDEX_all_formats.json against the packs it summarises by gHashTag · Pull Request #1578 · gHashTag/t27

gHashTag · 2026-07-31T10:16:13Z

INDEX_all_formats.json is the corpus's machine-readable summary — what consumers read, and what Paper B recomputes its coverage from. Nothing checked that it still agreed with the packs it summarises.

That gap has already cost a real regression. gen_all_formats.py hardcoded the tier for the six wide GoldenFloat rungs:

index.append({..., "kind": "bitexact_selfconsistent",
              "source": "... no independent second witness"})

That was correct when written. The six rungs then acquired independent second witnesses and were promoted in the pack files on 2026-07-05. The generator was never updated, so re-running it rewrote the index from 75/0/8 back to 69/6/8 — silently reverting an honesty-rule-#10 promotion the packs themselves record. It is fixed in #1576, but nothing would have caught the next one.

What the gate checks

A every index entry names a pack file that exists
B the recorded sha256 matches the file on disk
C kind agrees with the pack: bitexact + populated witnesses[] ⇒ "bitexact"; a pack that is not bit-exact is not labelled otherwise
D the index witness count equals len(pack["witnesses"])
E header totals equal the entries actually present
F no pack file in the directory is missing from the index

Deliberately not checked: a bit-exact pack with an empty witnesses[]. That is the normal case, not a violation — the ~60 uncontested packs were bit-precise from the start, their independent reference codec being part of how the vectors were generated. witnesses[] records the packs whose promotion was contested. A check demanding a witness everywhere would fail the whole corpus and would misstate rule #10. The asymmetry is the point: a recorded witness must not be ignored, but its absence is not evidence of anything.

Also not treated as failure: five hand-curated packs (bfloat16, fp8_e4m3, fp8_e5m2, mxfp4, gf16) carry no bitexact key at all while the index labels them bit-exact. An absent flag is unknown, not false; reading silence as denial would make the gate assert something the pack never said. They are reported as notes.

Verification

Follows the wp18_selftest_gate.py convention — the gate proves itself falsifiable before it is trusted:

[PASS] A missing pack file is caught
[PASS] B digest drift is caught
[PASS] C witnessed pack demoted in index is caught
[PASS] D witness count mismatch is caught
[PASS] E header total drift is caught
[PASS] F pack absent from the index is caught
selftest OK: the gate is falsifiable.

And replayed against the live corpus — demoting the six rungs exactly as the old generator would:

failures: 6
  C gf48: pack is bitexact with 3 witness(es) but index says kind='bitexact_selfconsistent'
  ... (all six)
verdict: FAIL     gate exit code: 1

Clean on current master (failures: 0, 5 notes), so this stands alone and does not depend on #1576.

Closes #1577

…marises The index is what consumers read and what Paper B recomputes its coverage from, and nothing checked that it still agreed with the packs underneath it. That gap already cost a regression. gen_all_formats.py hardcoded the tier for the six wide GoldenFloat rungs -- correct when written, but the rungs later acquired independent second witnesses and were promoted in the pack files. Re-running the generator rewrote the index from 75/0/8 back to 69/6/8, silently reverting an honesty-rule-#10 promotion the packs themselves record. #1576 fixes that instance; this closes the class. Checks: entry names a file that exists; recorded sha256 matches disk; kind agrees with the pack's own bitexact + witnesses state; index witness count matches the pack; header totals match the entries; no pack file missing from the index. Two things are deliberately NOT failures. A bitexact pack with an empty witnesses[] is the normal case -- the ~60 uncontested packs were bit-precise from the start, and witnesses[] records the promotions that were CONTESTED; demanding a witness everywhere would fail the whole corpus and misstate rule #10. And five hand-curated packs carry no bitexact key at all while the index labels them bitexact: an absent flag is unknown, not false, so reading silence as denial would make the gate assert something the pack never said. Both are reported as notes. Follows the wp18_selftest_gate.py convention -- --selftest plants a mutant per check and fails if any survives, so the gate proves itself falsifiable before CI trusts it. All six mutants are killed. Replayed against the live corpus by demoting the six rungs exactly as the old generator would, the gate fails with exit 1 and names each rung with its witness count. Clean on current master, so it stands alone and does not depend on #1576. Closes #1577

github-actions · 2026-07-31T10:16:28Z

📓 NotebookLM Notebook linked to this PR

Notebook ID: b83263109fb055dc
URL: https://notebooklm.google.com/notebook/b83263109fb055dc

This notebook contains session context, decisions, and artifacts for this work.

github-actions · 2026-07-31T10:16:33Z

PR Dashboard

Generated at: 2026-07-31 10:16:27 UTC

ci(conformance): gate INDEX_all_formats.json against the packs it summarises #1578 ci(conformance): gate INDEX_all_formats.json against the packs it summarises | Created: 2026-07-31 | Updated: 2026-07-31 | 9 failing / 18 total
fix(conformance): make the pack generator runnable from a clean checkout #1576 fix(conformance): make the pack generator runnable from a clean checkout | Created: 2026-07-31 | Updated: 2026-07-31 | 1 failing / 23 total
feat(igla): Wave Loop 822 — module-scope [463][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1572 feat(igla): Wave Loop 822 — module-scope [463][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-30 | Updated: 2026-07-30 | 1 failing / 3 total
feat(igla): Wave Loop 821 — module-scope [461][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1571 feat(igla): Wave Loop 821 — module-scope [461][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-30 | Updated: 2026-07-30 | 1 failing / 3 total
feat(igla): Wave Loop 820 — module-scope [459][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1569 feat(igla): Wave Loop 820 — module-scope [459][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-30 | Updated: 2026-07-30 | 1 failing / 3 total
feat(igla): Wave Loop 819 — module-scope [457][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1567 feat(igla): Wave Loop 819 — module-scope [457][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-30 | Updated: 2026-07-30 | 1 failing / 3 total
feat(igla): Wave Loop 818 — module-scope [455][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1566 feat(igla): Wave Loop 818 — module-scope [455][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-30 | Updated: 2026-07-30 | 1 failing / 3 total
feat(igla): Wave Loop 817 — module-scope [453][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1563 feat(igla): Wave Loop 817 — module-scope [453][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-29 | Updated: 2026-07-29 | 1 failing / 3 total
feat(igla): Wave Loop 816 — module-scope [451][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1560 feat(igla): Wave Loop 816 — module-scope [451][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-29 | Updated: 2026-07-29 | 1 failing / 3 total
feat(igla): Wave Loop 815 — module-scope [449][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1557 feat(igla): Wave Loop 815 — module-scope [449][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-29 | Updated: 2026-07-29 | 5 failing / 25 total
feat(igla): Wave Loop 814 — module-scope [447][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1556 feat(igla): Wave Loop 814 — module-scope [447][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-29 | Updated: 2026-07-29 | 5 failing / 25 total
feat(igla): Wave Loop 813 — module-scope [445][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1555 feat(igla): Wave Loop 813 — module-scope [445][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-28 | Updated: 2026-07-28 | 7 failing / 25 total
feat(igla): Wave Loop 812 — module-scope [443][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1554 feat(igla): Wave Loop 812 — module-scope [443][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-28 | Updated: 2026-07-28 | 7 failing / 26 total
feat(igla): Wave Loop 811 — module-scope [441][2]^6 Pt packed AoS variable from call with indexed signed writes #1552 feat(igla): Wave Loop 811 — module-scope [441][2]^6 Pt packed AoS variable from call with indexed signed writes | Created: 2026-07-28 | Updated: 2026-07-28 | 6 failing / 26 total
feat(igla): Wave Loop 810 — module-scope [439][2]^6 Pt packed AoS variable from call with indexed signed writes #1550 feat(igla): Wave Loop 810 — module-scope [439][2]^6 Pt packed AoS variable from call with indexed signed writes | Created: 2026-07-27 | Updated: 2026-07-27 | 6 failing / 26 total
feat(igla): Wave Loop 809 — module-scope [437][2]^6 Pt packed AoS variable from call with indexed signed writes #1548 feat(igla): Wave Loop 809 — module-scope [437][2]^6 Pt packed AoS variable from call with indexed signed writes | Created: 2026-07-27 | Updated: 2026-07-27 | 6 failing / 26 total
feat(igla): Wave Loop 808 — [435][2]^6 Pt module-scope AoS variable from call with indexed signed writes #1546 feat(igla): Wave Loop 808 — [435][2]^6 Pt module-scope AoS variable from call with indexed signed writes | Created: 2026-07-26 | Updated: 2026-07-26 | 6 failing / 26 total
feat(igla): Wave Loop 807 — [433][2]^6 Pt module-scope AoS variable from call with indexed signed writes #1544 feat(igla): Wave Loop 807 — [433][2]^6 Pt module-scope AoS variable from call with indexed signed writes | Created: 2026-07-26 | Updated: 2026-07-26 | 6 failing / 26 total
feat(igla): Wave Loop 806 — [431][2]^6 Pt module-scope AoS variable from call with indexed signed writes #1542 feat(igla): Wave Loop 806 — [431][2]^6 Pt module-scope AoS variable from call with indexed signed writes | Created: 2026-07-26 | Updated: 2026-07-26 | 6 failing / 26 total
feat(igla): Wave Loop 805 — [429][2]^6 Pt AoS var call signed writes #1540 feat(igla): Wave Loop 805 — [429][2]^6 Pt AoS var call signed writes | Created: 2026-07-26 | Updated: 2026-07-26 | 6 failing / 26 total
feat(igla): Wave Loop 804 — [427][2]^6 Pt AoS var call signed writes #1538 feat(igla): Wave Loop 804 — [427][2]^6 Pt AoS var call signed writes | Created: 2026-07-26 | Updated: 2026-07-26 | 6 failing / 26 total
feat(igla): Wave Loop 803 — [425][2]^6 Pt AoS var call signed writes #1536 feat(igla): Wave Loop 803 — [425][2]^6 Pt AoS var call signed writes | Created: 2026-07-25 | Updated: 2026-07-25 | 5 failing / 26 total
feat(igla): Wave Loop 802 — [423][2]^6 Pt AoS var call signed writes #1534 feat(igla): Wave Loop 802 — [423][2]^6 Pt AoS var call signed writes | Created: 2026-07-25 | Updated: 2026-07-25 | 5 failing / 26 total
feat(igla): Wave Loop 801 — module-scope [421][2]^6 Pt non-power-of-two outer-dimension AoS variable from call with indexed signed writes #1532 feat(igla): Wave Loop 801 — module-scope [421][2]^6 Pt non-power-of-two outer-dimension AoS variable from call with indexed signed writes | Created: 2026-07-25 | Updated: 2026-07-25 | 5 failing / 25 total
feat(igla): Wave Loop 800 — module-scope [419][2]^6 Pt non-power-of-two outer-dimension AoS variable from call with indexed signed writes #1530 feat(igla): Wave Loop 800 — module-scope [419][2]^6 Pt non-power-of-two outer-dimension AoS variable from call with indexed signed writes | Created: 2026-07-25 | Updated: 2026-07-25 | 5 failing / 25 total
feat(igla): Wave Loop 799 — module-scope [417][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1528 feat(igla): Wave Loop 799 — module-scope [417][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-25 | Updated: 2026-07-25 | 0 failing / 8 total
feat(igla): Wave Loop 798 — module-scope [415][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1526 feat(igla): Wave Loop 798 — module-scope [415][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-25 | Updated: 2026-07-25 | 5 failing / 25 total
feat(igla): Wave Loop 797 — module-scope [413][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1524 feat(igla): Wave Loop 797 — module-scope [413][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-25 | Updated: 2026-07-25 | 4 failing / 25 total
feat(igla): Wave Loop 796 — module-scope [411][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1522 feat(igla): Wave Loop 796 — module-scope [411][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-25 | Updated: 2026-07-25 | 4 failing / 25 total
Wave Loop 795 — module-scope [409][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1520 Wave Loop 795 — module-scope [409][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-25 | Updated: 2026-07-25 | 4 failing / 25 total
Wave Loop 794 — module-scope [407][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1518 Wave Loop 794 — module-scope [407][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
Wave Loop 793 — module-scope [405][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1516 Wave Loop 793 — module-scope [405][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
docs(trinity): Wave Loop 792 closeout — [403][2]^6 Pt module-scope AoS (Closes #1513) #1514 docs(trinity): Wave Loop 792 closeout — [403][2]^6 Pt module-scope AoS (Closes #1513) | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
Wave Loop 791 — module-scope [401][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1512 Wave Loop 791 — module-scope [401][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
feat(igla): Wave Loop 790 — module-scope [399][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1510 feat(igla): Wave Loop 790 — module-scope [399][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
feat(igla): Wave Loop 789 — module-scope [397][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1508 feat(igla): Wave Loop 789 — module-scope [397][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
feat(igla): Wave Loop 788 — module-scope [395][2]^6 Pt non-power-of-two outer-dimension AoS var from call #1506 feat(igla): Wave Loop 788 — module-scope [395][2]^6 Pt non-power-of-two outer-dimension AoS var from call | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
feat(igla): Wave Loop 787 — module-scope [393][2]^6 Pt non-power-of-two outer-dimension AoS var from call #1504 feat(igla): Wave Loop 787 — module-scope [393][2]^6 Pt non-power-of-two outer-dimension AoS var from call | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
Wave Loop 786: module-scope [391][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable #1502 Wave Loop 786: module-scope [391][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
Wave Loop 785: module-scope [389][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable #1500 Wave Loop 785: module-scope [389][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
feat(igla): Wave Loop 784 — module-scope [387][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1498 feat(igla): Wave Loop 784 — module-scope [387][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
feat(igla): Wave Loop 783 — module-scope [385][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1496 feat(igla): Wave Loop 783 — module-scope [385][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
feat(igla): Wave Loop 782 — module-scope [383][2]^6 Pt non-power-of-two outer-dimension AoS variable from call with indexed signed writes #1494 feat(igla): Wave Loop 782 — module-scope [383][2]^6 Pt non-power-of-two outer-dimension AoS variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
feat(igla): Wave Loop 777 — module-scope [373][2]^6 Pt non-power-of-two packed AoS variable with indexed signed writes #1491 feat(igla): Wave Loop 777 — module-scope [373][2]^6 Pt non-power-of-two packed AoS variable with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 3 failing / 25 total
Merge Wave Loop Ladder W774-W776 + README update #1489 Merge Wave Loop Ladder W774-W776 + README update | Created: 2026-07-24 | Updated: 2026-07-24 | 3 failing / 25 total
feat(igla): Wave Loop 776 — module-scope [371][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1488 feat(igla): Wave Loop 776 — module-scope [371][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 3 failing / 25 total
feat(igla): Wave Loop 775 — module-scope [369][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1486 feat(igla): Wave Loop 775 — module-scope [369][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
feat(igla): Wave Loop 774 — module-scope [367][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes #1484 feat(igla): Wave Loop 774 — module-scope [367][2]^6 Pt non-power-of-two outer-dimension array-of-struct variable from call with indexed signed writes | Created: 2026-07-24 | Updated: 2026-07-24 | 4 failing / 25 total
chore(deps)(deps): bump candle-core from 0.10.2 to 0.11.0 #1470 chore(deps)(deps): bump candle-core from 0.10.2 to 0.11.0 | Created: 2026-07-13 | Updated: 2026-07-13 | 3 failing / 25 total
chore(deps)(deps): bump ed25519-dalek from 2.2.0 to 3.0.0 #1469 chore(deps)(deps): bump ed25519-dalek from 2.2.0 to 3.0.0 | Created: 2026-07-13 | Updated: 2026-07-13 | 3 failing / 25 total

Summary

Status	Count
Total Open PRs	50
PRs with Failing Checks	41
PRs with All Checks Green	9
READY	1
FAILING	41
PENDING	0

Seal Status

⚠️ STALE -- sha256(compiler.rs)=d6b51a3bcc20 != manifest seal=87e5cbd3ad94.
The committed NMSE numbers were certified against an older compiler.rs.
Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Jul 31, 2026

fix(witness): resolve the wide-rung pack defaults inside the repository #1582

Open

fix(ci): track the paper count the published version actually declares #1584

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci(conformance): gate INDEX_all_formats.json against the packs it summarises - #1578

ci(conformance): gate INDEX_all_formats.json against the packs it summarises#1578
gHashTag wants to merge 1 commit into
masterfrom
gate/pack-index-consistency

gHashTag commented Jul 31, 2026

Uh oh!

github-actions Bot commented Jul 31, 2026

Uh oh!

github-actions Bot commented Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

gHashTag commented Jul 31, 2026

What the gate checks

Verification

Uh oh!

github-actions Bot commented Jul 31, 2026

Uh oh!

github-actions Bot commented Jul 31, 2026

PR Dashboard

Summary

Seal Status

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant