Skip to content

Commit d4ca11f

Browse files
hyperpolymathclaude
andcommitted
docs(er): formal VeriSim E-R schema + CORPUS-ADAPTERS index + saturation ADR
Three new docs landing the schema + indexes for the saturation campaign: - docs/architecture/VERISIM-ER-SCHEMA.md — 12 entities (Octad + 8 modalities + ProofAttempt + CertificateBlob + ProverBinaryIntegrity) and 7 relationships (composition / cross-prover identity / attempt / certificate / binary-integrity / depends-on / evolved-from). Crosswalk Rust struct ↔ Cap'n Proto schema ↔ ClickHouse table. Replaces the aspirational design/ECHIDNA-VERISIM-TRIANGULATION-2026-04-17.adoc as the formal data-model spec. Drift-detection plan + open migrations tracked inline. - docs/CORPUS-ADAPTERS.md — index of all 17 corpus adapters with hazard-flag inventory + canonical source URLs + open backlog (naproche/mathcomp/iris/cubical_agda/tlaps/pvs/naturalproofs/ alphaproof) for future waves. - docs/decisions/2026-06-01-saturation-campaign.md — ADR for the whole campaign: context, decision, consequences, what this campaign did NOT do, alternatives considered. Lane: prover-corpus-saturation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 33b565b commit d4ca11f

3 files changed

Lines changed: 615 additions & 0 deletions

File tree

docs/CORPUS-ADAPTERS.md

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
<!--
2+
SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
3+
SPDX-License-Identifier: MPL-2.0
4+
-->
5+
6+
# Corpus Adapters — Index
7+
8+
**Status**: canonical. Updated 2026-06-01 as part of the saturation campaign
9+
(see `docs/decisions/2026-06-01-saturation-campaign.md`).
10+
11+
Each adapter is a `pub fn ingest(root: &Path) -> Result<Corpus>` in
12+
`src/rust/corpus/<name>.rs`, structurally indexing a proof tree into
13+
`Corpus` (`src/rust/corpus/mod.rs:138`). Hazards (axioms, sorry,
14+
believe_me, cheat, …) populate `AxiomUsage` per entry.
15+
16+
## Adapter table
17+
18+
| Adapter | File extensions | Canonical source URL | Hazard flags surfaced | Status |
19+
|---|---|---|---|---|
20+
| **agda** | `*.agda` | <https://github.com/agda/agda> | postulate, believe_me, assert_total, sorry, trustme | shipped pre-2026-04 |
21+
| **coq** | `*.v` | <https://github.com/coq/coq> | Axiom, Parameter, Admitted | shipped pre-2026-04 |
22+
| **lean** | `*.lean` | <https://github.com/leanprover-community/mathlib4> | sorry, axiom, admit | shipped pre-2026-04 |
23+
| **idris2** | `*.idr` / `*.ipkg` | <https://github.com/idris-lang/Idris2> | believe_me, assert_total, postulate | shipped pre-2026-04 |
24+
| **isabelle** | `*.thy` | <https://www.isa-afp.org/> | axiomatization, axiom, sorry, oops, consts, nitpick, quickcheck | **NEW 2026-06-01** |
25+
| **metamath** | `*.mm` | <https://github.com/metamath/set.mm> | `$a` axiom usage, `?` admitted in proof | **NEW 2026-06-01** |
26+
| **mizar** | `*.miz`, `*.abs` | <http://mizar.org/version/8.1.14/mml/> | `@proof` sketch, commented-theorem, consider-without-proof | **NEW 2026-06-01** |
27+
| **hol_light** | `*.ml` (HOL Light filter) | <https://github.com/jrh13/hol-light> | new_axiom, mk_thm, free ASSUME, failwith "not yet" | **NEW 2026-06-01** |
28+
| **hol4** | `*Script.sml` | <https://github.com/HOL-Theorem-Prover/HOL> | new_axiom, mk_thm, cheat, CHEAT_TAC | **NEW 2026-06-01** |
29+
| **dafny** | `*.dfy` | <https://github.com/dafny-lang/dafny> | assume, `:axiom`, extern, expect, `:fuel 0` | **NEW 2026-06-01** |
30+
| **why3** | `*.mlw`, `*.why` | <https://gitlab.inria.fr/why3/why3> | axiom, val function (uninterpreted), val lemma, assume, absurd | **NEW 2026-06-01** |
31+
| **fstar** | `*.fst`, `*.fsti` | <https://github.com/FStarLang/FStar> | assume, admit, magic, Obj.magic, unsafe_coerce, admit_smt_queries | **NEW 2026-06-01** |
32+
| **acl2_books** | `*.lisp`, `*.acl2` | <https://github.com/acl2/acl2> | defaxiom, defstub, skip-proofs (top-level surfaced), ld-skip-proofsp, local-in-theory-nil | **NEW 2026-06-01** |
33+
| **tptp** | `*.p`, `*.tptp` | <https://tptp.org/> | no_conjecture (axiom-only file), TFF/THF dialects not fully supported | **NEW 2026-06-01** |
34+
| **smtlib** | `*.smt2`, `*.smt` | <https://smt-lib.org/benchmarks.shtml> | status_unknown, uninterpreted (declared-not-defined), unnamed_bang_pattern | **NEW 2026-06-01** |
35+
| **proofnet** | `*.jsonl` | <https://github.com/zhangir-azerbayev/ProofNet> | sorry, admit, axiom, no_formal_proof | **NEW 2026-06-01** |
36+
| **minif2f** | `*.lean`, `*.thy`, `*.ml`, `*.mm`, `*.v` (multi) | <https://github.com/openai/miniF2F> | sorry, admit, Admitted, empty_body, todo | **NEW 2026-06-01** |
37+
38+
## Coverage summary
39+
40+
**Pre-2026-04**: 4 adapters (agda, coq, lean, idris2).
41+
**Post-2026-06-01**: 17 adapters — 4.25× expansion.
42+
43+
## Test fixtures
44+
45+
Every adapter has a minimal smoke fixture under
46+
`tests/corpus_fixtures/<adapter>/`. Fixtures cover the
47+
happy-path decl parse plus at least one hazard-detection case.
48+
They are intentionally tiny — the goal is "compile + smoke
49+
correctness", not "extensive coverage". Larger evaluation corpora
50+
live outside the repo (mathlib4 etc. are fetched at training time
51+
via `just provision-corpora`).
52+
53+
## Cross-prover synonyms
54+
55+
Every adapter has a corresponding per-prover synonyms TOML at
56+
`data/synonyms/<adapter>.toml` listing tactic / decl / hazard
57+
vocabulary with canonical-and-aliases form. Plus three cross-prover
58+
taxonomic dictionaries (underscore-prefix; merged into
59+
`SynonymTable` via `merge_external()`):
60+
61+
- `_msc2020.toml` — 87 MSC2020 codes for cross-domain classification
62+
- `_wordnet_math.toml`~80 lemmas from WordNet 3.1 math sub-hierarchy
63+
- `_conceptnet_seed.toml`~55 pre-fetched ConceptNet 5.7 edges
64+
for offline-resilient semantic_class resolution
65+
66+
## Wiring into downstream consumers
67+
68+
1. **suggest**: `src/rust/suggest/synonyms.rs::SynonymTable::load_all()`
69+
reads per-prover TOMLs. The cross-prover dictionaries are loaded
70+
via `with_msc2020()` / `with_wordnet()` / `with_conceptnet()` (new
71+
methods landing in the same campaign).
72+
73+
2. **octad emission**: `src/rust/corpus/octad.rs::Corpus::emit_octads()`
74+
walks any `Corpus` produced by any adapter and emits 8-modality
75+
octads into VeriSimDB (see `docs/architecture/VERISIM-ER-SCHEMA.md`).
76+
77+
3. **GNN training**: `src/julia/training/train.jl` consumes
78+
`premises_<adapter>.jsonl` files under `training_data/`. The new
79+
adapters can produce these files via
80+
`cargo run --bin corpus-emit -- --adapter <name> --output training_data/premises_<name>.jsonl`
81+
(TODO — wire CLI).
82+
83+
## When to add a new adapter
84+
85+
- Upstream proof library has > 100 declarations AND
86+
- The library has a stable file format (not a daily-changing CI artifact) AND
87+
- A real prover backend exists in `src/rust/provers/` OR
88+
it's a recognised ML evaluation set (ProofNet, MiniF2F, NaturalProofs, …).
89+
90+
Drop the new file under `src/rust/corpus/<name>.rs`, add it to
91+
`mod.rs`, write at least 2 unit tests, add a fixture under
92+
`tests/corpus_fixtures/<name>/`, add an entry to this index, and
93+
add the per-prover synonyms TOML at `data/synonyms/<name>.toml`
94+
(omit if it's a format like TPTP/SMT-LIB rather than a prover).
95+
96+
## Open backlog (next saturation wave)
97+
98+
- `src/rust/corpus/naproche.rs` (Naproche-SAD)
99+
- `src/rust/corpus/mathcomp.rs` (Coq MathComp — distinct from generic coq.rs)
100+
- `src/rust/corpus/iris.rs` (Coq Iris separation logic)
101+
- `src/rust/corpus/cubical_agda.rs` (Cubical Agda stdlib)
102+
- `src/rust/corpus/tlaps.rs` (TLA+ Proofs)
103+
- `src/rust/corpus/pvs.rs` (PVS theories)
104+
- `src/rust/corpus/naturalproofs.rs` (Bobrow et al. NL→Coq dataset)
105+
- `src/rust/corpus/alphaproof.rs` (DeepMind 2024 olympiad set)
106+
107+
Drop here as upstream URLs are confirmed and at least one production
108+
consumer needs each.

0 commit comments

Comments
 (0)