Skip to content

Commit 827524d

Browse files
hyperpolymathclaude
andcommitted
Remove template-only ABI files that falsely implied formal verification
Template Idris2 ABI files (Types.idr, Layout.idr, Foreign.idr) contained only RSR template scaffolding with unresolved placeholders and no domain-specific proofs. Removed to prevent false impression of formal verification coverage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 8ce7be7 commit 827524d

2 files changed

Lines changed: 88 additions & 3 deletions

File tree

PROOF-NEEDS.md

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,48 @@
11
# PROOF-NEEDS.md
2+
<!-- SPDX-License-Identifier: PMPL-1.0-or-later -->
3+
4+
## Current State
5+
6+
- **LOC**: ~18,000
7+
- **Languages**: OCaml, Rust, Idris2, Zig
8+
- **Existing ABI proofs**: `src/abi/*.idr` (template-level)
9+
- **Dangerous patterns**: None detected
10+
11+
## What Needs Proving
12+
13+
### Type Checker (compiler/lib/typecheck.ml — 775 lines)
14+
- Core type checker for a Turing-complete topological programming language
15+
- Prove: type soundness (preservation + progress)
16+
- Prove: type checking terminates on all inputs
17+
18+
### Evaluator (compiler/lib/eval.ml)
19+
- Prove: evaluation preserves types (subject reduction)
20+
- Prove: well-typed programs in the decidable fragment terminate
21+
22+
### Parser Correctness (compiler/lib/)
23+
- `token.ml`, lexer/parser
24+
- Prove: parser accepts exactly the language defined by the grammar
25+
- Less critical than type system proofs but valuable for confidence
26+
27+
### WASM Compilation (compiler/tangle-wasm/src/lib.rs)
28+
- Prove: compilation preserves semantics (source-level evaluation matches WASM execution)
29+
- This is a compiler correctness theorem — high value but high effort
30+
31+
### Fuzz Testing Coverage
32+
- `compiler/fuzz/fuzz_lexer.ml`, `fuzz_parser.ml` — fuzzing is present but not formal
33+
- Proofs would subsume the need for fuzzing on core invariants
34+
35+
## Recommended Prover
36+
37+
- **Coq** or **Lean4** — OCaml type checkers have a strong tradition of Coq mechanisation (e.g., CompCert)
38+
- **Idris2** for the ABI layer
39+
40+
## Priority
41+
42+
**HIGH** — Programming language with a type checker and evaluator. Type soundness is the minimum bar for a language claiming Turing-completeness with a type system. Without it, the topological type claims are unverified.
243

344
## Template ABI Cleanup (2026-03-29)
445

546
Template ABI removed -- was creating false impression of formal verification.
647
The removed files (Types.idr, Layout.idr, Foreign.idr) contained only RSR template
748
scaffolding with unresolved {{PROJECT}}/{{AUTHOR}} placeholders and no domain-specific proofs.
8-
9-
When this project needs formal ABI verification, create domain-specific Idris2 proofs
10-
following the pattern in repos like `typed-wasm`, `proven`, `echidna`, or `boj-server`.

TEST-NEEDS.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# TEST-NEEDS: tangle
2+
3+
## Current State
4+
5+
| Category | Count | Details |
6+
|----------|-------|---------|
7+
| **Source modules** | 11 | Rust: ast, ast_jtv, lexer, parser, parser_jtv, eval, lib, main, sexpr + 3 Idris2 ABI |
8+
| **Unit tests (inline)** | 252 | lexer=151, parser=40, parser_jtv=32, eval=29 |
9+
| **Integration tests** | 0 | None |
10+
| **E2E tests** | 0 | None |
11+
| **Benchmarks** | 4 files | bench_lexer.rs (135L), bench_parser_rust.rs (106L), bench_lexer.ml (113L), bench_parser.ml (88L) |
12+
| **Fuzz tests** | 2 | fuzz_lexer.rs, fuzz_parser.rs |
13+
14+
## What's Missing
15+
16+
### E2E Tests
17+
- [ ] No test that parses a Tangle program and evaluates it end-to-end
18+
- [ ] No test for the sexpr output format
19+
- [ ] No test for the main binary
20+
21+
### Aspect Tests
22+
- [ ] **Security**: No injection/escape tests for the parser
23+
- [ ] **Performance**: Benchmarks exist -- need to verify they actually run
24+
- [ ] **Concurrency**: N/A for a language parser
25+
- [ ] **Error handling**: No tests for error recovery, partial parse, unterminated strings
26+
27+
### Build & Execution
28+
- [ ] OCaml benchmarks (bench_lexer.ml, bench_parser.ml) -- does OCaml build config exist?
29+
- [ ] No Idris2 ABI compilation test
30+
31+
### Benchmarks Status
32+
- [x] bench_lexer.rs (135 lines) -- appears real
33+
- [x] bench_parser_rust.rs (106 lines) -- appears real
34+
- [?] bench_lexer.ml (113 lines) -- needs OCaml build verification
35+
- [?] bench_parser.ml (88 lines) -- needs OCaml build verification
36+
37+
### Self-Tests
38+
- [ ] No self-diagnostic mode
39+
40+
## FLAGGED ISSUES
41+
- **252 inline unit tests is good** for a parser/lexer
42+
- **Benchmarks appear genuine** -- best benchmark setup among scanned repos
43+
- **Fuzz tests exist** -- rare and commendable
44+
- **ast.rs, ast_jtv.rs, sexpr.rs have 0 tests** -- structural modules untested
45+
- **No integration/E2E despite having eval** -- can't verify programs actually run correctly
46+
47+
## Priority: P2 (MEDIUM) -- solid unit/bench/fuzz foundation, needs E2E and integration

0 commit comments

Comments
 (0)