|
6 | 6 |
|
7 | 7 | | Category | Count | Details | |
8 | 8 | |----------|-------|---------| |
9 | | -| **Source modules** | 21 | 11 Idris2 ABI (Region, TypedAccess, Levels, Pointer, Effects, Lifetime, Linear, MultiModule, Proofs, Tropical, Epistemic), 3 ReScript parser (Ast, Parser, Lexer), 3 Idris2 interface ABI, 2 Zig FFI + cache | |
10 | | -| **Unit tests** | 1 file | ParserTests.res (~82 assertions) | |
11 | | -| **Integration tests** | 0 | None | |
12 | | -| **E2E tests** | 1 | e2e-smoke.mjs (~43 assertions) | |
13 | | -| **Property-based tests** | 1 | property_test.mjs (10 properties, ~40+ assertions) | |
14 | | -| **Benchmarks** | 0 | None | |
15 | | -| **ECHIDNA harness** | 1 | echidna-harness.mjs (7 assertions) | |
| 9 | +| **Source modules** | 21 | 11 Idris2 ABI (Region, TypedAccess, Levels, Pointer, Effects, Lifetime, Linear, MultiModule, Proofs, Tropical, Epistemic), 4 ReScript parser (Ast, Parser, Lexer, Checker), 3 Idris2 interface ABI, 2 Zig FFI + cache, 1 Rust verifier crate (typed-wasm-verify, ~1.6k LOC + 53 tests) | |
| 10 | +| **Unit tests** | 2 files | ParserTests.res (88 assertions), crates/typed-wasm-verify (43 unit + 10 cross-compat) | |
| 11 | +| **Integration tests** | 1 | tests/contracts/airborne-step-state-contract.mjs (14 assertions) | |
| 12 | +| **E2E tests** | 2 | tests/smoke/e2e-smoke.mjs (40 assertions), tests/e2e/e2e-driver.mjs (corpus driver) | |
| 13 | +| **Per-level tests** | 10 | tests/levels/L1.mjs .. L10.mjs (56 assertions total) | |
| 14 | +| **Aspect tests** | 1 | tests/aspect/claim-envelope.mjs (49 assertions — added 2026-05 to catch cross-doc claim drift after deep audit found 5 such drifts) | |
| 15 | +| **Property-based tests** | 0 | tests/property/property_test.mjs claimed DONE 2026-04-04 but file is absent — stale entry being scrubbed | |
| 16 | +| **Benchmarks** | 1 | benchmarks/parser-bench.mjs (per-example wallclock; median/p95/min/throughput; JSON summary on stderr; added 2026-05) | |
| 17 | +| **ECHIDNA harness** | 1 | tests/echidna/echidna-harness.mjs (659 LOC, 124 local assertions, remote prover-wars submission) | |
16 | 18 |
|
17 | 19 | ## What's Missing |
18 | 20 |
|
19 | 21 | ### P2P Tests |
20 | | -- [x] **DONE 2026-04-04**: Property-based tests added (`tests/property/property_test.mjs`, 10 properties, ~40+ assertions) |
| 22 | +- [ ] **REVOKED 2026-05**: prior "DONE 2026-04-04 property tests" entry was |
| 23 | + false — `tests/property/property_test.mjs` was never committed. |
| 24 | + Aspect test now catches this drift class automatically. |
21 | 25 | - [ ] No tests for Idris2 ABI type checking with Zig FFI |
22 | 26 | - [ ] No tests for ReScript parser feeding into Idris2 type checker |
23 | 27 |
|
24 | 28 | ### E2E Tests |
25 | | -- [ ] e2e-smoke exists but only 43 assertions for a type system with 10 safety levels |
26 | | -- [ ] No WASM module compilation and execution test |
27 | | -- [ ] No multi-module linking test (MultiModule.idr untested) |
| 29 | +- [x] **DONE 2026-05**: `tests/e2e/e2e-driver.mjs` exercises every |
| 30 | + example through parse + check with skip/expect-clean/expect-diagnostic |
| 31 | + pragmas. Smoke test still narrow (40 assertions) but now augmented |
| 32 | + by the per-level suite (56 more) and the aspect test (49 more). |
| 33 | +- [ ] No WASM module compilation and execution test (blocked on codegen) |
| 34 | +- [ ] No multi-module linking test (MultiModule.idr untested at runtime) |
28 | 35 |
|
29 | 36 | ### Aspect Tests |
30 | | -- [ ] **Security**: No memory safety violation detection tests (this IS the product's purpose) |
31 | | -- [ ] **Performance**: No benchmarks for type checking overhead vs raw WASM |
| 37 | +- [x] **DONE 2026-05**: `tests/aspect/claim-envelope.mjs` — 49 checks that |
| 38 | + cross-document claims (README/ROADMAP/LEVEL-STATUS/EXPLAINME) stay |
| 39 | + consistent with actual artefacts (ipkg, Rust constants, CI pins, |
| 40 | + example corpus, RSR surface). Built in response to a deep audit |
| 41 | + finding five drifts the test now catches. |
| 42 | +- [ ] **Security**: No memory safety violation detection tests (this IS the |
| 43 | + product's purpose) |
| 44 | +- [ ] **Performance**: see Benchmarks below |
32 | 45 | - [ ] **Concurrency**: No concurrent WASM module compilation tests |
33 | | -- [ ] **Error handling**: No tests for invalid type annotations, malformed WASM |
| 46 | +- [ ] **Error handling**: 10/10 per-level test suites (`tests/levels/L*.mjs`) |
| 47 | + include negative cases — partial coverage |
34 | 48 |
|
35 | 49 | ### Build & Execution |
36 | 50 | - [ ] 11 Idris2 modules with 0 Idris2-level tests -- are proofs checked? |
| 51 | + (PROOF-NEEDS.md 2026-05-18 reconciliation: ipkg builds clean, but |
| 52 | + no Idris2 *test* layer beyond compile-time totality / type-checking) |
37 | 53 | - [ ] Zig FFI integration_test.zig likely a template placeholder |
38 | 54 |
|
39 | | -### Benchmarks Needed (CRITICAL) |
40 | | -- [ ] Type checking overhead per WASM instruction |
41 | | -- [ ] Memory region tracking performance |
42 | | -- [ ] Lifetime analysis scaling with module size |
| 55 | +### Benchmarks |
| 56 | +- [x] **DONE 2026-05**: `benchmarks/parser-bench.mjs` — per-example parse + |
| 57 | + check wallclock with median / p95 / min / throughput and JSON summary |
| 58 | + for trend tracking. Only the parser is end-to-end today, so that's |
| 59 | + where benchmark evidence has to start. |
| 60 | +- [ ] Type-checking overhead per WASM instruction (blocked on codegen + |
| 61 | + Zig FFI runtime path) |
| 62 | +- [ ] Memory region tracking performance (blocked on codegen) |
| 63 | +- [ ] Lifetime analysis scaling with module size (blocked on codegen) |
43 | 64 | - [ ] Comparison: typed-wasm overhead vs raw WASM execution |
| 65 | + (blocked on codegen) |
44 | 66 |
|
45 | 67 | ### Self-Tests |
46 | 68 | - [ ] No type system self-consistency check |
|
0 commit comments