Skip to content

Commit 83e3395

Browse files
hyperpolymathclaude
andcommitted
docs: add TEST-NEEDS.md and/or PROOF-NEEDS.md from audit
Documents testing and proof gaps identified during batch audit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 634f572 commit 83e3395

2 files changed

Lines changed: 97 additions & 0 deletions

File tree

PROOF-NEEDS.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# PROOF-NEEDS.md
2+
<!-- SPDX-License-Identifier: PMPL-1.0-or-later -->
3+
4+
## Current State
5+
6+
- **LOC**: ~94,700
7+
- **Languages**: Elixir, ReScript, Gleam, Julia, Rust, Idris2, Zig
8+
- **Existing ABI proofs**: `src/abi/*.idr` (template-level)
9+
- **Dangerous patterns**: 20+ `Obj.magic` in ReScript connector clients (VeriSimDrift, VeriSimFederation, VeriSimHexad, VeriSimProvenance, VeriSimSearch, VeriSimVql, Registry)
10+
11+
## What Needs Proving
12+
13+
### VQL Type Checker (src/vql/)
14+
- `VQLTypeChecker.res`, `VQLSubtyping.res`, `VQLProofObligation.res` — a query type system
15+
- Type checking correctness and subtyping transitivity should be formally proven
16+
- `VQLBidir.res` — bidirectional type inference needs soundness proof
17+
18+
### Connector Client Type Safety (connectors/clients/rescript/)
19+
- Every client module uses `json->Obj.magic` to cast JSON responses to typed records
20+
- VeriSimDrift, VeriSimFederation, VeriSimHexad, VeriSimProvenance, VeriSimSearch, VeriSimVql all affected
21+
- Prove: API response schemas match expected ReScript types
22+
23+
### Registry / Consensus (src/registry/)
24+
- `KRaftCluster.res` — Raft consensus implementation
25+
- `KRaftSerializer.res` — serialization for consensus protocol
26+
- Consensus algorithms have well-known formal specifications — this should be provably correct
27+
- `MetadataLog.res` — log integrity is critical for database correctness
28+
29+
### Federation Protocol
30+
- VeriSimFederation handles cross-instance data replication
31+
- Prove: federation preserves data consistency (eventual consistency guarantees)
32+
33+
### Provenance Chain
34+
- VeriSimProvenance tracks data lineage
35+
- Prove: provenance chains are append-only and tamper-evident
36+
37+
## Recommended Prover
38+
39+
- **Idris2** for VQL type system soundness and ABI contracts
40+
- **Lean4** for Raft consensus correctness (existing Lean4 Raft formalisations exist)
41+
- **Agda** for provenance chain integrity (algebraic properties)
42+
43+
## Priority
44+
45+
**HIGH** — Database system used as the persistence layer across the entire ecosystem. VQL type checking, Raft consensus, and provenance integrity are all areas where formal proofs would prevent data corruption or consistency violations.

TEST-NEEDS.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# TEST-NEEDS: verisimdb
2+
3+
## Current State
4+
5+
| Category | Count | Details |
6+
|----------|-------|---------|
7+
| **Source modules** | 18 | ReScript VQL (10: Bidir, Circuit, Context, Error, Explain, Parser, ProofObligation, Subtyping, TypeChecker, Types), Registry (4: KRaftCluster, KRaftSerializer, MetadataLog, Registry), Parser_test, 3 Idris2 ABI |
8+
| **Unit tests** | 1 file | VQLParser_test.res (~110 test/assert references) |
9+
| **Integration tests** | 1 file | integration_test.rs (0 #[test] -- may be stub) |
10+
| **E2E tests** | 0 | None |
11+
| **Benchmarks** | 2 files | modality_benchmarks.rs (104 bench/criterion refs), src/lib.rs |
12+
13+
## What's Missing
14+
15+
### P2P Tests (CRITICAL)
16+
- [ ] No tests for VQL type checker <-> subtyping <-> proof obligation pipeline
17+
- [ ] No tests for Registry <-> KRaft cluster interaction
18+
- [ ] No tests for bidirectional type inference (VQLBidir)
19+
20+
### E2E Tests (CRITICAL)
21+
- [ ] No test that stores data, queries it via VQL, and validates results
22+
- [ ] No test for the full modality verification pipeline
23+
- [ ] No test for KRaft consensus with multiple nodes
24+
25+
### Aspect Tests
26+
- [ ] **Security**: Database with no injection tests, no access control tests
27+
- [ ] **Performance**: Benchmark files exist -- need to verify they run
28+
- [ ] **Concurrency**: No concurrent read/write tests, no transaction isolation tests
29+
- [ ] **Error handling**: No tests for storage corruption, query timeout, cluster partition
30+
31+
### Build & Execution
32+
- [ ] integration_test.rs has 0 #[test] -- appears to be a stub
33+
- [ ] No Idris2 ABI compilation test
34+
35+
### Benchmarks Status
36+
- [x] modality_benchmarks.rs (104 criterion refs) -- appears genuine
37+
- [ ] No query throughput benchmark
38+
- [ ] No storage I/O benchmark
39+
- [ ] No cluster consensus latency benchmark
40+
41+
### Self-Tests
42+
- [ ] No database self-consistency check
43+
- [ ] No `verisimdb --self-test` mode
44+
45+
## FLAGGED ISSUES
46+
- **A DATABASE with 1 test file** -- VQLParser_test.res covers parsing but nothing else
47+
- **integration_test.rs is a STUB** with 0 actual test functions
48+
- **Registry/KRaft (distributed consensus) has 0 tests** -- consensus code is untested
49+
- **VQLBidir, VQLCircuit, VQLExplain, VQLProofObligation all have 0 tests**
50+
- **Benchmarks appear real** (104 criterion references) -- one bright spot
51+
52+
## Priority: P0 (CRITICAL)

0 commit comments

Comments
 (0)