Skip to content

Commit 610d632

Browse files
hyperpolymathclaude
andcommitted
Remove template-only ABI files that falsely implied formal verification
Template Idris2 ABI files (Types.idr, Layout.idr, Foreign.idr) contained only RSR template scaffolding with unresolved placeholders and no domain-specific proofs. Removed to prevent false impression of formal verification coverage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 8e805a2 commit 610d632

5 files changed

Lines changed: 107 additions & 626 deletions

File tree

PROOF-NEEDS.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Proof Requirements
2+
3+
## Current state
4+
- `src/abi/Types.idr` — Feedback types
5+
- `src/abi/Layout.idr` — Memory layout
6+
- `src/abi/Foreign.idr` — FFI declarations
7+
- No dangerous patterns in ABI layer
8+
- Claims: "formal verification via Idris2 for critical ABI definitions and memory safety proofs", "Production Ready"
9+
- 92K lines of source
10+
11+
## What needs proving
12+
- **Deduplication correctness**: Prove the deduplicator never drops unique feedback entries and never allows true duplicates through
13+
- **Submission atomicity**: Prove feedback submissions either fully succeed or fully fail (no partial submissions that corrupt state)
14+
- **Memory layout proofs**: The README claims "memory safety proofs" — these should be materialized in the ABI layer with actual dependent type proofs for layout correctness
15+
- **Rate limiting fairness**: Prove rate limiting does not permanently block legitimate users
16+
17+
## Recommended prover
18+
- **Idris2** — Already used for ABI; the claim of "memory safety proofs" needs to be substantiated with actual proofs in the existing `.idr` files
19+
20+
## Priority
21+
- **MEDIUM** — The README explicitly claims formal verification and memory safety proofs. If those proofs do not exist in substance, the claim is misleading. Priority is to either write the proofs or soften the claims.
22+
23+
## Template ABI Cleanup (2026-03-29)
24+
25+
Template ABI removed -- was creating false impression of formal verification.
26+
The removed files (Types.idr, Layout.idr, Foreign.idr) contained only RSR template
27+
scaffolding with unresolved {{PROJECT}}/{{AUTHOR}} placeholders and no domain-specific proofs.

TEST-NEEDS.md

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Test & Benchmark Requirements
2+
3+
## Current State
4+
- Unit tests: NONE (0 test files found)
5+
- Integration tests: NONE
6+
- E2E tests: NONE
7+
- Benchmarks: NONE
8+
- panic-attack scan: NEVER RUN
9+
10+
## What's Missing
11+
### Point-to-Point (P2P)
12+
34 Elixir source files with ZERO test files:
13+
- feedback_a_tron/application.ex — no tests
14+
- feedback_a_tron/deduplicator.ex — no tests
15+
- feedback_a_tron/mcp/tools/submit_feedback.ex — no tests
16+
- feedback_a_tron/mcp/tools/migration_observe.ex — no tests
17+
- feedback_a_tron/mcp/server.ex — no tests
18+
- feedback_a_tron/network_verifier.ex — no tests
19+
- feedback_a_tron/submitter.ex — no tests
20+
- feedback_a_tron/audit_log.ex — no tests
21+
- feedback_a_tron/cli.ex — no tests
22+
- feedback_a_tron/migration_observer.ex — no tests
23+
- feedback_a_tron/verisim_writer.ex — no tests
24+
- feedback_a_tron/batch_reviewer.ex — no tests
25+
- feedback_a_tron/report_generator.ex — no tests
26+
- feedback_a_tron/pipeline/supervisor.ex — no tests
27+
- feedback_a_tron/pipeline/producer.ex — no tests
28+
- feedback_a_tron/pipeline/verisim_consumer.ex — no tests
29+
- feedback_a_tron/pipeline/review_consumer.ex — no tests
30+
- feedback_a_tron/secure_dns.ex — no tests
31+
- feedback_a_tron/channels/nntp.ex — no tests
32+
- feedback_a_tron/channels/discourse.ex — no tests
33+
- Plus ~14 more modules
34+
- 3 Idris2 ABI files — no tests
35+
- 1 Julia file — no tests
36+
37+
### End-to-End (E2E)
38+
- Submit feedback -> deduplicate -> review -> write to VeriSimDB
39+
- MCP server: receive tool call -> process -> respond
40+
- Batch review: load batch -> review all -> generate report
41+
- Pipeline: produce events -> consume -> write to VeriSimDB
42+
- Migration observation: detect migration -> observe -> record
43+
- NNTP channel: receive -> process -> forward
44+
- Discourse channel: receive -> process -> forward
45+
- CLI: submit/review/report commands
46+
47+
### Aspect Tests
48+
- [ ] Security (MCP tool injection, network verifier bypass, NNTP injection, audit log tampering)
49+
- [ ] Performance (batch review throughput, pipeline backpressure, VeriSimDB write latency)
50+
- [ ] Concurrency (GenStage pipeline, concurrent submissions, supervisor restart)
51+
- [ ] Error handling (VeriSimDB unavailable, network failures, malformed feedback)
52+
- [ ] Accessibility (N/A)
53+
54+
### Build & Execution
55+
- [ ] mix compile — not verified
56+
- [ ] mix test — not verified (no test files to run)
57+
- [ ] MCP server starts — not verified
58+
- [ ] CLI --help works — not verified
59+
- [ ] Self-diagnostic — none
60+
61+
### Benchmarks Needed
62+
- Feedback submission throughput
63+
- Deduplication speed and accuracy
64+
- Pipeline end-to-end latency
65+
- VeriSimDB write performance
66+
- Batch review performance at scale
67+
68+
### Self-Tests
69+
- [ ] panic-attack assail on own repo
70+
- [ ] MCP server health check
71+
- [ ] Pipeline self-test
72+
73+
## Priority
74+
- **HIGH** — 34 Elixir source files with ZERO tests. This is a feedback processing system with MCP integration, GenStage pipeline, multi-channel input (NNTP, Discourse), VeriSimDB integration, and audit logging. Not a single module has a test file. The pipeline supervisor/producer/consumer pattern especially needs testing for correctness under load and failure conditions.
75+
76+
## FAKE-FUZZ ALERT
77+
78+
- `tests/fuzz/placeholder.txt` is a scorecard placeholder inherited from rsr-template-repo — it does NOT provide real fuzz testing
79+
- Replace with an actual fuzz harness (see rsr-template-repo/tests/fuzz/README.adoc) or remove the file
80+
- Priority: P2 — creates false impression of fuzz coverage

src/abi/Foreign.idr

Lines changed: 0 additions & 217 deletions
This file was deleted.

0 commit comments

Comments
 (0)