|
| 1 | +# TEST-NEEDS.md — maa-framework |
| 2 | + |
| 3 | +> Generated 2026-03-29 by punishing audit. |
| 4 | +
|
| 5 | +## Current State |
| 6 | + |
| 7 | +| Category | Count | Notes | |
| 8 | +|-------------|-------|-------| |
| 9 | +| Unit tests | 0 | No inline tests in source | |
| 10 | +| Integration | 3 | absolute-zero FFI integration_test.zig, aletheia FFI integration_test.zig, rhodibot integration_tests.rs | |
| 11 | +| E2E | 0 | None | |
| 12 | +| Benchmarks | 1 | aletheia/benches/verification_benchmark.rs — uses std::time, not criterion (manual benchmark, not automated) | |
| 13 | + |
| 14 | +**Source modules:** ~23 across 2 subsystems. absolute-zero: brainfuck interpreter, whitespace interpreter, main + ABI/FFI. aletheia: checks, config, main, output, types + rhodibot + rhodium-pipeline + ABI/FFI. |
| 15 | + |
| 16 | +## What's Missing |
| 17 | + |
| 18 | +### P2P (Property-Based) Tests |
| 19 | +- [ ] absolute-zero: arbitrary brainfuck program fuzzing (already has fuzz target, good) |
| 20 | +- [ ] aletheia: arbitrary repo structure verification fuzzing |
| 21 | +- [ ] Config parsing: property tests for all config formats |
| 22 | + |
| 23 | +### E2E Tests |
| 24 | +- [ ] absolute-zero: compile -> execute -> verify program output |
| 25 | +- [ ] aletheia: full RSR verification pipeline (scan repo -> check rules -> report) |
| 26 | +- [ ] rhodibot: full bot operation cycle |
| 27 | +- [ ] rhodium-pipeline: extraction pipeline end-to-end |
| 28 | + |
| 29 | +### Aspect Tests |
| 30 | +- **Security:** No tests for aletheia bypass (can a repo trick the verifier?), rhodibot credential handling |
| 31 | +- **Performance:** Benchmark exists but is manual (std::time), not integrated into CI, not criterion-based |
| 32 | +- **Concurrency:** No tests for parallel verification of multiple repos |
| 33 | +- **Error handling:** No tests for malformed repos, missing files, invalid configurations |
| 34 | + |
| 35 | +### Build & Execution |
| 36 | +- [ ] `cargo test` for all Rust crates |
| 37 | +- [ ] `zig build test` for FFI tests |
| 38 | +- [ ] `cargo fuzz` for absolute-zero fuzz targets |
| 39 | + |
| 40 | +### Benchmarks Needed |
| 41 | +- [ ] Verification speed per repo size/complexity (aletheia) |
| 42 | +- [ ] Brainfuck/whitespace interpreter throughput (absolute-zero) |
| 43 | +- [ ] rhodibot processing rate |
| 44 | +- [ ] Pipeline extraction throughput |
| 45 | + |
| 46 | +### Self-Tests |
| 47 | +- [ ] aletheia: verify itself against RSR standards |
| 48 | +- [ ] Config schema validation |
| 49 | +- [ ] ABI version agreement |
| 50 | + |
| 51 | +## Priority |
| 52 | + |
| 53 | +**HIGH.** An RSR compliance verification framework that cannot verify itself is the definition of irony. 23 source modules with 0 unit tests. The 3 integration tests are a start but cover only FFI seams. The benchmark is manual and not CI-integrated. Fuzz targets for absolute-zero are good but need complementary structured tests. |
| 54 | + |
| 55 | +## FAKE-FUZZ ALERT |
| 56 | + |
| 57 | +- `tests/fuzz/placeholder.txt` is a scorecard placeholder inherited from rsr-template-repo — it does NOT provide real fuzz testing |
| 58 | +- Replace with an actual fuzz harness (see rsr-template-repo/tests/fuzz/README.adoc) or remove the file |
| 59 | +- Priority: P2 — creates false impression of fuzz coverage |
0 commit comments