|
| 1 | +# TEST-NEEDS: SMTLib.jl |
| 2 | + |
| 3 | +## Current State |
| 4 | + |
| 5 | +| Category | Count | Details | |
| 6 | +|----------|-------|---------| |
| 7 | +| **Source modules** | 12 | 3,586 lines | |
| 8 | +| **Test files** | 1 | 1,578 lines, 650 @test/@testset | |
| 9 | +| **Benchmarks** | 0 | None | |
| 10 | +| **E2E tests** | 0 | None | |
| 11 | + |
| 12 | +## What's Missing |
| 13 | + |
| 14 | +### E2E Tests |
| 15 | +- [ ] No test running SMTLib against actual SMT solvers (Z3, CVC5) |
| 16 | +- [ ] No roundtrip test (generate SMTLib2 -> parse back) |
| 17 | + |
| 18 | +### Aspect Tests |
| 19 | +- [ ] **Performance**: SMT solving is performance-critical, 0 benchmarks |
| 20 | +- [ ] **Error handling**: No tests for unsatisfiable formulas, timeout, solver crashes |
| 21 | +- [ ] **Concurrency**: No parallel solving tests |
| 22 | + |
| 23 | +### Benchmarks Needed (CRITICAL) |
| 24 | +- [ ] Formula generation throughput |
| 25 | +- [ ] Solver invocation latency |
| 26 | +- [ ] Scaling with formula complexity |
| 27 | + |
| 28 | +### Self-Tests |
| 29 | +- [ ] No solver availability self-check |
| 30 | + |
| 31 | +## FLAGGED ISSUES |
| 32 | +- **650 tests is excellent** -- second highest test count |
| 33 | +- **Single test file for 12 modules** -- should be split |
| 34 | +- **0 benchmarks for performance-critical SMT library** -- major gap |
| 35 | +- **No solver integration tests** -- generates SMTLib2 but never verifies it works |
| 36 | + |
| 37 | +## Priority: P2 (MEDIUM) -- strong unit tests, needs benchmarks and solver E2E |
| 38 | + |
| 39 | +## FAKE-FUZZ ALERT |
| 40 | + |
| 41 | +- `tests/fuzz/placeholder.txt` is a scorecard placeholder inherited from rsr-template-repo — it does NOT provide real fuzz testing |
| 42 | +- Replace with an actual fuzz harness (see rsr-template-repo/tests/fuzz/README.adoc) or remove the file |
| 43 | +- Priority: P2 — creates false impression of fuzz coverage |
0 commit comments