|
| 1 | +# TEST-NEEDS: vql-ut |
| 2 | + |
| 3 | +## Current State |
| 4 | + |
| 5 | +| Category | Count | Details | |
| 6 | +|----------|-------|---------| |
| 7 | +| **Source modules** | 27 | Idris2 core (4: Checker, Grammar, Levels, Schema), ReScript bridges (3: Ast, Bridge, Parser), ReScript definitions/errors (2), Rust interfaces (LSP 2, DAP 2, fmt 2, lint 2, lib), 3 Idris2 interface ABI, Zig FFI | |
| 8 | +| **Unit tests** | 1 file | integration_test.rs (49 #[test]) | |
| 9 | +| **Integration tests** | 0 | Despite the filename, these are unit tests | |
| 10 | +| **E2E tests** | 0 | None | |
| 11 | +| **Benchmarks** | 0 | None | |
| 12 | +| **Fuzz tests** | 0 | None | |
| 13 | + |
| 14 | +## What's Missing |
| 15 | + |
| 16 | +### P2P Tests (CRITICAL) |
| 17 | +- [ ] No tests for ReScript parser -> Idris2 checker pipeline |
| 18 | +- [ ] No tests for LSP server handling real editor requests |
| 19 | +- [ ] No tests for DAP server with real debugger |
| 20 | +- [ ] No tests for fmt/lint tools on actual VQL-UT code |
| 21 | + |
| 22 | +### E2E Tests (CRITICAL) |
| 23 | +- [ ] No test that parses VQL-UT, type-checks it through all 10 levels, and executes it |
| 24 | +- [ ] No test for LSP completion/hover/diagnostics |
| 25 | +- [ ] No test for DAP breakpoints/stepping |
| 26 | + |
| 27 | +### Aspect Tests |
| 28 | +- [ ] **Security**: Query language with no injection tests |
| 29 | +- [ ] **Performance**: No benchmarks for type checking, parsing throughput |
| 30 | +- [ ] **Concurrency**: No concurrent query compilation tests |
| 31 | +- [ ] **Error handling**: No tests for malformed VQL-UT, type errors at each level |
| 32 | + |
| 33 | +### Build & Execution |
| 34 | +- [ ] 4 Idris2 core modules with 0 Idris2-level tests -- are proofs checked? |
| 35 | +- [ ] 4 Rust tool interfaces (LSP, DAP, fmt, lint) with 0 tests each |
| 36 | +- [ ] Zig FFI integration_test.zig likely template placeholder |
| 37 | + |
| 38 | +### Benchmarks Needed |
| 39 | +- [ ] VQL-UT parsing throughput |
| 40 | +- [ ] Type checking per level (L1-L10) |
| 41 | +- [ ] LSP response latency |
| 42 | +- [ ] Query compilation time |
| 43 | + |
| 44 | +### Self-Tests |
| 45 | +- [ ] No VQL-UT self-consistency check |
| 46 | + |
| 47 | +## FLAGGED ISSUES |
| 48 | +- **49 tests for 27 source modules** = thin coverage |
| 49 | +- **4 developer tools (LSP, DAP, fmt, lint) with 0 tests** -- tools that developers will use are untested |
| 50 | +- **10-level type system with 0 level-specific tests** -- can't verify any level works |
| 51 | +- **Idris2 formal core is unverified** -- the proofs exist but nobody tests that they check |
| 52 | + |
| 53 | +## Priority: P0 (CRITICAL) |
| 54 | + |
| 55 | +## FAKE-FUZZ ALERT |
| 56 | + |
| 57 | +- `tests/fuzz/placeholder.txt` is a scorecard placeholder inherited from rsr-template-repo — it does NOT provide real fuzz testing |
| 58 | +- Replace with an actual fuzz harness (see rsr-template-repo/tests/fuzz/README.adoc) or remove the file |
| 59 | +- Priority: P2 — creates false impression of fuzz coverage |
0 commit comments