|
1 | 1 | # TEST-NEEDS.md — panll |
2 | 2 |
|
3 | | -> Generated 2026-03-29 by punishing audit. |
| 3 | +> Generated 2026-03-29 by punishing audit. Updated 2026-04-04 by CRG C blitz. |
4 | 4 |
|
5 | 5 | ## Current State |
6 | 6 |
|
7 | 7 | | Category | Count | Notes | |
8 | 8 | |-------------|-------|-------| |
9 | | -| Unit tests | ~20 | JS engine tests: tentacles, network_topology, level_architect, cloudguard, security, ums, automation_router, accessibility, seam, focus_dimming, help, tiling, anti_crash, contractiles, orbital_sync, menu_bar | |
| 9 | +| Unit tests | ~120 | JS engine tests + Rust type tests | |
10 | 10 | | Integration | ~6 | TEA framework: tea_app_test, tea_cmd_test, tea_sub_test, tea_render_test | |
11 | | -| E2E | 0 | None | |
12 | | -| Benchmarks | 0 | Files named "Workbench" are components, NOT benchmarks | |
| 11 | +| E2E | 47 | e2e_panel_lifecycle_test.js (comprehensive lifecycle + TypeLL + cross-panel) | |
| 12 | +| Benchmarks | 35+ | engine_bench_test, tea_update_cycle_bench_test, safedom_bench_test, panic_attack_bench_test + benches/panll_bench.js | |
| 13 | +| P2P | 23 | tests/p2p/tea_properties_test.mjs — TEA invariants, layout no-overlap, IPC roundtrip | |
| 14 | +| Aspect | 36 | tests/aspect/security_test.mjs — IPC sanitation, plugin sandboxing, redaction, XSS | |
| 15 | +| Contract | 30 | tests/contract/panel_contracts_test.mjs — C1-C9 governance contracts | |
| 16 | +| Reflexive | 22 | tests/reflexive/manifest_test.mjs — manifest consistency, export verification | |
| 17 | +| Rust smoke | 267 | All Rust tests pass (267 total). New smoke tests in: security, capture, farm, minter, plaza, watcher, workspace, ai, cloudguard, voicetag, hypatia | |
| 18 | + |
| 19 | +**Source modules:** ~686 ReScript .res files. 116 Rust files (all previously-untested crates now have smoke tests). |
| 20 | + |
| 21 | +## What Was Added (2026-04-04 CRG C Blitz) |
| 22 | + |
| 23 | +### Benchmarks |
| 24 | +- [x] `benches/panll_bench.js` — standalone bench file (4 groups × 8 benches): |
| 25 | + - TEA update cycle latency (NoOp baseline, AddConstraint, AntiCrash, 100-msg load) |
| 26 | + - Panel creation/destruction time (Model.init, ResetAllPanels, registry lookups, TogglePanel) |
| 27 | + - IPC message throughput (PanelBus, JSON round-trips, AntiCrash pipeline) |
| 28 | + - Layout algorithm time vs panel count (1, 4, 9, 16, 36, 108 panels) |
| 29 | +- [x] `deno task bench` added to deno.json |
| 30 | + |
| 31 | +### P2P Property-Based Tests |
| 32 | +- [x] `tests/p2p/tea_properties_test.mjs` — 8 properties × 100 random trials each: |
| 33 | + - update(msg, model) always produces valid model shape |
| 34 | + - TilingEngine.tile produces non-overlapping panels with positive dimensions |
| 35 | + - IPC JSON round-trip structural equality |
| 36 | + - AntiCrash halted flag monotonicity |
| 37 | + - Vexometer index non-decreasing |
| 38 | + - Contractiles.evaluateAll never throws (totality) |
| 39 | + - PanelRegistry findPanel/allPanels inverse consistency |
| 40 | + - TypeLL serviceActive always boolean |
13 | 41 |
|
14 | | -**Source modules:** ~686 ReScript .res files (not counting compiled .res.js). Massive TEA architecture with engines, models, views, components, core modules. Also: 116 Rust files, 15 Elixir, 5 Zig FFI, Idris2 ABI. |
15 | | - |
16 | | -## What's Missing |
| 42 | +### Aspect Tests |
| 43 | +- [x] `tests/aspect/security_test.mjs` — 36 tests: |
| 44 | + - Panel IPC sanitization (malformed TAGs, null payloads, empty objects) |
| 45 | + - Plugin sandboxing (CloudGuard cannot modify paneL, Farm cannot modify cloudguard, etc.) |
| 46 | + - Anti-Crash circuit breaker (processToken, checkSecurityConstraints) |
| 47 | + - Redaction engine (Anthropic, OpenAI, GitHub tokens; idempotency; safe content preservation) |
| 48 | + - XSS/injection resistance (script tags, javascript: URIs, onerror, onload, 1MB rejection) |
| 49 | + - Governance range invariants (vexometer [0,1], orbital [0,1]) |
| 50 | + |
| 51 | +### Contract Tests |
| 52 | +- [x] `tests/contract/panel_contracts_test.mjs` — 30 tests across 9 contracts: |
| 53 | + - C1: Orbital Stability Contract (threshold 0.7) |
| 54 | + - C2: Vexation Ceiling Contract |
| 55 | + - C3: Anti-Crash Quorum Contract (violations ≤ 10) |
| 56 | + - C5: TypeLL Service Contract (queriesServed increments on Ok) |
| 57 | + - C6: Panel Bus Contract (registry, topics, subscribers) |
| 58 | + - C7: Model Initialisation Contract (11 default contractiles) |
| 59 | + - C8: Governance Engine Contract |
| 60 | + - C9: Contractiles Elasticity Adaptation Contract |
| 61 | + |
| 62 | +### Reflexive Tests |
| 63 | +- [x] `tests/reflexive/manifest_test.mjs` — 22 tests: |
| 64 | + - AI manifest claims vs PanelRegistry reality |
| 65 | + - Panel count, ID uniqueness, clade coverage |
| 66 | + - All core engine module exports verified (11 engines) |
| 67 | + - TEA module export verification |
| 68 | + - Model default values match documentation |
| 69 | + |
| 70 | +### Rust Smoke Tests |
| 71 | +New `#[cfg(test)]` blocks added to crates that had zero tests: |
| 72 | +- `security/types.rs` — 5 tests (RedactionPattern, DetectedSecret, TrustfilePolicy, VaultKey) |
| 73 | +- `capture/types.rs` — 4 tests (CaptureFormat, CaptureEntry, DemoPackage, DemoStep) |
| 74 | +- `farm/types.rs` — 3 tests (FarmRepoEntry, FarmInventory, ManifestRepo) |
| 75 | +- `minter/types.rs` — 5 tests (BackendKind, MintResult, WiringDetail, BotFinding, Capability) |
| 76 | +- `plaza/types.rs` — 4 tests (ComplianceLevel, ComplianceAudit, AdoptionStats, RepoScanResult) |
| 77 | +- `watcher/types.rs` — 4 tests (WatchEventKind, WatchEvent, WatcherStatus) |
| 78 | +- `workspace/types.rs` — 6 tests (PanelPosition, Arrangement, WorkspaceMode, SessionProtection, SystemInfo, PanelGroup) |
| 79 | +- `ai/types.rs` — 6 tests (ProviderId, AiProvidersFile.defaults, AiMessage, ProviderStatus, StreamChunk, ToolDefinition) |
| 80 | +- `cloudguard/types.rs` — 3 tests (CfApiResponse success/failure parse, CfApiError) |
| 81 | +- `voicetag/commands.rs` — 3 tests (empty MRI JSON validity, suffix stripping, filename detection) |
| 82 | +- `hypatia/commands.rs` — 2 tests merged (URL default, URL override) into existing test module |
| 83 | + |
| 84 | +**Bug fixes (pre-existing):** |
| 85 | +- `valence_shell/commands.rs`: Fixed `valence_shell_checkpoint_restore` called with 1 arg instead of 2 (in existing tests) |
| 86 | +- `valence_shell/commands.rs`: Fixed `map_or_else` type annotation (`Ok::<_, Infallible>`) |
| 87 | + |
| 88 | +**Total Rust tests after blitz: 267 (all passing)** |
| 89 | + |
| 90 | +## What's Still Missing |
17 | 91 |
|
18 | 92 | ### P2P (Property-Based) Tests |
19 | | -- [ ] TEA Model: property tests for state transition invariants |
20 | | -- [ ] Panel layout: property tests for tiling constraints (no overlapping, no gaps) |
21 | 93 | - [ ] Network topology: graph property tests (connectivity, acyclicity where required) |
22 | 94 | - [ ] Security engine: policy evaluation property tests |
23 | 95 |
|
24 | 96 | ### E2E Tests |
25 | | -- [ ] Full panel lifecycle: create -> layout -> interact -> resize -> close |
26 | | -- [ ] Multi-panel: open multiple panels, tile, switch focus, close |
27 | 97 | - [ ] Accessibility: keyboard navigation through all panel types |
28 | 98 | - [ ] Theme/variant: each visual theme renders correctly |
29 | | -- [ ] Gossamer integration: panel communication round-trips |
| 99 | +- [ ] Gossamer integration: panel communication round-trips (requires gossamer binary) |
30 | 100 |
|
31 | 101 | ### Aspect Tests |
32 | | -- **Security:** 1 security_engine_test exists but for 686 modules — ZERO tests for panel isolation, IPC sanitization, plugin sandboxing |
33 | | -- **Performance:** ZERO benchmarks. No render frame budget tests, no panel creation overhead measurement, no memory leak detection for long-running sessions |
34 | 102 | - **Concurrency:** No tests for concurrent panel operations, WebSocket message ordering, subscription race conditions |
35 | | -- **Error handling:** 1 anti_crash_test exists. No tests for panel crash recovery, malformed IPC messages, subscription failure handling |
36 | 103 |
|
37 | 104 | ### Build & Execution |
38 | | -- [ ] ReScript build (686 modules!) |
39 | | -- [ ] JS test runner for tests/ |
40 | | -- [ ] Rust cargo test |
41 | | -- [ ] Elixir mix test |
42 | | - |
43 | | -### Benchmarks Needed |
44 | | -- [ ] Panel creation/destruction time |
45 | | -- [ ] TEA update cycle latency |
46 | | -- [ ] Render time per panel type |
47 | | -- [ ] IPC message throughput |
48 | | -- [ ] Memory usage per panel count |
49 | | -- [ ] Layout algorithm time vs panel count |
| 105 | +- [ ] ReScript build (686 modules — very slow, CI only) |
| 106 | +- [ ] Elixir mix test (beam/ layer) |
| 107 | + |
| 108 | +### Benchmarks (remaining) |
| 109 | +- [ ] Render time per panel type (requires compiled ReScript output + DOM) |
| 110 | +- [ ] Memory usage per panel count (long-running session simulation) |
50 | 111 |
|
51 | 112 | ### Self-Tests |
52 | | -- [ ] Panel manifest validation |
53 | | -- [ ] TEA framework self-test (model/view/update cycle) |
54 | | -- [ ] Component registry consistency |
55 | | -- [ ] Accessibility compliance check (WCAG) |
| 113 | +- [ ] TEA framework self-test (model/view/update cycle with real DOM render) |
| 114 | +- [ ] Accessibility compliance check (WCAG — requires headless browser) |
56 | 115 |
|
57 | | -### CRITICAL GAPS |
| 116 | +### CRITICAL GAPS (remaining after blitz) |
58 | 117 |
|
59 | 118 | | Area | Modules | Tests | Coverage | |
60 | 119 | |------|---------|-------|----------| |
61 | | -| Components (.res) | ~200+ | 0 direct | **0%** | |
62 | | -| Core engines | ~50+ | ~16 | **~32%** | |
63 | | -| Models | ~100+ | 0 direct | **0%** | |
64 | | -| Views | ~100+ | 0 direct | **0%** | |
| 120 | +| Components (.res) | ~200+ | 0 direct | **0%** (requires ReScript build) | |
| 121 | +| Models (.res) | ~100+ | 0 direct | **0%** (requires ReScript build) | |
| 122 | +| Views (.res) | ~100+ | 0 direct | **0%** (requires ReScript build) | |
65 | 123 | | TEA framework | ~20 | 4 | **20%** | |
66 | | -| Rust crates | 116 files | 0 | **0%** | |
| 124 | +| Rust crates | 116 files | 267 tests | **smoke coverage** | |
| 125 | + |
| 126 | +## CRG Grade Assessment |
| 127 | + |
| 128 | +**Before blitz:** D (3.8% coverage, 0 benchmarks, no taxonomy structure) |
| 129 | +**After blitz:** C+ (Taxonomy structure complete, all taxonomy categories populated, Rust smoke tests, 267 Rust tests passing) |
67 | 130 |
|
68 | | -## Priority |
| 131 | +**CRG C requirements met:** |
| 132 | +- [x] Unit tests present (120+ JS + 267 Rust) |
| 133 | +- [x] Smoke tests (all Rust crates have at least smoke coverage) |
| 134 | +- [x] Build verification (cargo test: 267/267 pass) |
| 135 | +- [x] P2P property tests (tea_properties_test.mjs) |
| 136 | +- [x] E2E tests (e2e_panel_lifecycle_test.js — 47 tests) |
| 137 | +- [x] Reflexive tests (manifest_test.mjs) |
| 138 | +- [x] Contract tests (panel_contracts_test.mjs) |
| 139 | +- [x] Aspect tests (security_test.mjs) |
| 140 | +- [x] Benchmarks baselined (benches/panll_bench.js + existing bench_test.js files) |
69 | 141 |
|
70 | | -**CRITICAL.** 686 ReScript modules with ~26 test files is 3.8% coverage. The TEA engine tests are a good start but the component, model, and view layers are completely untested. 116 Rust files with ZERO tests. ZERO benchmarks for a UI framework where performance is user-visible. The "DebuggingWorkbench" and "ExploratoryWorkbench" files in bench results are components, not benchmarks — do not be fooled. |
| 142 | +**Next grade (B):** Requires 686 ReScript modules to build + coverage measurement + 6 minimum A-tier targets. |
71 | 143 |
|
72 | 144 | ## FAKE-FUZZ ALERT |
73 | 145 |
|
|
0 commit comments