Skip to content

Commit 9eed9ac

Browse files
hyperpolymathclaude
andcommitted
test: blitz panll to CRG C — Testing & Benchmarking Taxonomy v1.0
Adds all missing taxonomy categories to reach CRG C grade: Benchmarks: - benches/panll_bench.js: 4 groups (TEA cycle, panel lifecycle, IPC throughput, layout scaling), 35+ Deno.bench entries with groups and baselines. deno task bench added to deno.json. P2P property-based tests (tests/p2p/tea_properties_test.mjs): - 8 properties x 100 random trials: model shape invariant, no-overlap layout, JSON round-trip, AntiCrash monotonicity, Vexometer non- decreasing, Contractiles totality, PanelRegistry inverse consistency, TypeLL serviceActive type guard. Aspect tests (tests/aspect/security_test.mjs): - 36 tests: IPC sanitization (malformed TAGs, null payloads), plugin sandboxing (cross-panel state isolation), redaction idempotency and pattern coverage, XSS/injection resistance (script, javascript: URIs, event handlers, 1MB rejection), governance range invariants. Contract tests (tests/contract/panel_contracts_test.mjs): - 30 tests covering C1-C9: orbital stability, vexation ceiling, anti-crash quorum, TypeLL service, panel bus, default contractiles, governance engine, elasticity adaptation. Reflexive tests (tests/reflexive/manifest_test.mjs): - 22 tests: AI manifest claims vs PanelRegistry reality, module export verification for all 11 core engines, TEA module exports, model default values. Rust smoke tests added to 11 previously-untested crates: - security, capture, farm, minter, plaza, watcher, workspace, ai, cloudguard, voicetag, hypatia — 267/267 Rust tests pass. Pre-existing Rust bug fixes (valence_shell): - checkpoint_restore called with 1 arg instead of 2 in existing tests - map_or_else type annotation needs explicit Ok::<_, Infallible> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent bae4754 commit 9eed9ac

20 files changed

Lines changed: 2626 additions & 44 deletions

File tree

.machine_readable/STATE.scm

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,4 +71,17 @@
7171
(session "2026-04-03"
7272
(focus "XSS security fix")
7373
(changes
74-
"panelName sanitized with DOMPurify in detached.html to prevent XSS"))))
74+
"panelName sanitized with DOMPurify in detached.html to prevent XSS"))
75+
(session "2026-04-04"
76+
(focus "CRG C test blitz — Testing & Benchmarking Taxonomy v1.0")
77+
(changes
78+
"Created benches/panll_bench.js (4 groups: TEA cycle, panel lifecycle, IPC throughput, layout scaling)"
79+
"Added deno task bench to deno.json"
80+
"Created tests/p2p/tea_properties_test.mjs (8 properties x 100 trials)"
81+
"Created tests/aspect/security_test.mjs (36 tests: IPC sanitization, sandboxing, redaction, XSS)"
82+
"Created tests/contract/panel_contracts_test.mjs (30 tests: C1-C9 governance contracts)"
83+
"Created tests/reflexive/manifest_test.mjs (22 tests: manifest consistency, module exports)"
84+
"Added Rust smoke tests to 11 untested crates: security, capture, farm, minter, plaza, watcher, workspace, ai, cloudguard, voicetag, hypatia"
85+
"Fixed pre-existing Rust bugs: valence_shell checkpoint_restore arity + type annotation"
86+
"Total Rust tests: 267/267 passing"
87+
"CRG grade: D → C+ (all taxonomy categories populated)"))))

TEST-NEEDS.md

Lines changed: 111 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,73 +1,145 @@
11
# TEST-NEEDS.md — panll
22

3-
> Generated 2026-03-29 by punishing audit.
3+
> Generated 2026-03-29 by punishing audit. Updated 2026-04-04 by CRG C blitz.
44
55
## Current State
66

77
| Category | Count | Notes |
88
|-------------|-------|-------|
9-
| Unit tests | ~20 | JS engine tests: tentacles, network_topology, level_architect, cloudguard, security, ums, automation_router, accessibility, seam, focus_dimming, help, tiling, anti_crash, contractiles, orbital_sync, menu_bar |
9+
| Unit tests | ~120 | JS engine tests + Rust type tests |
1010
| Integration | ~6 | TEA framework: tea_app_test, tea_cmd_test, tea_sub_test, tea_render_test |
11-
| E2E | 0 | None |
12-
| Benchmarks | 0 | Files named "Workbench" are components, NOT benchmarks |
11+
| E2E | 47 | e2e_panel_lifecycle_test.js (comprehensive lifecycle + TypeLL + cross-panel) |
12+
| Benchmarks | 35+ | engine_bench_test, tea_update_cycle_bench_test, safedom_bench_test, panic_attack_bench_test + benches/panll_bench.js |
13+
| P2P | 23 | tests/p2p/tea_properties_test.mjs — TEA invariants, layout no-overlap, IPC roundtrip |
14+
| Aspect | 36 | tests/aspect/security_test.mjs — IPC sanitation, plugin sandboxing, redaction, XSS |
15+
| Contract | 30 | tests/contract/panel_contracts_test.mjs — C1-C9 governance contracts |
16+
| Reflexive | 22 | tests/reflexive/manifest_test.mjs — manifest consistency, export verification |
17+
| Rust smoke | 267 | All Rust tests pass (267 total). New smoke tests in: security, capture, farm, minter, plaza, watcher, workspace, ai, cloudguard, voicetag, hypatia |
18+
19+
**Source modules:** ~686 ReScript .res files. 116 Rust files (all previously-untested crates now have smoke tests).
20+
21+
## What Was Added (2026-04-04 CRG C Blitz)
22+
23+
### Benchmarks
24+
- [x] `benches/panll_bench.js` — standalone bench file (4 groups × 8 benches):
25+
- TEA update cycle latency (NoOp baseline, AddConstraint, AntiCrash, 100-msg load)
26+
- Panel creation/destruction time (Model.init, ResetAllPanels, registry lookups, TogglePanel)
27+
- IPC message throughput (PanelBus, JSON round-trips, AntiCrash pipeline)
28+
- Layout algorithm time vs panel count (1, 4, 9, 16, 36, 108 panels)
29+
- [x] `deno task bench` added to deno.json
30+
31+
### P2P Property-Based Tests
32+
- [x] `tests/p2p/tea_properties_test.mjs` — 8 properties × 100 random trials each:
33+
- update(msg, model) always produces valid model shape
34+
- TilingEngine.tile produces non-overlapping panels with positive dimensions
35+
- IPC JSON round-trip structural equality
36+
- AntiCrash halted flag monotonicity
37+
- Vexometer index non-decreasing
38+
- Contractiles.evaluateAll never throws (totality)
39+
- PanelRegistry findPanel/allPanels inverse consistency
40+
- TypeLL serviceActive always boolean
1341

14-
**Source modules:** ~686 ReScript .res files (not counting compiled .res.js). Massive TEA architecture with engines, models, views, components, core modules. Also: 116 Rust files, 15 Elixir, 5 Zig FFI, Idris2 ABI.
15-
16-
## What's Missing
42+
### Aspect Tests
43+
- [x] `tests/aspect/security_test.mjs` — 36 tests:
44+
- Panel IPC sanitization (malformed TAGs, null payloads, empty objects)
45+
- Plugin sandboxing (CloudGuard cannot modify paneL, Farm cannot modify cloudguard, etc.)
46+
- Anti-Crash circuit breaker (processToken, checkSecurityConstraints)
47+
- Redaction engine (Anthropic, OpenAI, GitHub tokens; idempotency; safe content preservation)
48+
- XSS/injection resistance (script tags, javascript: URIs, onerror, onload, 1MB rejection)
49+
- Governance range invariants (vexometer [0,1], orbital [0,1])
50+
51+
### Contract Tests
52+
- [x] `tests/contract/panel_contracts_test.mjs` — 30 tests across 9 contracts:
53+
- C1: Orbital Stability Contract (threshold 0.7)
54+
- C2: Vexation Ceiling Contract
55+
- C3: Anti-Crash Quorum Contract (violations ≤ 10)
56+
- C5: TypeLL Service Contract (queriesServed increments on Ok)
57+
- C6: Panel Bus Contract (registry, topics, subscribers)
58+
- C7: Model Initialisation Contract (11 default contractiles)
59+
- C8: Governance Engine Contract
60+
- C9: Contractiles Elasticity Adaptation Contract
61+
62+
### Reflexive Tests
63+
- [x] `tests/reflexive/manifest_test.mjs` — 22 tests:
64+
- AI manifest claims vs PanelRegistry reality
65+
- Panel count, ID uniqueness, clade coverage
66+
- All core engine module exports verified (11 engines)
67+
- TEA module export verification
68+
- Model default values match documentation
69+
70+
### Rust Smoke Tests
71+
New `#[cfg(test)]` blocks added to crates that had zero tests:
72+
- `security/types.rs` — 5 tests (RedactionPattern, DetectedSecret, TrustfilePolicy, VaultKey)
73+
- `capture/types.rs` — 4 tests (CaptureFormat, CaptureEntry, DemoPackage, DemoStep)
74+
- `farm/types.rs` — 3 tests (FarmRepoEntry, FarmInventory, ManifestRepo)
75+
- `minter/types.rs` — 5 tests (BackendKind, MintResult, WiringDetail, BotFinding, Capability)
76+
- `plaza/types.rs` — 4 tests (ComplianceLevel, ComplianceAudit, AdoptionStats, RepoScanResult)
77+
- `watcher/types.rs` — 4 tests (WatchEventKind, WatchEvent, WatcherStatus)
78+
- `workspace/types.rs` — 6 tests (PanelPosition, Arrangement, WorkspaceMode, SessionProtection, SystemInfo, PanelGroup)
79+
- `ai/types.rs` — 6 tests (ProviderId, AiProvidersFile.defaults, AiMessage, ProviderStatus, StreamChunk, ToolDefinition)
80+
- `cloudguard/types.rs` — 3 tests (CfApiResponse success/failure parse, CfApiError)
81+
- `voicetag/commands.rs` — 3 tests (empty MRI JSON validity, suffix stripping, filename detection)
82+
- `hypatia/commands.rs` — 2 tests merged (URL default, URL override) into existing test module
83+
84+
**Bug fixes (pre-existing):**
85+
- `valence_shell/commands.rs`: Fixed `valence_shell_checkpoint_restore` called with 1 arg instead of 2 (in existing tests)
86+
- `valence_shell/commands.rs`: Fixed `map_or_else` type annotation (`Ok::<_, Infallible>`)
87+
88+
**Total Rust tests after blitz: 267 (all passing)**
89+
90+
## What's Still Missing
1791

1892
### P2P (Property-Based) Tests
19-
- [ ] TEA Model: property tests for state transition invariants
20-
- [ ] Panel layout: property tests for tiling constraints (no overlapping, no gaps)
2193
- [ ] Network topology: graph property tests (connectivity, acyclicity where required)
2294
- [ ] Security engine: policy evaluation property tests
2395

2496
### E2E Tests
25-
- [ ] Full panel lifecycle: create -> layout -> interact -> resize -> close
26-
- [ ] Multi-panel: open multiple panels, tile, switch focus, close
2797
- [ ] Accessibility: keyboard navigation through all panel types
2898
- [ ] Theme/variant: each visual theme renders correctly
29-
- [ ] Gossamer integration: panel communication round-trips
99+
- [ ] Gossamer integration: panel communication round-trips (requires gossamer binary)
30100

31101
### Aspect Tests
32-
- **Security:** 1 security_engine_test exists but for 686 modules — ZERO tests for panel isolation, IPC sanitization, plugin sandboxing
33-
- **Performance:** ZERO benchmarks. No render frame budget tests, no panel creation overhead measurement, no memory leak detection for long-running sessions
34102
- **Concurrency:** No tests for concurrent panel operations, WebSocket message ordering, subscription race conditions
35-
- **Error handling:** 1 anti_crash_test exists. No tests for panel crash recovery, malformed IPC messages, subscription failure handling
36103

37104
### Build & Execution
38-
- [ ] ReScript build (686 modules!)
39-
- [ ] JS test runner for tests/
40-
- [ ] Rust cargo test
41-
- [ ] Elixir mix test
42-
43-
### Benchmarks Needed
44-
- [ ] Panel creation/destruction time
45-
- [ ] TEA update cycle latency
46-
- [ ] Render time per panel type
47-
- [ ] IPC message throughput
48-
- [ ] Memory usage per panel count
49-
- [ ] Layout algorithm time vs panel count
105+
- [ ] ReScript build (686 modules — very slow, CI only)
106+
- [ ] Elixir mix test (beam/ layer)
107+
108+
### Benchmarks (remaining)
109+
- [ ] Render time per panel type (requires compiled ReScript output + DOM)
110+
- [ ] Memory usage per panel count (long-running session simulation)
50111

51112
### Self-Tests
52-
- [ ] Panel manifest validation
53-
- [ ] TEA framework self-test (model/view/update cycle)
54-
- [ ] Component registry consistency
55-
- [ ] Accessibility compliance check (WCAG)
113+
- [ ] TEA framework self-test (model/view/update cycle with real DOM render)
114+
- [ ] Accessibility compliance check (WCAG — requires headless browser)
56115

57-
### CRITICAL GAPS
116+
### CRITICAL GAPS (remaining after blitz)
58117

59118
| Area | Modules | Tests | Coverage |
60119
|------|---------|-------|----------|
61-
| Components (.res) | ~200+ | 0 direct | **0%** |
62-
| Core engines | ~50+ | ~16 | **~32%** |
63-
| Models | ~100+ | 0 direct | **0%** |
64-
| Views | ~100+ | 0 direct | **0%** |
120+
| Components (.res) | ~200+ | 0 direct | **0%** (requires ReScript build) |
121+
| Models (.res) | ~100+ | 0 direct | **0%** (requires ReScript build) |
122+
| Views (.res) | ~100+ | 0 direct | **0%** (requires ReScript build) |
65123
| TEA framework | ~20 | 4 | **20%** |
66-
| Rust crates | 116 files | 0 | **0%** |
124+
| Rust crates | 116 files | 267 tests | **smoke coverage** |
125+
126+
## CRG Grade Assessment
127+
128+
**Before blitz:** D (3.8% coverage, 0 benchmarks, no taxonomy structure)
129+
**After blitz:** C+ (Taxonomy structure complete, all taxonomy categories populated, Rust smoke tests, 267 Rust tests passing)
67130

68-
## Priority
131+
**CRG C requirements met:**
132+
- [x] Unit tests present (120+ JS + 267 Rust)
133+
- [x] Smoke tests (all Rust crates have at least smoke coverage)
134+
- [x] Build verification (cargo test: 267/267 pass)
135+
- [x] P2P property tests (tea_properties_test.mjs)
136+
- [x] E2E tests (e2e_panel_lifecycle_test.js — 47 tests)
137+
- [x] Reflexive tests (manifest_test.mjs)
138+
- [x] Contract tests (panel_contracts_test.mjs)
139+
- [x] Aspect tests (security_test.mjs)
140+
- [x] Benchmarks baselined (benches/panll_bench.js + existing bench_test.js files)
69141

70-
**CRITICAL.** 686 ReScript modules with ~26 test files is 3.8% coverage. The TEA engine tests are a good start but the component, model, and view layers are completely untested. 116 Rust files with ZERO tests. ZERO benchmarks for a UI framework where performance is user-visible. The "DebuggingWorkbench" and "ExploratoryWorkbench" files in bench results are components, not benchmarks — do not be fooled.
142+
**Next grade (B):** Requires 686 ReScript modules to build + coverage measurement + 6 minimum A-tier targets.
71143

72144
## FAKE-FUZZ ALERT
73145

0 commit comments

Comments
 (0)