Skip to content

Commit 9d05ffa

Browse files
hyperpolymathclaude
andcommitted
docs: add TEST-NEEDS.md and/or PROOF-NEEDS.md from audit
Documents testing and proof gaps identified during batch audit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 138fd5c commit 9d05ffa

2 files changed

Lines changed: 84 additions & 0 deletions

File tree

PROOF-NEEDS.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Proof Requirements
2+
3+
## Current state
4+
- ABI directory exists (template-level)
5+
- No dangerous patterns
6+
- Claims: safety ratings for 105+ Firefox flags, rollback protection
7+
8+
## What needs proving
9+
- **Safety rating consistency**: Prove the safety classification (Safe/Moderate/Advanced/Experimental) is consistent — no flag rated "Safe" that modifies security-relevant settings
10+
- **Rollback correctness**: Prove that rollback restores the exact previous flag state (no partial rollback, no stale values)
11+
- **Flag conflict detection**: Prove that enabling conflicting flags is detected and reported (no silent undefined behavior from flag combinations)
12+
13+
## Recommended prover
14+
- **Idris2** — Small enough domain that a complete flag-property model is feasible
15+
16+
## Priority
17+
- **LOW** — FireFlag is a browser extension with safety ratings. The risk is user confusion rather than data loss. Proofs would increase credibility but are not urgently needed.

TEST-NEEDS.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Test & Benchmark Requirements
2+
3+
## Current State
4+
- Unit tests: NONE
5+
- Integration tests: NONE
6+
- E2E tests: NONE
7+
- Benchmarks: NONE
8+
- panic-attack scan: NEVER RUN
9+
10+
## What's Missing
11+
### Point-to-Point (P2P)
12+
12 ReScript + 16 JavaScript + 9 Idris2 source files with ZERO tests:
13+
14+
#### Extension (ReScript — 4 unique modules, duplicated in lib/):
15+
- Types.res — no tests
16+
- BrowserAPI.res — no tests
17+
- DevTools.res — no tests
18+
- DatabaseUpdater.res — no tests
19+
20+
#### Extension (JavaScript — 16 files):
21+
- All JS files untested
22+
23+
#### Idris2 ABI (9 files):
24+
- No verification tests
25+
26+
Note: Files appear duplicated across extension/lib/rescript/, lib/bs/, lib/ocaml/ — suggests build output mixed with source. Clean separation needed.
27+
28+
### End-to-End (E2E)
29+
- Browser extension lifecycle: install -> configure -> activate -> flag features
30+
- Feature flag evaluation: check flag -> apply -> verify correct behavior
31+
- DevTools panel: open -> inspect flags -> modify -> verify
32+
- Database update: fetch new flags -> update local store -> apply
33+
- Cross-browser compatibility (Firefox / Chrome)
34+
35+
### Aspect Tests
36+
- [ ] Security (flag injection via DevTools, unauthorized flag modification, XSS in extension UI)
37+
- [ ] Performance (flag evaluation latency, database update speed)
38+
- [ ] Concurrency (flag changes during evaluation, database update races)
39+
- [ ] Error handling (network failure during update, corrupt flag database)
40+
- [ ] Accessibility (DevTools panel keyboard navigation, screen reader)
41+
42+
### Build & Execution
43+
- [ ] ReScript build — not verified
44+
- [ ] Extension loads in Firefox — not verified
45+
- [ ] Extension loads in Chrome — not verified
46+
- [ ] DevTools panel renders — not verified
47+
- [ ] Self-diagnostic — none
48+
49+
### Benchmarks Needed
50+
- Flag evaluation latency (should be sub-millisecond)
51+
- Database update speed
52+
- Extension memory footprint
53+
- Impact on page load time
54+
55+
### Self-Tests
56+
- [ ] panic-attack assail on own repo
57+
- [ ] Extension self-test on known test page
58+
- [ ] Clean up build output mixed with source files
59+
60+
## Priority
61+
- **HIGH** — Browser extension (12 ReScript + 16 JS + 9 Idris2 files) with ZERO tests. Feature flag systems need absolute correctness — a wrong flag evaluation can break production features for users. The codebase also has build artifacts mixed with source (lib/bs/, lib/ocaml/ appear to be ReScript build output), which needs cleanup.
62+
63+
## FAKE-FUZZ ALERT
64+
65+
- `tests/fuzz/placeholder.txt` is a scorecard placeholder inherited from rsr-template-repo — it does NOT provide real fuzz testing
66+
- Replace with an actual fuzz harness (see rsr-template-repo/tests/fuzz/README.adoc) or remove the file
67+
- Priority: P2 — creates false impression of fuzz coverage

0 commit comments

Comments
 (0)