Merge pull request #18 from SharedIntellect/fix/stale-content-cleanup

fix: stale content cleanup — 13 findings + remove stale external reviews

Author: dacervera

.clawhubignore

@@ -0,0 +1,8 @@
+reference-implementation/venv/
+reference-implementation/quorum.egg-info/
+golden-test-set/artifacts/
+golden-test-set/annotations/
+golden-test-set/scripts/
+.hypothesis/
+.pytest_cache/
+__pycache__/

SKILL.md

@@ -33,8 +33,8 @@ python -m quorum.cli run --target <path-to-artifact> --rubric <rubric-name>
 ### Depth Profiles
 
 - `quick` — 2 critics (correctness, completeness) + pre-screen, ~5-10 min
-- `standard` — 6 critics (+ security, code_hygiene, cross_consistency†, tester) + pre-screen, ~15-30 min (default)
-- `thorough` — all 6 shipped critics + pre-screen + fix loops, ~30-60 min
+- `standard` — 4 active (correctness, completeness, security + tester) + pre-screen, ~15-30 min (default)
+- `thorough` — 5 active (+ code_hygiene) + pre-screen + fix loops, ~30-60 min
 
 *†Cross-Consistency requires `--relationships` flag with a relationships manifest.*
 

SPEC.md

@@ -103,7 +103,7 @@ Three execution profiles balance rigor, speed, and cost:
 | Depth | Critics | Fix Loops | Runtime | Use Case |
 |-------|---------|-----------|---------|----------|
 | **quick** | Correctness, Completeness | 0 | 5-10 min | Fast feedback; low stakes |
-| **standard** | + Security, Code Hygiene | 0 | 15-30 min | Most work; default |
+| **standard** | + Security | 0 | 15-30 min | Most work; default |
 | **thorough** | All shipped critics | 1 (apply + re-verify) | 30-60 min | Critical decisions; production |
 
 Pre-screen (10 built-in checks + optional DevSkim SAST, §3.1) runs before LLM critics at all depth levels. Fix loops are implemented (Phase 1.5): the Fixer proposes text replacements for CRITICAL/HIGH findings, applies them, and re-runs critics to verify resolution. Full 9-critic panels are the roadmap target for thorough depth.
@@ -159,7 +159,7 @@ Supervisor (Orchestrator)
 ├─ Architecture Critic (Tier 2)     [SPECIFIED, not yet built]
 ├─ Delegation Critic (Tier 1)       [SPECIFIED, not yet built]
 ├─ Style Critic (Tier 2)            [SPECIFIED, not yet built]
-├─ Tester (Tier 2, tools: grep/web/exec) [SPECIFIED, not yet built]
+├─ Tester (Tier 2, tools: grep/web/exec) [IMPLEMENTED] — L1 deterministic + L2 LLM claim verification; see docs/critics/TESTER_CRITIC_BRIEF.md
 ├─ Fixer (Tier 1, optional)         [IMPLEMENTED — proposal mode + re-validation loops]
 ├─ Aggregator (Tier 1)              [IMPLEMENTED]
 └─ Supervisor (Tier 1, final)       [IMPLEMENTED]
@@ -386,7 +386,7 @@ This follows Tomasev's "trust as runtime primitive" principle.
 | Component | Cost | Amortization |
 |-----------|------|--------------|
 | Per-run setup (Supervisor intake) | $0.02 | 1 run |
-| 4 shipped critics (parallel, max 30min; 9 at full build-out) | $0.15-0.45 | 1 run |
+| 6 shipped critics (parallel, max 30min; 9 at full build-out) | $0.15-0.45 | 1 run |
 | Aggregator synthesis | $0.01 | 1 run |
 | Tester tools (grep, git, web, exec) | $0.00 | amortized |
 | Learning update (`known_issues.json`) | $0.00 | amortized |
@@ -398,7 +398,7 @@ Additional runs on related artifacts reuse critic prompts and tools, amortizing
 
 ## 8. Implementation Checklist
 
-Status as of v0.5.0 (reference implementation):
+Status as of v0.7.2 (reference implementation):
 
 - [x] LLM provider — LiteLLM universal provider (100+ models, any tier combination)
 - [x] File-based artifact passing (no in-memory state between agents)
@@ -411,7 +411,7 @@ Status as of v0.5.0 (reference implementation):
 - [x] Depth preset system (quick/standard/thorough YAML configs)
 - [x] Path traversal security (boundary enforcement)
 - [x] Exit codes (0/1/2)
-- [x] Parallel critic dispatch (ThreadPoolExecutor, max 4 critics; batch files max 3)
+- [x] Parallel critic dispatch (ThreadPoolExecutor, max 6 critics; batch files max 3)
 - [x] Fixer agent — proposal mode + re-validation loops (Phase 1.5; proposes and applies text replacements for CRITICAL/HIGH, then re-runs critics to verify)
 - [x] Python code rubric (25 criteria, PC-001–PC-025, auto-detects on .py files)
 - [x] Learning memory system (known_issues.json frequency tracking + mandatory check promotion — shipped in v0.5.3)
@@ -444,7 +444,7 @@ Quorum is built on these peer-reviewed papers:
 - Rubric panel is **static** (doesn't specialize per artifact type dynamically)
 - **No critic-to-critic debate** (relies on Aggregator to resolve conflicts)
 - Learning is **frequency-based** only (no semantic deduplication of patterns yet)
-- **Confidence calibration** is not yet implemented
+- **Confidence calibration** replaced by criteria coverage counts in v0.6.1
 - **Trust/monitoring system** is not yet implemented
 
 ### Planned
@@ -470,7 +470,7 @@ Quorum is built on these peer-reviewed papers:
 ---
 
 **Quorum is a production-oriented, early-stage validation framework.**  
-*Architecture is sound and tested, with 6 critics shipped (5 callable in the pipeline), learning memory, fix re-validation, and batch processing all production-ready. Trust and monitoring systems are specified but not yet wired.*
+*Architecture is sound and tested, with 6 critics shipped (all callable), learning memory, fix re-validation, batch processing, and the Tester verification layer all production-ready. Trust and monitoring systems are specified but not yet wired.*
 
 
 ---

docs/CHANGELOG.md

@@ -6,6 +6,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [0.7.2] — 2026-03-12
 
+### Documentation — Stale Content Cleanup (PR #18)
+
+- **External reviews removed**: 3 frontier model review files (GPT-5.4, Opus 4.6, Grok 4.20) and ideas-backlog.md deleted — reviews were run against v0.5.3 and no longer reflect current capabilities. `EXTERNAL_REVIEWS.md` replaced with community invitation template.
+- **13 stale content findings resolved** across SPEC.md, IMPLEMENTATION.md, CONFIG_REFERENCE.md, TUTORIAL.md, SKILL.md, RUBRIC_BUILDING_GUIDE.md, SECURITY_CRITIC_FRAMEWORK.md, CODE_HYGIENE_FRAMEWORK.md, TESTER_CRITIC_BRIEF.md, and INSTALLATION.md:
+  - Tester marked as shipped (was "SPECIFIED, not yet built")
+  - Critic counts updated to 6 shipped across all docs
+  - Learning memory marked as shipped (v0.5.3) across all docs
+  - Confidence scores replaced with coverage language (v0.6.1)
+  - Version headers updated to v0.7.2
+  - Depth profile critic counts aligned with critic-status.yaml
+  - Broken cross-reference paths fixed (TESTER_CRITIC_BRIEF, INSTALLATION)
+  - Security and Code Hygiene framework status checklists updated
+
 ### Documentation — Docs Restructure & README Rewrite (PR #16)
 
 - **README.md** rewritten: ~400 lines → ~120 lines. Focused on what Quorum is, how to run it, and where to go next. First-person voice preserved.
@@ -14,7 +27,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
   - `architecture/` — IMPLEMENTATION, THE_NINE
   - `critics/` — all critic-specific docs and frameworks
   - `guides/` — RUBRIC_BUILDING_GUIDE, CROSS_ARTIFACT_DESIGN
-  - `reviews/` — external review snapshots
+  - `reviews/` — community and practitioner evaluations
   - `configuration/` — CONFIG_REFERENCE
 - **New files:** `docs/getting-started/QUICK_START.md` (zero to running in <5 min), `docs/README.md` (navigation hub)
 - All cross-references updated across 30+ files

docs/README.md

@@ -53,7 +53,7 @@ Everything you need to use, configure, and extend Quorum.
 
 | Doc | Description |
 |-----|-------------|
-| [External Reviews](reviews/EXTERNAL_REVIEWS.md) | Independent evaluations of Quorum by other models |
+| [External Reviews](reviews/EXTERNAL_REVIEWS.md) | Community and practitioner evaluations |
 | [Changelog](CHANGELOG.md) | Version history and release notes |
 | [Code of Conduct](CODE_OF_CONDUCT.md) | Community guidelines |
 

docs/architecture/IMPLEMENTATION.md

@@ -2,7 +2,7 @@
 
 This guide walks through building Quorum from the architectural spec. It's structured as a reference walkthrough, not a copy-paste tutorial. The goal is understanding the patterns so you can adapt them to your stack.
 
-> **Implementation Status (v0.6.0):** This guide describes the full target architecture. The reference implementation ships 6 critics (5 callable in the pipeline), the fixer (proposal mode), parallel execution, batch processing, and pre-screen integration. Sections marked 🔜 describe planned components. See `critic-status.yaml` for the authoritative status matrix.
+> **Implementation Status (v0.7.2):** This guide describes the full target architecture. All 6 critics are shipped and callable: Correctness, Completeness, Security, Code Hygiene, Cross-Artifact Consistency, and Tester. Also shipped: the fixer (proposal mode), parallel execution, batch processing, and pre-screen integration. See `critic-status.yaml` for the authoritative status matrix. Sections below marked 🔜 describe components not yet built (Architecture, Delegation, Style).
 
 ---
 
@@ -127,7 +127,7 @@ Your job is assignment, coordination, and synthesis — not evaluation.
 
 ### 2.2 The Critics
 
-> ✅ **Shipped:** Correctness, Security, Completeness, Code Hygiene (+ Cross-Artifact Consistency in batch mode)
+> ✅ **Shipped:** Correctness, Completeness, Security, Code Hygiene, Cross-Artifact Consistency, Tester
 > 🔜 **Planned:** Architecture, Delegation & Coordination, Style
 
 Each critic receives:
@@ -196,9 +196,9 @@ The hardest agent to get right. Its job:
 
 ---
 
-## Phase 3: The Learning System 🔜
+## Phase 3: The Learning System
 
-> **Status:** Specified in architecture, not yet implemented. The mechanism below describes the target design.
+> **Status:** Shipped in v0.5.3. The `known_issues.json` system tracks failure patterns across runs with frequency-based promotion to mandatory checks.
 
 ### 3.1 Extracting Lessons
 
@@ -247,8 +247,8 @@ This is how past failures automatically improve future validation.
 ### 4.1 Quick (5-10 min)
 
 ```yaml
-critics: [correctness, security, completeness]
-tester: enabled
+critics: [correctness, completeness]
+tester: disabled
 fixer: disabled
 aggregator: simplified (no conflict resolution)
 fix_loops: 0
@@ -260,7 +260,7 @@ Use for: iterative development, fast feedback loops, low-stakes work.
 ### 4.2 Standard (15-30 min)
 
 ```yaml
-critics: [all 5 critics + tester]
+critics: [correctness, completeness, security + tester]
 fixer: enabled for CRITICAL only
 aggregator: full
 fix_loops: 1
@@ -272,7 +272,7 @@ Use for: most production work, configuration reviews, research validation.
 ### 4.3 Thorough (45-90 min)
 
 ```yaml
-critics: [all 5 critics + tester]
+critics: [correctness, completeness, security, code_hygiene + tester]
 fixer: enabled for CRITICAL + HIGH
 aggregator: full + external validator
 fix_loops: 2
@@ -282,6 +282,8 @@ cost_ceiling: $1.50
 
 Use for: critical decisions, pre-launch reviews, irreversible actions.
 
+**Note:** Cross-Artifact Consistency is a separate mode activated with the `--relationships` flag — it's additive at any depth, not part of the base critic panels above.
+
 ---
 
 ## Phase 5: Integration Patterns

docs/configuration/CONFIG_REFERENCE.md

@@ -1,12 +1,12 @@
 # Configuration Reference
 
-> **Implementation Status (v0.5.2):** This document describes both shipped and planned features. Items marked 🔜 are specified but not yet implemented.
+> **Implementation Status (v0.7.2):** This document describes both shipped and planned features. Items marked 🔜 are specified but not yet implemented.
 
 ## Depth Profiles
 
 | Profile | Critics | Fix Rounds | Time | Use Case | Status |
 |---------|---------|------------|------|----------|--------|
-| **quick** | Shipped critics (currently 4) | 0 | 5–15 min | Iteration, drafts | ✅ Shipped |
+| **quick** | Shipped critics (currently 2: correctness, completeness) | 0 | 5–15 min | Iteration, drafts | ✅ Shipped |
 | **standard** | Shipped critics + fixer | 1 on CRITICAL | 15–30 min | Pre-merge, reviews | ✅ Shipped |
 | **thorough** | All shipped critics + fixer | ≤2 on CRITICAL/HIGH | 30–60 min | Critical decisions, production | ✅ Shipped |
 | *standard (full)* | *6 (+ Architecture, Delegation, Style)* | *1 on CRITICAL* | *15–30 min* | *Pre-merge, reviews* | 🔜 Planned |
@@ -68,16 +68,14 @@ Rubrics are JSON files with the following structure:
 
 | Verdict | Meaning |
 |---------|---------|
-| **PASS** | No CRITICAL or HIGH issues; confidence ≥ 0.8 |
-| **PASS_WITH_NOTES** | No CRITICAL issues; ≤ 2 HIGH issues with mitigations; confidence ≥ 0.6 |
+| **PASS** | No CRITICAL or HIGH issues; coverage ≥ 80% of criteria evaluated |
+| **PASS_WITH_NOTES** | No CRITICAL issues; ≤ 2 HIGH issues with mitigations; coverage ≥ 60% of criteria evaluated |
 | **REVISE** | CRITICAL issues found but fixable; or > 2 HIGH issues |
 | **REJECT** | Fundamental architectural or security issues; requires redesign |
 
-## Known Issues / Learning Memory 🔜
+## Known Issues / Learning Memory
 
-> **Status:** Specified in architecture, not yet wired into production runs.
-
-The `known_issues.json` file will accumulate failure patterns across validation runs:
+> **Status:** Shipped in v0.5.3. The `known_issues.json` file accumulates failure patterns across validation runs:
 
 ```json
 {

docs/critics/CODE_HYGIENE_FRAMEWORK.md

@@ -21,19 +21,19 @@ The boundary is clean: when code hygiene detects a pattern that has security imp
 
 ## Status
 
-**v0.5.1 State:** Framework design & reference implementation with partial feature completion.
+**v0.7.2 State:** Framework design complete. All features shipped.
 
 - [x] Framework design and documentation
 - [x] 12 evaluation categories specified
 - [x] ISO/IEC 25010:2023 (Maintainability + Reliability) grounding
 - [x] Two-layer architecture (deterministic + LLM) with delegation boundaries
-- [ ] Full SAST tool integration (Ruff/Pylint) — Milestone #15, v0.5.2
-- [ ] Business logic validation workflow (SEC-02) — v0.5.3 planned
-- [ ] Learning memory wiring — v0.6+
+- [x] Full SAST tool integration (Ruff/Pylint) — shipped in pre-screen
+- [x] Business logic validation workflow (SEC-02) — shipped
+- [x] Learning memory wiring — shipped v0.5.3
 
 **Known Limitations:**
-- Pre-screen layer runs 10 regex checks; deterministic Python analyzer not fully integrated
-- Business logic checks require specification/requirements context (not yet automated)
+- Ruff and Bandit run in pre-screen; full deterministic coverage continues to expand
+- Business logic checks require specification/requirements context
 
 ---
 

docs/critics/SECURITY_CRITIC_FRAMEWORK.md

@@ -11,20 +11,19 @@
 
 ## Status
 
-**v0.5.1 State:** Framework design & reference implementation with partial feature completion.
+**v0.7.2 State:** Framework design complete. All features shipped.
 
 - [x] Framework design and documentation
 - [x] 14 evaluation categories (SEC-01–SEC-14) specified
 - [x] OWASP ASVS 5.0, CWE Top 25, NIST SP 800-53 SA-11 grounding
 - [x] Detection capability matrix for SAST vs LLM judgment boundaries
-- [ ] Full SAST tool integration (Ruff/Bandit/PSScriptAnalyzer) — Milestone #15, v0.5.2
-- [ ] Threat model context feeding for SEC-04 (Authorization) — schema defined in `quorum-relationships.example.yaml` (`threat_context` type); runtime integration v0.5.3 planned
-- [ ] Learning memory wiring (issue tracking) — v0.6+
+- [x] Full SAST tool integration (Ruff/Bandit/PSScriptAnalyzer) — shipped in pre-screen
+- [x] Threat model context feeding for SEC-04 (Authorization) — `threat_context` type in relationship manifest, runtime integration shipped
+- [x] Learning memory wiring (issue tracking) — shipped v0.5.3
 
 **Known Limitations:**
-- Pre-screen layer runs 10 regex checks; 80+ referenced SAST rules not yet integrated
-- Authorization review (SEC-04) is speculative without threat model context — users can now define `threat_context` in the relationship manifest (roles, trust boundaries, sensitive operations); runtime feeding to the critic is planned for v0.5.3
-- PowerShell coverage ~70% vs 85%+ for Python (tooling ecosystem gap)
+- SAST rule coverage varies by language: Python ~85%+, PowerShell ~70% (tooling ecosystem gap)
+- Authorization review (SEC-04) benefits from threat model context — users can define `threat_context` in the relationship manifest (roles, trust boundaries, sensitive operations)
 
 ---
 

docs/critics/TESTER_CRITIC_BRIEF.md

@@ -1,8 +1,8 @@
 # Tester Critic — Design Brief
 
-**Status:** Pre-spec. This document describes what to build, not how. A full SPEC section should be written from this brief before any code.
+**Status:** Shipped (v0.6.0). L1 deterministic + L2 LLM verification. Wired into pipeline as Phase 3 (v0.7.0). This document is the original design brief that guided implementation.
 
-**Why this matters:** Three independent external reviews (GPT-5.4, Grok 4.20 Heavy, Opus 4.6 Extended — 2026-03-09) converged on the same conclusion: the Tester critic is Quorum's single most important gap. Without it, evidence grounding is "an LLM judging another LLM's citations" — rhetorically deterministic, not actually deterministic.
+**Why this matters:** Three independent external reviews (2026-03-09) converged on the Tester as Quorum's most important gap. The Tester shipped in v0.6.0.
 
 **Design principle reminder:** Quorum is the gate, not the road.
 
@@ -30,19 +30,19 @@ It answers: "The Correctness critic says line 247 of cli.py has an unchecked ret
 - Open file, read the cited line(s)
 - Fuzzy match cited evidence against actual content
 - **Catches:** hallucinated line numbers, wrong file references, stale loci
-- **Ship in:** v0.6.0
+- **Shipped:** v0.6.0
 
 ### Level 2: Claim Verification (hybrid — deterministic + light LLM)
 - Read the code/text surrounding the locus
 - One focused LLM call: "Given this excerpt and this claim, is the claim accurate?"
 - **Catches:** mischaracterized code, overstated severity, correct line but wrong conclusion
-- **Ship in:** v0.6.0
+- **Shipped:** v0.6.0
 
 ### Level 3: Execution Verification (deterministic, sandboxed)
 - For executable claims (regex patterns, import checks, syntax validation): actually run them
 - For coverage claims: run coverage analysis
 - Requires sandboxing — significant security surface
-- **Ship in:** v0.7.0 (deferred, gated behind `--tester-execute`)
+- **Status:** Deferred, gated behind `--tester-execute`
 
 ## Output: Verification Tags
 
@@ -101,7 +101,6 @@ Before building the Tester itself:
 - ~3 focused sessions, not one marathon
 
 ## References
-- Opus 4.6 review: `docs/external-reviews/2026-03-09-opus-extended-review.md`
-- GPT-5.4 review: `docs/external-reviews/2026-03-09-gpt54-review.md`
+- Three independent external reviews (2026-03-09) converged on the Tester as Quorum's most important gap. The Tester shipped in v0.6.0.
 - Current critic architecture: `SPEC.md` §Critics, §Aggregator
 - Finding/Evidence/Locus models: `quorum/models.py`