Skip to content

Commit 4b9bd5a

Browse files
Akkariclaude
andcommitted
fix: Quorum v0.5.3 self-validation — correct stale docs and code bugs
learning.py: clean up .tmp file if atomic rename fails in save() tests: exclude surrogate chars (Cs) from Hypothesis unicode strategy SPEC.md: fix 8 stale contradictions — re-validation loops and learning memory both shipped in v0.5.3; update §2.5, §3.5, §8, §10, footer CHANGELOG.md: correct planned PyPI package name quorum-ai → quorum-validator in [0.5.2] and [0.5.1] roadmap entries Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 20f71f6 commit 4b9bd5a

4 files changed

Lines changed: 13 additions & 14 deletions

File tree

SPEC.md

Lines changed: 8 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -104,9 +104,9 @@ Three execution profiles balance rigor, speed, and cost:
104104
|-------|---------|-----------|---------|----------|
105105
| **quick** | Correctness, Completeness | 0 | 5-10 min | Fast feedback; low stakes |
106106
| **standard** | + Security, Code Hygiene | 0 | 15-30 min | Most work; default |
107-
| **thorough** | All shipped critics | 1 (proposal mode) | 30-60 min | Critical decisions; production |
107+
| **thorough** | All shipped critics | 1 (apply + re-verify) | 30-60 min | Critical decisions; production |
108108

109-
Pre-screen (10 built-in checks + optional DevSkim SAST, §3.1) runs before LLM critics at all depth levels. Fix loops are implemented (Phase 1.5 — proposal mode). The Fixer proposes text replacements for CRITICAL/HIGH findings when max_fix_loops > 0; re-validation loops (apply → re-run) are deferred. Full 9-critic panels are the roadmap target for thorough depth.
109+
Pre-screen (10 built-in checks + optional DevSkim SAST, §3.1) runs before LLM critics at all depth levels. Fix loops are implemented (Phase 1.5): the Fixer proposes text replacements for CRITICAL/HIGH findings, applies them, and re-runs critics to verify resolution. Full 9-critic panels are the roadmap target for thorough depth.
110110

111111
### 2.6 Transparency Over Convenience
112112

@@ -265,7 +265,7 @@ relationships:
265265
- **PASS_WITH_NOTES** — Issues found, all addressable, recommendations provided
266266
- **REVISE** — HIGH/CRITICAL issues require rework; Supervisor provides guidance
267267
- **REJECT** — Unfixable architectural problems; restart required
268-
8. **Learning** — System extracts and logs new failure patterns *(specified, not yet wired up)*
268+
8. **Learning** — System extracts and logs new failure patterns *(implemented in v0.5.3)*
269269

270270
### 3.6 File-Based Artifact Passing
271271

@@ -413,11 +413,10 @@ Status as of v0.5.0 (reference implementation):
413413
- [x] Path traversal security (boundary enforcement)
414414
- [x] Exit codes (0/1/2)
415415
- [x] Parallel critic dispatch (ThreadPoolExecutor, max 4 critics; batch files max 3)
416-
- [x] Fixer agent — proposal mode (Phase 1.5; proposes text replacements for CRITICAL/HIGH)
416+
- [x] Fixer agent — proposal mode + re-validation loops (Phase 1.5; proposes and applies text replacements for CRITICAL/HIGH, then re-runs critics to verify)
417417
- [x] Python code rubric (25 criteria, PC-001–PC-025, auto-detects on .py files)
418+
- [x] Learning memory system (known_issues.json frequency tracking + mandatory check promotion — shipped in v0.5.3)
418419
- [ ] Remaining 5 critics (Architecture, Delegation, Style, Tester, full Fixer)
419-
- [ ] Re-validation loops (apply fixes → re-run critics → verify)
420-
- [ ] Learning memory system (Issue model exists, not wired up)
421420
- [ ] Trust/monitoring system (per-critic accuracy tracking)
422421

423422
See IMPLEMENTATION.md for a reference walkthrough.
@@ -443,17 +442,16 @@ Quorum is built on these peer-reviewed papers:
443442
### Current Limitations (v3.0)
444443

445444
- Only **4 of 9 critics are implemented** (Architecture, Delegation, Style, Tester are specified but not built)
446-
- **Fix loops are proposal-mode only** — Fixer proposes text replacements; re-validation loops (apply → re-run) are not yet implemented
447-
- **Learning memory is not wired up** — the `Issue` model exists; pattern accumulation is not connected
448445
- Rubric panel is **static** (doesn't specialize per artifact type dynamically)
449446
- **No critic-to-critic debate** (relies on Aggregator to resolve conflicts)
450447
- Learning is **frequency-based** only (no semantic deduplication of patterns yet)
451448
- **Confidence calibration** is not yet implemented
449+
- **Trust/monitoring system** is not yet implemented
452450

453451
### Planned
454452

455453
- Remaining critics: Architecture, Delegation, Style, Tester
456-
- Re-validation loops — apply Fixer proposals, re-run critics, verify resolution
454+
- ~~Re-validation loops~~ → **Shipped in v0.5.3**
457455
- ~~Learning memory~~ → **Shipped in v0.5.3**
458456
- Dynamic critic specialization (spawn domain-specific critics on-demand)
459457
- Critic debate mode (when two critics conflict, run a structured debate)
@@ -473,7 +471,7 @@ Quorum is built on these peer-reviewed papers:
473471
---
474472

475473
**Quorum is a production-oriented, early-stage validation framework.**
476-
*Architecture is sound and tested, but the product is still growing: no test suite, no CI, 4/9 critics shipped, learning/trust/monitoring systems specified but not wired.*
474+
*Architecture is sound and tested, but the product is still growing: 4/9 critics shipped, trust/monitoring systems specified but not wired; learning memory and fix re-validation shipped in v0.5.3.*
477475

478476

479477
---

docs/CHANGELOG.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
132132
- [ ] Tester critic (Milestone #10)
133133
- [ ] Confidence calibration (Milestone #6b)
134134
- [ ] **Learning memory** (Milestone #7) — wire up known_issues.json frequency tracking + mandatory check promotion
135-
- [ ] **PyPI publish** (Milestone #14) — `pip install quorum-ai` instead of clone + install
135+
- [ ] **PyPI publish** (Milestone #14) — `pip install quorum-validator` instead of clone + install (package name finalized as `quorum-validator`)
136136
- [ ] Delegation critic (Milestone #11)
137137
- [ ] Style critic (Milestone #12)
138138
- [ ] Documentation headers adoption (Phase 1–4)
@@ -257,7 +257,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
257257
- [ ] Tester critic (Milestone #10)
258258
- [ ] Confidence calibration (Milestone #6b)
259259
- [ ] **Learning memory** (Milestone #7) — wire up known_issues.json frequency tracking + mandatory check promotion
260-
- [ ] **PyPI publish** (Milestone #14) — `pip install quorum-ai` instead of clone + install
260+
- [ ] **PyPI publish** (Milestone #14) — `pip install quorum-validator` instead of clone + install (package name finalized as `quorum-validator`)
261261
- [ ] Delegation critic (Milestone #11)
262262
- [ ] Style critic (Milestone #12)
263263
- [ ] Documentation headers adoption (Phase 1–4)

reference-implementation/quorum/learning.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,14 +64,15 @@ def load(self) -> list[Issue]:
6464

6565
def save(self, issues: list[Issue]) -> None:
6666
"""Write issues to JSON using atomic tmp+rename to prevent corruption."""
67+
tmp = self._path.with_suffix(".tmp")
6768
try:
6869
self._path.parent.mkdir(parents=True, exist_ok=True)
6970
data = [issue.to_dict() for issue in issues]
70-
tmp = self._path.with_suffix(".tmp")
7171
tmp.write_text(json.dumps(data, indent=2, default=str), encoding="utf-8")
7272
tmp.replace(self._path)
7373
logger.debug("Saved %d known issues to %s", len(issues), self._path)
7474
except Exception as e:
75+
tmp.unlink(missing_ok=True) # Clean up partial write on failure
7576
logger.warning("Failed to save learning memory to %s: %s", self._path, e)
7677

7778
def update_from_findings(

reference-implementation/tests/test_prescreen_properties.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ def test_prescreen_handles_any_yaml(self, text):
7878
finally:
7979
tmp.unlink(missing_ok=True)
8080

81-
@given(st.text(alphabet=st.characters(blacklist_categories=("Cc",)), max_size=3000))
81+
@given(st.text(alphabet=st.characters(blacklist_categories=("Cc", "Cs")), max_size=3000))
8282
@settings(max_examples=30, deadline=5000)
8383
def test_prescreen_unicode_no_crash(self, text):
8484
"""Unicode text must never cause a crash in any check."""

0 commit comments

Comments
 (0)