Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Improvements to docs, tutorials, and examples are always welcome. If something w

### 🔬 External Reviews

Run Quorum against your own system and share the results. See `docs/EXTERNAL_REVIEWS.md` for how to submit.
Run Quorum against your own system and share the results. See `docs/reviews/EXTERNAL_REVIEWS.md` for how to submit.

## Pull Request Process

Expand Down
247 changes: 57 additions & 190 deletions README.md

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions SHIPPING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ Run `python tools/validate-docs.py` to find stale references, then fix:
- [ ] `README.md` — "What's coming" section (remove shipped items)
- [ ] `SPEC.md` — status matrix / implementation checklist updated
- [ ] `SPEC.md` — shipped critic counts in prose
- [ ] `docs/FOR_BEGINNERS.md` — critic counts and cost estimates
- [ ] `docs/CONFIG_REFERENCE.md` — critic status markers
- [ ] `docs/TUTORIAL.md` — critic descriptions and status markers
- [ ] `docs/IMPLEMENTATION.md` — critic status markers and depth profiles
- [ ] `docs/getting-started/FOR_BEGINNERS.md` — critic counts and cost estimates
- [ ] `docs/configuration/CONFIG_REFERENCE.md` — critic status markers
- [ ] `docs/getting-started/TUTORIAL.md` — critic descriptions and status markers
- [ ] `docs/architecture/IMPLEMENTATION.md` — critic status markers and depth profiles
- [ ] `SKILL.md` — critic counts in depth descriptions

**⚠️ Note on Cross-Consistency:** Cross-Consistency is a shipped critic but requires the `--relationships` flag. When updating counts, use language like: "6 shipped critics (including Cross-Consistency, activated with `--relationships`)".
Expand Down
8 changes: 4 additions & 4 deletions SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: quorum
description: Multi-agent validation framework — 4 independent AI critics evaluate artifacts against rubrics with evidence-grounded findings.
description: Multi-agent validation framework — 6 independent AI critics evaluate artifacts against rubrics with evidence-grounded findings.
metadata: {"openclaw":{"requires":{"bins":["python3","pip"],"env":["ANTHROPIC_API_KEY","OPENAI_API_KEY"]},"install":[{"id":"clone-repo","kind":"shell","command":"git clone https://github.com/SharedIntellect/quorum.git /tmp/quorum-install && cd /tmp/quorum-install/reference-implementation && pip install -r requirements.txt","label":"Clone Quorum repo and install Python dependencies"}],"source":"https://github.com/SharedIntellect/quorum"}}
---

Expand Down Expand Up @@ -100,9 +100,9 @@ Each finding includes: severity (CRITICAL/HIGH/MEDIUM/LOW), evidence citations p
## More Information

- [SPEC.md](https://github.com/SharedIntellect/quorum/blob/main/SPEC.md) — Full architectural specification
- [MODEL_REQUIREMENTS.md](https://github.com/SharedIntellect/quorum/blob/main/docs/MODEL_REQUIREMENTS.md) — Supported models and tiers
- [CONFIG_REFERENCE.md](https://github.com/SharedIntellect/quorum/blob/main/docs/CONFIG_REFERENCE.md) — All configuration options
- [FOR_BEGINNERS.md](https://github.com/SharedIntellect/quorum/blob/main/docs/FOR_BEGINNERS.md) — New to agent validation? Start here
- [MODEL_REQUIREMENTS.md](https://github.com/SharedIntellect/quorum/blob/main/docs/getting-started/MODEL_REQUIREMENTS.md) — Supported models and tiers
- [CONFIG_REFERENCE.md](https://github.com/SharedIntellect/quorum/blob/main/docs/configuration/CONFIG_REFERENCE.md) — All configuration options
- [FOR_BEGINNERS.md](https://github.com/SharedIntellect/quorum/blob/main/docs/getting-started/FOR_BEGINNERS.md) — New to agent validation? Start here


---
Expand Down
8 changes: 4 additions & 4 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
**Version:** 3.0
**Last Updated:** March 2026
**Status:** Documented and partially implemented — see §3 for implementation status per component
**Platform:** Designed for [OpenClaw](https://openclaw.ai) agent systems. Cross-platform compatibility with other agent frameworks is under active exploration — see [MODEL_REQUIREMENTS.md](docs/MODEL_REQUIREMENTS.md) for supported models and platforms.
**Platform:** Designed for [OpenClaw](https://openclaw.ai) agent systems. Cross-platform compatibility with other agent frameworks is under active exploration — see [MODEL_REQUIREMENTS.md](docs/getting-started/MODEL_REQUIREMENTS.md) for supported models and platforms.

---

Expand Down Expand Up @@ -153,7 +153,7 @@ Pre-screen results are written to `prescreen.json` in the run directory. If any
Supervisor (Orchestrator)
├─ Correctness Critic (Tier 2) [IMPLEMENTED]
├─ Completeness Critic (Tier 2) [IMPLEMENTED]
├─ Security Critic (Tier 1) [IMPLEMENTED] — grounded: OWASP ASVS 5.0, CWE Top 25, NIST SA-11; see docs/SEC02_BUSINESS_LOGIC_VALIDATION.md for business logic workflow
├─ Security Critic (Tier 1) [IMPLEMENTED] — grounded: OWASP ASVS 5.0, CWE Top 25, NIST SA-11; see docs/critics/SEC02_BUSINESS_LOGIC_VALIDATION.md for business logic workflow
├─ Code Hygiene Critic (Tier 2) [IMPLEMENTED] — grounded: ISO 25010:2023, CISQ
├─ Cross-Artifact Consistency [IMPLEMENTED] — Phase 2, separate from BaseCritic
├─ Architecture Critic (Tier 2) [SPECIFIED, not yet built]
Expand Down Expand Up @@ -462,9 +462,9 @@ Quorum is built on these peer-reviewed papers:

## 11. Getting Started

1. Read [IMPLEMENTATION.md](docs/IMPLEMENTATION.md) for a reference walkthrough
1. Read [IMPLEMENTATION.md](docs/architecture/IMPLEMENTATION.md) for a reference walkthrough
2. Review [examples/](examples/) for your use case
3. Adapt built-in rubrics or build custom ones (see [RUBRIC_BUILDING_GUIDE.md](docs/RUBRIC_BUILDING_GUIDE.md))
3. Adapt built-in rubrics or build custom ones (see [RUBRIC_BUILDING_GUIDE.md](docs/guides/RUBRIC_BUILDING_GUIDE.md))
4. Run the tutorial: `quorum run --target examples/sample-research.md --depth quick`

---
Expand Down
2 changes: 1 addition & 1 deletion critic-status.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Used by tools/validate-docs.py to detect stale documentation.
# Update this file FIRST when shipping a critic or feature.

version: "0.7.0"
version: "0.7.2"
spec_version: "3.0"

critics:
Expand Down
23 changes: 23 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,29 @@ All notable changes to Quorum will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [0.7.2] — 2026-03-12

### Documentation — Docs Restructure & README Rewrite (PR #16)

- **README.md** rewritten: ~400 lines → ~120 lines. Focused on what Quorum is, how to run it, and where to go next. First-person voice preserved.
- **23 docs reorganized** into 6 categorized subdirectories under `docs/`:
- `getting-started/` — QUICK_START, INSTALLATION, TUTORIAL, FOR_BEGINNERS, MODEL_REQUIREMENTS
- `architecture/` — IMPLEMENTATION, THE_NINE
- `critics/` — all critic-specific docs and frameworks
- `guides/` — RUBRIC_BUILDING_GUIDE, CROSS_ARTIFACT_DESIGN
- `reviews/` — external review snapshots
- `configuration/` — CONFIG_REFERENCE
- **New files:** `docs/getting-started/QUICK_START.md` (zero to running in <5 min), `docs/README.md` (navigation hub)
- All cross-references updated across 30+ files
- `validate-docs.py` exclusion updated to match new `docs/reviews/` path

### Fixed
- 7 broken relative links caused by docs moving one directory deeper (CONFIG_REFERENCE, SPEC, CONTRIBUTING, reference-implementation paths)
- `SKILL.md` frontmatter: "4 independent AI critics" → "6" (matched shipped state)
- `README.md` version badge: v0.7.0 → v0.7.2

---

## [0.7.1] — 2026-03-12

### Fixed — Self-Validation Findings (PR #14)
Expand Down
14 changes: 7 additions & 7 deletions docs/DOCUMENTATION_STANDARDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Every source file gets a metadata block at the top, using `@KEY: value` pairs in
```python
# @module: quorum.critics.code_hygiene
# @purpose: Code quality evaluation grounded in ISO 25010 + CISQ
# @grounding: CODE_HYGIENE_FRAMEWORK.md
# @grounding: critics/CODE_HYGIENE_FRAMEWORK.md
# @owner: code-hygiene-critic
# @version: 0.3.0
# @tier: tier-2 (default model assignment)
Expand All @@ -34,7 +34,7 @@ Every source file gets a metadata block at the top, using `@KEY: value` pairs in

**Markdown:**
```markdown
<!-- @doc: CROSS_ARTIFACT_DESIGN.md -->
<!-- @doc: guides/CROSS_ARTIFACT_DESIGN.md -->
<!-- @purpose: Architectural decisions for cross-file consistency checking -->
<!-- @status: design-complete -->
<!-- @version: 1.0 -->
Expand All @@ -47,7 +47,7 @@ Every source file gets a metadata block at the top, using `@KEY: value` pairs in
# @config: quorum-relationships.yaml
# @purpose: Declared file relationships for cross-artifact consistency
# @schema-version: 1.0
# @see: CROSS_ARTIFACT_DESIGN.md
# @see: guides/CROSS_ARTIFACT_DESIGN.md
```

**Shell:**
Expand Down Expand Up @@ -199,9 +199,9 @@ This enables a future pre-screen check (`PS-011: Header validation`) that mechan
- [ ] `prescreen.py`

### Phase 3: Framework & Design Docs
- [ ] `CODE_HYGIENE_FRAMEWORK.md`
- [ ] `SECURITY_CRITIC_FRAMEWORK.md`
- [ ] `CROSS_ARTIFACT_DESIGN.md`
- [ ] `critics/CODE_HYGIENE_FRAMEWORK.md`
- [ ] `critics/SECURITY_CRITIC_FRAMEWORK.md`
- [ ] `guides/CROSS_ARTIFACT_DESIGN.md`
- [ ] `SPEC.md`
- [ ] `CHANGELOG.md`

Expand All @@ -219,7 +219,7 @@ This enables a future pre-screen check (`PS-011: Header validation`) that mechan
| Pipeline & CLI | `docs/PIPELINE_REFERENCE.md` | High — users need this |
| Pre-Screen System | `docs/PRESCREEN_REFERENCE.md` | High — 10 checks need documentation |
| Critic Framework | `docs/CRITIC_ARCHITECTURE.md` | Medium — developers extending Quorum |
| Rubric System | `docs/RUBRIC_BUILDING_GUIDE.md` | **Already exists** ✅ |
| Rubric System | `docs/guides/RUBRIC_BUILDING_GUIDE.md` | **Already exists** ✅ |
| Cross-Artifact | `docs/CROSS_ARTIFACT_DESIGN.md` | **Already exists** ✅ |

---
Expand Down
97 changes: 71 additions & 26 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,73 @@
# Quorum Documentation Index
# Quorum Documentation

Everything you need to use, configure, and extend Quorum.

---

## Getting Started
- **[INSTALLATION.md](INSTALLATION.md)** — Setup, dependencies, and first run
- **[TUTORIAL.md](TUTORIAL.md)** — Step-by-step walkthrough
- **[FOR_BEGINNERS.md](FOR_BEGINNERS.md)** — What Quorum does and why, for newcomers
- **[CONFIG_REFERENCE.md](CONFIG_REFERENCE.md)** — Depth profiles, model tiers, and all configuration options

## Architecture & Design
- **[THE_NINE.md](THE_NINE.md)** — The nine-agent target architecture
- **[IMPLEMENTATION.md](IMPLEMENTATION.md)** — Current implementation status and technical details
- **[MODEL_REQUIREMENTS.md](MODEL_REQUIREMENTS.md)** — Why Quorum needs capable models and what "Tier 1/Tier 2" means
- **[CROSS_ARTIFACT_DESIGN.md](CROSS_ARTIFACT_DESIGN.md)** — Phase 2 cross-artifact consistency validation

## Security & Code Quality Frameworks
- **[SECURITY_CRITIC_FRAMEWORK.md](SECURITY_CRITIC_FRAMEWORK.md)** — SEC-01 through SEC-14: the full security critic rule set
- **[SEC02_BUSINESS_LOGIC_VALIDATION.md](SEC02_BUSINESS_LOGIC_VALIDATION.md)** — Business logic validation workflow (requirements → critic path, threat_context integration)
- **[CODE_HYGIENE_FRAMEWORK.md](CODE_HYGIENE_FRAMEWORK.md)** — Code quality rules grounded in ISO 25010:2023 and CISQ
- **[POWERSHELL_COVERAGE.md](POWERSHELL_COVERAGE.md)** — Honest assessment of PowerShell SAST coverage (~70%) and landscape gaps

## Building Rubrics
- **[RUBRIC_BUILDING_GUIDE.md](RUBRIC_BUILDING_GUIDE.md)** — How to create domain-specific rubrics for Quorum

## Project
- **[CHANGELOG.md](CHANGELOG.md)** — Release history
- **[EXTERNAL_REVIEWS.md](EXTERNAL_REVIEWS.md)** — Independent evaluations of Quorum's architecture
- **[DOCUMENTATION_STANDARDS.md](DOCUMENTATION_STANDARDS.md)** — Standards for Quorum's own documentation
- **[CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)** — Community guidelines

| Doc | Description |
|-----|-------------|
| [Quick Start](getting-started/QUICK_START.md) | Your first validation in 60 seconds |
| [Installation](getting-started/INSTALLATION.md) | Detailed setup, dependencies, and troubleshooting |
| [For Beginners](getting-started/FOR_BEGINNERS.md) | New to AI agent tooling? Start here |
| [Tutorial](getting-started/TUTORIAL.md) | Walkthrough of a full validation workflow |
| [Model Requirements](getting-started/MODEL_REQUIREMENTS.md) | Which models work with Quorum and why |

## Configuration

| Doc | Description |
|-----|-------------|
| [Config Reference](configuration/CONFIG_REFERENCE.md) | Every config option, rubric format, and CLI flag |

## Guides

| Doc | Description |
|-----|-------------|
| [Rubric Building Guide](guides/RUBRIC_BUILDING_GUIDE.md) | How to build rubrics for new domains step by step |
| [Cross-Artifact Design](guides/CROSS_ARTIFACT_DESIGN.md) | Checking consistency between related files |

## Architecture

| Doc | Description |
|-----|-------------|
| [SPEC.md](../SPEC.md) | Full architectural specification |
| [The Nine Critics](architecture/THE_NINE.md) | The critic architecture — what each one does and why |
| [Implementation Notes](architecture/IMPLEMENTATION.md) | Technical decisions and internals |

## Critics — Deep Dives

| Doc | Description |
|-----|-------------|
| [Security Critic Framework](critics/SECURITY_CRITIC_FRAMEWORK.md) | OWASP ASVS 5.0, CWE Top 25, NIST SA-11 coverage |
| [Code Hygiene Framework](critics/CODE_HYGIENE_FRAMEWORK.md) | ISO 25010:2023, CISQ quality model |
| [Tester Critic Brief](critics/TESTER_CRITIC_BRIEF.md) | L1 deterministic + L2 LLM claim verification |
| [Business Logic Validation](critics/SEC02_BUSINESS_LOGIC_VALIDATION.md) | SEC-02 workflow for business logic review |
| [Evidence Integrity](critics/SEC03_EVIDENCE_INTEGRITY.md) | SEC-03 evidence grounding and integrity |
| [Pipeline Resilience](critics/SEC05_PIPELINE_RESILIENCE.md) | SEC-05 crash recovery and fault tolerance |
| [Rubric Framework](critics/SEC06_RUBRIC_FRAMEWORK.md) | SEC-06 rubric architecture and extensibility |
| [Self-Validation Convergence](critics/SEC07_SELF_VALIDATION_CONVERGENCE.md) | SEC-07 Quorum validating itself |
| [PowerShell Coverage](critics/POWERSHELL_COVERAGE.md) | PSScriptAnalyzer integration details |

## Reviews & Research

| Doc | Description |
|-----|-------------|
| [External Reviews](reviews/EXTERNAL_REVIEWS.md) | Independent evaluations of Quorum by other models |
| [Changelog](CHANGELOG.md) | Version history and release notes |
| [Code of Conduct](CODE_OF_CONDUCT.md) | Community guidelines |

## Platform Ports

Quorum runs natively in other agent platforms:

| Port | Description |
|------|-------------|
| [Copilot CLI](../ports/copilot-cli/) | GitHub Copilot skill — stdlib-only pre-screen + 5 LLM critics |
| [Claude Code](../ports/claude-code/) | Claude Code subscription — zero API cost validation |

## Contributing

→ [CONTRIBUTING.md](../CONTRIBUTING.md)

We especially welcome rubric contributions for new domains.
File renamed without changes.
2 changes: 1 addition & 1 deletion docs/THE_NINE.md → docs/architecture/THE_NINE.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ That's why there are nine.

---

*The characters in this story correspond to the nine agents defined in the [Quorum Specification](../SPEC.md). Their behaviors, evidence requirements, and interaction patterns are real. The drama is only slightly exaggerated.*
*The characters in this story correspond to the nine agents defined in the [Quorum Specification](../../SPEC.md). Their behaviors, evidence requirements, and interaction patterns are real. The drama is only slightly exaggerated.*

---

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ That's it. No build step. No configuration files to wrestle with. No Docker cont

## "But where's the code?"

Right here: [`reference-implementation/`](../reference-implementation/)
Right here: [`reference-implementation/`](../../reference-implementation/)

```bash
cd reference-implementation
Expand Down
4 changes: 2 additions & 2 deletions docs/INSTALLATION.md → docs/getting-started/INSTALLATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ models:
depth: standard # quick | standard | thorough
```

See [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md) for full options and [MODEL_REQUIREMENTS.md](MODEL_REQUIREMENTS.md) for supported models.
See [CONFIG_REFERENCE.md](../configuration/CONFIG_REFERENCE.md) for full options and [MODEL_REQUIREMENTS.md](MODEL_REQUIREMENTS.md) for supported models.

## Requirements

Expand All @@ -66,7 +66,7 @@ See [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md) for full options and [MODEL_REQUI

## Troubleshooting

**"No model configured"** — Run `quorum run` once and follow the first-run setup prompts, or manually create `quorum-config.yaml` (see [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md)).
**"No model configured"** — Run `quorum run` once and follow the first-run setup prompts, or manually create `quorum-config.yaml` (see [CONFIG_REFERENCE.md](../configuration/CONFIG_REFERENCE.md)).

**"API key not found"** — Set your provider's API key as an environment variable:
```bash
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Quorum's critics need models that can reason about complex artifacts, follow str

## Tier System

Quorum uses a two-tier architecture. **Tier 1** handles judgment-heavy roles (supervisor, aggregator, security critic). **Tier 2** handles pattern-matching roles (correctness, completeness, architecture critics). See [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md) for which roles map to which tier.
Quorum uses a two-tier architecture. **Tier 1** handles judgment-heavy roles (supervisor, aggregator, security critic). **Tier 2** handles pattern-matching roles (correctness, completeness, architecture critics). See [CONFIG_REFERENCE.md](../configuration/CONFIG_REFERENCE.md) for which roles map to which tier.

### Tier 1 — Recommended (Judgment Roles)

Expand Down Expand Up @@ -49,7 +49,7 @@ models:
tier_2: anthropic/claude-sonnet-4-6 # Evaluation roles (correctness, completeness, architecture)
```

For per-critic overrides, see [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md).
For per-critic overrides, see [CONFIG_REFERENCE.md](../configuration/CONFIG_REFERENCE.md).

**First-run behavior:** If no config exists, Quorum auto-detects your available model, asks two setup questions (preferred tier 1 and tier 2 models), writes the config, and proceeds immediately. Zero-config users get sensible defaults.

Expand Down
69 changes: 69 additions & 0 deletions docs/getting-started/QUICK_START.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Quick Start

Get your first Quorum validation running in 60 seconds.

## 1. Install

```bash
pip install quorum-validator
```

## 2. Set your model provider

```bash
export ANTHROPIC_API_KEY=your-key
# or: export OPENAI_API_KEY=your-key
# Quorum uses LiteLLM — 100+ models supported
```

## 3. Run your first validation

```bash
quorum run --target your-file.py --depth quick
```

That's it. Quorum auto-detects the file type, picks the right rubric, spawns critics, and gives you a verdict.

## What You'll See

```
Running Quorum (quick depth, critics: correctness, completeness) ...

◆ QUORUM VERDICT: PASS_WITH_NOTES
──────────────────────────────────
Issues: 0 HIGH · 1 MEDIUM · 2 LOW/INFO (3 total)

1. [MEDIUM] Function `process_data` has no error handling for
empty input — raises unhandled IndexError at line 47
2. [INFO] Docstring for `validate` mentions param `strict`
which doesn't exist in the function signature
3. [INFO] TODO marker at line 12 — "TODO: add retry logic"

Cost: $0.14 (8,200 prompt + 1,840 completion tokens)
```

## Depth Levels

| Depth | Critics | Cost* | Use when |
|-------|---------|-------|----------|
| `quick` | 2 (correctness, completeness) | ~$0.15 | Sanity check before continuing |
| `standard` | 6 (+ security, code hygiene, cross-consistency, tester) | ~$0.50 | Default for most work |
| `thorough` | 6 + fix loops + full audit trail | ~$1.50+ | Production-bound, can't be wrong |

*Estimates on Claude Sonnet. Scales with model and artifact size.*

## Next Steps

- **Validate a directory:** `quorum run --target ./src/ --depth standard`
- **Use a custom rubric:** `quorum run --target report.md --rubric my-rubric.json`
- **Check cross-artifact consistency:** `quorum run --target ./docs/ --relationships quorum-relationships.yaml`
- **Build your own rubric:** [Rubric Building Guide](../guides/RUBRIC_BUILDING_GUIDE.md)
- **Understand the critics:** [The Nine](../architecture/THE_NINE.md)

## Troubleshooting

First run will prompt you to configure your model. If you hit issues:

- [Full Installation Guide](INSTALLATION.md)
- [Model Requirements](MODEL_REQUIREMENTS.md)
- [Config Reference](../configuration/CONFIG_REFERENCE.md)
File renamed without changes.
File renamed without changes.
Loading
Loading