|
| 1 | +# Cross-Model Reviewer Agent |
| 2 | + |
| 3 | +Orchestrates adversarial code review across multiple AI models (Codex + Claude) and computes consensus. |
| 4 | + |
| 5 | +## Purpose |
| 6 | + |
| 7 | +Provide higher-confidence code review by running independent reviews from different model families, then applying conservative consensus logic. If models agree, confidence is high. If they disagree, the conflict is surfaced for human decision. |
| 8 | + |
| 9 | +## Protocol |
| 10 | + |
| 11 | +### Step 1: Dispatch Codex Adversarial Review |
| 12 | + |
| 13 | +Run `flowctl codex adversarial --base <branch>` to get the Codex model's adversarial review. This model actively tries to break the code, looking for bugs, race conditions, security vulnerabilities, and edge cases. |
| 14 | + |
| 15 | +### Step 2: Dispatch Claude Review |
| 16 | + |
| 17 | +Write a structured review prompt and either: |
| 18 | +- Let the orchestrator (skill layer) invoke Claude directly, or |
| 19 | +- Pre-populate a result file at `$TMPDIR/flowctl-cross-model-claude-result.json` |
| 20 | + |
| 21 | +The Claude review focuses on correctness, security, performance, and maintainability. |
| 22 | + |
| 23 | +### Step 3: Compute Consensus |
| 24 | + |
| 25 | +Use `flowctl codex cross-model --base <branch>` which: |
| 26 | +1. Runs both reviews |
| 27 | +2. Parses each into a `ModelReview` struct with verdict, findings, and confidence |
| 28 | +3. Applies the conservative consensus algorithm: |
| 29 | + - All agree on SHIP → **Consensus(SHIP)** — safe to proceed |
| 30 | + - Any says NEEDS_WORK → **Consensus(NEEDS_WORK)** — conservative block |
| 31 | + - Mixed/unclear → **Conflict** — human must decide |
| 32 | + - Insufficient data → **InsufficientReviews** — retry or escalate |
| 33 | + |
| 34 | +### Step 4: Store Results |
| 35 | + |
| 36 | +Combined review is saved to `.flow/reviews/cross-model-YYYYMMDD-HHMMSS.json` with: |
| 37 | +- Both model reviews (verdict, findings, confidence) |
| 38 | +- Consensus result |
| 39 | +- Timestamp and base branch |
| 40 | +- Path to the Claude prompt file (for audit) |
| 41 | + |
| 42 | +## MCP Integration |
| 43 | + |
| 44 | +The `flowctl_review` MCP tool exposes cross-model review: |
| 45 | + |
| 46 | +```json |
| 47 | +{ |
| 48 | + "name": "flowctl_review", |
| 49 | + "arguments": { |
| 50 | + "base": "main", |
| 51 | + "focus": "security" |
| 52 | + } |
| 53 | +} |
| 54 | +``` |
| 55 | + |
| 56 | +## Review Types |
| 57 | + |
| 58 | +### ReviewFinding |
| 59 | +Individual issue with severity (critical/warning/info), category, description, and optional file/line. |
| 60 | + |
| 61 | +### ReviewVerdict |
| 62 | +- **SHIP**: Code is ready |
| 63 | +- **NEEDS_WORK**: Code needs fixes |
| 64 | +- **ABSTAIN**: Model cannot determine (excluded from consensus) |
| 65 | + |
| 66 | +### ConsensusResult |
| 67 | +- **Consensus**: All voting models agree (with averaged confidence) |
| 68 | +- **Conflict**: Models disagree (reviews included for inspection) |
| 69 | +- **InsufficientReviews**: Fewer than 2 reviews or all abstained |
| 70 | + |
| 71 | +## Usage |
| 72 | + |
| 73 | +```bash |
| 74 | +# Full cross-model review (JSON output) |
| 75 | +flowctl codex cross-model --base main --json |
| 76 | + |
| 77 | +# With focus area |
| 78 | +flowctl codex cross-model --base main --focus "authentication" --json |
| 79 | + |
| 80 | +# Via MCP |
| 81 | +echo '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"flowctl_review","arguments":{"base":"main"}}}' | flowctl mcp |
| 82 | +``` |
| 83 | + |
| 84 | +## Pre-populated Claude Results |
| 85 | + |
| 86 | +For environments where Claude is already available (e.g., Claude Code), the orchestrating skill can pre-populate the Claude review result before invoking `flowctl codex cross-model`: |
| 87 | + |
| 88 | +```bash |
| 89 | +# Write Claude's review result |
| 90 | +cat > /tmp/flowctl-cross-model-claude-result.json << 'EOF' |
| 91 | +{ |
| 92 | + "model": "claude/opus-4", |
| 93 | + "verdict": "SHIP", |
| 94 | + "confidence": 0.92, |
| 95 | + "review": "Code looks correct. No critical issues found." |
| 96 | +} |
| 97 | +EOF |
| 98 | + |
| 99 | +# Then run cross-model (will pick up the pre-populated result) |
| 100 | +flowctl codex cross-model --base main --json |
| 101 | +``` |
| 102 | + |
| 103 | +## Design Decisions |
| 104 | + |
| 105 | +- **Conservative consensus**: Any NEEDS_WORK blocks, even if other models say SHIP. This prevents false confidence from a single agreeing model. |
| 106 | +- **Abstain handling**: Models that fail or cannot determine a verdict are excluded from the vote, not counted as disagreement. |
| 107 | +- **Two-model minimum**: Consensus requires at least 2 non-abstaining reviews. |
| 108 | +- **Structured findings**: Every finding has severity, category, and description — enabling automated triage and gap registration. |
0 commit comments