You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
4.`{PRESCREEN_EVIDENCE}` ← pre-screen JSON output (security and code_hygiene critics; empty for correctness and completeness)
121
126
5.`{EXTRA_CONTEXT}` ← mandatory known-issue patterns + any additional context
122
127
123
128
**Dispatch via `task` tool:**
@@ -130,8 +135,8 @@ Each critic is dispatched as a task agent. The system prompt comes from the crit
130
135
131
136
**Progress indicators:** After dispatching each critic, inform the user:
132
137
- "🔍 Correctness critic dispatched (1 of 4)..."
133
-
- "🔍 Security critic dispatched (2 of 4)..."
134
-
- "🔍 Completeness critic dispatched (3 of 4)..."
138
+
- "🔍 Completeness critic dispatched (2 of 4)..."
139
+
- "🔍 Security critic dispatched (3 of 4)..."
135
140
- "🔍 Code Hygiene critic dispatched (4 of 4)..."
136
141
- "⏳ Waiting for critic results..."
137
142
@@ -148,6 +153,7 @@ After all critics return, aggregate findings into a verdict.
148
153
Parse each critic's JSON output. Validate that every finding has:
149
154
-`severity` (CRITICAL/HIGH/MEDIUM/LOW/INFO)
150
155
-`description` (non-empty)
156
+
-`evidence_tool` (non-empty — how the finding was verified)
151
157
-`evidence_result` (non-empty — reject findings without evidence)
152
158
153
159
**Reject ungrounded findings.** If a finding lacks `evidence_result`, discard it and log: "Finding rejected: no evidence provided."
@@ -169,9 +175,9 @@ Apply rules from `verdict-rules.yaml` in order:
169
175
| 3+ HIGH findings |**REVISE**|
170
176
| Any HIGH (fewer than 3) |**PASS_WITH_NOTES**|
171
177
| Only MEDIUM/LOW |**PASS_WITH_NOTES**|
172
-
| No findings with evidence|**PASS**|
178
+
| No findings (or only INFO)|**PASS**|
173
179
174
-
**Escalation:** If cross-artifact relationship checks found HIGH or CRITICAL issues, escalate verdict by one level (PASS → PASS_WITH_NOTES, PASS_WITH_NOTES → REVISE).
180
+
**Escalation:** If cross-artifact relationship checks found HIGH or CRITICAL issues, escalate verdict by one level (PASS → PASS_WITH_NOTES, PASS_WITH_NOTES → REVISE). Note: relationship checks are not yet ported; this rule is reserved for future use.
175
181
176
182
**REJECT** is never assigned automatically — it requires your supervisor judgment that the artifact is fundamentally unsalvageable.
177
183
@@ -247,10 +253,12 @@ After verdict, update `learning/known_issues.json`:
247
253
| Failure | Action |
248
254
|---------|--------|
249
255
| Target not found | Abort: "❌ Target file not found: {path}" |
250
-
| Rubric not found | Abort: "❌ Rubric not found: {name}. Available: {list}" |
256
+
| Rubric not found | Abort: "❌ Rubric not found: {name}. Available rubrics: [list names only]" |
| Critic returns invalid JSON | Treat as critic failure (DEGRADED). Log warning, proceed with remaining critics. |
251
259
| Pre-screen script missing | Warn, continue without pre-screen |
252
-
|1-2 critics fail/time out | DEGRADED: produce verdict from remaining critics |
253
-
|3 critics fail | PARTIAL: produce verdict from sole remaining critic |
260
+
|Some critics fail/time out | DEGRADED: produce verdict from remaining critics. Apply standard verdict rules to available findings. Tag output with "⚠️ DEGRADED: N of M critics returned."|
261
+
|All but one critic fail | PARTIAL: produce verdict from sole remaining critic. Tag output with "⚠️ PARTIAL." Verdict reflects only that critic's coverage.|
254
262
| All critics fail | Abort: "❌ QUORUM_FAILED: All critics failed" |
**This is not** the full Quorum reference implementation. The CLI version has additional capabilities (batch processing, fix loops, cost tracking, tester verification). This skill covers the highest-value portion: deterministic pre-screen → parallel critic dispatch → evidence-grounded findings → deterministic verdict.
296
304
297
-
**Capability coverage:**~75% of reference implementation. Full 4-critic suite (correctness, completeness, security, code hygiene). Missing: tester (L1/L2 verification), fixer (automated remediation), batch mode, cost tracking, structured output artifacts. These are planned for future releases.
305
+
**Capability coverage:**~75% of reference implementation. Includes all four core evaluation critics. Not yet ported: L1/L2 verification, automated remediation, batch mode, cost tracking, structured output artifacts. These are planned for future releases.
0 commit comments