You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: add prerequisite gate and orphan check to check-issue skill (#264)
- Add Rule Check 0: require source/target implementations before evaluating
[Rule] issues. Fails with `Blocked` label and skips remaining checks.
- Add Rule Check 3e: cross-validate issue content against actual code
implementations (fields, methods, objective direction).
- Add Model Check 1.5: verify at least one [Rule] issue references the
model to prevent orphan nodes. Fails with `Orphan` label.
- Update flow diagram, report templates, and label application section.
Closes#264
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
**Goal:** Is this reduction novel or does it improve on an existing one?
63
+
**Goal:** Are both source and target problems implemented in the codebase?
64
+
65
+
A reduction rule cannot be implemented if the source or target problem does not exist in code. This is a **hard prerequisite** — if either is missing, skip all remaining checks.
57
66
58
67
1. Parse **Source** and **Target** problem names from the issue body.
59
68
60
-
2. Resolve problem aliases (the issue may say "MIS" but `pred` needs "MaximumIndependentSet"):
69
+
2. Resolve problem aliases and verify implementation:
61
70
```bash
62
71
pred show <source> --json 2>/dev/null
63
72
pred show <target> --json 2>/dev/null
64
73
```
65
-
If either fails, try the full name. If still fails, **Warn** (unknown problem — may be a new model).
74
+
Try aliases first (e.g., "MIS" → "MaximumIndependentSet"), then the full name.
75
+
76
+
3. Decision:
77
+
-**Both succeed** → **Pass**. Store the JSON output for use in subsequent checks (Check 1 overhead comparison, Check 3e cross-validation, Check 4c metric names).
78
+
-**Either fails** → **Fail** with label `Blocked`
79
+
- Message: "Cannot evaluate: `<problem>` is not implemented in the codebase. The corresponding [Model] issue must be merged first."
80
+
-**Skip all remaining checks** (Check 1–4). The report should only contain Check 0.
**Goal:** Is this reduction novel or does it improve on an existing one?
87
+
88
+
1. Check existing path (using resolved names from Check 0):
68
89
```bash
69
90
pred path <source><target> --json
70
91
```
71
92
72
-
4. Decision (principle: new rule must reduce the reduction overhead):
93
+
2. Decision (principle: new rule must reduce the reduction overhead):
73
94
-**No path exists** → **Pass** (novel reduction)
74
95
-**Path exists** → compare overhead:
75
96
- Parse the proposed overhead from the issue's "Size Overhead" table
@@ -78,7 +99,7 @@ Applies when the title contains `[Rule]`.
78
99
- If overhead is **equal or higher** on all dimensions → **Fail**
79
100
- If overhead comparison is ambiguous (different dimensions, incomparable expressions) → **Warn** with explanation
80
101
81
-
5. Check **Motivation** field: if empty, placeholder, or just "enables X" without explaining *why this path matters* → **Warn**
102
+
3. Check **Motivation** field: if empty, placeholder, or just "enables X" without explaining *why this path matters* → **Warn**
82
103
83
104
---
84
105
@@ -139,6 +160,28 @@ While searching, if you find:
139
160
140
161
→ Include in the report as a **Recommendation** (not a failure). Example: "Note: Smith et al. (2024) improve on this with O(n) overhead instead of O(n^2)."
141
162
163
+
### 3e: Cross-validate Against Code Implementation
164
+
165
+
Using the `pred show` JSON stored from Check 0, verify the issue's description is consistent with the actual code:
166
+
167
+
1.**Source problem consistency:**
168
+
- Compare the issue's description of the source problem (input structure, objective, constraints) against the code's actual implementation (fields, `direction()`, `evaluate()` semantics)
169
+
- If the issue says "MIS maximizes vertex count" but the code uses weighted sums → **Fail**
170
+
171
+
2.**Target problem consistency:**
172
+
- Same check for the target problem
173
+
- Verify the target's field names and semantics match what the issue's reduction algorithm constructs
- Every field name referenced in the algorithm (e.g., "add edge to E'", "set weight w_i") must correspond to actual fields/methods on the source or target type
177
+
- Check against `size_fields` and schema from `pred show --json`
178
+
179
+
4.**Size overhead metric names match target:**
180
+
- Every metric in the "Size Overhead" table must be a valid getter on the target problem
181
+
- This overlaps with Check 4c but is a **correctness** concern (wrong metric names = wrong reduction), not just a writing concern
182
+
183
+
If any mismatch found → **Fail** with specifics about what the issue claims vs. what the code actually implements.
A model without any reduction rules is an orphan node in the reduction graph — it adds no value. Verify that at least one `[Rule]` issue exists that references this model as source or target.
259
+
260
+
```bash
261
+
# Search for [Rule] issues referencing this problem name
0 commit comments