You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
refactor(skills): separate Expected Outcome from Example Instance and add alias check
Split the combined "example with known optimal solution" into distinct
Example Instance (problem data only) and Expected Outcome (solution/value)
sections across issue templates, check-issue, add-model, propose, and
final-review skills. Differentiates satisfaction (valid config + justification)
from optimization (optimal config + objective value) throughout. Also adds
an alias sanity check to final-review's model completeness checklist.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: .claude/skills/add-model/SKILL.md
+9-3Lines changed: 9 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,9 +26,15 @@ Before any implementation, collect all required information. If called from `iss
26
26
| 9 |**Best known exact algorithm**| Complexity with variable definitions | "O(1.1996^n) by Xiao & Nagamochi (2017), where n = \|V\|" |
27
27
| 10 |**Solving strategy**| How it can be solved | "BruteForce works; ILP reduction available" |
28
28
| 11 |**Category**| Which sub-module under `src/models/`|`graph`, `formula`, `set`, `algebraic`, `misc`|
29
+
| 12 |**Expected outcome from the issue**| Concrete outcome for the issue's example instance | Optimization: one optimal solution + optimal value. Satisfaction: one valid/satisfying solution + why it is valid |
29
30
30
31
If any item is missing, ask the user to provide it. Do NOT proceed until the checklist is complete.
31
32
33
+
The issue's **Expected Outcome** section is the source of truth for the implementation-facing example.
34
+
- For optimization problems, use the issue's optimal solution and optimal objective value.
35
+
- For satisfaction problems, use the issue's valid / satisfying solution and its justification.
36
+
- Do not invent or replace the expected outcome during implementation unless the issue is corrected first.
37
+
32
38
### Associated Rule Check
33
39
34
40
Before implementation, verify that at least one reduction rule exists or is planned for this problem — otherwise it will be an orphan node in the reduction graph.
@@ -185,14 +191,14 @@ Required tests:
185
191
-`test_<name>_direction` -- verify optimization direction (if optimization problem)
186
192
-`test_<name>_serialization` -- round-trip serde test (optional but recommended)
-`test_<name>_paper_example` -- **use the same instance from the paper example** (Step 6), verify the claimed solution is valid/optimal and the solution count matches
194
+
-`test_<name>_paper_example` -- **use the same instance from the paper example** (Step 6), verify the issue's expected outcome is valid/optimal and the solution count matches
189
195
190
196
The `test_<name>_paper_example` test is critical for consistency between code and paper. It must:
191
197
1. Construct the exact same instance shown in the paper's example figure
192
-
2. Evaluate the solution shown in the paper and assert it is valid (and optimal for optimization problems)
198
+
2. Evaluate the solution from the issue's **Expected Outcome** section as shown in the paper and assert it is valid (and optimal for optimization problems)
193
199
3. Use `BruteForce` to find all optimal/satisfying solutions and assert the count matches the paper's claim
194
200
195
-
This test should be written **after** Step 6 (paper entry), once the example instance and solution are finalized. If writing tests before the paper, use the same instance you plan to use in the paper and come back to verify consistency.
201
+
This test should be written **after** Step 6 (paper entry), once the example instance and expected outcome are finalized. If writing tests before the paper, use the issue's Example Instance + Expected Outcome as the source of truth and come back to verify consistency.
196
202
197
203
Link the test file via `#[cfg(test)] #[path = "..."] mod tests;` at the bottom of the model file.
Copy file name to clipboardExpand all lines: .claude/skills/check-issue/SKILL.md
+5-2Lines changed: 5 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -306,7 +306,8 @@ Check all template sections are present and substantive:
306
306
| Schema | Type name, variants, field table |
307
307
| Complexity | Best known algorithm with citation **and** a concrete complexity expression in terms of problem parameters (e.g., `q^n`, `2^{0.8765n}`) |
308
308
| How to solve | At least one solver method checked |
309
-
| Example Instance | Concrete instance with known solution |
309
+
| Example Instance | Concrete instance that exercises the core structure |
310
+
| Expected Outcome | Satisfaction: one valid / satisfying solution with brief justification. Optimization: one optimal solution with the optimal objective value |
310
311
311
312
Missing or placeholder sections → list them as **Fail** items.
312
313
@@ -328,7 +329,9 @@ The formal definition must be **precise and implementable**:
328
329
329
330
-**Non-trivial**: Enough vertices/variables to exercise constraints meaningfully (not just a triangle)
330
331
-**Exercises core structure**: Examples must use the defining features of the problem. For instance, a "MultivariateQuadratic" example that only has linear terms does not exercise the quadratic structure → **Fail**. If the problem's name or definition highlights a specific structural feature (quadratic, k-colorable, bipartite, etc.), at least one example must exercise that feature.
331
-
-**Known optimal solution provided**: Must state the optimal value, not just the instance
332
+
-**Expected outcome provided**:
333
+
- Satisfaction problems must include a concrete valid / satisfying solution and say why it is valid
334
+
- Optimization problems must include a concrete optimal solution and the optimal objective value
332
335
-**Detailed enough for paper**: This example will appear in the paper — it needs to be illustrative
333
336
-**Round-trip testable**: The example must be complex enough that a round-trip test (construct instance → solve → verify) can catch implementation bugs. A too-simple instance (e.g., 2 vertices, a single clause) may have a trivially correct solution that passes even with a wrong implementation. The example should have multiple feasible configurations with different objective values (for optimization) or a mix of satisfying and non-satisfying configurations (for satisfaction problems), so that correctness is meaningfully tested. Rule of thumb: the instance should have at least 2 suboptimal feasible solutions in addition to the optimal one.
Copy file name to clipboardExpand all lines: .claude/skills/final-review/SKILL.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -184,6 +184,7 @@ Verify the PR includes all required components. Check:
184
184
-[ ] Paper section in `docs/paper/reductions.typ` (`problem-def` entry)
185
185
-[ ]`display-name` entry in paper
186
186
-[ ]`trait_consistency.rs` entry in `src/unit_tests/trait_consistency.rs` (`test_all_problems_implement_trait_correctly`, plus `test_direction` for optimization)
187
+
-[ ] Aliases: if provided, verify they are standard literature abbreviations (not made up); if empty, confirm no well-known abbreviation is missing; check no conflict with existing aliases
Copy file name to clipboardExpand all lines: .claude/skills/propose/SKILL.md
+11-5Lines changed: 11 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -312,7 +312,9 @@ Work through these topics in order, using `AskUserQuestion` where multiple-choic
312
312
313
313
If the user picks "Generate new batch", create 3 new examples with different sizes/structures and re-present.
314
314
315
-
After the user picks a concrete example, provide a complete instance with its known optimal solution.
315
+
After the user picks a concrete example, provide a complete instance with its expected outcome.
316
+
- For optimization problems: give at least one optimal solution and the optimal objective value
317
+
- For satisfaction problems: give at least one valid / satisfying solution and explain briefly why it is valid
316
318
- Must exercise the problem's core structure
317
319
- Must be small enough to verify by hand
318
320
@@ -488,7 +490,7 @@ If the reduction is well-known, use the literature to **pre-fill** answers in St
488
490
- Must define all symbols before using them
489
491
- Must be detailed enough that someone could implement it
490
492
491
-
3.**Explanation** — Present a correctness argument explaining why the reduction preserves optimal solutions, then ask for feedback via `AskUserQuestion`:
493
+
3.**Explanation** — Present a correctness argument explaining why the reduction preserves feasibility (for satisfaction problems) or optimality (for optimization problems), then ask for feedback via `AskUserQuestion`:
492
494
```
493
495
AskUserQuestion:
494
496
question: "How does this explanation look?"
@@ -525,7 +527,8 @@ If the reduction is well-known, use the literature to **pre-fill** answers in St
525
527
526
528
If the user picks "Generate new batch", create 3 new examples with different sizes/structures and re-present.
527
529
528
-
After the user picks a concrete example, fully work out the example: show source instance, each construction step, resulting target instance, and the optimal solution.
530
+
After the user picks a concrete example, fully work out the example: show source instance, each construction step, and the resulting target instance.
531
+
- Do not ask the user to provide solved witnesses manually
529
532
- Must be non-trivial but hand-verifiable
530
533
- Must exercise the core structure of the reduction
531
534
@@ -637,14 +640,17 @@ If proposing a model + rules, present all drafts together:
637
640
- Reduction Rule Crossref (linking to companion rule issues or noting planned rules)
638
641
- How to solve (brute-force, ILP, or other — if ILP/QUBO, must cross-reference rule issue)
639
642
- Example Instance
643
+
- Expected Outcome
644
+
- Optimization problems: optimal solution + optimal objective value
- Reduction Algorithm (numbered steps, all symbols defined)
645
651
- Size Overhead (table with target metrics and formulas)
646
652
- Validation Method
647
-
- Example (fully worked)
653
+
- Example (fully worked: source instance, construction, target instance)
648
654
- BibTeX (include the BibTeX entry for the reference at the end of the issue)
649
655
650
656
---
@@ -665,7 +671,7 @@ Apply all 4 checks from `/check-issue` against the draft content:
665
671
1.**Usefulness:**`pred show <name>` must fail (problem doesn't exist). At least one reduction planned.
666
672
2.**Non-trivial:** Not isomorphic to existing problem.
667
673
3.**Correctness:** Complexity expression verified against literature.
668
-
4.**Well-written:** All template sections present, symbols consistent, example exercises core structure.
674
+
4.**Well-written:** All template sections present, symbols consistent, example exercises core structure, and Expected Outcome matches the problem type (valid solution for satisfaction, optimal solution/value for optimization).
669
675
670
676
**If any check fails:** Fix the draft automatically if possible. If user input is needed, ask. Loop back to Step 4 with the corrected draft.
0 commit comments