Skip to content

Commit 9eaa786

Browse files
GiggleLiuclaude
andauthored
Refactor generated doc exports and remove centralized trait tests (#676)
* initial * clean up reduction_graph.json * refactor-remove-conflicting files * update skills and prompts * relax model test naming to flexible coverage with >= 3 test floor Replace rigid per-model test function name checklist with flexible coverage guidance enforced by a minimum of 3 test functions. All three files (CLAUDE.md, add-model skill, structural reviewer) are now consistent on the threshold. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * remove conflicting generated files These files were intentionally removed earlier in this branch; re-delete after merge conflict resolution. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent da12028 commit 9eaa786

20 files changed

Lines changed: 101 additions & 2706 deletions

File tree

.claude/CLAUDE.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -207,7 +207,7 @@ New code must have >95% test coverage. Run `make coverage` to check.
207207
### Naming
208208

209209
- Reduction tests: `test_<source>_to_<target>_closed_loop`
210-
- Model tests: `test_<model>_basic`, `test_<model>_serialization`
210+
- Model tests: descriptive names — e.g., `test_<model>_creation`, `test_<model>_evaluate_*`, `test_<model>_direction`, `test_<model>_solver`, `test_<model>_serialization`. Use whichever are relevant; there is no fixed per-model naming set.
211211
- Solver tests: `test_<solver>_<problem>`
212212

213213
### Key Testing Patterns
@@ -218,6 +218,8 @@ See Key Patterns above for solver API signatures. Follow the reference files for
218218

219219
Unit tests in `src/unit_tests/` linked via `#[path]` (see Core Modules above). Integration tests in `tests/suites/`, consolidated through `tests/main.rs`. Canonical example-db coverage lives in `src/unit_tests/example_db.rs`.
220220

221+
Model review automation checks for a dedicated test file under `src/unit_tests/models/...` with at least 3 test functions. The exact split of coverage is judged per model during review.
222+
221223
## Documentation Locations
222224
- `README.md` — Project overview and quickstart
223225
- `.claude/` — Claude Code instructions and skills
@@ -258,7 +260,7 @@ Also add to the `display-name` dictionary:
258260
]
259261
```
260262

261-
Every directed reduction in the graph needs its own `reduction-rule` entry. The paper auto-checks completeness against `reduction_graph.json`.
263+
Every directed reduction in the graph needs its own `reduction-rule` entry. The paper auto-checks completeness against the generated `reduction_graph.json` export.
262264

263265
## Complexity Verification Requirements
264266

.claude/skills/add-model/SKILL.md

Lines changed: 12 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,6 @@ Before implementing, make sure the plan explicitly covers these items that struc
7575
- CLI discovery and `pred create <ProblemName>` support are included where applicable
7676
- A canonical model example is registered for example-db / `pred create --example`
7777
- `docs/paper/reductions.typ` adds both the display-name dictionary entry and the `problem-def(...)`
78-
- `src/unit_tests/trait_consistency.rs` is updated
7978

8079
## Step 1: Determine the category
8180

@@ -196,32 +195,24 @@ This example is now the canonical source for:
196195

197196
Create `src/unit_tests/models/<category>/<name>.rs`:
198197

199-
Required tests:
200-
- `test_<name>_creation` -- construct an instance, verify dimensions
201-
- `test_<name>_evaluation` -- verify `evaluate()` on valid and invalid configs
202-
- `test_<name>_direction` -- verify optimization direction (if optimization problem)
203-
- `test_<name>_serialization` -- round-trip serde test (optional but recommended)
204-
- `test_<name>_solver` -- verify brute-force solver finds correct solutions
205-
- `test_<name>_paper_example` -- **use the same instance from the paper example** (Step 6), verify the issue's expected outcome is valid/optimal and the solution count matches
198+
Every model needs **at least 3 test functions** (the structural reviewer enforces this). Choose from the coverage areas below — pick whichever are relevant to the model:
206199

207-
The `test_<name>_paper_example` test is critical for consistency between code and paper. It must:
208-
1. Construct the exact same instance shown in the paper's example figure
200+
- **Creation/basic** — exercise constructor inputs, key accessors, `dims()` / `num_variables()`.
201+
- **Evaluation** — valid and invalid configs so the feasibility boundary is explicit.
202+
- **Direction** — verify optimization direction (optimization problems only).
203+
- **Solver** — brute-force solver finds correct solutions (when the model is small enough).
204+
- **Serialization** — round-trip serde (when the model is used in CLI/example-db flows).
205+
- **Paper example** — verify the worked example from the paper entry (see below).
206+
207+
When you add `test_<name>_paper_example`, it should:
208+
1. Construct the same instance shown in the paper's example figure
209209
2. Evaluate the solution from the issue's **Expected Outcome** section as shown in the paper and assert it is valid (and optimal for optimization problems)
210-
3. Use `BruteForce` to find all optimal/satisfying solutions and assert the count matches the paper's claim
210+
3. Use `BruteForce` to confirm the claimed optimum/satisfying solution count when the instance is small enough for unit tests
211211

212-
This test should be written **after** Step 6 (paper entry), once the example instance and expected outcome are finalized. If writing tests before the paper, use the issue's Example Instance + Expected Outcome as the source of truth and come back to verify consistency.
212+
This test is usually written **after** Step 6 (paper entry), once the example instance and expected outcome are finalized. If writing tests before the paper, use the issue's Example Instance + Expected Outcome as the source of truth and come back to verify consistency.
213213

214214
Link the test file via `#[cfg(test)] #[path = "..."] mod tests;` at the bottom of the model file.
215215

216-
## Step 5.5: Add trait_consistency entry
217-
218-
Add the new problem to `src/unit_tests/trait_consistency.rs`:
219-
220-
1. **`test_all_problems_implement_trait_correctly`** — add a `check_problem_trait(...)` call with a small instance
221-
2. **`test_direction`** (optimization problems only) — add an `assert_eq!(...direction(), Direction::Minimize/Maximize)` entry
222-
223-
This is **required** for every new model — it ensures the Problem trait implementation is well-formed.
224-
225216
## Step 6: Document in paper
226217

227218
Write a `problem-def` entry in `docs/paper/reductions.typ`. **Reference example:** search for `problem-def("MaximumIndependentSet")` to see the gold-standard entry — use it as a template.
@@ -294,5 +285,4 @@ If running standalone (not inside `make run-plan`), invoke [review-implementatio
294285
| Missing from CLI help table | Must add entry to "Flags by problem type" table in `cli.rs` `after_help` |
295286
| Schema lists derived fields | Schema should list constructor params, not internal fields (e.g., `matrix, k` not `matrix, m, n, k`) |
296287
| Missing canonical model example | Add a builder in `src/example_db/model_builders.rs` and keep it aligned with paper/example workflows |
297-
| Forgetting trait_consistency | Must add entry in `test_all_problems_implement_trait_correctly` (and `test_direction` for optimization) in `src/unit_tests/trait_consistency.rs` |
298288
| Paper example not tested | Must include `test_<name>_paper_example` that verifies the exact instance, solution, and solution count shown in the paper |

.claude/skills/add-rule/SKILL.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -170,8 +170,8 @@ Checklist: notation self-contained, complexity cited, overhead consistent, examp
170170
## Step 6: Regenerate exports and verify
171171

172172
```bash
173-
cargo run --example export_graph # Update reduction_graph.json
174-
cargo run --example export_schemas # Update problem schemas
173+
cargo run --example export_graph # Generate reduction_graph.json for docs/paper builds
174+
cargo run --example export_schemas # Generate problem schemas for docs/paper builds
175175
make test clippy # Must pass
176176
```
177177

.claude/skills/final-review/SKILL.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -184,7 +184,6 @@ Verify the PR includes all required components. Check:
184184
- [ ] Canonical model example function in the model file
185185
- [ ] Paper section in `docs/paper/reductions.typ` (`problem-def` entry)
186186
- [ ] `display-name` entry in paper
187-
- [ ] `trait_consistency.rs` entry in `src/unit_tests/trait_consistency.rs` (`test_all_problems_implement_trait_correctly`, plus `test_direction` for optimization)
188187
- [ ] Aliases: if provided, verify they are standard literature abbreviations (not made up); if empty, confirm no well-known abbreviation is missing; check no conflict with existing aliases
189188

190189
**For [Rule] PRs:**

.claude/skills/issue-to-pr/SKILL.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ The plan MUST reference the appropriate implementation skill and follow its step
9797
Include the concrete details from the issue (problem definition, reduction algorithm, example, etc.) mapped onto each step.
9898

9999
**Plan batching:** The paper writing step (add-model Step 6 / add-rule Step 5) MUST be in a **separate batch** from the implementation steps, so it gets its own subagent with fresh context. It depends on the implementation being complete (needs exports). Example batch structure for a `[Model]` plan:
100-
- Batch 1: Steps 1-5.5 (implement model, register, CLI, tests, trait_consistency)
100+
- Batch 1: Steps 1-5.5 (implement model, register, CLI, tests)
101101
- Batch 2: Step 6 (write paper entry — depends on batch 1 for exports)
102102

103103
**Solver rules:**
@@ -223,12 +223,11 @@ EOF
223223
python3 scripts/pipeline_pr.py comment --repo "$REPO" --pr "$PR" --body-file "$COMMENT_FILE"
224224
rm -f "$COMMENT_FILE"
225225

226-
# Repo verification may regenerate tracked exports (notably after `make paper`).
226+
# Repo verification may regenerate ignored doc exports (notably after `make paper`).
227227
# Inspect the tree once more before pushing.
228228
git status --short
229229

230-
# If expected generated exports changed, stage them before push.
231-
git add docs/src/reductions/problem_schemas.json docs/src/reductions/reduction_graph.json
230+
# Generated doc exports under docs/src/reductions/ are ignored; do not stage them.
232231

233232
# The issue plan file must be gone before push.
234233
test ! -e docs/plans/<plan-file>.md

.claude/skills/review-implementation/structural-reviewer-prompt.md

Lines changed: 9 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -33,17 +33,15 @@ Given: problem name `P` = `{PROBLEM_NAME}`, category `C` = `{CATEGORY}`, file st
3333
| 5 | `OptimizationProblem` or `SatisfactionProblem` impl | `Grep("(OptimizationProblem|SatisfactionProblem).*for.*{P}", file)` |
3434
| 6 | `#[cfg(test)]` + `#[path = "..."]` test link | `Grep("#\\[path =", file)` |
3535
| 7 | Test file exists | `Glob("src/unit_tests/models/{C}/{F}.rs")` |
36-
| 8 | Test has creation test | `Grep("fn test_.*creation|fn test_{F}.*basic", test_file)` |
37-
| 9 | Test has evaluation test | `Grep("fn test_.*evaluat", test_file)` |
38-
| 10 | Registered in `{C}/mod.rs` | `Grep("mod {F}", "src/models/{C}/mod.rs")` |
39-
| 11 | Re-exported in `models/mod.rs` | `Grep("{P}", "src/models/mod.rs")` |
40-
| 12 | `declare_variants!` entry exists | `Grep("declare_variants!|default opt|default sat|opt {P}|sat {P}", file)` |
41-
| 13 | CLI `resolve_alias` entry | `Grep("{P}", "problemreductions-cli/src/problem_name.rs")` |
42-
| 14 | CLI `create` support | `Grep('"{P}"', "problemreductions-cli/src/commands/create.rs")` |
43-
| 15 | Canonical model example registered | `Grep("{P}", "src/example_db/model_builders.rs")` |
44-
| 16 | Paper `display-name` entry | `Grep('"{P}"', "docs/paper/reductions.typ")` |
45-
| 17 | Paper `problem-def` block | `Grep('problem-def.*"{P}"', "docs/paper/reductions.typ")` |
46-
| 18 | `trait_consistency` entry | `Grep("{P}", "src/unit_tests/trait_consistency.rs")` |
36+
| 8 | Test file has >= 3 test functions | `Grep("fn test_", test_file)` — count matches, FAIL if < 3 |
37+
| 9 | Registered in `{C}/mod.rs` | `Grep("mod {F}", "src/models/{C}/mod.rs")` |
38+
| 10 | Re-exported in `models/mod.rs` | `Grep("{P}", "src/models/mod.rs")` |
39+
| 11 | `declare_variants!` entry exists | `Grep("declare_variants!|default opt|default sat|opt {P}|sat {P}", file)` |
40+
| 12 | CLI `resolve_alias` entry | `Grep("{P}", "problemreductions-cli/src/problem_name.rs")` |
41+
| 13 | CLI `create` support | `Grep('"{P}"', "problemreductions-cli/src/commands/create.rs")` |
42+
| 14 | Canonical model example registered | `Grep("{P}", "src/example_db/model_builders.rs")` |
43+
| 15 | Paper `display-name` entry | `Grep('"{P}"', "docs/paper/reductions.typ")` |
44+
| 16 | Paper `problem-def` block | `Grep('problem-def.*"{P}"', "docs/paper/reductions.typ")` |
4745

4846
## Rule Checklist
4947

.claude/skills/review-pipeline/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -329,7 +329,7 @@ Completed: 2/2 | All moved to Final review
329329
| Guessing on an issue card with multiple linked repo PRs | Stop, show options to the user, and recommend the most likely correct OPEN PR |
330330
| Picking a PR before Copilot has reviewed | Inspect the checked-out diff and PR body first. If the PR is incomplete, comment and move it back to Ready. If it is review-ready, request Copilot review and switch to another item instead of waiting |
331331
| Missing project scopes | Run `gh auth refresh -s read:project,project` |
332-
| Skipping review-implementation | Always run structural completeness check in Step 2b — it catches gaps Copilot misses (paper entries, CLI registration, trait_consistency) |
332+
| Skipping review-implementation | Always run structural completeness check in Step 2b — it catches gaps Copilot misses (paper entries, CLI registration, example-db wiring) |
333333
| Skipping agentic tests | Always run test-feature even if CI is green |
334334
| Not checking out the right branch | Use `gh pr view` to get the exact branch name |
335335
| Waiting idle for Copilot | Request the review, leave the PR in Review pool, and keep triaging other items in the same run |

docs/src/design.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -230,7 +230,8 @@ Exported files:
230230
- [reduction_graph.json](reductions/reduction_graph.json) — all problem variants and reduction edges
231231
- [problem_schemas.json](reductions/problem_schemas.json) — field definitions for each problem type
232232

233-
Regenerate with `cargo run --example export_graph` and `cargo run --example export_schemas`.
233+
These JSON assets are generated during `make doc`, `make mdbook`, and `make paper`; they are build artifacts, not committed source files.
234+
Generate them manually with `cargo run --example export_graph` and `cargo run --example export_schemas` when you need the raw exports locally.
234235

235236
### Path finding
236237

docs/src/getting-started.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,7 @@ problemreductions = { version = "0.2", default-features = false }
207207

208208
The library exports machine-readable metadata useful for tooling and research:
209209

210+
These files are generated when you build the docs locally.
210211
- [reduction_graph.json](reductions/reduction_graph.json) lists all problem variants and reduction edges
211212
- [problem_schemas.json](reductions/problem_schemas.json) lists field definitions for each problem type
212213

0 commit comments

Comments
 (0)