Commit 4417858
feat: US-011 - Task generator CLI
Implements scripts/generate_mcp_unique_tasks.py — CLI tool that reads
the use case registry and repo-set fixtures, fills templates, and writes
task directories to benchmarks/<mcp_suite>/<task_slug>/.
Key features:
- Filter by --use-case-ids, --category, --family, or --all
- --dry-run shows what would be generated without writing files
- --include-stubs generates oracle_type='tbd' entries (skipped by default)
- --curate-oracle auto-populates oracle_answer.json via curate_oracle.py
- --validate runs syntactic checks (bash -n, json.load, py_compile) post-generation
- Copies oracle_checks.py to tests/ for Harbor /tests/oracle_checks.py access
- Registry fallback: searches both configs/ and ralph-mcp-unique/configs/
Tested: --help, --dry-run, actual generation + validation for use_case_id=1.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1 parent 9f82070 commit 4417858
File tree
3 files changed
+624
-1
lines changed- ralph-mcp-unique
- scripts
3 files changed
+624
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
257 | 257 | | |
258 | 258 | | |
259 | 259 | | |
260 | | - | |
| 260 | + | |
261 | 261 | | |
262 | 262 | | |
263 | 263 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
160 | 160 | | |
161 | 161 | | |
162 | 162 | | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
163 | 181 | | |
164 | 182 | | |
165 | 183 | | |
| |||
0 commit comments