Skip to content

Commit b4be4e5

Browse files
committed
fix(explore): synthesize final idea artifacts
1 parent 1e521c9 commit b4be4e5

11 files changed

Lines changed: 448 additions & 31 deletions

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,11 +51,11 @@ Requires [codex CLI](https://github.com/openai/codex) for review. See the full [
5151
```bash
5252
/humanize:explore-idea .humanize/ideas/<slug>-<timestamp>.directions.json
5353
```
54-
Dispatches bounded parallel prototype workers (one per direction), each running in an isolated git worktree. After all workers complete, synthesizes a two-tier report ranking the best product direction and the most implementation-ready prototype.
54+
Dispatches bounded parallel prototype workers (one per direction), each running in an isolated git worktree. After all workers complete, writes `.humanize/explore/<run-id>/explore-report.md` for audit/ranking details and `.humanize/explore/<run-id>/final-idea.md` as the plan-ready synthesis.
5555

56-
3. **Generate a plan** from your draft:
56+
3. **Generate a plan** from your draft or explored final idea:
5757
```bash
58-
/humanize:gen-plan --input draft.md --output docs/plan.md
58+
/humanize:gen-plan --input .humanize/explore/<run-id>/final-idea.md --output docs/plan.md
5959
```
6060

6161
4. **Refine an annotated plan** before implementation when reviewers add comments (`CMT:` ... `ENDCMT`, `<cmt>` ... `</cmt>`, or `<comment>` ... `</comment>`):

commands/explore-idea.md

Lines changed: 80 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
description: "Launch bounded parallel prototype workers for idea directions and synthesize a two-tier report"
2+
description: "Launch bounded parallel prototype workers for idea directions and synthesize canonical explore artifacts"
33
argument-hint: "<draft-or-directions-json> [--directions ids] [--concurrency N] [--max-worker-iterations N] [--worker-timeout-min N] [--codex-timeout-min N]"
44
allowed-tools:
55
- "Bash(${CLAUDE_PLUGIN_ROOT}/scripts/validate-explore-idea-io.sh:*)"
@@ -25,11 +25,13 @@ Read and execute below with ultrathink.
2525
- MUST NOT run workers until the user explicitly confirms the dispatch.
2626
- MUST NOT push any branch to any remote at any point.
2727
- MUST write `manifest.json` to the run directory BEFORE dispatching any worker.
28+
- MUST write canonical artifacts to `explore-report.md` and `final-idea.md`; do not create any legacy compatibility alias.
2829
- MUST NOT invoke nested Skills or slash commands inside worker prompts.
2930
- MUST NOT use `--effort max` (not supported by `ask-codex.sh`).
3031
- Worker branches follow the format `explore/<RUN_ID>/<dir_slug>` exactly, and MUST be created by running `git checkout -b` from the current HEAD after asserting `HEAD == <BASE_COMMIT>`; workers MUST NOT run `git checkout <BASE_BRANCH>` (that branch is already checked out in the coordinator worktree, and Git forbids two worktrees from checking out the same branch simultaneously); a HEAD mismatch is a fatal worker error.
3132
- Workers MUST run only targeted tests for the files they touched, not the full test suite.
3233
- Worker Codex calls must be scoped to the worker worktree root via `CLAUDE_PROJECT_DIR="$PWD"`.
34+
- Worker Codex review calls must use the validation-provided `CODEX_REVIEW_MODEL_SPEC` exactly. The generated value is expected to be `gpt-5.5:xhigh`.
3335
- All worker results must be recorded in `worker-results.jsonl`; no result may be silently dropped.
3436

3537
## Worker Constraint Sync
@@ -57,17 +59,20 @@ Run:
5759
Handle exit codes:
5860
- `0`: Parse stdout to extract all `KEY: value` pairs:
5961
`DIRECTIONS_JSON_FILE`, `DRAFT_PATH`, `RUN_ID`, `RUN_DIR`, `BASE_BRANCH`, `BASE_COMMIT`,
62+
`RUN_SLUG`, `CODEX_REVIEW_MODEL`, `CODEX_REVIEW_EFFORT`, `CODEX_REVIEW_MODEL_SPEC`,
63+
`REPORT_PATH`, `FINAL_IDEA_PATH`, `FINAL_IDEA_TEMPLATE`,
6064
`SELECTED_DIRECTION_IDS`, `EFFECTIVE_CONCURRENCY`, `MAX_WORKER_ITERATIONS`,
6165
`WORKER_TIMEOUT_MIN`, `CODEX_TIMEOUT_MIN`, `WORKER_PROMPT_TEMPLATE`, `REPORT_TEMPLATE`.
6266
Continue to Phase 2.
67+
Parse values by splitting each line on the first literal `": "` only. Values can contain additional colons, for example `CODEX_REVIEW_MODEL_SPEC: gpt-5.5:xhigh`.
6368
- `1`: Report "No input path provided" and stop.
6469
- `2`: Report "Input file not found" and stop.
6570
- `3`: Report "Companion .directions.json missing — regenerate the idea draft with `/humanize:gen-idea`" and stop.
6671
- `4`: Report "Input must be a .directions.json or .md file" and stop.
6772
- `5`: Report "Directions JSON failed schema validation" and stop.
6873
- `6`: Report the specific cap or argument error from stderr and stop.
6974
- `7`: Report "Main checkout has uncommitted tracked changes — commit or stash before exploring" and stop.
70-
- `8`: Report "Run directory collision — wait one second and retry" and stop.
75+
- `8`: Report "Run directory collision — retry to generate a fresh run id" and stop.
7176
- `9`: Report "Template file missing — plugin configuration error" and stop.
7277

7378
Load the directions JSON:
@@ -87,8 +92,11 @@ Display a pre-dispatch summary to the user and require explicit confirmation bef
8792
Input: <DIRECTIONS_JSON_FILE>
8893
Draft: <DRAFT_PATH or "(direct .directions.json input)">
8994
Run directory: <RUN_DIR>
95+
Run slug: <RUN_SLUG>
9096
Base branch: <BASE_BRANCH>
9197
Base commit: <BASE_COMMIT>
98+
Explore report: <REPORT_PATH>
99+
Final idea: <FINAL_IDEA_PATH>
92100
93101
Selected directions (<N> of <total>):
94102
[1] <direction_id>: <name>
@@ -99,6 +107,9 @@ Effective concurrency: <EFFECTIVE_CONCURRENCY>
99107
Worker iteration cap: <MAX_WORKER_ITERATIONS>
100108
Worker timeout: <WORKER_TIMEOUT_MIN> min
101109
Codex timeout: <CODEX_TIMEOUT_MIN> min
110+
Codex review model: <CODEX_REVIEW_MODEL>
111+
Codex review effort: <CODEX_REVIEW_EFFORT>
112+
Codex review model spec: <CODEX_REVIEW_MODEL_SPEC>
102113
103114
WARNING: Workers will create local git worktrees, branches, and commits.
104115
Workers will run targeted tests and invoke Codex.
@@ -140,6 +151,7 @@ For each selected direction (in `SELECTED_DIRECTION_IDS`):
140151
- `<CONFIDENCE>``confidence`
141152
- `<MAX_WORKER_ITERATIONS>``MAX_WORKER_ITERATIONS`
142153
- `<CODEX_TIMEOUT_MIN>``CODEX_TIMEOUT_MIN`
154+
- `<CODEX_REVIEW_MODEL_SPEC>``CODEX_REVIEW_MODEL_SPEC` from validation stdout (expected rendered value: `gpt-5.5:xhigh`)
143155
- `<BASE_BRANCH>``BASE_BRANCH`
144156
- `<BASE_COMMIT>``BASE_COMMIT`
145157
- `<ORIGINAL_IDEA>``original_idea` from the directions JSON
@@ -163,6 +175,10 @@ Write `<RUN_DIR>/manifest.json` with all coordinator fields:
163175
"max_worker_iterations": <MAX_WORKER_ITERATIONS>,
164176
"worker_timeout_min": <WORKER_TIMEOUT_MIN>,
165177
"codex_timeout_min": <CODEX_TIMEOUT_MIN>,
178+
"codex_review_model": "<CODEX_REVIEW_MODEL>",
179+
"codex_review_effort": "<CODEX_REVIEW_EFFORT>",
180+
"report_path": "<REPORT_PATH>",
181+
"final_idea_path": "<FINAL_IDEA_PATH>",
166182
"expected_worker_count": <selected count>,
167183
"runtime_spike_status": "not_validated",
168184
"workers": [
@@ -204,7 +220,7 @@ The agent must create a branch named `explore/<RUN_ID>/<dir_slug>` in its worktr
204220

205221
If any agent fails to start, record a coordinator-generated failure row in `worker-results.jsonl`:
206222
```json
207-
{"schema_version": 1, "run_id": "<RUN_ID>", "direction_id": "<id>", "dir_slug": "<slug>", "task_status": "failed", "error": "worker failed to start", "codex_final_verdict": "unavailable", "rounds_used": 0, "tests_passed": 0, "tests_failed": 0, "worktree_path": "", "branch_name": "explore/<RUN_ID>/<slug>", "commit_sha": "", "commit_count": 0, "dirty_state": "unknown", "commit_status": "none", "summary_markdown": "", "what_worked": [], "what_didnt": [], "bitlesson_action": "none"}
223+
{"schema_version": 1, "run_id": "<RUN_ID>", "direction_id": "<id>", "dir_slug": "<slug>", "task_status": "failed", "error": "worker failed to start", "expected_codex_review_model": "<CODEX_REVIEW_MODEL>", "expected_codex_review_effort": "<CODEX_REVIEW_EFFORT>", "codex_review_model": "", "codex_review_effort": "", "codex_review_metadata_path": "", "codex_final_verdict": "unavailable", "rounds_used": 0, "tests_passed": 0, "tests_failed": 0, "worktree_path": "", "branch_name": "explore/<RUN_ID>/<slug>", "commit_sha": "", "commit_count": 0, "dirty_state": "unknown", "commit_status": "none", "summary_markdown": "", "what_worked": [], "what_didnt": [], "bitlesson_action": "none"}
208224
```
209225

210226
---
@@ -226,7 +242,7 @@ For each worker agent result:
226242
3. If parsing succeeds, append the JSON object as one line to `<RUN_DIR>/worker-results.jsonl`.
227243
4. If JSON parsing fails or sentinels are absent, append a coordinator-generated `no_summary` row:
228244
```json
229-
{"schema_version": 1, "run_id": "<RUN_ID>", "direction_id": "<id>", "dir_slug": "<slug>", "task_status": "no_summary", "error": "worker did not emit valid JSON result", "codex_final_verdict": "unavailable", "rounds_used": 0, "tests_passed": 0, "tests_failed": 0, "worktree_path": "", "branch_name": "explore/<RUN_ID>/<slug>", "commit_sha": "", "commit_count": 0, "dirty_state": "unknown", "commit_status": "none", "summary_markdown": "", "what_worked": [], "what_didnt": [], "bitlesson_action": "none"}
245+
{"schema_version": 1, "run_id": "<RUN_ID>", "direction_id": "<id>", "dir_slug": "<slug>", "task_status": "no_summary", "error": "worker did not emit valid JSON result", "expected_codex_review_model": "<CODEX_REVIEW_MODEL>", "expected_codex_review_effort": "<CODEX_REVIEW_EFFORT>", "codex_review_model": "", "codex_review_effort": "", "codex_review_metadata_path": "", "codex_final_verdict": "unavailable", "rounds_used": 0, "tests_passed": 0, "tests_failed": 0, "worktree_path": "", "branch_name": "explore/<RUN_ID>/<slug>", "commit_sha": "", "commit_count": 0, "dirty_state": "unknown", "commit_status": "none", "summary_markdown": "", "what_worked": [], "what_didnt": [], "bitlesson_action": "none"}
230246
```
231247

232248
### 5.2: Coordinator Error Handling
@@ -246,18 +262,23 @@ After collecting all results, update the `workers` array in `manifest.json` to s
246262

247263
---
248264

249-
## Phase 6: Report Synthesis
265+
## Phase 6: Artifact Synthesis
250266

251-
Generate `<RUN_DIR>/report.md` by reading `REPORT_TEMPLATE` and synthesizing results.
267+
Generate the canonical run artifacts:
268+
- `<REPORT_PATH>` (`explore-report.md`) by reading `REPORT_TEMPLATE` and synthesizing results.
269+
- `<FINAL_IDEA_PATH>` (`final-idea.md`) by reading `FINAL_IDEA_TEMPLATE` and producing a plan-ready synthesis for `/humanize:gen-plan`.
270+
271+
Do not create any legacy compatibility alias for the report.
252272

253273
### 6.1: Load Results
254274

255275
Read `<RUN_DIR>/worker-results.jsonl` (one JSON object per line).
256276
Read the full directions JSON from `DIRECTIONS_JSON_FILE`.
277+
Read `REPORT_TEMPLATE` and `FINAL_IDEA_TEMPLATE`.
257278

258279
### 6.2: Two-Tier Ranking
259280

260-
The report contains two ranking sections:
281+
The explore report contains two ranking sections:
261282

262283
**Tier 1: Best Product Direction**
263284
Rank all directions (even failed workers) on:
@@ -277,6 +298,28 @@ Rank only workers that produced a result on:
277298
- `dirty_state` (clean > dirty > unknown)
278299
- `rounds_used` (fewer is better, given same quality)
279300

301+
Template substitutions for `REPORT_TEMPLATE` include:
302+
- `<RUN_ID>``RUN_ID`
303+
- `<BASE_BRANCH>``BASE_BRANCH`
304+
- `<BASE_COMMIT>``BASE_COMMIT`
305+
- `<CREATED_AT>` → the report creation timestamp
306+
- `<REPORT_PATH>``REPORT_PATH`
307+
- `<FINAL_IDEA_PATH>``FINAL_IDEA_PATH`
308+
- `<SUMMARY_PARAGRAPH>` → run summary
309+
- `<PRODUCT_DIRECTION_RANKING_ROWS>` → Tier 1 rows
310+
- `<PRODUCT_DIRECTION_RATIONALE>` → Tier 1 rationale
311+
- `<IMPLEMENTATION_RANKING_ROWS>` → Tier 2 rows
312+
- `<IMPLEMENTATION_RANKING_RATIONALE>` → Tier 2 rationale
313+
- `<WORKER_RESULT_ENTRIES>` → summarized worker results
314+
- `<WINNER_WORKTREE_PATH>` → winning worker worktree path
315+
- `<WINNER_BRANCH_NAME>` → winning worker branch name
316+
- `<WINNER_COMMIT_SHA>` → winning worker commit SHA
317+
- `<COMMIT_SHA>` → prototype commit SHA for cherry-pick examples
318+
- `<CLEANUP_COMMANDS>` → cleanup commands for non-adopted prototypes
319+
- `<ALL_WORKER_DETAILS>` → complete worker details
320+
- `<ALL_WORKTREE_REMOVE_COMMANDS>` → worktree removal commands
321+
- `<ALL_BRANCH_DELETE_COMMANDS>` → branch deletion commands
322+
280323
### 6.3: Adoption Paths
281324

282325
For each worker result, include an adoption path section with:
@@ -285,7 +328,31 @@ For each worker result, include an adoption path section with:
285328
- Commit SHA: `commit_sha`
286329
- Suggested next command (e.g., `cd <worktree_path> && /humanize:start-rlcr-loop`)
287330

288-
### 6.4: Cleanup Guidance
331+
### 6.4: Final Idea Synthesis
332+
333+
Write `<FINAL_IDEA_PATH>` from `FINAL_IDEA_TEMPLATE`. It must be a plan-ready synthesis, not another audit report:
334+
- Select the final recommended direction, or explicitly state that no direction is ready if evidence does not support adoption.
335+
- Carry forward the winning direction's rationale, approach summary, objective evidence, constraints, and known risks.
336+
- Summarize explore outcomes from `worker-results.jsonl`: worker status, Codex verdict, tests, commits, dirty state, and relevant implementation findings.
337+
- Include cross-direction learnings that affect the final implementation plan.
338+
- Include the command `/humanize:gen-plan --input <FINAL_IDEA_PATH> --output <plan-path>`.
339+
340+
Template substitutions for `FINAL_IDEA_TEMPLATE` include:
341+
- `<TITLE>` → a concise title for the synthesized final approach
342+
- `<RUN_ID>``RUN_ID`
343+
- `<DIRECTIONS_JSON_FILE>``DIRECTIONS_JSON_FILE`
344+
- `<REPORT_PATH>``REPORT_PATH`
345+
- `<FINAL_IDEA_PATH>``FINAL_IDEA_PATH`
346+
- `<FINAL_RECOMMENDATION>` → the chosen plan-ready recommendation
347+
- `<RATIONALE>` → synthesis rationale
348+
- `<APPROACH_SUMMARY>` → final approach summary
349+
- `<OBJECTIVE_EVIDENCE>` → evidence list
350+
- `<EXPLORE_OUTCOMES>` → worker-derived outcomes
351+
- `<CONSTRAINTS>` → implementation constraints
352+
- `<KNOWN_RISKS>` → risk list
353+
- `<CROSS_DIRECTION_LEARNINGS>` → learnings from non-adopted directions
354+
355+
### 6.5: Cleanup Guidance
289356

290357
Include shell commands to remove non-adopted worktrees and branches:
291358
```bash
@@ -294,13 +361,15 @@ git worktree remove --force <worktree_path>
294361
git branch -D <branch_name>
295362
```
296363

297-
### 6.5: Failure Report
364+
### 6.6: Failure Artifacts
298365

299-
If all workers failed (`.failed` exists), still write `report.md` with:
366+
If all workers failed (`.failed` exists), still write `<REPORT_PATH>` with:
300367
- Failure summary table (direction_id, dir_slug, task_status, error)
301368
- Cleanup guidance for any partially created worktrees
302369
- No ranking sections
303370

371+
Also write `<FINAL_IDEA_PATH>` with a clear "no adoption recommended" final recommendation and the evidence needed before retrying or planning.
372+
304373
---
305374

306375
## Error Handling Summary
@@ -311,5 +380,5 @@ If all workers failed (`.failed` exists), still write `report.md` with:
311380
| User denies confirmation | Stop. No manifest, no worktrees. |
312381
| `manifest.json` write fails | Write `.failed`. Stop. |
313382
| One worker fails | Record failure row. Continue remaining workers. |
314-
| All workers fail | Write `.failed`. Update manifest. Write failure report. |
383+
| All workers fail | Write `.failed`. Update manifest. Write failure artifacts. |
315384
| Result collection error for one worker | Record error row. Continue. |

docs/usage.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ Generates a repo-grounded idea draft using directed-diversity exploration. A lea
9191
/humanize:explore-idea <draft.md | draft.directions.json> [--directions ids] [--concurrency N] [--max-worker-iterations N] [--worker-timeout-min N] [--codex-timeout-min N]
9292
```
9393

94-
Launches bounded parallel prototype workers — one per selected direction — each running in an isolated git worktree. After all workers complete, synthesizes a two-tier ranking report:
94+
Launches bounded parallel prototype workers — one per selected direction — each running in an isolated git worktree. After all workers complete, synthesizes an explore report plus a plan-ready final idea:
9595
- **Tier 1**: Best product direction (ranked by user value, evidence, strategic fit)
9696
- **Tier 2**: Most implementation-ready prototype (ranked by outcome: task status, Codex verdict, tests, commits)
9797

@@ -106,7 +106,8 @@ Launches bounded parallel prototype workers — one per selected direction — e
106106
- `manifest.json` — coordinator state and per-worker metadata
107107
- `dispatch-prompts/` — exact prompts sent to each worker
108108
- `worker-results.jsonl` — machine-readable result rows
109-
- `report.md` — synthesis report with two-tier rankings and adoption paths
109+
- `explore-report.md` — audit report with two-tier rankings, adoption paths, and cleanup guidance
110+
- `final-idea.md` — plan-ready synthesis artifact for `/humanize:gen-plan`
110111

111112
### start-rlcr-loop
112113

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# <TITLE>
2+
3+
## Run Context
4+
5+
- Run ID: <RUN_ID>
6+
- Directions JSON: <DIRECTIONS_JSON_FILE>
7+
- Explore Report: <REPORT_PATH>
8+
- Final Idea: <FINAL_IDEA_PATH>
9+
10+
## Final Recommendation
11+
12+
<FINAL_RECOMMENDATION>
13+
14+
## Rationale
15+
16+
<RATIONALE>
17+
18+
## Approach Summary
19+
20+
<APPROACH_SUMMARY>
21+
22+
## Objective Evidence
23+
24+
<OBJECTIVE_EVIDENCE>
25+
26+
## Explore Outcomes
27+
28+
<EXPLORE_OUTCOMES>
29+
30+
## Constraints
31+
32+
<CONSTRAINTS>
33+
34+
## Known Risks
35+
36+
<KNOWN_RISKS>
37+
38+
## Cross-Direction Learnings
39+
40+
<CROSS_DIRECTION_LEARNINGS>
41+
42+
## Suggested Gen-Plan Command
43+
44+
```bash
45+
/humanize:gen-plan --input <FINAL_IDEA_PATH> --output <plan-path>
46+
```

prompt-template/explore/report-template.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
1-
# explore-idea Run Report
1+
# explore-idea Explore Report
22

33
**Run ID:** <RUN_ID>
44
**Base Branch:** <BASE_BRANCH>
55
**Base Commit:** <BASE_COMMIT>
66
**Created At:** <CREATED_AT>
7+
**Explore Report:** <REPORT_PATH>
8+
**Final Idea:** <FINAL_IDEA_PATH>
79

810
---
911

@@ -64,12 +66,12 @@ cd <WINNER_WORKTREE_PATH>
6466
/humanize:start-rlcr-loop --skip-impl
6567
```
6668

67-
### Restart From Plan
69+
### Generate Plan From Final Idea
6870

69-
Use the winning direction's approach summary as input to `/humanize:gen-plan`:
71+
Use the plan-ready final idea synthesis as input to `/humanize:gen-plan`:
7072

7173
```bash
72-
/humanize:gen-plan --input <DRAFT_PATH> --output <plan-path>
74+
/humanize:gen-plan --input <FINAL_IDEA_PATH> --output <plan-path>
7375
```
7476

7577
### Cherry-Pick Prototype
@@ -106,7 +108,8 @@ All explore run artifacts are stored in:
106108
manifest.json — coordinator state and per-worker metadata
107109
dispatch-prompts/ — exact prompts sent to each worker
108110
worker-results.jsonl — machine-readable result rows
109-
report.md — this report
111+
explore-report.md — audit, ranking, adoption, and cleanup report
112+
final-idea.md — plan-ready synthesis artifact for gen-plan
110113
```
111114

112115
To remove all local explore artifacts for this run:

0 commit comments

Comments
 (0)