Skip to content

Commit 45b63fc

Browse files
SmithSmith
authored andcommitted
feat(planner): update artifact persistence semantics and introduce revision policies
- Changed goal descriptions in planner-schema-output and planner-terminal-status-artifacts scenarios to reflect "persisted_artifact" instead of "plan_file_created". - Enhanced planner-terminal-status-artifacts to handle in-place updates without forcing new paths. - Updated robustness-paraphrase scenario to use "persisted_artifact". - Added tests for read-only edit denylist to ensure no edit tools are granted to specific agents. - Introduced a new architecture decision record for plan artifact revision policy, detailing in-place updates and supersession modes. - Created scenarios for planner in-place plan revision and read-only agent tool denylist to validate new policies. - Updated schemas to enforce new revision mode requirements and clarify artifact handling in orchestration.
1 parent e241c77 commit 45b63fc

35 files changed

Lines changed: 725 additions & 83 deletions

NOTES.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,6 @@
22

33
Repo-persistent active-objective state only. See `docs/agent-engineering/MEMORY-ARCHITECTURE.md` for the three-layer memory model; task-episodic history lives under `plans/artifacts/<task-slug>/`.
44

5-
- Active objective: comprehensive project optimization complete; awaiting user review of completion summary.
5+
- Active objective: plan editing policy reform complete; awaiting user review.
66
- Blockers: none.
7-
- Pending: user review before any commit or package-release action.
7+
- Pending: user review before any commit.

Orchestrator.agent.md

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,14 @@ Do NOT use `vscode/askQuestions` for questions answerable from codebase evidence
7474
- Include `trace_id`, `iteration_index`, and `max_iterations` in every gate-event emission per `schemas/orchestrator.gate-event.schema.json`.
7575
- Purpose: enable log correlation across multi-agent orchestration chains.
7676

77+
### Planner Revision Modes
78+
- Use `revision_mode: initial_create` when no active plan exists.
79+
- Use `revision_mode: in_place_update` for ordinary PLAN_REVIEW fixes to an active draft/current plan. The payload-selected path is `active_plan_path`, and Planner must return the same `plan_path`.
80+
- Use `revision_mode: new_artifact_supersession` only for accepted-baseline replacement, user-requested new artifacts, material invalidation, or independent citation needs. The payload-selected path is `existing_plan_path`, and the new Planner output should set `revision_of` to that prior path.
81+
- Apply the Universal Model Resolution Rule before every Planner dispatch. For replan/update dispatches, the outer `agent/runSubagent` call must include the resolved outer `model`, and the Planner payload must include payload-level `model`, `trace_id`, review-loop `iteration_index`, `revision_mode`, `revision_reason`, and exactly the selected path field for the mode: `active_plan_path` for `in_place_update` or `existing_plan_path` for `new_artifact_supersession`.
82+
- Serialize write-capable Planner revisions by `(trace_id, active_plan_path)`. Never run two write-capable Planner updates to the same plan in parallel; parallel review agents may read the same `plan_path` but must not edit it.
83+
- Phase 3 structural validation is not behavior-complete. `cd evals && npm run test:structural` confirms schema structure and legacy compatibility only; Phase 4 owns conditional enforcement behavior tests and scenario migration for `revision_mode`, selected path fields, `trace_id`, and `iteration_index`.
84+
7785
## Archive
7886

7987
### Context Compaction Policy
@@ -193,6 +201,7 @@ For `CodeReviewer-subagent`, `PlanAuditor-subagent`, and `AssumptionVerifier-sub
193201
**Dispatch:**
194202
- Apply the Universal Model Resolution Rule. Before a plan `complexity_tier` exists (no `plan_path` yet), use Planner's top-level `primary` model — never omit `model` because tier context is missing.
195203
- Dispatch Planner as an **entry-point delegate, not a phase executor**, with:
204+
- `revision_mode: initial_create` because no active plan exists. Phase 3 keeps the schema field optional for legacy fixture compatibility, but live initial planning dispatch uses this mode.
196205
- The original user request.
197206
- The current `trace_id` and known constraints.
198207
- Any evidence already gathered.
@@ -238,12 +247,12 @@ For `CodeReviewer-subagent`, `PlanAuditor-subagent`, and `AssumptionVerifier-sub
238247
4. If PlanAuditor `APPROVED` AND (AssumptionVerifier not dispatched OR zero BLOCKING mirages):
239248
- If ExecutabilityVerifier is in scope for the current tier or HIGH-risk override: dispatch ExecutabilityVerifier-subagent (apply Universal Model Resolution Rule) with `plan_path`.
240249
- If ExecutabilityVerifier `PASS` or not in scope → plan APPROVED, exit loop.
241-
- If ExecutabilityVerifier `FAIL`/`WARN` → route findings to Planner, increment `iteration_index`.
242-
5. If PlanAuditor `NEEDS_REVISION` or AssumptionVerifier has BLOCKING mirages → route combined findings to Planner, increment `iteration_index`.
250+
- If ExecutabilityVerifier `FAIL`/`WARN`increment `iteration_index` and route findings to Planner using `revision_mode: in_place_update` unless the Planner Revision Modes criteria require `new_artifact_supersession`.
251+
5. If PlanAuditor `NEEDS_REVISION` or AssumptionVerifier has BLOCKING mirages → increment `iteration_index` and route combined findings to Planner using `revision_mode: in_place_update` unless the Planner Revision Modes criteria require `new_artifact_supersession`.
243252
6. **Convergence Detection:** If `iteration_index ≥ 3` and score improvement over previous 2 iterations < 5% → stagnation. Present findings summary to user with `WAITING_APPROVAL`.
244253
7. If `iteration_index > max_iterations` → present best plan version and unresolved issues to user.
245254
- **Regression Tracking:** At `iteration_index > 1`, load verified items from previous iteration. Pass to PlanAuditor as context. Any previously verified item that now fails → automatic BLOCKING regression issue.
246-
- **Lineage Contract:** When incrementing `iteration_index` and routing a REPLAN-with-new-plan-path, the new plan SHOULD carry `revision_of` set to the prior plan path. Auditor outputs that mark a same-finding recurrence SHOULD carry `regression_iteration` + `regression_finding_id` on the relevant finding object to enable per-finding regression tracing across iterations.
255+
- **Lineage Contract:** `revision_of` is supersession lineage only. Use it when `revision_mode: new_artifact_supersession` creates a replacement plan artifact; do not require it for `revision_mode: in_place_update`, where successive review iterations may refer to the same `plan_path`. Auditor outputs that mark a same-finding recurrence SHOULD carry `regression_iteration` + `regression_finding_id` on the relevant finding object to enable per-finding regression tracing across iterations.
247256
- If trigger conditions are not met: skip directly to Implementation Loop.
248257

249258
5. **Implementation Loop (Per Phase)**
@@ -327,7 +336,7 @@ When a subagent returns a `failure_classification`, Orchestrator routes automati
327336
| `escalate` | STOP — transition to `WAITING_APPROVAL`, present to user | 0 |
328337
| `model_unavailable` | Retry the same agent up to `retry_budgets.model_unavailable_max` times; on exhaustion, escalate to user via `WAITING_APPROVAL` | retry_budgets.model_unavailable_max |
329338

330-
If retry limit is exhausted, escalate to user with accumulated failure evidence. For all dispatch actions in this table (retry or replan), apply the Universal Model Resolution Rule to resolve the `model` parameter — including needs_replan Planner dispatch.
339+
If retry limit is exhausted, escalate to user with accumulated failure evidence. For all dispatch actions in this table (retry or replan), apply the Universal Model Resolution Rule to resolve the `model` parameter — including needs_replan Planner dispatch. A `needs_replan` Planner dispatch that updates an active plan must follow Planner Revision Modes: include outer `model`, payload-level `model`, `trace_id`, review-loop `iteration_index`, `revision_mode`, `revision_reason`, and either `active_plan_path` for `in_place_update` or `existing_plan_path` for `new_artifact_supersession`.
331340

332341
### Diagnosis Packet (MEDIUM/LARGE — Fixable Retries)
333342

Planner.agent.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
description: 'Autonomous planner that writes comprehensive implementation plans and feeds them to Orchestrator'
3-
tools: [read/readFile, agent, agent/runSubagent, edit/createFile, search/codebase, search/fileSearch, search/listDirectory, search/textSearch, search/usages, web/fetch, web/githubRepo, vscode/askQuestions, vscode/getProjectSetupInfo, io.github.upstash/context7/get-library-docs, io.github.upstash/context7/resolve-library-id]
3+
tools: [read/readFile, agent, agent/runSubagent, edit/createFile, edit/editFiles, search/codebase, search/fileSearch, search/listDirectory, search/textSearch, search/usages, web/fetch, web/githubRepo, vscode/askQuestions, vscode/getProjectSetupInfo, io.github.upstash/context7/get-library-docs, io.github.upstash/context7/resolve-library-id]
44
agents: ["CodeMapper-subagent", "Researcher-subagent"]
55
model: GPT-5.5 (copilot)
66
model_role: capable-planner
@@ -25,6 +25,7 @@ Produce implementation plans that are deterministic, schema-compliant, and execu
2525
- No direct implementation.
2626
- No code execution.
2727
- No edits outside plan artifacts.
28+
- For `revision_mode: in_place_update`, edit only the supplied `active_plan_path`; any other file edit is out of scope unless separately authorized by an implementation phase plan.
2829
- No ownership of PLAN_REVIEW, approval gates, execution gating, or todo lifecycle — those belong to Orchestrator.
2930
- No invoking PlanAuditor-subagent, AssumptionVerifier-subagent, or ExecutabilityVerifier-subagent as part of standard plan generation. The `complexity_tier` field in the plan output signals to Orchestrator which review agents to activate.
3031
- No delegation to agents outside the project-internal delegation roster documented in `plans/project-context.md`.
@@ -33,7 +34,8 @@ Produce implementation plans that are deterministic, schema-compliant, and execu
3334
- Output must conform to `schemas/planner.plan.schema.json`.
3435
- Every phase MUST declare exactly one machine-readable `executor_agent` from the supported executor set in `plans/project-context.md`.
3536
- If confidence is below 0.9 (see `governance/runtime-policy.json` `confidence_threshold`) or evidence is missing, set status to `ABSTAIN` or `REPLAN_REQUIRED`. Use `ABSTAIN` when evidence is insufficient to decompose even after clarification and research. Use `REPLAN_REQUIRED` when scope is understood but the current design is invalidated (dependency changed, architectural assumption reversed). Both statuses require a markdown plan artifact with diagnostics and a recovery next step.
36-
- **Lineage:** When producing a revised plan in response to a REPLAN-with-new-plan-path, set the optional `revision_of` field to the prior plan's path. This establishes iter-N traceability without breaking iter-1 fixtures (field is optional).
37+
- **Revision modes:** `initial_create` creates the first plan artifact when no active plan exists. `in_place_update` applies ordinary PLAN_REVIEW fixes only to the supplied `active_plan_path` and returns the same `plan_path`. `new_artifact_supersession` creates a new plan artifact when Orchestrator requests accepted-baseline replacement, user-requested new artifacts, material invalidation, or independent citation.
38+
- **Lineage:** Set the optional `revision_of` field only when producing a new superseding plan artifact under `revision_mode: new_artifact_supersession`. This establishes supersession traceability without breaking iter-1 fixtures (field is optional) and is not used for `in_place_update`.
3739

3840
### Mandatory Workflow Procedure
3941
1. Idea Interview Gate: BEFORE the Clarification Gate, evaluate whether the user request is vague or abstract. Trigger condition: the request contains **all three** of — (a) no specific file names or paths, (b) no concrete acceptance criteria, (c) no explicit technology or constraint named. If triggered, load `skills/patterns/idea-to-prompt.md` and execute the 5-step interview protocol using `vscode/askQuestions`. Replace the original vague request with the structured prompt assembled at the end of Step 5. Skip this gate entirely if any single concrete signal is present (a file path, an agent name, a schema reference, or a measurable goal).
@@ -63,7 +65,7 @@ For all other scopes, record applicability, impact, evidence source, and disposi
6365
- **LARGE:** Always include a Mermaid `sequenceDiagram` alongside the phase dependency DAG.
6466
- Record design decisions in the plan artifact's "Design Decisions" section (see plan document template).
6567
8. Planning (phase decomposition with quality gates).
66-
9. Handoff (artifact-first plan file plus `plan_path` handoff for Orchestrator; PLAN_REVIEW ownership remains with Orchestrator).
68+
9. Handoff (artifact-first plan persistence plus `plan_path` handoff for Orchestrator; PLAN_REVIEW ownership remains with Orchestrator).
6769
- For MEDIUM/LARGE plans where Researcher produced a non-trivial evidence packet, set `context_packet_path` in the plan to the research digest artifact path so downstream executors can consume it without re-investigation.
6870

6971
### Clarification Policy
@@ -123,7 +125,7 @@ Agent-specific additions:
123125
- `vscode/getProjectSetupInfo` for automatic project stack detection (framework, language, package manager).
124126
- `vscode/askQuestions` for resolving mandatory clarification classes — present structured options before planning.
125127
- `io.github.upstash/context7/resolve-library-id` and `io.github.upstash/context7/get-library-docs` for third-party library documentation lookup when plans depend on external frameworks or APIs.
126-
- Markdown plan file creation in plan directory.
128+
- Markdown plan file creation in the plan directory and scoped `in_place_update` edits to the Orchestrator-supplied `active_plan_path` only.
127129

128130
### Disallowed
129131
- Any implementation or code execution action.
@@ -149,7 +151,7 @@ When a plan depends on third-party library behavior, framework APIs, or MCP inte
149151
## Output Requirements
150152

151153
When complete, follow this output procedure **in mandatory order** — the artifact must be saved before any chat response is produced:
152-
1. **Create the markdown plan file first.** Save it at `<plan-directory>/<task-name>-plan.md` using `plans/templates/plan-document-template.md` as the authoritative artifact structure. The plan file must remain consistent with `schemas/planner.plan.schema.json`. Do not produce any chat output until the file is saved.
154+
1. **Persist the markdown plan artifact first.** For `initial_create`, create a new file at `<plan-directory>/<task-name>-plan.md` using `plans/templates/plan-document-template.md` as the authoritative artifact structure. For `in_place_update`, edit only the supplied `active_plan_path` and return that same path as `plan_path`. For `new_artifact_supersession`, create a new plan artifact and set `revision_of` to the prior `existing_plan_path`. The plan file must remain consistent with `schemas/planner.plan.schema.json`. Do not produce any chat output until the file is saved.
153155
2. **Then provide a concise handoff message.** The handoff message must include: the saved plan file path, a one-paragraph approach summary, and the recommended first phase. It must NOT contain inline phase breakdowns, risk tables, plan bodies, or todo/checklist management language. All plan detail belongs in the saved artifact, not in chat.
154156

155157
### Plan Document Template

docs/agent-engineering/ADR-PROCESS.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,13 +30,13 @@ ADRs must strictly adhere to the `ADR-TEMPLATE.md` format. They must include exa
3030

3131
## Storage Convention
3232

33-
ADRs are stored in the `docs/architecture-decisions/` directory using the naming format `NNNN-kebab-title.md` (where `NNNN` is a zero-padded sequential number, e.g., `0001-initial-architecture.md`). The directory will be created when the first ADR is written.
33+
ADRs are stored as individual append-only records in the `docs/architecture-decisions/` directory using the naming format `NNNN-kebab-title.md` (where `NNNN` is a zero-padded sequential number, e.g., `0001-initial-architecture.md`). The directory will be created when the first ADR is written. Unlike active plans which undergo `in_place_update` revisions, ADRs preserve historical context.
3434

3535
## Status Lifecycle
3636

3737
1. **Proposed**: The decision is drafted and under review along with the associated plan or PR.
3838
2. **Accepted**: The decision has been approved and is being implemented or has been implemented.
39-
3. **Superseded**: The decision is no longer active and has been replaced by a newer decision (accompanied by a link to the superseding ADR).
39+
3. **Superseded**: The decision is no longer active and has been replaced by a newer decision (accompanied by a link to the superseding ADR, following a `new_artifact_supersession` model).
4040

4141
## Review Expectations
4242

docs/agent-engineering/AGENT-AS-TOOL.md

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,32 @@
11
# Agent-as-Tool — Subagent Input Contract (Spec Only)
22

33
## Purpose
4+
45
Codify the minimum input contract every ControlFlow subagent must accept, so that each agent can be surfaced in the future as an MCP tool or native VS Code chat tool without re-designing its signature. This document is specification-only; no runtime wiring is introduced by Phase 8.
56

67
## Scope
8+
79
Applies to all 12 subagents reachable from the Orchestrator (see `plans/project-context.md`). The Orchestrator itself is an orchestration surface, not a tool.
810

911
## Required Input Fields
1012

1113
| Field | Type | Description |
12-
|-------|------|-------------|
14+
| ----- | ---- | ----------- |
1315
| `scope` | string (prose) | Concrete description of the delegated unit of work. Bounded, verifiable, and scoped to a single phase or sub-phase. |
1416
| `context_refs` | array of strings | File paths or artifact URIs the agent must read before acting. Relative to repo root. Empty array allowed only for purely generative tasks. |
1517
| `trace_id` | UUIDv4 string | Per-task correlation identifier generated by the Orchestrator. See `docs/agent-engineering/OBSERVABILITY.md`. |
1618

17-
Absence of any required field ⇒ subagent returns status `NEEDS_INPUT` with a structured `clarification_request` per `docs/agent-engineering/CLARIFICATION-POLICY.md`.
19+
Absence of any required field implies missing context:
20+
21+
- Tool/native wrapper validation should catch missing required fields before invocation where the agent schema lacks `NEEDS_INPUT`.
22+
- Agents with `NEEDS_INPUT` in their schema may use `NEEDS_INPUT` with a structured `clarification_request` per `docs/agent-engineering/CLARIFICATION-POLICY.md`.
23+
- Read-only/review agents should use their schema-supported `ABSTAIN`, `INSUFFICIENT_EVIDENCE`, or failure path.
1824

1925
## Recommended Input Fields
2026

2127
| Field | Type | Description |
22-
|-------|------|-------------|
23-
| `iteration_index` | integer ≥ 0 | Current iteration in a bounded review loop. |
28+
| ----- | ---- | ----------- |
29+
| `iteration_index` | integer ≥ 0 | Current iteration in a bounded review loop. Propagated through Planner replan/update payloads. |
2430
| `max_iterations` | integer ≥ 1 | Cap from `runtime-policy.json → max_iterations_by_tier`. |
2531
| `retry_attempt` | integer ≥ 0 | Attempt counter for reliability signaling, drawn from existing `retry_budgets` (see `RELIABILITY-GATES.md`). Not a new counter. |
2632
| `budget_context` | object | Optional resource budget envelope per `skills/patterns/budget-tracking.md`. |
@@ -29,6 +35,7 @@ Absence of any required field ⇒ subagent returns status `NEEDS_INPUT` with a s
2935
Recommended fields improve determinism and observability but are not required for contract conformance.
3036

3137
## Return Shape
38+
3239
Every subagent returns a structured execution report that **already** satisfies the agent-as-tool return contract:
3340

3441
- `schemas/core-implementer.execution-report.schema.json`
@@ -47,6 +54,7 @@ Every subagent returns a structured execution report that **already** satisfies
4754
Phase 5 added optional `trace_id` to every report schema, which closes the correlation loop required for tool surfacing. No additional schema change is required.
4855

4956
## Forward Compatibility — MCP / Native Tool Surfacing
57+
5058
Each subagent is designed to be surfaced as a tool with the following frontmatter-described I/O contract:
5159

5260
```yaml
@@ -67,12 +75,14 @@ output_schema: schemas/<agent>.<report|verdict|discovery|plan|plan-audit>.schema
6775
When Phase N (future) implements surfacing, no contract-breaking field changes are needed — only the registration wrapper.
6876
6977
## Non-Goals (Phase 8)
78+
7079
- No MCP server is registered.
7180
- No VS Code tool manifest is changed.
7281
- No existing agent frontmatter is mutated.
7382
- No schema `required` arrays are changed.
7483

7584
## Cross-References
85+
7686
- `docs/agent-engineering/OBSERVABILITY.md` — `trace_id` generation and propagation.
7787
- `docs/agent-engineering/TOOL-ROUTING.md` — tool selection policy once agents are surfaced as tools.
7888
- `docs/agent-engineering/RELIABILITY-GATES.md` — retry/escalation semantics carried by `retry_attempt`.

0 commit comments

Comments
 (0)