You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add ControlFlow planning and review skills with supporting documentation
- Introduced complexity tiers for task planning in complexity-tiers.md.
- Added ControlFlow portability notes for adapting practices in controlflow-portability.md.
- Established LLM behavior guidelines to enhance coding practices in llm-behavior-guidelines.md.
- Created a comprehensive plan template in plan-template.md for structured task execution.
- Defined a planner output contract to standardize plan artifacts in planner-output-contract.md.
- Developed a semantic risk taxonomy to categorize planning risks in semantic-risk-taxonomy.md.
- Implemented controlflow-review skill to facilitate code and phase reviews in SKILL.md.
- Established evidence discipline and review checklists to enhance review quality in evidence-discipline.md and review-checklist.md.
- Created security review discipline guidelines for focused security assessments in security-review-discipline.md.
- Defined validation status labels for findings in validation-status.md.
- Added controlflow-router skill to streamline task routing to appropriate workflows in SKILL.md.
- Introduced controlflow-spec skill for capturing specifications before planning in SKILL.md.
- Established controlflow-strict-workflow for a comprehensive workflow process in SKILL.md.
- Created templates for assumption verifier, executability verifier, and plan audit reports.
- Added tests to validate generated assets and ensure compliance with expected outputs.
Copy file name to clipboardExpand all lines: NOTES.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,6 @@
2
2
3
3
Repo-persistent active-objective state only. See `docs/agent-engineering/MEMORY-ARCHITECTURE.md` for the three-layer memory model; task-episodic history lives under `plans/artifacts/<task-slug>/`.
4
4
5
-
- Active objective: model-routing mismatch root-cause fix complete; awaiting user review of completion summary.
5
+
- Active objective: comprehensive project optimization complete; awaiting user review of completion summary.
6
6
- Blockers: none.
7
-
- Pending: user review before any commit or merge action.
7
+
- Pending: user review before any commit or package-release action.
Copy file name to clipboardExpand all lines: Orchestrator.agent.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -103,11 +103,11 @@ Maintain awareness of current orchestration state at all times:
103
103
-**Failure Retries:** Count of retries per classification for current phase (if any).
104
104
- Todo Management Protocol:
105
105
- At plan start, create a todo item for each phase using the format `Phase {N} — {Title}`.
106
-
- At phase completion, mark the corresponding todo item as completed immediately after the phase review gate passes.
106
+
- At phase completion, after the phase review gate passes, use the `#todos` tool to mark exactly that phase's todo item completed before any approval pause.
107
107
- At wave completion, verify all todo items for that wave are marked completed before advancing.
108
108
- At plan completion, verify all phase todo items are marked completed during the Completion Gate.
109
109
-**No batching of completions.** Each phase's todo item must be marked in its own `#todos` call as soon as that phase's verification checklist passes. Holding completions for a later bulk update is non-compliant — even if intermediate phases are obvious successes.
110
-
-**Context-compaction reconciliation.** Immediately after any context summarization, conversation resumption, or session restart, the first action before any other phase work MUST be a `#todos` reconciliation pass: compare the current todo list against the actual state of plan artifacts (created files, completed phases per `plans/<task>-plan.md`) and update statuses to match reality. Resuming work without reconciliation is non-compliant.
110
+
-**Context-compaction reconciliation.** Immediately after any context summarization, conversation resumption, or session restart, the first action before any other phase work MUST be a `#todos` reconciliation pass: compare the current todo list against the actual state of plan artifacts (created files, completed phases per `plans/<task>-plan.md`) and update statuses to match reality. This applies even when the active phase is Phase 1; resume, review, or implementation work for Phase 1 cannot continue until reconciliation is complete. Resuming work without reconciliation is non-compliant.
111
111
112
112
### Observability Sink
113
113
When emitting gate events, optionally also append one NDJSON line per event to `plans/artifacts/observability/<task-id>.ndjson`. See [docs/agent-engineering/OBSERVABILITY.md](docs/agent-engineering/OBSERVABILITY.md).
@@ -164,8 +164,8 @@ Before every `agent/runSubagent` call, regardless of dispatch context, apply thi
164
164
1. Load `governance/model-routing.json`.
165
165
2. Look up the target agent name in the top-level `agent_role_index` map to get its role.
166
166
3. Read `roles[role].by_tier[complexity_tier]`. If the entry is `{ "inherit_from": "default" }`, use the role's top-level `primary` model; otherwise use the tier-specific `primary`.
167
-
4. Pass the resolved `primary` model string as the `model` parameter to `agent/runSubagent`. Never omit `model`.
168
-
5. For initial planning dispatches before any plan `complexity_tier` exists, use the target role's top-level `primary` model. For replan/planning dispatches after a plan exists, use the active plan's `complexity_tier`. Never omit `model` because tier context is missing.
167
+
4. Pass the exact target as the outer `agentName` parameter and the resolved `primary` model string as the outer `model` parameter to `agent/runSubagent`. Never omit either outer field.
168
+
5. For initial planning dispatches before any plan `complexity_tier` exists, use the target role's top-level `primary` model. For replan/planning dispatches after a plan exists, use the active plan's `complexity_tier`. Never omit `model` because tier context is missing; missing tier context changes the resolution source, not the outer tool-call contract.
169
169
170
170
This rule covers all dispatch paths without exception: Plan Review Gate reviewers (PlanAuditor, AssumptionVerifier, ExecutabilityVerifier), phase CodeReviewer dispatch, final CodeReviewer dispatch, failure-classification retry dispatch, needs_replan Planner dispatch, and Implementation Loop executor dispatch.
171
171
@@ -247,7 +247,7 @@ For `CodeReviewer-subagent`, `PlanAuditor-subagent`, and `AssumptionVerifier-sub
247
247
- If trigger conditions are not met: skip directly to Implementation Loop.
248
248
249
249
5.**Implementation Loop (Per Phase)**
250
-
-**Pre-Phase Gate (phases after Phase 1):** Before starting any phase after Phase 1, verify the previous phase's todo item is marked completed. If it is not, mark it via the `#todos`tool before proceeding.
250
+
-**Pre-Phase Gate:** Before starting any phase or advancing a wave, verify all prior phase todo items are marked completed. If an open prior phase todo exists, use the `#todos` tool to reconcile the open prior phase todo before any phase or wave advancement. P2 or P3 must not start while the P1 todo remains open. For Phase 1 after context compaction, conversation resumption, or session restart, run the `#todos`reconciliation pass before resuming Phase 1 work.
251
251
- Run PreFlect gate.
252
252
- Resolve the phase owner from `phase.executor_agent`. This field is authoritative for delegation and approval summaries.
253
253
- If a legacy phase omits `executor_agent`, do not infer silently. Route the plan back through `REPLAN` to Planner and stop the implementation batch until the phase is reissued with an explicit executor.
@@ -257,7 +257,7 @@ For `CodeReviewer-subagent`, `PlanAuditor-subagent`, and `AssumptionVerifier-sub
257
257
- Delegate to CodeReviewer-subagent for phase code review (apply Universal Model Resolution Rule). Code review is mandatory for all complexity tiers — see `governance/runtime-policy.json → review_pipeline_by_tier.code_review`. Pass the changed files list, phase scope, and executor agent execution report.
258
258
- Block only on `validated_blocking_issues` from CodeReviewer-subagent verdict — not on raw unvalidated CRITICAL/MAJOR findings. If `validated_blocking_issues` is empty, the phase may proceed even if unvalidated issues exist.
259
259
- If CodeReviewer-subagent review status is not `APPROVED`, loop with targeted revision context.
260
-
-Mark the completed phase's todo item as completed using the `#todos` tool.
260
+
-After the phase review gate passes, mark the completed phase's todo item as completed using a separate `#todos` tool call before the approval pause.
@@ -50,7 +50,7 @@ For all other scopes, record applicability, impact, evidence source, and disposi
50
50
4. Include selected skill file paths in each applicable phase's `skill_references` array.
51
51
Implementation agents load referenced skills before executing phase tasks.
52
52
6. Research (delegate CodeMapper-subagent/Researcher-subagent when scope is large).
53
-
-**Model Resolution:** For every `agent/runSubagent` dispatch to `CodeMapper-subagent` or `Researcher-subagent`, load `governance/model-routing.json`, resolve the subagent role via the top-level `agent_role_index`, then apply `roles[role].by_tier[complexity_tier]`. If the tier entry is `{ "inherit_from": "default" }`, inherit the role's default `primary` model and default `fallbacks`; otherwise use the tier-specific `primary` and tier-specific `fallbacks` when present. Every research dispatch must pass the exact target as the outer `agentName` field and the resolved `primary` as the outer `model` field. Only pass a fallback list if/when `agent/runSubagent` explicitly supports one; otherwise pass only the resolved primary model string. A payload-level `model` in the prompt/delegation payload is contract and audit context; it does not by itself select the runtime model and is not a substitute for the outer `model` field.
53
+
- **Model Resolution:** For every `agent/runSubagent` dispatch to `CodeMapper-subagent` or `Researcher-subagent`, load `governance/model-routing.json`, resolve the subagent role via the top-level `agent_role_index`, then apply `roles[role].by_tier[complexity_tier]`. If the tier entry is `{ "inherit_from": "default" }`, inherit the role's default `primary` model and default `fallbacks`; otherwise use the tier-specific `primary` and tier-specific `fallbacks` when present. If `complexity_tier` is temporarily unavailable during a recovery/terminal-status path, use the target role's top-level `primary` model. Never omit `model` because tier context is missing. Every research dispatch must pass the exact target as the outer `agentName` field and the resolved `primary` as the outer `model` field. Only pass a fallback list if/when `agent/runSubagent` explicitly supports one; otherwise pass only the resolved primary model string. A payload-level `model` in the prompt/delegation payload is contract and audit context; it does not by itself select the runtime model and is not a substitute for the outer `model` field.
54
54
7. Design (structured design decisions and diagram selection):
55
55
-**Design Decisions Checklist** — Before proceeding to Planning (Step 8), explicitly address four dimensions:
56
56
1.**Boundary changes** — Does the task change system boundaries, add new actors, or modify integration points? If no boundary changes, state "No boundary changes."
- Premium tokens are spent on planning and on finding flaws.
51
51
- Routine orchestration and implementation stay cheaper, managed through dynamic subagent dispatch logic.
52
52
53
-
During the rollout window, agents add a `model_role:` line to their frontmatter **alongside** the existing `model:` line:
53
+
During the compatibility window, agents carry a `model_role:` line in their frontmatter **alongside** the existing `model:` line:
54
54
55
55
```yaml
56
56
---
@@ -71,15 +71,28 @@ When Orchestrator or Planner dispatch a subagent via `agent/runSubagent`, they a
71
71
72
72
1. They load `governance/model-routing.json`.
73
73
2. They look up the target agent in `agent_role_index`.
74
-
3. They apply the `by_tier` complexity rule to determine the required model string.
75
-
4. They pass the resolved `primary` explicitly as the outer `model` parameter to `agent/runSubagent`, and MUST supply the verified target-agent field `agentName` at the outer tool-call boundary, overriding the agent's frontmatter at call time.
74
+
3. They apply the `by_tier` complexity rule to determine the required model string. If no `complexity_tier` exists yet, they use the target role's top-level `primary` model rather than omitting model selection.
75
+
4. They pass the verified target-agent field as the outer `agentName` parameter and the resolved `primary` explicitly as the outer `model` parameter to `agent/runSubagent`, overriding the agent's frontmatter at call time.
76
76
77
77
### Delegation Payload Contract
78
78
79
79
The schema definition in `schemas/orchestrator.delegation-protocol.schema.json` requires a nested payload-level `model` field in all delegation objects. These payload-level fields carry resolved model context for validation, audit, and prompt-visible traceability, but they do not by themselves enforce the runtime model override. The outer `model` parameter on the tool call is the strict enforcement point.
80
80
81
81
While global VS Code Copilot execution (e.g., triggering an agent directly from chat) still relies on the frontmatter fallback, all internal orchestrated pipeline dispatches strictly enforce the logical routing graph dynamically. It is important to note that offline evals do not prove live `runSubagent` execution; we distinguish structural tests and tool/API-shape evidence from real live subagent dispatch (as proven by the existing model override spike).
82
82
83
+
### Offline Regression Coverage
84
+
85
+
Structural and behavior evals validate the dispatch contract shape and the prompt-visible routing rules. They intentionally do not claim to observe live `agent/runSubagent` runtime parameters.
86
+
87
+
Current negative cases are documented in `evals/scenarios/orchestrator-model-resolution.json` and validated by `evals/validate.mjs`, `evals/tests/orchestration-handoff-contract.test.mjs`, and `evals/tests/drift-detection.test.mjs`:
88
+
89
+
- Missing outer `agentName`, even if a payload/prose target is present.
90
+
- Missing outer `model`, even if `agentName` is present.
91
+
- Payload-only `model`, where the nested delegation payload carries model context but the outer runtime selector is omitted.
92
+
- Wrong effective review tier, especially unresolved HIGH risk that must route capable-reviewer dispatch through `LARGE`.
93
+
- Unconfigured fallback on `model_unavailable`, where retry models must come from the configured fallback list for the same effective tier.
94
+
- Omitted model due to missing tier context, where the correct behavior is to use the target role's top-level `primary` model.
95
+
83
96
## Matrix shape (Stage C/D)
84
97
85
98
The `by_tier` object describes model overrides based on the complexity tier of the task (`TRIVIAL`, `SMALL`, `MEDIUM`, `LARGE`). Because internal control plane logic resolves this matrix dynamically during subagent dispatch, this is an **active runtime switch** for Orchestrator and Planner.
@@ -181,6 +194,6 @@ The field lives at the **per-role** level, as a sibling of `primary`, `fallbacks
181
194
## Cross-references
182
195
183
196
- Repository agent-engineering index: `docs/agent-engineering/README.md` (authored in Phase 10).
184
-
- Drift detection: `evals/validate.mjs` and the upcoming `evals/scenarios/model-routing-alignment.json`.
197
+
- Drift detection: `evals/validate.mjs`, `evals/scenarios/model-routing-alignment.json`, and `evals/scenarios/orchestrator-model-resolution.json`.
0 commit comments