Skip to content

Commit e241c77

Browse files
SmithSmith
authored andcommitted
Add ControlFlow planning and review skills with supporting documentation
- Introduced complexity tiers for task planning in complexity-tiers.md. - Added ControlFlow portability notes for adapting practices in controlflow-portability.md. - Established LLM behavior guidelines to enhance coding practices in llm-behavior-guidelines.md. - Created a comprehensive plan template in plan-template.md for structured task execution. - Defined a planner output contract to standardize plan artifacts in planner-output-contract.md. - Developed a semantic risk taxonomy to categorize planning risks in semantic-risk-taxonomy.md. - Implemented controlflow-review skill to facilitate code and phase reviews in SKILL.md. - Established evidence discipline and review checklists to enhance review quality in evidence-discipline.md and review-checklist.md. - Created security review discipline guidelines for focused security assessments in security-review-discipline.md. - Defined validation status labels for findings in validation-status.md. - Added controlflow-router skill to streamline task routing to appropriate workflows in SKILL.md. - Introduced controlflow-spec skill for capturing specifications before planning in SKILL.md. - Established controlflow-strict-workflow for a comprehensive workflow process in SKILL.md. - Created templates for assumption verifier, executability verifier, and plan audit reports. - Added tests to validate generated assets and ensure compliance with expected outputs.
1 parent 8732082 commit e241c77

90 files changed

Lines changed: 4499 additions & 36 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

NOTES.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,6 @@
22

33
Repo-persistent active-objective state only. See `docs/agent-engineering/MEMORY-ARCHITECTURE.md` for the three-layer memory model; task-episodic history lives under `plans/artifacts/<task-slug>/`.
44

5-
- Active objective: model-routing mismatch root-cause fix complete; awaiting user review of completion summary.
5+
- Active objective: comprehensive project optimization complete; awaiting user review of completion summary.
66
- Blockers: none.
7-
- Pending: user review before any commit or merge action.
7+
- Pending: user review before any commit or package-release action.

Orchestrator.agent.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -103,11 +103,11 @@ Maintain awareness of current orchestration state at all times:
103103
- **Failure Retries:** Count of retries per classification for current phase (if any).
104104
- Todo Management Protocol:
105105
- At plan start, create a todo item for each phase using the format `Phase {N} — {Title}`.
106-
- At phase completion, mark the corresponding todo item as completed immediately after the phase review gate passes.
106+
- At phase completion, after the phase review gate passes, use the `#todos` tool to mark exactly that phase's todo item completed before any approval pause.
107107
- At wave completion, verify all todo items for that wave are marked completed before advancing.
108108
- At plan completion, verify all phase todo items are marked completed during the Completion Gate.
109109
- **No batching of completions.** Each phase's todo item must be marked in its own `#todos` call as soon as that phase's verification checklist passes. Holding completions for a later bulk update is non-compliant — even if intermediate phases are obvious successes.
110-
- **Context-compaction reconciliation.** Immediately after any context summarization, conversation resumption, or session restart, the first action before any other phase work MUST be a `#todos` reconciliation pass: compare the current todo list against the actual state of plan artifacts (created files, completed phases per `plans/<task>-plan.md`) and update statuses to match reality. Resuming work without reconciliation is non-compliant.
110+
- **Context-compaction reconciliation.** Immediately after any context summarization, conversation resumption, or session restart, the first action before any other phase work MUST be a `#todos` reconciliation pass: compare the current todo list against the actual state of plan artifacts (created files, completed phases per `plans/<task>-plan.md`) and update statuses to match reality. This applies even when the active phase is Phase 1; resume, review, or implementation work for Phase 1 cannot continue until reconciliation is complete. Resuming work without reconciliation is non-compliant.
111111

112112
### Observability Sink
113113
When emitting gate events, optionally also append one NDJSON line per event to `plans/artifacts/observability/<task-id>.ndjson`. See [docs/agent-engineering/OBSERVABILITY.md](docs/agent-engineering/OBSERVABILITY.md).
@@ -164,8 +164,8 @@ Before every `agent/runSubagent` call, regardless of dispatch context, apply thi
164164
1. Load `governance/model-routing.json`.
165165
2. Look up the target agent name in the top-level `agent_role_index` map to get its role.
166166
3. Read `roles[role].by_tier[complexity_tier]`. If the entry is `{ "inherit_from": "default" }`, use the role's top-level `primary` model; otherwise use the tier-specific `primary`.
167-
4. Pass the resolved `primary` model string as the `model` parameter to `agent/runSubagent`. Never omit `model`.
168-
5. For initial planning dispatches before any plan `complexity_tier` exists, use the target role's top-level `primary` model. For replan/planning dispatches after a plan exists, use the active plan's `complexity_tier`. Never omit `model` because tier context is missing.
167+
4. Pass the exact target as the outer `agentName` parameter and the resolved `primary` model string as the outer `model` parameter to `agent/runSubagent`. Never omit either outer field.
168+
5. For initial planning dispatches before any plan `complexity_tier` exists, use the target role's top-level `primary` model. For replan/planning dispatches after a plan exists, use the active plan's `complexity_tier`. Never omit `model` because tier context is missing; missing tier context changes the resolution source, not the outer tool-call contract.
169169

170170
This rule covers all dispatch paths without exception: Plan Review Gate reviewers (PlanAuditor, AssumptionVerifier, ExecutabilityVerifier), phase CodeReviewer dispatch, final CodeReviewer dispatch, failure-classification retry dispatch, needs_replan Planner dispatch, and Implementation Loop executor dispatch.
171171

@@ -247,7 +247,7 @@ For `CodeReviewer-subagent`, `PlanAuditor-subagent`, and `AssumptionVerifier-sub
247247
- If trigger conditions are not met: skip directly to Implementation Loop.
248248

249249
5. **Implementation Loop (Per Phase)**
250-
- **Pre-Phase Gate (phases after Phase 1):** Before starting any phase after Phase 1, verify the previous phase's todo item is marked completed. If it is not, mark it via the `#todos` tool before proceeding.
250+
- **Pre-Phase Gate:** Before starting any phase or advancing a wave, verify all prior phase todo items are marked completed. If an open prior phase todo exists, use the `#todos` tool to reconcile the open prior phase todo before any phase or wave advancement. P2 or P3 must not start while the P1 todo remains open. For Phase 1 after context compaction, conversation resumption, or session restart, run the `#todos` reconciliation pass before resuming Phase 1 work.
251251
- Run PreFlect gate.
252252
- Resolve the phase owner from `phase.executor_agent`. This field is authoritative for delegation and approval summaries.
253253
- If a legacy phase omits `executor_agent`, do not infer silently. Route the plan back through `REPLAN` to Planner and stop the implementation batch until the phase is reissued with an explicit executor.
@@ -257,7 +257,7 @@ For `CodeReviewer-subagent`, `PlanAuditor-subagent`, and `AssumptionVerifier-sub
257257
- Delegate to CodeReviewer-subagent for phase code review (apply Universal Model Resolution Rule). Code review is mandatory for all complexity tiers — see `governance/runtime-policy.json → review_pipeline_by_tier.code_review`. Pass the changed files list, phase scope, and executor agent execution report.
258258
- Block only on `validated_blocking_issues` from CodeReviewer-subagent verdict — not on raw unvalidated CRITICAL/MAJOR findings. If `validated_blocking_issues` is empty, the phase may proceed even if unvalidated issues exist.
259259
- If CodeReviewer-subagent review status is not `APPROVED`, loop with targeted revision context.
260-
- Mark the completed phase's todo item as completed using the `#todos` tool.
260+
- After the phase review gate passes, mark the completed phase's todo item as completed using a separate `#todos` tool call before the approval pause.
261261
- Pause for user commit/continue approval.
262262

263263
6. **Completion Gate**

Planner.agent.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
description: 'Autonomous planner that writes comprehensive implementation plans and feeds them to Orchestrator'
3-
tools: [read/readFile, agent/runSubagent, edit/createFile, search/codebase, search/fileSearch, search/listDirectory, search/textSearch, search/usages, web/fetch, web/githubRepo, vscode/askQuestions, vscode/getProjectSetupInfo, io.github.upstash/context7/get-library-docs, io.github.upstash/context7/resolve-library-id]
3+
tools: [read/readFile, agent, agent/runSubagent, edit/createFile, search/codebase, search/fileSearch, search/listDirectory, search/textSearch, search/usages, web/fetch, web/githubRepo, vscode/askQuestions, vscode/getProjectSetupInfo, io.github.upstash/context7/get-library-docs, io.github.upstash/context7/resolve-library-id]
44
agents: ["CodeMapper-subagent", "Researcher-subagent"]
55
model: GPT-5.5 (copilot)
66
model_role: capable-planner
@@ -50,7 +50,7 @@ For all other scopes, record applicability, impact, evidence source, and disposi
5050
4. Include selected skill file paths in each applicable phase's `skill_references` array.
5151
Implementation agents load referenced skills before executing phase tasks.
5252
6. Research (delegate CodeMapper-subagent/Researcher-subagent when scope is large).
53-
- **Model Resolution:** For every `agent/runSubagent` dispatch to `CodeMapper-subagent` or `Researcher-subagent`, load `governance/model-routing.json`, resolve the subagent role via the top-level `agent_role_index`, then apply `roles[role].by_tier[complexity_tier]`. If the tier entry is `{ "inherit_from": "default" }`, inherit the role's default `primary` model and default `fallbacks`; otherwise use the tier-specific `primary` and tier-specific `fallbacks` when present. Every research dispatch must pass the exact target as the outer `agentName` field and the resolved `primary` as the outer `model` field. Only pass a fallback list if/when `agent/runSubagent` explicitly supports one; otherwise pass only the resolved primary model string. A payload-level `model` in the prompt/delegation payload is contract and audit context; it does not by itself select the runtime model and is not a substitute for the outer `model` field.
53+
- **Model Resolution:** For every `agent/runSubagent` dispatch to `CodeMapper-subagent` or `Researcher-subagent`, load `governance/model-routing.json`, resolve the subagent role via the top-level `agent_role_index`, then apply `roles[role].by_tier[complexity_tier]`. If the tier entry is `{ "inherit_from": "default" }`, inherit the role's default `primary` model and default `fallbacks`; otherwise use the tier-specific `primary` and tier-specific `fallbacks` when present. If `complexity_tier` is temporarily unavailable during a recovery/terminal-status path, use the target role's top-level `primary` model. Never omit `model` because tier context is missing. Every research dispatch must pass the exact target as the outer `agentName` field and the resolved `primary` as the outer `model` field. Only pass a fallback list if/when `agent/runSubagent` explicitly supports one; otherwise pass only the resolved primary model string. A payload-level `model` in the prompt/delegation payload is contract and audit context; it does not by itself select the runtime model and is not a substitute for the outer `model` field.
5454
7. Design (structured design decisions and diagram selection):
5555
- **Design Decisions Checklist** — Before proceeding to Planning (Step 8), explicitly address four dimensions:
5656
1. **Boundary changes** — Does the task change system boundaries, add new actors, or modify integration points? If no boundary changes, state "No boundary changes."

docs/agent-engineering/MODEL-ROUTING.md

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Model Routing
22

3-
**Status:** Phase 4 spike (single-agent rollout)
3+
**Status:** Active logical-role routing with structural offline regression coverage
44
**File:** `governance/model-routing.json` is the canonical source of truth for internal model selection.
55
**trace_id:** 7d3f5a2e-1b4c-4e9f-9a8b-2c5d8e1f3a7b
66

@@ -50,7 +50,7 @@ This yields a pragmatic split:
5050
- Premium tokens are spent on planning and on finding flaws.
5151
- Routine orchestration and implementation stay cheaper, managed through dynamic subagent dispatch logic.
5252

53-
During the rollout window, agents add a `model_role:` line to their frontmatter **alongside** the existing `model:` line:
53+
During the compatibility window, agents carry a `model_role:` line in their frontmatter **alongside** the existing `model:` line:
5454

5555
```yaml
5656
---
@@ -71,15 +71,28 @@ When Orchestrator or Planner dispatch a subagent via `agent/runSubagent`, they a
7171

7272
1. They load `governance/model-routing.json`.
7373
2. They look up the target agent in `agent_role_index`.
74-
3. They apply the `by_tier` complexity rule to determine the required model string.
75-
4. They pass the resolved `primary` explicitly as the outer `model` parameter to `agent/runSubagent`, and MUST supply the verified target-agent field `agentName` at the outer tool-call boundary, overriding the agent's frontmatter at call time.
74+
3. They apply the `by_tier` complexity rule to determine the required model string. If no `complexity_tier` exists yet, they use the target role's top-level `primary` model rather than omitting model selection.
75+
4. They pass the verified target-agent field as the outer `agentName` parameter and the resolved `primary` explicitly as the outer `model` parameter to `agent/runSubagent`, overriding the agent's frontmatter at call time.
7676

7777
### Delegation Payload Contract
7878

7979
The schema definition in `schemas/orchestrator.delegation-protocol.schema.json` requires a nested payload-level `model` field in all delegation objects. These payload-level fields carry resolved model context for validation, audit, and prompt-visible traceability, but they do not by themselves enforce the runtime model override. The outer `model` parameter on the tool call is the strict enforcement point.
8080

8181
While global VS Code Copilot execution (e.g., triggering an agent directly from chat) still relies on the frontmatter fallback, all internal orchestrated pipeline dispatches strictly enforce the logical routing graph dynamically. It is important to note that offline evals do not prove live `runSubagent` execution; we distinguish structural tests and tool/API-shape evidence from real live subagent dispatch (as proven by the existing model override spike).
8282

83+
### Offline Regression Coverage
84+
85+
Structural and behavior evals validate the dispatch contract shape and the prompt-visible routing rules. They intentionally do not claim to observe live `agent/runSubagent` runtime parameters.
86+
87+
Current negative cases are documented in `evals/scenarios/orchestrator-model-resolution.json` and validated by `evals/validate.mjs`, `evals/tests/orchestration-handoff-contract.test.mjs`, and `evals/tests/drift-detection.test.mjs`:
88+
89+
- Missing outer `agentName`, even if a payload/prose target is present.
90+
- Missing outer `model`, even if `agentName` is present.
91+
- Payload-only `model`, where the nested delegation payload carries model context but the outer runtime selector is omitted.
92+
- Wrong effective review tier, especially unresolved HIGH risk that must route capable-reviewer dispatch through `LARGE`.
93+
- Unconfigured fallback on `model_unavailable`, where retry models must come from the configured fallback list for the same effective tier.
94+
- Omitted model due to missing tier context, where the correct behavior is to use the target role's top-level `primary` model.
95+
8396
## Matrix shape (Stage C/D)
8497

8598
The `by_tier` object describes model overrides based on the complexity tier of the task (`TRIVIAL`, `SMALL`, `MEDIUM`, `LARGE`). Because internal control plane logic resolves this matrix dynamically during subagent dispatch, this is an **active runtime switch** for Orchestrator and Planner.
@@ -181,6 +194,6 @@ The field lives at the **per-role** level, as a sibling of `primary`, `fallbacks
181194
## Cross-references
182195

183196
- Repository agent-engineering index: `docs/agent-engineering/README.md` (authored in Phase 10).
184-
- Drift detection: `evals/validate.mjs` and the upcoming `evals/scenarios/model-routing-alignment.json`.
197+
- Drift detection: `evals/validate.mjs`, `evals/scenarios/model-routing-alignment.json`, and `evals/scenarios/orchestrator-model-resolution.json`.
185198
- Plan: `plans/controlflow-comprehensive-revision-plan.md` Phase 4.
186199
- Spike record: `plans/artifacts/model-resolver/phase-1-spike.md`.

0 commit comments

Comments
 (0)