You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: Enhance drift detection and validation mechanisms
- Added tests for frontmatter model defaults validation in drift-detection.test.mjs, ensuring that frontmatter models align with role defaults and do not conflict with tier-specific overrides.
- Implemented validation for payload model description semantics, ensuring that payload models are correctly defined and do not override frontmatter models.
- Updated orchestration-handoff-contract.test.mjs to include checks for outer dispatch contracts, ensuring separation of outer tool-call fields and payload-level models.
- Enhanced validation logic in validate.mjs to incorporate checks for frontmatter model defaults and payload model semantics.
- Revised project context documentation to reflect updated model routing roles for agents.
- Documented lessons learned in session outcomes regarding model routing and frontmatter defaults.
- Updated orchestrator delegation protocol schema to clarify payload model descriptions and their intended use.
- Introduced a new structural reference fixture in model-routing-alignment.json to ensure alignment across governance role defaults, direct-invocation frontmatter, and payload-level delegation schema.
Copy file name to clipboardExpand all lines: NOTES.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,6 @@
2
2
3
3
Repo-persistent active-objective state only. See `docs/agent-engineering/MEMORY-ARCHITECTURE.md` for the three-layer memory model; task-episodic history lives under `plans/artifacts/<task-slug>/`.
4
4
5
-
- Active objective: Claude Code plugin implementation complete; awaiting user review.
5
+
- Active objective: model-routing mismatch root-cause fix complete; awaiting user review of completion summary.
6
6
- Blockers: none.
7
-
- Pending: user review of final summary and any follow-up Claude CLI validation when available.
7
+
- Pending: user review before any commit or merge action.
Copy file name to clipboardExpand all lines: Orchestrator.agent.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -166,13 +166,13 @@ Before every `agent/runSubagent` call, regardless of dispatch context, apply thi
166
166
4. Pass the resolved `primary` model string as the `model` parameter to `agent/runSubagent`. Never omit `model`.
167
167
5. For initial planning dispatches before any plan `complexity_tier` exists, use the target role's top-level `primary` model. For replan/planning dispatches after a plan exists, use the active plan's `complexity_tier`. Never omit `model` because tier context is missing.
168
168
169
-
This rule covers all dispatch paths without exception: Plan Review Gate reviewers (PlanAuditor, AssumptionVerifier, ExecutabilityVerifier), phase CodeReviewer dispatch, final CodeReviewer dispatch, needs_replan Planner dispatch, and Implementation Loop executor dispatch.
169
+
This rule covers all dispatch paths without exception: Plan Review Gate reviewers (PlanAuditor, AssumptionVerifier, ExecutabilityVerifier), phase CodeReviewer dispatch, final CodeReviewer dispatch, failure-classification retry dispatch, needs_replan Planner dispatch, and Implementation Loop executor dispatch.
170
170
171
171
### Dispatch Tool-Call Contract (Required Fields)
172
172
173
173
Every `agent/runSubagent` call must include these outer tool-call fields:
174
174
-**`agentName`** — the verified target-agent field (string). Placing the agent name only inside prompt prose or a delegation payload is non-compliant.
175
-
-**`model`** — the resolved primary model string from the Universal Model Resolution Rule. Never omit.
175
+
-**`model`** — the resolved primary model string from the Universal Model Resolution Rule, passed as the outer `model` field at the tool-call boundary. Never omit. A payload-level `model` inside the prompt/delegation payload remains useful for schema validation and audit context, but it does not by itself select the runtime model and is not a substitute for this outer field.
176
176
-**Prompt/context payload** — scope, deliverables, and relevant context references.
Copy file name to clipboardExpand all lines: Planner.agent.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -50,7 +50,7 @@ For all other scopes, record applicability, impact, evidence source, and disposi
50
50
4. Include selected skill file paths in each applicable phase's `skill_references` array.
51
51
Implementation agents load referenced skills before executing phase tasks.
52
52
6. Research (delegate CodeMapper-subagent/Researcher-subagent when scope is large).
53
-
-**Model Resolution:** For every `agent/runSubagent` dispatch to `CodeMapper-subagent` or `Researcher-subagent`, load `governance/model-routing.json`, resolve the subagent role via the top-level `agent_role_index`, then apply `roles[role].by_tier[complexity_tier]`. If the tier entry is `{ "inherit_from": "default" }`, inherit the role's default `primary` model and default `fallbacks`; otherwise use the tier-specific `primary` and tier-specific `fallbacks` when present. Pass the resolved `primary` as the `model`parameter. Only pass a fallback list if/when `agent/runSubagent` explicitly supports one; otherwise pass only the resolved primary model string.
53
+
-**Model Resolution:** For every `agent/runSubagent` dispatch to `CodeMapper-subagent` or `Researcher-subagent`, load `governance/model-routing.json`, resolve the subagent role via the top-level `agent_role_index`, then apply `roles[role].by_tier[complexity_tier]`. If the tier entry is `{ "inherit_from": "default" }`, inherit the role's default `primary` model and default `fallbacks`; otherwise use the tier-specific `primary` and tier-specific `fallbacks` when present. Every research dispatch must pass the exact target as the outer `agentName` field and the resolved `primary` as the outer `model`field. Only pass a fallback list if/when `agent/runSubagent` explicitly supports one; otherwise pass only the resolved primary model string. A payload-level `model` in the prompt/delegation payload is contract and audit context; it does not by itself select the runtime model and is not a substitute for the outer `model` field.
54
54
7. Design (structured design decisions and diagram selection):
55
55
-**Design Decisions Checklist** — Before proceeding to Planning (Step 8), explicitly address four dimensions:
56
56
1.**Boundary changes** — Does the task change system boundaries, add new actors, or modify integration points? If no boundary changes, state "No boundary changes."
@@ -72,7 +72,11 @@ When Orchestrator or Planner dispatch a subagent via `agent/runSubagent`, they a
72
72
1. They load `governance/model-routing.json`.
73
73
2. They look up the target agent in `agent_role_index`.
74
74
3. They apply the `by_tier` complexity rule to determine the required model string.
75
-
4. They pass the resolved `primary` explicitly as the `model` parameter to `agent/runSubagent`, and MUST supply the verified target-agent field `agentName` at the tool-call boundary, overriding the agent's frontmatter at call time.
75
+
4. They pass the resolved `primary` explicitly as the outer `model` parameter to `agent/runSubagent`, and MUST supply the verified target-agent field `agentName` at the outer tool-call boundary, overriding the agent's frontmatter at call time.
76
+
77
+
### Delegation Payload Contract
78
+
79
+
The schema definition in `schemas/orchestrator.delegation-protocol.schema.json` requires a nested payload-level `model` field in all delegation objects. These payload-level fields carry resolved model context for validation, audit, and prompt-visible traceability, but they do not by themselves enforce the runtime model override. The outer `model` parameter on the tool call is the strict enforcement point.
76
80
77
81
While global VS Code Copilot execution (e.g., triggering an agent directly from chat) still relies on the frontmatter fallback, all internal orchestrated pipeline dispatches strictly enforce the logical routing graph dynamically. It is important to note that offline evals do not prove live `runSubagent` execution; we distinguish structural tests and tool/API-shape evidence from real live subagent dispatch (as proven by the existing model override spike).
78
82
@@ -114,17 +118,19 @@ fallback chain at every complexity tier.
114
118
The prompt-driven runtime resolution module (Orchestrator/Planner dynamic lookup) is complete.
115
119
116
120
Remaining prerequisites for Stage D (auto-tuning observability):
121
+
117
122
- (a) accumulate ≥50 task telemetry entries via the NDJSON sink at `plans/artifacts/observability/`
118
123
- (b) expand `governance/model-routing.json` schema with `inherit_from` targets beyond `"default"` (e.g., other roles or tier mixes)
Defines which LLM model handles which type of task. Does **not** contain API keys — only model names / capabilities.
110
118
111
119
**Typical sections:**
112
-
-`task_type_routing` — by task type (research, implementation, review, planning).
113
-
-`tier_routing` — by complexity tier.
114
-
-`fallback` — default model.
115
120
116
-
**Who uses it:** Orchestrator and individual agents via the `model_role` field in schema outputs. See also [docs/agent-engineering/MODEL-ROUTING.md](../agent-engineering/MODEL-ROUTING.md).
121
+
-`roles` — by role ID (e.g. `capable-planner`, `capable-implementer`), explicitly handling complexity tiers.
122
+
-`by_tier` — explicit capability overrides for routing models dynamically by complexity tier.
123
+
-`agent_role_index` — reverse-mapping index from agent name to active model role.
124
+
125
+
**Who uses it:** Orchestrator and Planner actively read `model-routing.json` for internal dispatch resolution (e.g. `agent/runSubagent`) to pass the resolved model as an outer tool-call parameter overriding the fallback frontmatter. See [docs/agent-engineering/MODEL-ROUTING.md](../agent-engineering/MODEL-ROUTING.md).
117
126
118
127
## rename-allowlist.json
119
128
120
129
Contains permitted file renames. Prevents accidental drift: if an agent tries to rename a file not on the allowlist — the drift check fails.
**Зачем:** decoupling агентских файлов от хардкоженных моделей. Можно менять модель в одном месте, не трогая 13 агентов.
164
175
165
-
**Сейчас:** runtime читает `model:` напрямую; `model_role`существует для логического индексирования, eval-выравнивания и будущей миграции.
176
+
**Сейчас:** runtime читает `model:` напрямую при внешнем вызове (от пользователя); при внутреннем вызове Orchestrator/Planner используют `governance/model-routing.json` для вычисления нужной модели и передают её явно через параметр `model` внешних вызовов (например, `agent/runSubagent`). Поля `model_role`существуют для логического индексирования, eval-выравнивания и будущей миграции.
0 commit comments