Skip to content

Commit 8b19d13

Browse files
SmithSmith
authored andcommitted
Refactor skills documentation and patterns
- Updated README.md to reflect new pattern files and their purposes. - Enhanced index.md with binding legends for applicable agents and updated skill file references. - Revised completeness-traceability.md to correct the number of skill files in the index. - Added decision rules and clarified usage in idea-to-prompt.md and spec-driven-development.md. - Improved llm-behavior-guidelines.md by condensing anti-pattern examples and enhancing clarity. - Updated memory-promotion-candidates.md to clarify provenance of the checklist. - Streamlined repo-memory-hygiene.md for clarity and conciseness in the pre-write checklist. - Adjusted security-review-discipline.md to specify provenance of the discipline.
1 parent 358eedb commit 8b19d13

41 files changed

Lines changed: 459 additions & 1044 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/copilot-instructions.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ When status is `FAILED`, `NEEDS_INPUT`, `NEEDS_REVISION`, or `REJECTED`, include
2626
- `fixable` — Small correctable issue (typo, missing import, config value); retry with fix hint.
2727
- `needs_replan` — Architecture mismatch or missing dependency; delegate to Planner for targeted replan.
2828
- `escalate` — Security vulnerability, data integrity risk, or unresolvable blocker; stop and await human approval.
29+
- `model_unavailable` — the routed/primary model is unavailable or unreachable; substitute per model-routing fallback and retry up to `model_unavailable_max` (see runtime-policy.json), then escalate. Distinct from `transient`; the PlanAuditor/AssumptionVerifier `transient`-exclusion does NOT exclude `model_unavailable`.
2930

3031
## NOTES.md
3132
Maintain/update `NOTES.md` for persistent state across context resets:
@@ -59,7 +60,7 @@ Agent engineering policies are in `docs/agent-engineering/`:
5960
- Failure taxonomy applies to all agents; PlanAuditor and AssumptionVerifier exclude `transient`.
6061
- P.A.R.T. section order in every agent file: **Prompt → Archive → Resources → Tools** (see `PART-SPEC.md`).
6162
- Orchestrator and Planner must delegate only to project-internal agents documented in `plans/project-context.md`; external/third-party agents are strictly prohibited.
62-
- Adding/editing agents or skills: follow the 4-step process in `CONTRIBUTING.md` (create agent file, schema, eval scenarios, register in `plans/project-context.md`).
63+
- Adding/editing agents or skills: follow the agent-contribution process in `CONTRIBUTING.md` (create agent file, schema, eval scenarios, register in `plans/project-context.md`).
6364
- Tool and agent permission grants are in `governance/` (`agent-grants.json`, `tool-grants.json`, `runtime-policy.json`, `rename-allowlist.json`). Update these when changing an agent's tool profile.
6465

6566
## Agent System

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@ yarn-error.log*
1010
# Ignore all plan files by default, but allow required project artifacts
1111
plans/*
1212
!plans/project-context.md
13-
!plans/pipeline-comparison.md
1413
!plans/session-outcomes.md
1514
!plans/templates/
1615
plans/templates/*
@@ -56,6 +55,8 @@ temp/
5655
*.swo
5756
# ── Eval warm cache ────────────────────────────────────────────
5857
evals/.cache/
58+
# ── Eval stdout dumps (debug scratch, not tracked) ────────────
59+
evals/out*.txt
5960
# ── Tool output spill directory ───────────────────────────────
6061
.cache/tool-output/
6162
# ── Windows OS artifacts ──────────────────────────────────

AssumptionVerifier-subagent.agent.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
description: 'Adversarial mirage detector that hunts assumption-fact confusion in plans using 17 systematic patterns'
33
tools: [read/readFile, search/codebase, search/fileSearch, search/listDirectory, search/textSearch, search/usages]
4-
model: Claude Opus 4.7 (copilot)
4+
model: Claude Opus 4.8 (copilot)
55
model_role: capable-reviewer
66
---
77
You are AssumptionVerifier, an adversarial mirage detector for plan verification.

CHANGELOG.md

Lines changed: 19 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -11,76 +11,38 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
1111
### Added
1212

1313
- **Cursor IDE Support**: Added `.cursor/rules` to provide high-level guidance to the Cursor agent, along with documentation in `docs/agent-engineering/CURSOR-SUPPORT.md`.
14-
15-
---
16-
17-
## [Unreleased] — Claude Code Plugin (Phase 6)
18-
19-
### Added - Claude Code Plugin
20-
2114
- **Claude Code Plugin**: Shipped a first-class plugin adaptation for Anthropic's Claude Code at `plugins/controlflow-claude-code/`.
2215
- Includes 10 ControlFlow workflow skills adapted for Claude native slash-invocation.
2316
- Adds 6 native plugin agents for isolated review, research, mapping, and verification tasks.
2417
- Ships local validation scripts and local development lifecycle documentation.
2518
- Adds project-level references inside `README.md`, `CONTRIBUTING.md`, and quickstart docs.
26-
27-
---
28-
29-
## [Unreleased] — Codex Plugin & Lifecycle Documentation Parity (Phases 4-8)
30-
31-
### Added - Codex Parity
32-
3319
- **BrowserTester Evidence Discipline**: Added snapshot, wait, console/network, visual regression, and untrusted content protocols (`BrowserTester-subagent.agent.md`).
3420
- **Living-Document Guidance**: Added "Living-Document and Restartability Guidance" section to `plans/templates/plan-document-template.md`.
35-
36-
### Changed - Codex Parity
37-
38-
- **Codex Plugin Lifecycle**: Introduced fixed lifecycle sections (`## Progress`, `## Discoveries`, `## Decision Log`, `## Outcomes`, `## Idempotence & Recovery`) in strict-plan dialect, enforced by `validate-strict-artifacts.ps1`.
39-
- **Plugin First-Class Status**: Upgraded `controlflow-codex` to a first-class plugin deliverable in `AGENTS.md` and `README.md`.
40-
41-
---
42-
43-
## [Unreleased] — Model Routing Stage C Matrix
44-
45-
### Added - Model Routing
46-
47-
- `feat: add optional final review gate via final_review_gate policy flag` — Adds opt-in Completion Gate sub-step that dispatches CodeReviewer with review_scope=final for holistic scope-drift detection. Auto-triggers for LARGE tier plans. Configured via governance/runtime-policy.json.
48-
- **Stage C Final (Consolidated)**: Model Routing Stage C has fully landed. Stage D is a forward pointer only.
21+
- **Optional final review gate** via the `final_review_gate` policy flag — adds an opt-in Completion Gate sub-step that dispatches CodeReviewer with `review_scope=final` for holistic scope-drift detection. Auto-triggers for LARGE tier plans. Configured via `governance/runtime-policy.json`.
22+
- **Model Routing Stage C Matrix (Consolidated)**: Model Routing Stage C has fully landed. Stage D is a forward pointer only.
4923
- All 13 agents now declare `model_role:` frontmatter.
5024
- `governance/model-routing.json` extended with optional `by_tier` matrix (10 roles × 4 complexity tiers with `inherit_from: "default"` for non-divergent cases).
5125
- `evals/drift-checks.mjs` gained `validateModelRole` + `validateByTierShape`.
5226
- Drift-check suite grew from 23 → 34 (+11): 4 Check #1 model_role tests (A/B/C/D), 4 matrix-completeness tests (N1/N2/N3/positive), 3 reference-integrity tests (RI-1/RI-2/RI-3).
53-
- Structural gained +1 (Check #1 real implementation).
54-
- Total eval suite: 358 → 370.
5527
- VS Code runtime tolerance PASS recorded in `plans/artifacts/model-routing-stage-c/phase-1-spike-result.md`.
28+
- **ControlFlow Comprehensive Revision** — ten-phase revision and modernization program delivered per `plans/controlflow-comprehensive-revision-plan.md`:
29+
- **Phase 1 — Researcher validation + `plans_classification`.** Evidence-first confirmation of all 10 audit findings and per-plan classification artifact gating downstream archival decisions.
30+
- **Phase 2 — CodeMapper drift inventory.** Mechanical cross-reference of agent model pins, tool grants, schema references, skill registrations, executor-enum alignment, and PreFlect baseline line counts.
31+
- **Phase 3 — Consistency fixes (relabel-only; archival gated).** Phase Executor Agents table reduced to 8 rows matching the `executor_agent` enum; three-way check-count reconciliation; in-place status relabels without destructive moves.
32+
- **Phase 4 — Model Routing (logical-index-only; spike deferred).** Published `governance/model-routing.json` with 10 logical roles covering all 13 agents and `docs/agent-engineering/MODEL-ROUTING.md`. Runtime opt-in via agent frontmatter `model_role:` is held pending VS Code frontmatter-tolerance verification; drift Check #1 remains gated off.
33+
- **Phase 5 — Observability & `trace_id` schema hardening.** UUIDv4 `trace_id` added as additive-optional field across 13 report / verdict / audit / discovery / research / clarification / plan schemas; `docs/agent-engineering/OBSERVABILITY.md` documents generation, propagation, and the NDJSON sink convention at `plans/artifacts/observability/<task-id>.ndjson`.
34+
- **Phase 6 — Agentic Memory layering.** Three-layer memory model (session / task-episodic / repo-persistent) codified in `docs/agent-engineering/MEMORY-ARCHITECTURE.md`; `NOTES.md` scoped to repo-persistent state with reversible migration manifest.
35+
- **Phase 7 — PreFlect canonicalization.** Single `skills/patterns/preflect-core.md` replaces per-agent PreFlect restatements with pointer + ≤5 domain lines; Orchestrator-enforced cross-plan entry gate against `plans/performance-optimization-plan.md`.
36+
- **Phase 8 — SOTA pattern integration (additive, no budget compounding).** `skills/patterns/reflection-loop.md` (pre-retry hook; default disabled; draws from existing `retry_budgets`), `skills/patterns/budget-tracking.md` (`budget_defaults` orthogonal to `retry_budgets`), and `docs/agent-engineering/AGENT-AS-TOOL.md` (MCP forward-compatible input contract).
37+
- **Phase 9 — Additive drift-detection evals.** Four new drift checks (roster ↔ enum bidirectional alignment, agent `Resources` ↔ schemas existence, cross-plan file-overlap with anchor-map gate, multi-doc check-count consistency) in `evals/tests/drift-detection.test.mjs`; `model_role` resolution check staged but gated off until Phase 4 spike re-enables it.
38+
- **Phase 10 — README & engineering-docs refresh.** Measured-value badges, new Feature rows for the six shipped capabilities, and `docs/agent-engineering/README.md` index for the 14 engineering documents.
39+
40+
### Changed
5641

57-
### Changed - Model Routing
58-
59-
- **Eval suite (410 checks)** — 227 structural + 78 behavior + 63 orchestration-handoff + 42 drift-detection. Advertised counts in `README.md`, `.github/copilot-instructions.md`, `CONTRIBUTING.md`, `evals/README.md`, `SECURITY.md`, and `.github/PULL_REQUEST_TEMPLATE.md` reconciled to the measured value and enforced by drift Check #5.
60-
61-
---
62-
63-
## [Unreleased] — ControlFlow Comprehensive Revision
64-
65-
Ten-phase revision and modernization program delivered per `plans/controlflow-comprehensive-revision-plan.md`.
66-
67-
### Added - Comprehensive Revision
68-
69-
- **Phase 1 — Researcher validation + `plans_classification`.** Evidence-first confirmation of all 10 audit findings and per-plan classification artifact gating downstream archival decisions.
70-
- **Phase 2 — CodeMapper drift inventory.** Mechanical cross-reference of agent model pins, tool grants, schema references, skill registrations, executor-enum alignment, and PreFlect baseline line counts.
71-
- **Phase 3 — Consistency fixes (relabel-only; archival gated).** Phase Executor Agents table reduced to 8 rows matching the `executor_agent` enum; three-way check-count reconciliation; in-place status relabels without destructive moves.
72-
- **Phase 4 — Model Routing (logical-index-only; spike deferred).** Published `governance/model-routing.json` with 10 logical roles covering all 13 agents and `docs/agent-engineering/MODEL-ROUTING.md`. Runtime opt-in via agent frontmatter `model_role:` is held pending VS Code frontmatter-tolerance verification; drift Check #1 remains gated off.
73-
- **Phase 5 — Observability & `trace_id` schema hardening.** UUIDv4 `trace_id` added as additive-optional field across 13 report / verdict / audit / discovery / research / clarification / plan schemas; `docs/agent-engineering/OBSERVABILITY.md` documents generation, propagation, and the NDJSON sink convention at `plans/artifacts/observability/<task-id>.ndjson`.
74-
- **Phase 6 — Agentic Memory layering.** Three-layer memory model (session / task-episodic / repo-persistent) codified in `docs/agent-engineering/MEMORY-ARCHITECTURE.md`; `NOTES.md` scoped to repo-persistent state with reversible migration manifest.
75-
- **Phase 7 — PreFlect canonicalization.** Single `skills/patterns/preflect-core.md` replaces per-agent PreFlect restatements with pointer + ≤5 domain lines; Orchestrator-enforced cross-plan entry gate against `plans/performance-optimization-plan.md`.
76-
- **Phase 8 — SOTA pattern integration (additive, no budget compounding).** `skills/patterns/reflection-loop.md` (pre-retry hook; default disabled; draws from existing `retry_budgets`), `skills/patterns/budget-tracking.md` (`budget_defaults` orthogonal to `retry_budgets`), and `docs/agent-engineering/AGENT-AS-TOOL.md` (MCP forward-compatible input contract).
77-
- **Phase 9 — Additive drift-detection evals.** Four new drift checks (roster ↔ enum bidirectional alignment, agent `Resources` ↔ schemas existence, cross-plan file-overlap with anchor-map gate, multi-doc check-count consistency) in `evals/tests/drift-detection.test.mjs`; `model_role` resolution check staged but gated off until Phase 4 spike re-enables it.
78-
- **Phase 10 — README & engineering-docs refresh.** Measured-value badges, new Feature rows for the six shipped capabilities, and `docs/agent-engineering/README.md` index for the 11 engineering documents.
79-
80-
### Changed - Comprehensive Revision
81-
82-
- **Eval suite (358 checks)** — 212 structural + 74 behavior + 49 orchestration-handoff + 23 drift-detection. Advertised counts in `README.md`, `.github/copilot-instructions.md`, `CONTRIBUTING.md`, `evals/README.md`, `SECURITY.md`, and `.github/PULL_REQUEST_TEMPLATE.md` reconciled to the measured value and enforced by drift Check #5.
83-
- Phase Executor Agents table reduced to 8 rows aligned with the `executor_agent` enum; AssumptionVerifier and ExecutabilityVerifier reclassified as review-only roles outside the executor enum.
42+
- **Codex Plugin Lifecycle**: Introduced fixed lifecycle sections (`## Progress`, `## Discoveries`, `## Decision Log`, `## Outcomes`, `## Idempotence & Recovery`) in strict-plan dialect, enforced by `validate-strict-artifacts.ps1`.
43+
- **Plugin First-Class Status**: Upgraded `controlflow-codex` to a first-class plugin deliverable in `AGENTS.md` and `README.md`.
44+
- **Eval suite count reconciliation** — Advertised eval counts in `README.md`, `.github/copilot-instructions.md`, `CONTRIBUTING.md`, `evals/README.md`, `SECURITY.md`, and `.github/PULL_REQUEST_TEMPLATE.md` were reconciled to the measured value. There is no live count-enforcement pass; the current total is reported by `cd evals && npm test`.
45+
- **Phase Executor Agents table** reduced to 8 rows aligned with the `executor_agent` enum; AssumptionVerifier and ExecutabilityVerifier reclassified as review-only roles outside the executor enum.
8446

8547
### Notes
8648

CONTRIBUTING.md

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ Thank you for your interest in contributing! This guide covers the key contribut
1010
- [Adding a new agent](#adding-a-new-agent)
1111
- [Editing an existing agent](#editing-an-existing-agent)
1212
- [Adding skills](#adding-skills)
13+
- [Adding or modifying eval scenarios and validator passes](#adding-or-modifying-eval-scenarios-and-validator-passes)
14+
- [Editing governance configuration (`governance/*.json`)](#editing-governance-configuration-governancejson)
1315
- [Proposing changes](#proposing-changes)
1416
- [Code of conduct](#code-of-conduct)
1517

@@ -60,8 +62,10 @@ npm run test:behavior
6062
- `ABSTAIN` / `NEEDS_INPUT` / failure classification behavior
6163
- Tool routing compliance if the agent uses external tools
6264

63-
5. **Register the agent in governance files**:
64-
- Add it to `governance/agent-grants.json` with its canonical tool grants.
65+
5. **Register the agent in governance files** (see AGENTS.md §4 editing checklist):
66+
- Add it to `governance/agent-grants.json` with its canonical agent/tool grants.
67+
- Add it to `governance/tool-grants.json` when the agent's tool surface changes — the eval suite enforces consistency.
68+
- Update `governance/runtime-policy.json` and `governance/rename-allowlist.json` where relevant (e.g. review routing/retry knobs or rename permissions).
6569
- Add it to `plans/project-context.md` (agent roster table).
6670

6771
6. **Update `README.md`**:
@@ -93,6 +97,24 @@ For project-wide orchestration audits, prefer `skills/patterns/orchestration-aud
9397

9498
---
9599

100+
## Adding or modifying eval scenarios and validator passes
101+
102+
The eval suite is the only verification gate (`cd evals && npm test`).
103+
104+
1. Scenario fixtures live in `evals/scenarios/`; add or edit the fixture that exercises the behavior, validated against the matching schema in `schemas/`.
105+
2. Validator passes live in `evals/validate.mjs` (structural/schema/P.A.R.T) and drift checks in `evals/drift-checks.mjs`. Add a pass only when an existing one cannot express the contract; keep new passes offline and network-free.
106+
3. Run `cd evals && npm test` (or the faster `test:structural` / `test:behavior`) and confirm green before opening a PR. See `evals/README.md` for pass descriptions.
107+
108+
---
109+
110+
## Editing governance configuration (`governance/*.json`)
111+
112+
1. Most knobs in `governance/` are referenced by Orchestrator/Planner prompt text — re-read the consumers and update the prompt wording when knob semantics change (per AGENTS.md §4).
113+
2. Keep grants consistent: `governance/agent-grants.json` and `governance/tool-grants.json` are enforced against agent frontmatter by the eval suite.
114+
3. Run `cd evals && npm test` after any governance edit to catch drift — changes here affect all agents.
115+
116+
---
117+
96118
## Contributing to Plugins (Codex and Claude Code)
97119

98120
Changes to files under `plugins/controlflow-codex/` and `plugins/controlflow-claude-code/` require their own validation in addition to the repository eval suite:

CodeReviewer-subagent.agent.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
description: 'Review code changes from a completed implementation phase.'
33
tools: ['search', 'usages', 'problems', 'changes', 'runCommands', 'runTasks', 'read/readFile']
4-
model: Claude Opus 4.7 (copilot)
4+
model: Claude Opus 4.8 (copilot)
55
model_role: capable-reviewer
66
---
77
You are CodeReviewer-subagent, the deterministic verification gate.
@@ -132,6 +132,7 @@ Agent-specific additions: _none_
132132
- `schemas/code-reviewer.verdict.schema.json`
133133
- `schemas/orchestrator.gate-event.schema.json`
134134
- `plans/project-context.md` (if present)
135+
- `skills/patterns/llm-behavior-guidelines.md` (load on non-trivial tasks — anti-pattern guardrails: scope drift, over-abstraction, silent assumptions, weak success criteria)
135136

136137
## Tools
137138

CoreImplementer-subagent.agent.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ Agent-specific fields:
7676
- `schemas/core-implementer.execution-report.schema.json`
7777
- `plans/project-context.md` (if present)
7878
- `docs/agent-engineering/TOOL-ROUTING.md`
79+
- `skills/patterns/llm-behavior-guidelines.md` (load on non-trivial tasks — anti-pattern guardrails: scope drift, over-abstraction, silent assumptions, weak success criteria)
7980

8081
## Tools
8182

NOTES.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,6 @@
22

33
Repo-persistent active-objective state only. See `docs/agent-engineering/MEMORY-ARCHITECTURE.md` for the three-layer memory model; task-episodic history lives under `plans/artifacts/<task-slug>/`.
44

5-
- Active objective: capability-matrix registry follow-up cleanup is complete; runtime uses registry metadata and strict file-level anchor-map coverage.
5+
- Active objective: comprehensive cleanup effort is complete; full eval gate (`cd evals && npm test`) is green.
66
- Blockers: none.
77
- Pending: optional follow-up to investigate pre-existing capability-matrix live-tree drift flags surfaced by the smoke test.

PlanAuditor-subagent.agent.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
description: 'Adversarial plan reviewer that audits architecture, security, and risk coverage before implementation begins.'
33
tools: ['read/readFile', 'read/problems', 'search/codebase', 'search/fileSearch', 'search/textSearch', 'search/listDirectory', 'search/usages']
4-
model: Claude Opus 4.7 (copilot)
4+
model: Claude Opus 4.8 (copilot)
55
model_role: capable-reviewer
66
---
77
You are PlanAuditor-subagent, the adversarial plan auditor.

Planner.agent.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,7 @@ Agent-specific additions:
114114
- `docs/agent-engineering/PROMPT-BEHAVIOR-CONTRACT.md`
115115
- `plans/project-context.md` (if present)
116116
- `skills/index.md` (domain skill mapping — read during Step 5)
117+
- `skills/patterns/llm-behavior-guidelines.md` (load on non-trivial tasks — anti-pattern guardrails: scope drift, over-abstraction, silent assumptions, weak success criteria)
117118
- Plan artifacts directory: `plans/` (default location for all plan and completion files)
118119

119120
## Tools

0 commit comments

Comments
 (0)