From 0d0bf2a9724e9412cfc18545897acba7814e03e3 Mon Sep 17 00:00:00 2001 From: jennypng <63012604+JennyPng@users.noreply.github.com> Date: Fri, 26 Jun 2026 16:39:24 -0700 Subject: [PATCH 01/21] plan --- agentic-workflow-design.md | 772 +++++++++++++++++++++++++++++++++++++ 1 file changed, 772 insertions(+) create mode 100644 agentic-workflow-design.md diff --git a/agentic-workflow-design.md b/agentic-workflow-design.md new file mode 100644 index 00000000000..8d32afca0fd --- /dev/null +++ b/agentic-workflow-design.md @@ -0,0 +1,772 @@ +# Agentic `research → plan → implement` workflow — design & implementation plan + +## 1. Problem + +When implementing features with AI agents, the highest-quality output comes from a +disciplined, multi-phase workflow where each phase runs in a **fresh agent session** (clean +context, no bias bleed from earlier reasoning) and hands off to the next phase through +**durable artifacts on disk**. Doing this by hand — opening new sessions, copying context, +re-prompting — is tedious and error-prone. + +**Goal:** chain the whole workflow behind a *single entrypoint* so one command takes a task +description and drives research → assumptions → classification → per-item research → plan → +implementation, producing reviewable artifacts at every step. + +## 2. Why the GitHub Copilot SDK (not a skill, not raw CLI) + +The workflow's hard requirement is **one fresh, isolated context per phase**. Evaluated +options: + +| Approach | Fresh context per phase? | Notes | +| --- | --- | --- | +| Single skill (`SKILL.md`) | ❌ | A skill runs inside one conversation/context. Can't isolate phases. | +| Sub-agents (`Task`/`/fleet`) | ✅ | Works, but orchestration logic lives in a long-lived parent context; gating/retries/parallelism are awkward to express. | +| Headless CLI (`copilot -p`) | ✅ | Works, but you hand-roll process lifecycle, parsing, retries. | +| **Copilot SDK** | ✅ | `client.createSession()` = a fresh isolated session, first-class. Same agent runtime as the CLI, exposed programmatically. | + +The [Copilot SDK](https://github.com/github/copilot-sdk) wins: it exposes the same agent +engine behind the CLI over JSON-RPC, manages the CLI process lifecycle automatically, and is +available for TS/Python/Go/.NET/Java/Rust. **We will build in TypeScript** (`@github/copilot-sdk`), +which bundles the CLI automatically and has the richest surface. + +What the SDK gives us that a script/skill can't easily: + +- **Fresh context per phase** — one `createSession()` per phase, on purpose. +- **Per-phase model selection** — cheap model for classification, strong model for plan/implement. +- **Custom tools** (`defineTool`) — give the agent a `write_artifact` / `validate_subitems` + tool so it *cannot* skip emitting an artifact or produce a malformed one. +- **Programmatic gating & retries** — our code inspects each artifact (exists? parses? schema-valid?) + before advancing, and re-runs a phase on bad output. +- **Parallel fan-out** — phase 4 spins up N concurrent sessions, one per sub-item. +- **Structured event capture** — subscribe to session events to auto-write the execution log, + token usage, and tool-call traces. +- **Checkpoint / resume** — persist "phase 3 complete" so a failed run resumes instead of + restarting from scratch. + +### SDK API surface we rely on (verified from SDK docs) + +```ts +import { CopilotClient, defineTool } from "@github/copilot-sdk"; + +const client = new CopilotClient(); // manages CLI process lifecycle +const session = await client.createSession({ // FRESH isolated context + model: "gpt-4.1", + streaming: true, + tools: [/* defineTool(...) */], + // onPermissionRequest: approveAll // for autonomous runs (verify exact option name) +}); +session.on("assistant.message_delta", e => process.stdout.write(e.data.deltaContent)); +session.on("session.idle", () => {}); +const res = await session.sendAndWait({ prompt }); +await client.stop(); +``` + +> Open item: confirm the exact TS option for auto-approving tool permissions in autonomous runs +> (Python exposes `PermissionHandler.approve_all`), **and** whether built-in file/shell tools can +> be scoped/disabled per session (the §6.1 enforcement model depends on it). Both are validated in +> the M0 capability spike (§11) before further build-out. + +## 3. The artifact contract (the heart of the system) + +Each run gets a working directory: `.agentic-workflow//` (see §3.1). Phases communicate +**only** through files in it. This makes every run inspectable, resumable, and lets us improve +any phase's prompt independently. + +| # | Phase | Fresh session | Reads | Writes | +| --- | --- | --- | --- | --- | +| 1 | research *(skippable, §3.2)* | ✅ | the codebase | `specs/architecture.md`, `specs/functional.md`, `specs/apispec.md` (only if an API definition is relevant to the task) | +| 2 | assumptions | ✅ | phase 1 specs *(or task + codebase if research skipped)* | `assumptions.md` | +| 3 | classify + split *(skippable, §3.2)* | ✅ | specs + assumptions *(or task + codebase if research skipped)* | `classification.md`, `subitems.json` | +| 4 | research-item (loop ×N, parallel) *(skippable, §3.2)* | ✅ each | one sub-item + specs | `research/.md` | +| 5 | plan | ✅ | specs + assumptions + all item research *(whatever exists)* | `plan.md` (structured, with checkpoint gates — see §5) | +| 6 | implement | ✅ | `plan.md` | code edits + `execution-log.md` | + +`specs/apispec.md` is **conditional**: phase 1 produces it only when the task touches an API +surface (REST/RPC/SDK contract, schema, public interface). It captures the relevant existing +API definition (endpoints/operations, request/response shapes, contracts) so later phases reason +about the real interface. To remove the "not relevant vs. forgotten/failed" ambiguity, phase 1 +**always records the decision explicitly** in a per-run `manifest.json`, e.g.: + +```jsonc +{ + "apispec": { "required": false, "reason": "Task does not touch any public API surface" } +} +``` + +Downstream phases read the manifest (not the mere presence/absence of the file) to decide +whether API concerns apply. + +### 3.1 Artifact storage & gitignore (not checked in by default) + +Run artifacts are **scratch by default** — they must not be committed unless the user opts in. + +- **Default location:** a single temp root `.agentic-workflow/` at the repo root, with one + subdirectory per run: `.agentic-workflow//`. (Leading dot keeps it out of the way; + a dedicated root — rather than reusing `.workflow/` — makes the ignore rule unambiguous.) +- **Auto-ignore without mutating tracked files:** the tool ignores the scratch tree **without + modifying the repo's tracked root `.gitignore`** (which would itself be a source-tree change + before phase 6 ever runs). Two mechanisms, in priority order: + 1. **Local ignore (default):** add `.agentic-workflow/` to `.git/info/exclude`, which is a + local, untracked ignore file — no committed change to the repo. + 2. **Self-contained inner `.gitignore`:** also write an inner `.agentic-workflow/.gitignore` + containing `*` and `!.gitignore`, so the whole tree is ignored and travels with the + directory (and the ignore file itself can be tracked if a user opts to commit artifacts). + + Mutating the repo-root `.gitignore` is **opt-in only** (`--write-root-gitignore`), never + automatic. Both steps no-op if already present or if the repo isn't a git repo. +- **Code edits are the exception:** phase 6 edits real source files in the repo — those are + *meant* to be committed (ideally on a feature branch via `--branch`). Only the + artifacts/logs under `.agentic-workflow/` are ignored, never the implementation changes. +- **Opt-in to keep artifacts:** `--out ` redirects the working-dir root to a committed + location (e.g. `docs/specs//`) when the artifacts themselves are a deliverable. When + `--out` points outside `.agentic-workflow/`, the tool does **not** auto-ignore it (the user is + explicitly choosing to track those files). +- **Cleanup:** `agentic-workflow clean []` removes a run's scratch dir; `--keep-days ` + config prunes old runs. Since it's all under the ignored root, deletion never affects git state. + +Resolution order for the working-dir root: `--out` flag → `AGENTIC_WORKFLOW_OUT` env var → +config file → default `.agentic-workflow/`. + +### 3.2 Skipping phases for simple tasks (`--skip-research`, `--skip-classify`) + +For small, well-understood tasks, some phases are pure overhead. Two independent skip flags let +the user trade rigor for tokens. Both record the decision in `manifest.json` (so `status` and +downstream phases can tell "skipped" apart from "missing/failed"), and both keep the plan phase's +full structured `plan.md` (incl. machine-readable gates) — skipping is an *input* shortcut, never +a reduction of plan rigor. + +**`--skip-research[=all|specs]`** (alias `--no-research`) — omit **phase 1** spec generation. + +- **What's skipped:** phase 1 writes no `specs/*`; `manifest.json` records + `{ "research": { "skipped": true, "reason": "" } }`. +- **Downstream adaptation:** phases that normally read `specs/*` (assumptions, classify, plan) + fall back to reading the **task description + the codebase directly**. Their prompt templates + branch on `manifest.research.skipped`. +- **Granularity:** `all` (default when no value) skips phase 1 **and** phase-4 per-item research + (the same "deep research" cost); `specs` skips only phase 1 but keeps phase-4. Bare + `--skip-research` ≡ `--skip-research=all`. +- **API safety valve:** skipping research loses `apispec.md`. The classify/plan prompts must still + set `manifest.apispec` from a quick inspection; if they detect likely API impact, they emit a + `blocking: true` assumption (§5) so the run pauses before proceeding without an API spec. + +**`--skip-classify`** (alias `--no-classify`) — omit **phase 3** classification & splitting for +tasks that are obviously a single, atomic unit of work. + +- **What's skipped:** no `classification.md`; the orchestrator **synthesizes a trivial + `subitems.json`** with exactly one item (the whole task) so the schema/contract downstream + phases rely on is preserved. `manifest.json` records + `{ "classify": { "skipped": true, "reason": "..." } }`. +- **Single-item synthesis:** the lone item gets `id: "main"`, `type` defaulted to the run's + `--type` flag if given (else `"feature"`), `dependsOn: []`, `overlapRisk: "low"`, and + `expectedFilesOrAreas`/`acceptanceCriteria` left to the plan phase to infer. +- **Phase-4 collapse:** with one item, the fan-out is a single (optional) research session, not a + parallel set — and is itself skipped when `--skip-research[=all]` is also set. +- **Plan reconciliation note:** the "research reconciliation" section of `plan.md` (§5) is a no-op + when there's a single item; the plan template notes this rather than inventing overlaps. + +**Combining for the smallest tasks:** `--skip-research --skip-classify` reduces the pipeline to +**assumptions → plan → implement** (a typo fix, a one-line guard, etc.). The orchestrator +**warns** (does not error) if either skip is combined with `--autopilot` on a task that turns out +to be larger than expected — e.g. classify is skipped but the plan phase proposes edits across +many unrelated areas — since skipping raises the risk of an under-informed plan. + +**`--simple` (convenience preset).** Because skipping both research and classify is the common +"this is a small task" case, `--simple` is a single flag that **defaults both +`--skip-research=all` and `--skip-classify` on**, yielding the assumptions → plan → implement +pipeline. It's purely a preset: + +- Equivalent to `--skip-research --skip-classify`; `manifest.json` records the same `skipped` + entries (with reason `"--simple"` unless `--reason` overrides). +- **Individual flags override the preset.** e.g. `--simple --no-skip-research` keeps research but + still skips classify; `--simple --skip-research=specs` downgrades the research skip to specs-only. + (Each skip exposes a `--no-skip-*` negation so the preset can be partially re-enabled.) +- Does **not** imply `--autopilot` — mode is still chosen separately; `--simple` only changes which + phases run, not whether the run pauses for review. + +### `subitems.json` schema (sketch) + +```jsonc +{ + "task": "string — original task description", + "classification": "feature | bug | refactor | mixed", + "items": [ + { + "id": "kebab-case-id", + "type": "feature | bug | refactor", + "title": "short title", + "description": "what this sub-item covers", + "rationale": "why it's a separate item", + "dependsOn": ["other-item-id"], // ordering / dependency awareness + "expectedFilesOrAreas": ["src/api/**"], // where this item is expected to touch + "acceptanceCriteria": ["..."], // how we know this item is done + "nonGoals": ["..."], // explicitly out of scope for this item + "overlapRisk": "low | medium | high" // risk of conflicting with sibling items + } + ] +} +``` + +The classify prompt must aim for **independent, non-overlapping** sub-items and populate +`dependsOn`/`overlapRisk` honestly. These fields let phase 4 order/parallelize safely and let +phase 5 reconcile sibling research (see §5, "research reconciliation"). + +## 4. Proposed code layout (to be built in a later pass) + +Following this repo's tool convention (`/tools//` self-contained with README, tests, CI): + +``` +tools/agentic-workflow/ +├── README.md usage + design summary +├── package.json / tsconfig.json / .gitignore +├── ci.yml (from eng/pipelines/templates — follow-up) +├── prompts/ versioned per-phase prompt templates (the IP) +│ ├── 01-research.md +│ ├── 02-assumptions.md +│ ├── 03-classify.md +│ ├── 04-research-item.md +│ ├── 05-plan.md +│ └── 06-implement.md +├── schema/ +│ └── subitems.schema.json +└── src/ + ├── index.ts CLI entrypoint: parse task → run workflow + ├── orchestrator.ts phase sequencing, checkpoint/resume, gate enforcement + ├── phase.ts runPhase: createSession → render prompt → sendAndWait → validate + ├── policy.ts per-phase tool/filesystem policy + post-phase git-diff guard (§6.1) + ├── gates.ts parse plan stages/gates; run gate commands; enforce allowed_files (§6.2) + ├── prompts.ts load + render templates (var substitution) + ├── artifacts.ts working-dir mgmt, atomic read/write, hashing, run lock, state file + ├── tools.ts custom SDK tools: write_artifact, validate_subitems + ├── config.ts per-phase model + options + └── types.ts shared types (Phase, SubItem, RunState, Stage, Gate…) +``` + +### Orchestrator flow (pseudocode) + +```ts +const client = new CopilotClient(); +const ctx = workingDir(slug(task)); // .agentic-workflow// +await acquireRunLock(ctx); // prevent concurrent resume/approve (§9.4) +const state = loadCheckpoint(ctx); // resume support + +await phase(client, "research", { reads: [], writes: ["specs/architecture.md","specs/functional.md","manifest.json"], policy: READ_ONLY }); +await phase(client, "assumptions", { reads: ["specs/*"], writes: ["assumptions.md"], policy: READ_ONLY }); +const items = await phase(client, "classify", { reads: ["specs/*","assumptions.md"], writes: ["subitems.json"], validate: subitemsSchema, policy: READ_ONLY }); + +// parallel, FRESH session each, bounded concurrency, each item write-scoped to its own note +await mapWithConcurrency(items, config.concurrency, it => + phase(client, "research-item", { item: it, writes: [`research/${it.id}.md`], policy: researchItemPolicy(it) })); + +await phase(client, "plan", { reads: ["specs/*","assumptions.md","research/*"], writes: ["plan.md"], validate: planSchema, policy: READ_ONLY }); + +// Phase 6 is STAGED: each plan stage runs in its own fresh sub-session, gated by the orchestrator. +const stages = parsePlanStages("plan.md"); // structured stages + machine-readable gates (§6.2) +for (const stage of stages) { + await phase(client, "implement-stage", { stage, reads: ["plan.md"], appends: ["execution-log.md"], updates: ["plan.md"], policy: implementPolicy(stage) }); + const dev = checkDeviation(stage); // edits outside expected_files must be documented (§5) + if (dev.undocumented) { markPaused(state, stage, dev); break; } // missing docs → pause for review + const result = runGate(stage.gate); // ORCHESTRATOR runs gate cmds, checks exit codes + appendGateResult("execution-log.md", result); + if (!result.passed) { markFailed(state, stage); break; } // halt at the failing gate + checkpoint(state, stage); +} + +await client.stop(); +``` + +`phase()` responsibilities: +1. `client.createSession({ model: config[phase].model, tools, onPermissionRequest, policy })` +2. Render the phase's prompt template with run vars (task, ctx paths, item). +3. `session.sendAndWait({ prompt })`, streaming events into the execution log. +4. **Validate** the expected output artifact(s): exists → parses → schema-valid. +5. **Enforce policy (§6.1):** for non-implementation phases, assert `git diff` is empty (no + source mutation) and that only the declared artifact paths were written; fail the phase if not. +6. On failure: retry up to N times by spawning a **fresh session each time** (preserving the + clean-context guarantee), seeding it with the original inputs **plus** the validator/policy + errors — never by continuing the failed session's transcript. Else abort the run. +7. Persist checkpoint (`state.json`) atomically: phase/item status + artifact hashes. + +## 5. Per-phase prompt design + +Each template is a small, versioned markdown file with explicit I/O contract. Pattern: + +- **Role/goal** for the phase. +- **Inputs**: exact file paths to read (no guessing). +- **Output**: exact file path(s) to write, required sections, format. +- **Constraints**: stay in scope; don't implement during research; cite code locations; etc. + +Examples of the intent per phase: + +- **01-research** *(skippable via `--skip-research`, §3.2)* — produce architecture + functional + specs of the *current* code relevant to the task. Read-only exploration; no design proposals + yet. **Conditionally** produce `specs/apispec.md` when the task touches an API surface + (REST/RPC/SDK contract, schema, public interface), documenting the relevant existing API + definition. +- **02-assumptions** — from the specs, enumerate baseline assumptions, unknowns, and risks the + later work will rely on. One assumption per line with rationale/confidence. **Clarification + stop:** if any assumption is *low-confidence and affects correctness, security, or API + behavior*, the prompt must flag it as `blocking: true`; the orchestrator then **pauses and asks + the user** before proceeding to classify/plan rather than inventing an answer (see §9.1, applies + in every mode including autopilot). +- **03-classify** *(skippable via `--skip-classify`, §3.2)* — classify the task as + bug/feature/refactor (or mixed) and split into **independent, non-overlapping** sub-items; emit + `subitems.json` (schema-validated, incl. `dependsOn`/`overlapRisk`/`expectedFilesOrAreas`) + a + human-readable `classification.md`. When skipped, the orchestrator synthesizes a single-item + `subitems.json` instead of running this phase. +- **04-research-item** — deep, isolated research on a single sub-item, with specs as context; + output a focused research note. Runs once per item, in parallel (ordered by `dependsOn`). +- **05-plan** — consume all research and produce a highly detailed, structured implementation + plan. `plan.md` must contain these sections, in order: + 0. **Research reconciliation** — reconcile the N independent phase-4 notes: call out overlaps, + contradictions, and duplicated work, and state the single coherent strategy chosen. (Comes + first because the rest of the plan depends on it.) + 1. **Decisions and rationale** — the choices made and *why*. + 2. **End-to-end approach** — the overall strategy, and explicitly how the success criterion + will be *proved*. + 3. **Step-by-step implementation plan** — ordered, concrete steps (file-by-file changes). + Steps are grouped into stages, each ending in a **checkpoint gate** (see below). + 4. **Stop/go gates** — explicit points where work pauses for validation before continuing. + 5. **Validation plan** — tests to run, tests to add, and observability checks. + 6. **Rollout strategy** — how the change is shipped/enabled. + 7. **Rollback plan** — how to safely revert. + 8. **Risks and mitigations**. + 9. **Definition of done** — concrete, checkable completion criteria. + 10. **Open questions**. + 11. **Out-of-scope observations** — things noticed but deliberately not addressed. + 12. **Plan changes** — initially empty; phase 6 appends entries here whenever implementation + deviates from the plan (what changed, why, impact), so `plan.md` stays the source of truth. + + **Machine-readable gates (required).** Beyond the prose, the plan must embed a structured, + machine-parseable block (validated by `planSchema`) that the orchestrator — not the agent — + enforces at runtime: + + ```yaml + stages: + - id: stage-1 + expected_files: ["src/foo.ts", "test/foo.test.ts"] # the plan's best guess at scope (advisory) + steps: + - { id: "1.1", description: "..." } + gate: + id: gate-1 + commands: ["npm test -- foo"] # orchestrator runs these + expected: exit_code_0 # pass criterion + ``` + + Each stage names its `expected_files` (the plan's anticipated scope — **advisory, not a hard + wall**, see phase 6) and a `gate` with concrete validation `commands` + expected result. + Free-form prose gates alone are insufficient because they're unenforceable. +- **06-implement (staged).** Implementation is **not** one big session. Each plan *stage* runs in + its **own fresh sub-session** (`implement-stage-1 → gate → implement-stage-2 → …`), preserving + the clean-context guarantee inside the riskiest phase and bounding context growth. Per stage: + - `expected_files` is the **anticipated** scope, not a lock. Implementation legitimately + surfaces plan gaps — the agent **may edit beyond `expected_files`** when needed to do the work + correctly, provided the deviation is **documented** (see deviation policy) and the scope is + **justified** in the execution log. The orchestrator does not hard-block out-of-scope edits. + - The agent appends to `execution-log.md`. For **every action** the log must (a) **justify the + scope** — why it was necessary — and (b) **map it to a concrete step** in `plan.md` (by + stage/step id). + - **Gate enforcement is the orchestrator's job, not the agent's:** after the sub-session ends, + the orchestrator runs the stage's `gate.commands`, checks `expected`, records pass/fail + + output in the log, and **halts at the first failing gate** rather than continuing. + - **Deviation policy (documented, not blocked):** deviating from the plan is **allowed** — it's + often the *correct* response to a gap discovered mid-implementation. The requirement is + transparency, not prevention: + - The agent appends a **"Plan changes"** entry to `plan.md` describing what changed and why + (e.g. additional files touched, a step reordered, an approach adjusted), keeping `plan.md` + the source of truth. + - The `execution-log.md` entry for the deviating action **justifies its scope** and maps it to + the originating step / plan-change entry. + - The orchestrator **detects** edits outside `expected_files` and **verifies the deviation was + documented** (a Plan-changes entry + a scope justification exist); if an out-of-scope edit + has no documentation, *that* is the failure — it pauses for review, not the edit itself. + - **Larger deviations** — a change to **architecture, public API, or overall test strategy** — + still **stop** and either request approval (review mode) or, if the divergence invalidates + the plan, return to the plan phase. The line is "did this break the plan's foundation?", + not "did this touch an unlisted file?". Silent, *undocumented* scope expansion remains + disallowed. + +## 6. Custom SDK tools (reliability) + +- `write_artifact(path, content)` — the only sanctioned way for a phase to emit its artifact; + lets us guarantee the file lands in the working dir with the right name (with path-traversal / + normalization checks so a phase can't escape its scope). +- `validate_subitems(json)` — validates phase-3 output against the schema and returns errors so + the agent can self-correct via its own tool call **within its turn** before the orchestrator + gates. (This is in-turn validation feedback, not a phase-level retry — phase retries still + spawn a fresh session per §4.) + +### 6.1 Per-phase tool & filesystem policy (enforced, not requested) + +The "phases communicate only through files" guarantee must be **enforced by the orchestrator**, +not merely asked of the agent. Each phase runs under an explicit policy: + +| Phase(s) | Repo source | Artifact writes | Network/shell | +| --- | --- | --- | --- | +| 1 research, 2 assumptions, 3 classify, 5 plan | **read-only** | only its declared artifacts via `write_artifact` | shell/network disabled (research may read code, not run it) | +| 4 research-item | **read-only** | only `research/.md` | as above | +| 6 implement-stage | **edit** (anywhere needed; edits beyond `expected_files` must be documented, §5) | append `execution-log.md`, update `plan.md` Plan-changes | shell allowed for gate/validation commands | + +Enforcement mechanisms (all orchestrator-side): +- **Scope built-in file/shell tools.** During the M0 spike, determine whether the SDK lets us + disable or path-scope the agent's built-in edit/shell tools per session. If it does, use it; if + not, fall back to running phases against a **read-only checkout / overlay** for non-impl phases. +- **`write_artifact` is the only write path** for phases 1–5, with path normalization + traversal + rejection so writes can't escape the run's working dir. +- **Post-phase git-diff guard.** After every non-implementation phase, assert `git diff` (and + untracked files) is empty; a non-empty diff fails the phase (the agent mutated source it + shouldn't have). +- **`expected_files` is advisory in phase 6.** After each stage, the orchestrator diffs the + changes against the stage's `expected_files`: edits *within* scope pass silently; edits *outside* + scope are allowed but **require a matching Plan-changes entry + execution-log justification** + (§5). An undocumented out-of-scope edit pauses the run for review — the missing documentation is + the failure, not the edit. Source edits in phase 6 are otherwise unrestricted. + +### 6.2 Orchestrator-enforced checkpoint gates + +Gates are **machine-checked by the orchestrator**, never self-reported by the agent: +- `gates.ts` parses the structured `stages:`/`gate:` block from `plan.md` (§5). +- After a stage's implement sub-session ends, the orchestrator **runs the gate's `commands`** + itself, compares against `expected` (e.g. `exit_code_0`), and records exit codes + output in + `execution-log.md`. +- On failure the run **halts at that gate** (autopilot stops too); the user can inspect, fix, and + `resume`. This removes reliance on the agent to honestly stop, validate, and report. + +## 7. Configuration + +`config.ts` maps each phase to a model + options, e.g.: + +| Phase | Model (suggested) | Why | +| --- | --- | --- | +| research / research-item | strong reasoning | comprehension quality matters most | +| assumptions | mid | structured enumeration | +| classify | cheap/fast | short structured output | +| plan | strongest | this is the highest-leverage artifact | +| implement | strong + tools | code edits + verification | + +Models, retry counts, parallelism cap, and working-dir root are all config-driven. + +## 8. Single entrypoint & optional skill front-door + +- Primary entrypoint: `npm run workflow -- ""` (or a thin `bin`). +- Optional **thin skill** (`SKILL.md`) whose only job is to invoke this tool, giving + discoverability inside interactive Copilot ("run the feature workflow on X"). The SDK program + remains the engine; the skill is just a friendly door. This is the recommended end state: + **SDK program = engine, skill = front door.** + +## 9. CLI design + +The single entrypoint is a CLI (`agentic-workflow`, also runnable via `npm run workflow --`). +It supports starting a new run, **per-phase gating** (where to pause vs. autopilot, §9.1), and +resuming an interrupted run. + +### Commands + +| Command | Purpose | +| --- | --- | +| `run ""` | Start a new workflow run for the given task. Default command. | +| `resume []` | Continue an existing run from its last completed phase (see §9.4). | +| `status []` | Show a run's progress: phases completed, current phase, artifacts written. | +| `list` | List known runs in the working-dir root with their state. | +| `approve []` | In review mode, mark the current paused stage approved and advance one phase. | +| `abort []` | Mark a run aborted (keeps artifacts for inspection). | +| `clean []` | Delete a run's scratch dir (or all old runs with `--keep-days`). Safe — the dir is gitignored. | + +`run` and `resume` are the core of the request; the others are thin helpers over the same +checkpoint state. + +### 9.1 Execution control: per-phase gating + +Rather than a single global mode, a run is controlled by a **gate set** — the set of phase +boundaries where the orchestrator **pauses for human review**. Every boundary *not* in the gate +set runs on autopilot. This lets the user place stops exactly where their judgment adds value +(e.g. review the plan, but auto-run the research) and let everything else flow. + +The familiar "modes" are just **presets** over this gate set: + +| Preset | Flag | Resulting gate set | +| --- | --- | --- | +| **Autopilot** | `--autopilot` | **∅** — no stops; runs end to end (auto-approve within guardrails). | +| **Review** (default) | `--review` | a gate **after every phase** — pause, inspect/edit, `resume`. | +| **Dry run** | `--dry-run` | orthogonal — renders prompts/flow with **no SDK calls or edits** at all. | + +**Fine-grained overrides** (applied on top of the preset, left to right): + +| Flag | Effect | +| --- | --- | +| `--pause-after ` | Add a gate **after** each listed phase. | +| `--pause-before ` | Add a gate **before** each listed phase. | +| `--auto ` | **Remove** the gate(s) around each listed phase (run it without stopping). | + +Phase names: `research`, `assumptions`, `classify`, `research-item`, `plan`, `implement`. +Because phase 6 is staged (§5), `implement:*` addresses *every* stage boundary and +`implement:` a specific one — so you can autopilot the implementation but pause between +stages, or vice-versa. + +**Resolution & persistence.** The orchestrator starts from the preset's gate set, then applies +the override flags in order, producing a concrete gate set that is **persisted in `state.json`**. +`resume` honors it; passing the same flags to `resume` adjusts it mid-run. `status` prints the +resolved gates so the user can see exactly where the run will stop next. + +**Examples:** + +```bash +# Auto-run research→plan, then review the plan and gate the implementation +agentic-workflow run "" --autopilot --pause-after plan --pause-before implement + +# Review everything EXCEPT the cheap early phases +agentic-workflow run "" --review --auto research,assumptions + +# Fully autopilot, but stop between each implementation stage to sanity-check +agentic-workflow run "" --autopilot --pause-after 'implement:*' + +# Only one stop in the whole run: right before code is touched +agentic-workflow run "" --autopilot --pause-before implement +``` + +Notes: +- `--review` is the safe default precisely because phase 6 edits real code; autopilot is opt-in. +- `--pause-before ` remains the common "auto-run up to here, then gate" shorthand; it's just + the single-phase case of the override flags above. +- **Blocking-clarification stop (independent of the gate set).** Even with an empty gate set + (autopilot), if phase 2 marks an assumption `blocking: true` (low-confidence and affecting + correctness/security/API behavior), the run pauses and asks the user rather than inventing an + answer — and likewise the orchestrator pauses on an **undocumented out-of-scope edit** or a + **failing gate** (§5/§6.2). These safety stops are not part of the user-defined gate set and + can't be removed by `--auto`; `--yes` suppresses only the *clarification* prompt for fully + unattended runs. + +### 9.2 Flags (full surface) + +``` +agentic-workflow run "" [options] + +Execution control (gate set; presets + per-phase overrides, §9.1): + --autopilot Preset: no stops (gate set = ∅). Auto-approve within guardrails. + --review Preset (default): pause after every phase. + --dry-run Render prompts/flow only; no SDK calls, no edits. + --pause-after Add a gate after each listed phase. + --pause-before Add a gate before each listed phase. + --auto Remove the gate(s) around each listed phase (run without stopping). + Phases: research, assumptions, classify, research-item, plan, + implement (and implement:* / implement: for stage gates). + +Run control: + --simple Preset for small tasks: defaults --skip-research=all and + --skip-classify on (assumptions → plan → implement); §3.2. + Override with --no-skip-research / --no-skip-classify. + --run-id Use/assign an explicit run id (default: slug of task + short hash). + --out Working-dir root (default: ./.agentic-workflow). Run lives in //. + --from Start at a specific phase (advanced; assumes prior artifacts exist). + --only Run just the listed phase(s), then stop. + --skip-research[=all|specs] Skip phase-1 spec generation (and phase-4 per-item research unless + =specs) for simple tasks to save tokens; §3.2. Alias: --no-research. + --skip-classify Skip phase-3 classification; synthesize a single-item subitems.json + for atomic tasks; §3.2. Alias: --no-classify. + --type Declared task type; used as the synthesized item type when + classify is skipped (default: feature). + --reason Reason recorded in manifest.json for a skipped phase. + +Execution tuning: + --model Override per-phase model (e.g. plan=gpt-5,classify=gpt-4.1-mini). + --concurrency Max parallel sessions for phase-4 fan-out (default: e.g. 3). + --max-retries Per-phase validation retry budget (default: e.g. 2). + --branch Git branch to run phase-6 edits on (recommended for safety). + +Permissions / guardrails (autopilot): + --allow-dir Additional directory the agent may read/write (repeatable). + --no-network Disallow network tools during the run. + --write-root-gitignore Opt in to adding `.agentic-workflow/` to the repo-root .gitignore + (default: ignore locally via .git/info/exclude only; §3.1). + --yes Non-interactive: assume "yes" to any orchestrator prompts + (also suppresses the blocking-clarification stop; §9.1). + +Output: + --json Machine-readable progress/events on stdout. + --quiet | --verbose Control log verbosity. +``` + +`resume`, `status`, `approve`, `abort` accept `[]` and `--out `; if `run-id` is +omitted they target the most recent run under the working-dir root. + +### 9.3 Gating behavior at a glance + +``` +GATE SET = the phase boundaries where the run PAUSES for review. Everything else autopilots. + +--autopilot (gates = ∅): + research → assumptions → classify → research-item×N → plan → implement → done (no stops) + +--review (gates = after every phase): + research ─┐ resume assumptions ─┐ resume … plan ─┐ resume implement → done + (PAUSE+exit after each phase; user inspects/edits artifact, then resume) + +--autopilot --pause-after plan --pause-before implement (gates = {after plan} only*): + research → assumptions → classify → research-item×N → plan ─┐ resume implement → done + (one PAUSE, right before code is touched) + +DRY RUN: print rendered prompt + planned session/artifact for every phase; make no calls + +* "after plan" and "before implement" are the same boundary; safety stops (blocking + clarification, undocumented out-of-scope edit, failing gate) fire regardless of the gate set. +``` + +### 9.4 Resume semantics + +State lives in `//state.json`, written **atomically** and updated after every phase +(and every phase-4 sub-item). It records: the resolved **gate set** (§9.1), task, run-id, +**per-phase status** (`pending` / `in_progress` / `failed` / `completed` — not just "last +completed"), per-item phase-4 status, retry counts, template versions, and a **content hash for +each produced artifact**. A **run lock** (e.g. `run.lock`) is held while a run mutates state to +prevent concurrent `resume`/`approve` from corrupting it or duplicating phase-4 work. + +`resume` rules: +- Reads `state.json`, finds the last `completed` (and validated) phase, and continues — running + straight through autopilot boundaries and stopping at the next gate in the persisted gate set. + Re-passing `--pause-*`/`--auto`/`--autopilot`/`--review` adjusts the gate set for the remainder. + A phase left `in_progress`/`failed` (e.g. after a crash) is re-run from scratch in a fresh + session — never half-trusted. +- **Revalidate before consuming.** On every resume, re-hash and re-validate the input artifacts. + If a hash differs from `state.json`, the artifact was edited since it was produced — in review + mode that's expected (record the new approved hash); the orchestrator re-checks it parses / + is schema-valid before downstream phases read it, so a corrupted/partial edit can't flow + through silently. +- Phase-4 fan-out resumes per item: items with a `completed` status **and** a matching hash are + skipped; missing/failed/edited items are re-run. +- In **review** mode, `resume` advances exactly one phase and pauses again. `approve` records the + current artifact's hash as approved and advances one phase. Repeated `resume`/`approve` walks + the user through the pipeline. +- In **autopilot** mode, `resume` (after a crash/failure) continues straight through to completion. +- The user may **edit an artifact** during a review pause; the next phase reads the edited file + from disk (and its new hash is recorded), so human corrections flow downstream naturally. +- `--from ` forces a starting phase (rewinding state and invalidating later artifacts' + hashes); guarded behind a confirmation since it discards downstream work. + +### 9.5 Exit codes (for scripting / CI) + +| Code | Meaning | +| --- | --- | +| `0` | Run **fully complete** (all phases done). | +| `10` | **Paused** — the run hit a gate in its gate set, a `--pause-*` boundary, a blocking clarification, an undocumented out-of-scope edit, or a failing gate, and awaits `resume`/`approve`. Distinct from completion. | +| `1` | Unrecoverable failure (validation exhausted retries, gate hard-failed with no path forward, SDK/auth error). | +| `2` | Bad usage / invalid flags. | + +### 9.6 Example sessions + +```bash +# True autopilot, isolated branch for the implementation phase +agentic-workflow run "Add rate limiting to the public API" \ + --autopilot --branch feat/rate-limiting + +# Stage-gated review: run one phase at a time, inspecting artifacts between +agentic-workflow run "Refactor auth module" --review +# → writes specs/, pauses. User reads specs/architecture.md, edits if needed. +agentic-workflow resume # → assumptions.md, pauses +agentic-workflow resume # → subitems.json + classification.md, pauses +agentic-workflow status # → shows phase 3 done, phase 4 next (N items) +agentic-workflow resume # → research/.md ×N (parallel), pauses +agentic-workflow approve # → plan.md, pauses (review the plan carefully) +agentic-workflow resume # → implements per plan, writes execution-log.md, done + +# Auto-run research→plan, but gate before touching code +agentic-workflow run "Migrate to new logging lib" --pause-before implement + +# Inspect prompts/flow without spending requests +agentic-workflow run "Add CSV export" --dry-run + +# Simple task: skip research/specs entirely to save tokens (straight to plan → implement) +agentic-workflow run "Fix typo in error message in src/auth.ts" --skip-research --autopilot + +# Tiny atomic task: --simple presets skip research + classify → assumptions → plan → implement +agentic-workflow run "Add a null guard in src/parse.ts:42" --simple --type bug --autopilot +``` + +## 10. Repo integration (per this repo's conventions) + +- Place under `tools/agentic-workflow/` with its own README and tests. +- Add `/tools/agentic-workflow/ @JennyPng` to `.github/CODEOWNERS`. +- Add a row to the root `README.md` tool index. +- Provide `ci.yml` from `eng/pipelines/templates` for build/test. +- Pipeline names: public `tools - agentic-workflow - ci`, internal `tools - agentic-workflow`. + +## 11. Phased build plan (milestones) + +1. **M0 — Capability spike (1 day):** stand up a minimal SDK program and **validate the + assumptions the whole design rests on** before building further. Confirm: + - `createSession()` truly isolates history (no bleed across sessions); + - multiple concurrent sessions per client work (needed for phase-4 fan-out); + - exact event names + streaming behavior; + - the TS permission hook / auto-approve option (Python has `PermissionHandler.approve_all`); + - **whether built-in file/shell tools can be scoped or disabled per session** (critical for + the §6.1 read-only policy) — if not, design the read-only-checkout fallback; + - working-directory behavior; custom-tool schema/validation support; model override; + - token/cost usage events; and that **code editing is actually available** via the SDK runtime. +2. **M1 — Artifact contract + working dir (½ day):** `artifacts.ts`, `types.ts`, `config.ts`, + `.agentic-workflow//` layout + local ignore (§3.1), atomic writes, hashing, run lock, + checkpoint `state.json` with per-phase status. +3. **M2 — `phase()` core + policy (1.5 days):** createSession → render prompt → sendAndWait → + validate → **policy/git-diff guard (§6.1)** → fresh-session retry → checkpoint. Stream events + to the execution log. +4. **M3 — Prompt templates (1 day):** author + iterate all 6 templates against a real task. +5. **M4 — Classification + parallel fan-out (1 day):** `subitems.json` schema (with deps/overlap) + + validator tool; parallel phase-4 sessions ordered by `dependsOn`, with a concurrency cap. +6. **M5 — Plan + staged implement + gates (1.5 days):** wire phase 5 (incl. machine-readable + `stages:`/`gate:` block + `planSchema`) and the **staged** phase 6 with orchestrator-enforced + gates and `allowed_files`; execution-log + deviation handling; end-to-end run on a sample. +7. **M6 — Hardening (1 day):** retries/backoff, resume revalidation + locking, dry-run, redaction. +8. **M7 — CLI surface (½ day):** wire commands (`run`/`resume`/`status`/`list`/`approve`/`abort`/ + `clean`), the gate-set resolution (presets + `--pause-after`/`--pause-before`/`--auto`), + blocking-clarification stop, exit codes. +9. **M8 — Packaging:** README, tests, `ci.yml`, CODEOWNERS, root README index. Optional skill + front-door. + +> **MVP cut (de-risk first):** a thinner v1 is worth landing before the full surface — +> research → (assumptions+classify) → optional per-item research → plan → **mandatory human +> approval** → staged implement with orchestrator-enforced gates. Defer `--only`, `--from`, +> `approve`-as-separate-from-`resume`, root-`.gitignore` mutation, and fully unattended phase 6 +> until the gate/policy enforcement is proven. The richer CLI above remains the target design. + +## 12. Open questions / risks + +- **Permissions in autonomous mode:** exact TS option to auto-approve tool/file permissions so + phases run unattended; what guardrails (allowed dirs, no network) to enforce. +- **Tool scoping (design-critical):** whether the SDK can disable/path-scope built-in file/shell + tools per session. The §6.1 enforcement model depends on this; the read-only-checkout fallback + exists if it can't. Resolve in M0. +- **Gate command safety:** orchestrator-run gate `commands` execute arbitrary shell from a + generated `plan.md`. Constrain to an allowlist / require approval of the gate block in autopilot. +- **Cost/quota:** each phase = ≥1 premium request; phase-4 fan-out and staged phase-6 multiply + this. Need a concurrency cap and per-run budget. +- **Context size for phase 5:** plan phase reads all specs + all item research — may need + summarization if large. +- **Implementation safety:** phase 6 edits real code. Run on a dedicated branch; gates + + `allowed_files` + deviation policy bound scope; human review of `plan.md` recommended. +- **Artifact location:** scratch by default — runs live under a locally-ignored `.agentic-workflow/` + root (`.git/info/exclude` + inner `.gitignore`; root-`.gitignore` only via opt-in flag); see §3.1. +- **Determinism:** prompt templates are versioned; the template version is stamped into each + artifact / `state.json` for traceability. + +## 13. Decisions locked so far + +- Language: **TypeScript** (`@github/copilot-sdk`). +- Mechanism: **SDK `createSession` per phase**, artifacts on disk as hand-off; **fresh session on + every retry** (no transcript reuse). +- **Guarantees are orchestrator-enforced, not prompt-only:** per-phase tool/FS policy + git-diff + guard (§6.1), machine-readable checkpoint gates the orchestrator runs (§6.2), and resume + revalidation + hashing + run lock (§9.4). +- **Phase 6 is staged:** one fresh sub-session per plan stage, each gated and confined to + `allowed_files`. +- Location: new tool at `tools/agentic-workflow/` (this design doc lives at repo root for now). +- End state: SDK engine + optional thin skill as the single entrypoint. +- CLI: `run` + `resume` core commands. Execution is controlled by a **per-phase gate set** — + the boundaries where the run pauses for review — with `--autopilot` (∅) and `--review` (every + phase) as presets, refined by `--pause-after`/`--pause-before`/`--auto` (incl. per-stage + `implement:*`). Safety stops (blocking clarification, undocumented out-of-scope edit, failing + gate) fire regardless. The resolved gate set persists in `state.json`; `--dry-run` is orthogonal. +- **Research is optional:** `--skip-research[=all|specs]` omits spec generation (and per-item + research unless `=specs`) for simple tasks; downstream prompts branch on `manifest.research.skipped` + and the API safety valve still guards against silently dropping an API spec (§3.2). +- **Classification is optional:** `--skip-classify` omits phase 3 for atomic tasks; the + orchestrator synthesizes a single-item `subitems.json` so the downstream contract is preserved. + Combined with `--skip-research`, the pipeline collapses to assumptions → plan → implement (§3.2). +- **`--simple` preset:** one flag defaults both skips on for small tasks; individual `--no-skip-*` + flags override it. It changes which phases run, not the execution mode (§3.2). From 7b57be308651d825aab476942abc6d3976672d09 Mon Sep 17 00:00:00 2001 From: jennypng <63012604+JennyPng@users.noreply.github.com> Date: Mon, 29 Jun 2026 14:09:21 -0700 Subject: [PATCH 02/21] initial impl --- .github/CODEOWNERS | 1 + .../extensions/agentic-workflow/extension.mjs | 91 + .plan.md | 479 ++++ README.md | 1 + agentic-workflow-design.md | 59 +- gao.md | 42 + thinnerplan.plan.md | 245 ++ tools/agentic-workflow/.gitignore | 6 + tools/agentic-workflow/.prettierignore | 6 + tools/agentic-workflow/.prettierrc.json | 6 + tools/agentic-workflow/EXECUTION-LOG.md | 184 ++ tools/agentic-workflow/README.md | 127 + tools/agentic-workflow/ci.yml | 57 + tools/agentic-workflow/package-lock.json | 2315 +++++++++++++++++ tools/agentic-workflow/package.json | 48 + tools/agentic-workflow/prompts/01-research.md | 31 + .../prompts/02-assumptions.md | 31 + tools/agentic-workflow/prompts/03-classify.md | 42 + .../prompts/04-research-item.md | 26 + tools/agentic-workflow/prompts/05-plan.md | 63 + .../agentic-workflow/prompts/06-implement.md | 52 + tools/agentic-workflow/prompts/critique.md | 26 + tools/agentic-workflow/prompts/revise.md | 30 + tools/agentic-workflow/spike/FINDINGS.md | 39 + tools/agentic-workflow/spike/package.json | 16 + tools/agentic-workflow/spike/spike.mjs | 81 + tools/agentic-workflow/src/artifacts.ts | 142 + tools/agentic-workflow/src/cli.ts | 164 ++ tools/agentic-workflow/src/gates.ts | 83 + tools/agentic-workflow/src/harness.ts | 108 + tools/agentic-workflow/src/index.ts | 7 + tools/agentic-workflow/src/orchestrator.ts | 385 +++ tools/agentic-workflow/src/prompts.ts | 34 + tools/agentic-workflow/src/session-options.ts | 129 + tools/agentic-workflow/src/state.ts | 53 + tools/agentic-workflow/src/types.ts | 80 + tools/agentic-workflow/src/validate.ts | 98 + tools/agentic-workflow/test/artifacts.test.ts | 73 + tools/agentic-workflow/test/gates.test.ts | 48 + .../test/orchestrator.test.ts | 227 ++ tools/agentic-workflow/test/prompts.test.ts | 43 + .../test/session-options.test.ts | 63 + tools/agentic-workflow/test/validate.test.ts | 102 + tools/agentic-workflow/tsconfig.json | 19 + 44 files changed, 5954 insertions(+), 8 deletions(-) create mode 100644 .github/extensions/agentic-workflow/extension.mjs create mode 100644 .plan.md create mode 100644 gao.md create mode 100644 thinnerplan.plan.md create mode 100644 tools/agentic-workflow/.gitignore create mode 100644 tools/agentic-workflow/.prettierignore create mode 100644 tools/agentic-workflow/.prettierrc.json create mode 100644 tools/agentic-workflow/EXECUTION-LOG.md create mode 100644 tools/agentic-workflow/README.md create mode 100644 tools/agentic-workflow/ci.yml create mode 100644 tools/agentic-workflow/package-lock.json create mode 100644 tools/agentic-workflow/package.json create mode 100644 tools/agentic-workflow/prompts/01-research.md create mode 100644 tools/agentic-workflow/prompts/02-assumptions.md create mode 100644 tools/agentic-workflow/prompts/03-classify.md create mode 100644 tools/agentic-workflow/prompts/04-research-item.md create mode 100644 tools/agentic-workflow/prompts/05-plan.md create mode 100644 tools/agentic-workflow/prompts/06-implement.md create mode 100644 tools/agentic-workflow/prompts/critique.md create mode 100644 tools/agentic-workflow/prompts/revise.md create mode 100644 tools/agentic-workflow/spike/FINDINGS.md create mode 100644 tools/agentic-workflow/spike/package.json create mode 100644 tools/agentic-workflow/spike/spike.mjs create mode 100644 tools/agentic-workflow/src/artifacts.ts create mode 100644 tools/agentic-workflow/src/cli.ts create mode 100644 tools/agentic-workflow/src/gates.ts create mode 100644 tools/agentic-workflow/src/harness.ts create mode 100644 tools/agentic-workflow/src/index.ts create mode 100644 tools/agentic-workflow/src/orchestrator.ts create mode 100644 tools/agentic-workflow/src/prompts.ts create mode 100644 tools/agentic-workflow/src/session-options.ts create mode 100644 tools/agentic-workflow/src/state.ts create mode 100644 tools/agentic-workflow/src/types.ts create mode 100644 tools/agentic-workflow/src/validate.ts create mode 100644 tools/agentic-workflow/test/artifacts.test.ts create mode 100644 tools/agentic-workflow/test/gates.test.ts create mode 100644 tools/agentic-workflow/test/orchestrator.test.ts create mode 100644 tools/agentic-workflow/test/prompts.test.ts create mode 100644 tools/agentic-workflow/test/session-options.test.ts create mode 100644 tools/agentic-workflow/test/validate.test.ts create mode 100644 tools/agentic-workflow/tsconfig.json diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index f64dbd4a71f..350afe67be7 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -65,6 +65,7 @@ # Azure SDK Tools MCP ##################### /eng/common/instructions/azsdk-tools/ @maririos @praveenkuttappan @MrJustinB +/tools/agentic-workflow/ @JennyPng # PRLabel: %azsdk-cli /tools/ai-evals/azsdk-mcp/ @jeo02 @smw-ms @praveenkuttappan @maririos # PRLabel: %azsdk-cli diff --git a/.github/extensions/agentic-workflow/extension.mjs b/.github/extensions/agentic-workflow/extension.mjs new file mode 100644 index 00000000000..92e2eb8f623 --- /dev/null +++ b/.github/extensions/agentic-workflow/extension.mjs @@ -0,0 +1,91 @@ +// Optional interactive front-door for the agentic workflow (thinnerplan T3.2). +// +// Thin shim ONLY: it registers a `/agentic-workflow` slash command + a `run_agentic_workflow` +// tool inside an interactive Copilot CLI session, and delegates to the *built* orchestrator. It +// contains no orchestration logic of its own — re-point the import and it still works. +import { joinSession } from "@github/copilot-sdk/extension"; +import { fileURLToPath } from "node:url"; +import * as path from "node:path"; + +const here = path.dirname(fileURLToPath(import.meta.url)); +// Resolve the built tool relative to this file: repo/.github/extensions/agentic-workflow -> tool. +const TOOL_DIST = path.resolve(here, "..", "..", "..", "tools", "agentic-workflow", "dist", "index.js"); + +let sessionRef; + +/** Parse the free-form arg string into RunOptions for the orchestrator. */ +function parseArgs(argstr) { + const tokens = (argstr ?? "").trim().match(/(?:[^\s"]+|"[^"]*")+/g) ?? []; + const opts = { task: "", judge: true }; + const taskWords = []; + for (let i = 0; i < tokens.length; i++) { + const t = tokens[i].replace(/^"|"$/g, ""); + if (t === "--simple") opts.simple = true; + else if (t === "--no-judge") opts.judge = false; + else if (t === "--judge-model") opts.judgeModel = tokens[++i]?.replace(/^"|"$/g, ""); + else if (t === "--run-id") opts.runId = tokens[++i]?.replace(/^"|"$/g, ""); + else taskWords.push(t); + } + opts.task = taskWords.join(" "); + return opts; +} + +async function execute(argstr) { + const session = sessionRef; + const opts = parseArgs(argstr); + if (!opts.task) { + await session?.log("agentic-workflow: provide a task, e.g. /agentic-workflow Add CSV export --simple"); + return; + } + let runWorkflow, SdkHarness; + try { + ({ runWorkflow, SdkHarness } = await import(TOOL_DIST)); + } catch { + await session?.log( + `agentic-workflow: build the tool first (cd tools/agentic-workflow && npm run build). Looked for ${TOOL_DIST}`, + ); + return; + } + await session?.log(`agentic-workflow: starting run for "${opts.task}"${opts.simple ? " (--simple)" : ""}…`); + const harness = new SdkHarness({ workingDirectory: process.cwd() }); + try { + const result = await runWorkflow(opts, harness); + await session?.log( + `agentic-workflow: ${result.message} (exit ${result.exitCode}) — artifacts at ${result.runDir}`, + ); + } catch (err) { + await session?.log(`agentic-workflow: run failed — ${err?.message ?? err}`); + } finally { + await harness.stop(); + } +} + +sessionRef = await joinSession({ + commands: [ + { + name: "agentic-workflow", + description: "Run the research -> plan -> implement workflow on a task.", + handler: (ctx) => execute(ctx.args), + }, + ], + tools: [ + { + name: "run_agentic_workflow", + description: + "Run the agentic research -> plan -> implement workflow headlessly on a task " + + "description. Accepts the same flags as the CLI (--simple, --no-judge, --judge-model).", + parameters: { + type: "object", + properties: { + task: { type: "string", description: "The task description" }, + args: { type: "string", description: "Optional flags, e.g. --simple --no-judge" }, + }, + required: ["task"], + }, + handler: async (a) => { + await execute(`${a?.task ?? ""} ${a?.args ?? ""}`); + return { content: [{ type: "text", text: "agentic-workflow run dispatched" }] }; + }, + }, + ], +}); diff --git a/.plan.md b/.plan.md new file mode 100644 index 00000000000..6a59fc3c653 --- /dev/null +++ b/.plan.md @@ -0,0 +1,479 @@ +# Implementation Plan — Agentic `research → plan → implement` Workflow + +> Derived from `agentic-workflow-design.md`. This is a **planning artifact only** — no code is +> implemented here. It translates the design into an ordered, gated build with explicit +> decisions, success proof, validation, and stop/go checkpoints. + +--- + +## 0. Decisions & rationale (locked for this build) + +| # | Decision | Rationale | +| --- | --- | --- | +| D1 | Build as a self-contained TS tool at `tools/agentic-workflow/` | Matches repo convention (`/tools//` with README/tests/CI); design §10/§13. | +| D2 | TypeScript + `@github/copilot-sdk` | Richest SDK surface, bundles CLI, ESM matches existing TS tools (`js-sdk-release-tools`). | +| D3 | Toolchain: `tsc` build, `vitest --run` tests, `prettier` format, ESM (`"type":"module"`) | Mirror `tools/js-sdk-release-tools` exactly to inherit known-good CI patterns. | +| D4 | **M0 capability spike gates everything** | The whole architecture (fresh-context isolation, tool scoping, permissions) rests on unverified SDK behavior. Build nothing further until proven. | +| D5 | Ship the **MVP cut first**, then widen to full CLI | Design §11 explicitly recommends de-risking gate/policy enforcement before the rich CLI surface. | +| D6 | Orchestrator-enforced guarantees, never prompt-only | Policy (§6.1), gates (§6.2), resume revalidation (§9.4) are code-checked. Prompts can't be trusted to self-police. | +| D7 | Fresh session per phase **and** per retry; staged phase-6 | Clean-context guarantee is the product's core value (design §13). | +| D8 | Artifacts scratch-by-default under locally-ignored `.agentic-workflow/` | No tracked-file mutation before phase 6 runs (§3.1). | +| D9 | **Orchestrator records the real diff, gate exit codes, and changed files; agents supply rationale only** | The execution log's *facts* (what changed, what gates returned) come from the orchestrator, not the agent's self-report. Keeps the audit trustworthy without an elaborate two-party ceremony. | +| D10 | **Validate machine artifacts with Zod at read/write boundaries** | `subitems.json` and the plan block are checked against Zod schemas in code; malformed output → fresh-session retry. No `schemaVersion` field and no generated JSON Schema until a format actually breaks (KISS). | +| D11 | **A failing gate pauses for human review (exit `10`) by default; hard failure (exit `1`) only under `--auto`/autopilot for that boundary or on retry exhaustion** | Resolves the `10`/`1` conflict consistently across §2, M2, M5, M7 (gao ambiguity: exit codes). | +| D12 | **Gate-command safety for v1 = human approval of the plan block in review mode** | The default review gate means a human reads and approves `plan.md` (including its gate commands) before implement runs. A full sandboxed/allow-listed executor is deferred until autopilot (D13). | +| D13 | **Review mode is the default; full autopilot (incl. gate sandbox + budget caps) is explicit opt-in** | Unattended phase 6 requires `--autopilot` and the extra safety machinery; the MVP doesn't pay for it. | + +**Build philosophy:** each milestone ends at a **stop/go gate** with a concrete, observable proof. +Milestones are ordered by **dependency, not duration** — an agent moves quickly, so the only thing +that throttles progress is whether the prerequisite gate is green. Do not start a milestone until +its predecessor gate passes. Where the dependency graph allows, work is parallelizable across +agents (noted per milestone — e.g. the six prompt templates in M3, and per-item research in M4). + +**Parallelization map (independent once their gate is green):** +- After **G1**: the M2 modules — `tools.ts`, `policy.ts`, and the `phase()` happy path — can be + authored concurrently, then integrated. +- After **G2**: the six M3 templates are independent files; author them in parallel. +- After **G3**: M4's schema (`subitems.schema.json`) and the fan-out runner are independent. +- M8 packaging items (README, `ci.yml`, CODEOWNERS, root index) are mutually independent. + +The critical path is strictly serial: **G0 → G1 → G2 → G3 → G4 → G5**, because each builds on the +prior's runtime contract. M6/M7 layer onto the M5 core and can interleave once G5 is green. + +--- + +## 1. End-to-end approach + +1. **Prove the platform (M0).** A throwaway spike answers the design-critical SDK questions. Its + findings either confirm the §6.1 enforcement model or trigger the **read-only-checkout fallback**. +2. **Lay the durable substrate (M1).** Working-dir, artifact I/O, hashing, run lock, `state.json`, + `manifest.json`, types, config. Everything else reads/writes through this. +3. **Build the phase engine (M2).** `phase()` = createSession → render → sendAndWait → validate → + policy/git-diff guard → fresh-session retry → checkpoint, with event streaming to the log. +4. **Author the IP (M3).** The six prompt templates — the highest-leverage, most-iterated asset. +5. **Add fan-out (M4)** and **plan + staged implement + gates (M5)** — the workflow's spine. +6. **Harden (M6)** resume/locking/dry-run/redaction, then **expose the CLI (M7)** and **package (M8).** + +Each phase communicates **only** through files in `.agentic-workflow//`, so any phase can be +re-run, inspected, or improved independently. + +--- + +## 2. How success will be proved + +The system's success criterion is: *one command drives research → … → implement, producing +reviewable, gated artifacts, with each phase isolated and each guarantee enforced by code.* This is +proved by layered, observable checks: + +| Claim | Proof | +| --- | --- | +| **Fresh context per phase** | M0 spike: a **single-use nonce** (random token never written to any artifact, log, or the repo) is revealed only inside session A; a fresh session B is asked for it and cannot produce it. "Isolation" is defined relative to SDK memory, external tools, and on-disk files — none may carry the nonce. Unit assertion on session isolation. | +| **Phases communicate only via files** | Post-phase git-diff guard (algorithm in M2.4) returns empty for phases 1–5 on a real run; integration test asserts no source mutation. | +| **Gates are orchestrator-run, not self-reported** | Inject a deliberately failing gate command → in default/review mode the run **pauses** at that stage with exit code `10`; under `--auto` for that boundary it **fails** with exit `1` (D11). The orchestrator runs the gate and records the captured exit code. | +| **Schema contracts hold** | `subitems.json` and the `plan.md` machine block validate against their Zod schemas (D10); malformed agent output triggers a fresh-session retry, proven by a fixture test. | +| **Audit records are trustworthy** | The execution log records the **actual diff, gate commands, and exit codes from the orchestrator** (D9); the agent only adds rationale. | +| **Resume is safe** | Kill a run mid-phase, `resume` re-runs the `in_progress` phase from scratch; edit an artifact between phases → new hash recorded, downstream phases re-run reading the edited content (M6.2). | +| **Skips preserve the contract** | `--simple` run produces a single-item `subitems.json` and a full structured `plan.md`; `manifest.json` records the skip reasons. | +| **End-to-end** | A real sample task runs to completion on a branch, produces all artifacts + an `execution-log.jsonl` mapping every edit to a plan step, and passes its own gates. | + +A run is "done" when: all phases `completed` in `state.json`, every gate passed, `execution-log.jsonl` +maps each action to a `plan.md` step, and exit code `0`. + +--- + +## 3. Step-by-step plan (by milestone) + +> Format per step — **What / Where / Why / Expected outcome.** Gates marked **⛔ STOP/GO**. + +### M0 — Capability spike *(blocks all other milestones)* + +- **M0.1 Stand up a minimal SDK program.** + - *What:* a single throwaway script that creates a `CopilotClient`, opens a session, sends a + prompt, streams deltas, and stops. + - *Where:* `tools/agentic-workflow/spike/` (deleted or archived before M8). + - *Why:* confirm the SDK installs, authenticates, and runs in this environment at all. + - *Expected:* a streamed assistant reply printed to stdout; clean `client.stop()`. + +- **M0.2 Verify session isolation.** + - *What:* generate a **single-use nonce** (random token), reveal it only inside session A, then in a *new* session B ask for it. The nonce is never written to any artifact, log, or the repo. + - *Why:* the entire design depends on `createSession()` giving a clean context; "isolation" must hold against SDK memory, external tools, and on-disk files. + - *Expected:* B cannot produce the nonce. Result documented verbatim in spike notes (without echoing the nonce itself). + +- **M0.3 Verify concurrency.** Open ≥3 sessions from one client simultaneously; confirm all + progress independently (needed for phase-4 fan-out). *Expected:* no cross-talk, no deadlock. + +- **M0.4 Resolve permissions hook.** Find the **exact TS option** to auto-approve tool/file + permissions (Python = `PermissionHandler.approve_all`). *Expected:* an autonomous session runs + without interactive prompts. + +- **M0.5 Resolve tool scoping (design-critical).** Determine whether built-in file/shell tools can + be **disabled or path-scoped per session**. *Expected:* a definitive yes/no. + - **yes →** §6.1 read-only policy uses native scoping. + - **no →** adopt the **read-only-checkout/overlay fallback** for non-impl phases. This must become a + **concrete M0 design output (`spike/FALLBACK.md`)**, not a yes/no branch, specifying: worktree/checkout + creation per phase, artifact handoff into the scoped tree, how implementation diffs are transferred + back to the real working dir (patch apply), cleanup, symlink handling, nested-repo behavior, and + treatment of ignored files. **This decision and its mechanism feed M2's policy design.** + +- **M0.5a (deferred to autopilot).** A standalone gate-command sandbox (allowlist/scrubbed env/ + no-TTY executor) is **not built for the MVP** — review-mode human approval of the plan block is the + v1 safety boundary (D12). Revisit when `--autopilot` is added. + +- **M0.6 Catalog the surface.** Record exact event names, streaming behavior, working-dir behavior, + custom-tool (`defineTool`) schema/validation support, model override, token/cost usage events, + and that **code editing is actually available** via the runtime. + +- **⛔ STOP/GO GATE G0:** A written `spike/FINDINGS.md` answers every §11-M0 / §12 question with a + concrete result, and (if scoping is unavailable) `spike/FALLBACK.md` exists. **Go** only if isolation + (M0.2), concurrency (M0.3), permissions (M0.4), and an enforcement path (M0.5, native *or* specified + fallback) are all confirmed. **No-go →** revise the design's enforcement model before proceeding (do + not build on assumptions). + +--- + +### M1 — Artifact contract + working dir *(after G0)* + +- **M1.1 Project scaffold.** + - *What:* `package.json` (ESM, scripts: `build`/`test`/`format` mirroring `js-sdk-release-tools`), + `tsconfig.json`, `vitest.config.ts`, tool-local `.gitignore`. + - *Where:* `tools/agentic-workflow/`. + - *Why:* inherit the repo's known-good TS tool conventions (D3). + - *Expected:* `npm install && npm run build` succeeds on an empty `src/index.ts`. + +- **M1.2 Shared types + schemas.** `src/types.ts` — `Phase`, `SubItem`, `RunState`, `Stage`, `Gate`, + `Manifest`, `Policy`, `GateSet`. Define machine artifacts with **Zod** and validate against those + schemas at boundaries (D10). No `schemaVersion` fields or generated JSON Schema files. *Why:* single + source of truth consumed by every module. + +- **M1.3 Working-dir + ignore management.** `src/artifacts.ts` — resolve root (`--out` → env → + config → default `.agentic-workflow/`), create `/`, write `.git/info/exclude` entry + + inner `.agentic-workflow/.gitignore` (`*` / `!.gitignore`); both no-op if present or non-git + (§3.1). *Expected:* artifacts dir exists and is git-ignored without touching tracked `.gitignore`. + +- **M1.4 Atomic I/O + hashing.** Atomic write (temp + rename), read, content-hash helpers in + `artifacts.ts`. *Why:* resume revalidation and crash-safety depend on atomicity + hashes. + +- **M1.5 Run lock + dirty-worktree preflight.** `run.lock` acquire/release around state mutation + (§9.4); a second concurrent run/resume on the same run-id fails fast. Plus a lightweight preflight: + warn (don't hard-block) if the worktree is dirty outside `.agentic-workflow/`, and record the branch + + base commit for the git-diff guard. *Why:* crash-safety and a clean base, without speculative + cross-run branch coordination. *Expected:* concurrent same-run-id access fails fast; a dirty worktree + warns. + +- **M1.6 Checkpoint state + manifest (orchestrator-written).** `state.json` (per-phase status, retry + counts, template versions, artifact hashes, resolved gate set) and `manifest.json` (apispec/research/ + classify decisions), **both written only by the orchestrator** from agent-supplied decision + artifacts (D9). Write atomically. *Why:* the spine of resume and skip-vs-missing disambiguation. + +- **M1.7 Config.** `src/config.ts` — per-phase model map, retry budget, concurrency cap, working-dir + root, all overridable. *Expected:* defaults load; `--model plan=…` style overrides parse. + +- **⛔ STOP/GO GATE G1:** Unit tests prove atomic write survives a simulated crash, hashes are + stable, the lock is exclusive, and a fresh `state.json` round-trips. **Go** when green. + +--- + +### M2 — `phase()` core + policy *(after G1)* + +- **M2.1 `phase()` happy path.** `src/phase.ts` — `createSession({model, tools, onPermissionRequest, + policy})` → render prompt → `sendAndWait` → stream events into an **orchestrator-owned, redacted + structured log** (`execution-log.jsonl`). *Where:* `phase.ts` + `prompts.ts` (template load/render + with var substitution) + `log.ts`. The orchestrator records the facts (events, tool calls, exit + codes, file changes); **redaction runs inline at write time** (secret/token patterns scrubbed before + anything touches disk — built in here, hardened in M6.4, not deferred) (gao crit. #5). *Expected:* a + stub phase produces a declared artifact via `write_artifact`, with no raw secrets in the log. + +- **M2.2 Validation pipeline.** exists → parses → schema-valid, per phase. *Why:* gate output + quality before the orchestrator advances. *Expected:* a malformed artifact is rejected. + +- **M2.3 Custom tools.** `src/tools.ts` — `write_artifact(path, content)` (path normalization + + traversal rejection) and `validate_subitems(json)`. *Why:* the only sanctioned write path for + phases 1–5; lets a phase self-correct in-turn (§6). + +- **M2.4 Policy enforcement.** `src/policy.ts` — apply the M0-chosen mechanism (native scoping *or* + read-only checkout per `spike/FALLBACK.md`) + a **post-phase git-diff guard** for non-impl phases: + 1. Record HEAD + tracked/untracked state **before** the phase (the M1.5 base). + 2. After the phase, compute changes via `git status --porcelain` so **untracked source files are + included** (plain `git diff` misses them). + 3. **Exclude only the artifact root** (`.agentic-workflow/`). + 4. Any other add/modify/delete vs. the pre-phase snapshot → **fail the phase** (§6.1). + *Expected:* a phase that creates or edits a tracked or untracked source file is caught and failed; + edits confined to `.agentic-workflow/` pass. + +- **M2.5 Fresh-session retry.** On validation/policy failure, spawn a **brand-new** session (never + continue the transcript), seeded with original inputs **plus** the validator/policy errors, up to + `--max-retries`. *Why:* preserves clean-context guarantee under retry (D7). *Expected:* a fixture + that fails once then succeeds proves the retry path; exhausting retries is a **hard failure → exit + `1`** (D11). + +- **M2.6 Checkpoint write.** Persist phase status + artifact hashes atomically after each phase. + +- **⛔ STOP/GO GATE G2:** With a stub prompt, a phase runs end-to-end: session → artifact → + validate → git-diff guard empty → checkpoint; an injected bad output triggers exactly one + fresh-session retry. **Go** when proven on a real (cheap-model) call. + +--- + +### M3 — Prompt templates *(after G2; templates parallelizable across agents)* + +- **M3.1–M3.6 Author the six templates** in `prompts/`: `01-research.md`, `02-assumptions.md`, + `03-classify.md`, `04-research-item.md`, `05-plan.md`, `06-implement.md`. + - *What:* each with Role/goal, exact input paths, exact output path(s)+sections, constraints + (stay in scope, don't implement during research, cite code locations). + - *Why:* the IP of the system; isolating them as versioned files lets each be iterated alone. + - *Key contract points to encode:* + - `01` conditionally writes `specs/apispec.md` and emits an apispec-decision note that the + **orchestrator** reads and records in `manifest.json` (D9). Branches on `manifest.research.skipped`. + - `02` flags `blocking: true` for low-confidence assumptions affecting correctness/security/API. + - `03` emits schema-valid `subitems.json` (deps/overlap/expectedFilesOrAreas) + `classification.md`. + - `05` produces the ordered prose sections **and** the machine-readable plan block: a single fenced + ` ```yaml agentic-plan ` block carrying `stages[]` and per-stage `gate{commands,expected}`, + validated against its Zod schema (D10); sizes stages cohesive + loosely coupled. + - `06` writes `execution-log` rationale + `handoff.md`; the orchestrator separately records the + actual diff and gate exit codes (D9). Deviations are logged as `plan.md` "Plan changes" entries. + - *Expected:* each template renders with run vars and, on a sample task, yields a well-formed + artifact. + +- **M3.7 Template versioning.** Stamp a template version into each artifact + `state.json` (§12 + determinism). *Expected:* version visible in produced artifacts. + +- **M3.8 Prompt-template regression tests.** Lightweight **golden-structure tests** assert each + rendered template contains its required sections and exact input/output paths, so prompt edits can't + silently drop the core contract (gao opp. 3). *Expected:* a structural change to any template fails + its golden test. + +- **⛔ STOP/GO GATE G3:** On one real sample task, phases 1→2 produce sensible `specs/*` + + `assumptions.md` (with at least the structure correct). **Go** when templates are stable enough to + build fan-out/plan on top. (Prompt iteration continues throughout later milestones.) + +--- + +### M4 — Classification + parallel fan-out *(after G3)* + +- **M4.1 `subitems.json` schema.** `schema/subitems` Zod schema per the §3 sketch (id/type/deps/ + overlapRisk/expectedFilesOrAreas/acceptanceCriteria/nonGoals). Wire into `validate_subitems`. The + `--concurrency` cap bounds fan-out cost; a hard token-budget ceiling is deferred (see §7). + +- **M4.2 Classify phase + skip synthesis.** Run phase 3; when `--skip-classify`, synthesize a + single-item `subitems.json` (`id:"main"`, type from `--type` or `feature`, `dependsOn:[]`, + `overlapRisk:"low"`) and record the skip in `manifest.json`. *Expected:* downstream contract + identical whether classify ran or was skipped. + +- **M4.3 Ordered parallel fan-out.** `mapWithConcurrency` over items, **fresh session each**, + ordered by `dependsOn`, bounded by `--concurrency`; each item write-scoped to `research/.md`. + **Dependency input contract:** `dependsOn` controls ordering *and* feeds context — when item B + depends on item A, B's research session receives A's completed `research/A.md` as read-only input + (independent items do not). *Why:* phase-4 deep research, safely parallel, without losing upstream + findings. *Expected:* N notes produced; a dependent item's note references its prerequisite; an item + failure doesn't corrupt siblings; per-item status tracked in `state.json`. + +- **M4.4 Skip-research collapse.** With `--skip-research=all`, phase 4 is omitted; `=specs` keeps it. + Single-item + skip-research → phase 4 is a no-op. + +- **⛔ STOP/GO GATE G4:** A multi-item sample produces independent, non-overlapping `research/*.md` + in parallel under the concurrency cap; a single-item skip path also works. **Go** when green. + +--- + +### M5 — Plan + staged implement + gates *(after G4 — the core)* + +- **M5.1 `planSchema` + plan phase.** Validate the embedded plan block (the fenced `yaml agentic-plan` + block, M3 format) against its Zod schema; phase 5 reads all available specs/assumptions/research and + writes the full structured `plan.md` (research reconciliation first, through to plan-changes log). + *Expected:* `plan.md` parses and the machine block validates. + +- **M5.2 Stage parser.** `src/gates.ts` — `parsePlanStages("plan.md")` → ordered stages with + `expected_files`, `context_needed`, steps, gate commands + expected. *Expected:* round-trips the + sample plan's stages. (Gate commands run only after the plan block is human-approved in review mode, + D12.) + +- **M5.3 Context pack builder.** `buildContextPack(ctx, stage)` — `plan.md` + prior `handoff.md` + + cumulative diff + `context_needed` files (§6.3). *Why:* fresh ≠ blank; no inter-stage knowledge + loss. *Expected:* pack contains exactly the curated inputs, bounded in size. + +- **M5.4 Staged implement loop.** For each stage: fresh sub-session with the pack → agent edits and + writes `execution-log` rationale + `handoff.md`; the **orchestrator** records the actual diff and + changed files (D9). *Expected:* each stage runs isolated; handoff carries forward. + +- **M5.5 Deviation detection.** Orchestrator diffs edits vs `expected_files`; out-of-scope edits are + **allowed but require** a matching plan-changes entry + execution-log justification. Undocumented + out-of-scope edit → **pause for review** (the missing docs are the failure, not the edit, §5/§6.1). + +- **M5.6 Orchestrator-run gates.** After each stage, the orchestrator (not the agent) runs + `gate.commands`, checks `expected` (e.g. `exit_code_0`), records exit code + output in the + execution log, and **stops at the first failing gate** (§6.2): in default/review mode it + **pauses → exit `10`**; under `--auto` for that boundary it **fails → exit `1`** (D11). *Expected:* + a deliberately failing gate pauses (review) or fails (auto) the run at that stage. + +- **M5.7 Orchestrator wiring.** `src/orchestrator.ts` + `src/index.ts` — sequence phases per the + §4 pseudocode; load checkpoint; acquire lock. + +- **⛔ STOP/GO GATE G5 (key milestone):** A **real sample task runs end-to-end** on a branch: + specs → assumptions → classify → research → plan → staged implement; every edit maps to a plan + step; gates run by the orchestrator; an injected failing gate halts correctly. **Go** when the + full happy path + the failing-gate path are both demonstrated. + +--- + +### M6 — Hardening *(after G5)* + +- **M6.1 Retry/backoff** on transient SDK/auth errors (distinct from validation retries). +- **M6.2 Resume revalidation.** On `resume`, re-hash + re-validate inputs; `in_progress`/`failed` + phases re-run from scratch in a fresh session. When an artifact's hash changed between runs, + **re-run the phases after it in the fixed pipeline order** (`research → assumptions/classify → + per-item research → plan → implement`) reading the edited content — no per-artifact dependency-graph + bookkeeping. New hashes are recorded and re-checked before downstream reads (§9.4). Phase-4 resumes + per item (completed + matching-hash items skipped). +- **M6.3 Dry-run.** Render prompts + planned sessions/artifacts for every phase with **no SDK calls + or edits** (orthogonal to gating). +- **M6.4 Redaction.** A basic secret/token regex scrubs log/artifact writes (built inline in M2.1); + M6 adds tests proving injected secrets in SDK output, tool args, and shell output don't reach disk. + *Why:* present from the first log write, not deferred — but kept to simple patterns, not exhaustive + detection. +- **⛔ STOP/GO GATE G6:** Kill a run mid-phase → `resume` recovers correctly; edit-between-phases + flows downstream; `--dry-run` makes zero calls. **Go** when proven. + +--- + +### M7 — CLI surface *(after G6)* + +- **M7.1 Commands.** `run` (default), `resume`, `status`, `list`, `approve`, `abort`, `clean` + over the same checkpoint state (§9). **Approval model:** an approval is an orchestrator-recorded + entry in `state.json` (which boundary, by whom/when, the approved artifact hash); `approve` records + it and `resume` continues from the paused boundary — re-validating that the approved artifact's hash + is unchanged before proceeding (gao ambiguity: approval model). +- **M7.2 Gate-set resolution (review is the default).** Default preset is `--review` (pause at every + phase boundary); `--autopilot` is **explicit opt-in** (D13). Apply `--pause-after`/`--pause-before`/ + `--auto` left-to-right (incl. `implement:*` / `implement:`) → persist resolved gate set in + `state.json`; `status` prints it. +- **M7.3 Skip/preset flags.** `--simple`, `--skip-research[=all|specs]`, `--skip-classify`, + `--no-skip-*` negations, `--type`, `--reason`. +- **M7.4 Safety stops.** Blocking-clarification (phase-2 `blocking:true`), undocumented out-of-scope + edit, and failing gate — fire **regardless** of gate set; `--yes` suppresses only the clarification. + A failing gate **pauses (exit `10`)** unless that boundary is `--auto`, where it **fails (exit `1`)** + (D11). +- **M7.5 Exit codes.** `0` complete / `10` paused (incl. pending approval, paused gate) / `1` failure + (incl. `--auto` gate failure, retry exhaustion) / `2` bad usage (§9.5). +- **⛔ STOP/GO GATE G7:** The §9.6 example sessions all behave as documented (autopilot, + stage-gated review, `--pause-before implement`, `--dry-run`, `--simple`). **Go** when green. + +--- + +### M8 — Packaging *(after G7)* + +- **M8.1** `README.md` (usage + design summary) and tests covering the success-proof scenarios in §2. +- **M8.2** `ci.yml` from `eng/pipelines/templates`; pipeline names `tools - agentic-workflow - ci` + (public) / `tools - agentic-workflow` (internal). +- **M8.3** Add `/tools/agentic-workflow/ @JennyPng` to `.github/CODEOWNERS`; add a row to root + `README.md` tool index. +- **M8.4** Optional thin `SKILL.md` front-door that only invokes the tool. +- **M8.5** Remove/archive the M0 `spike/`. +- **⛔ STOP/GO GATE G8:** CI green; README accurate; CODEOWNERS + index updated. **Done.** + +--- + +## 4. MVP cut (land first, per design §11) + +Before the full CLI, ship: research → (assumptions + classify) → optional per-item research → +plan → **mandatory human approval** → staged implement with orchestrator-enforced gates. The MVP is +**review-mode** (D13): the single hard-coded "approve before implement" gate stands in for M7's +gate-set engine, and the approval is recorded in `state.json` so `resume` can continue from it. +Gate-command safety for the MVP is simply that a human approves the plan block (including its gates) +before implement runs (D12) — no separate sandbox or budget machinery. +**Defer:** `--only`, `--from`, `approve`-separate-from-`resume`, root-`.gitignore` mutation, the gate +sandbox, the token-budget ceiling, and fully unattended phase 6 (autopilot). This sequences as +**M0 → M1 → M2 → M3 → M4(core) → M5**. + +--- + +## 5. Validation plan + +**Unit tests (vitest, mirroring `js-sdk-release-tools`):** +- `artifacts`: atomic write/crash-safety, hashing stability, lock exclusivity, dirty-worktree warning, + gitignore/exclude writing (and non-git no-op). +- `schema`: `subitems.json` and the plan block (Zod) accept valid fixtures, reject malformed ones with + useful errors. +- `gates`: `parsePlanStages` round-trips a sample plan; gate runner captures exit codes correctly. +- `policy`: the git-diff guard flags tracked + untracked source mutations but ignores + `.agentic-workflow/`; `write_artifact` rejects path traversal. +- `gate-set`: preset + override resolution produces the expected boundary set (table-driven), with + **review as default**. +- `manifest`: skip flags record the right `skipped`/`apispec` entries; only the orchestrator writes + `manifest.json`. +- `log`: redaction scrubs injected secrets from SDK output, tool args, and shell output. +- `prompts`: golden-structure regression tests for the six templates (M3.8). + +**Integration / scenario tests — *mocked, deterministic* (no live quota):** +- Session isolation assertion via nonce (carry-over of the M0 spike). +- Fresh-session retry: fail-once-then-succeed fixture; retry-exhaustion → exit `1`. +- Failing-gate halt: injected failing command → **pause (exit `10`) in review, fail (exit `1`) under + `--auto`**; log records it. +- Resume: kill mid-phase → recover; edit-between-phases → new hash + later pipeline phases re-run + reading the edited file. +- Skip paths: `--simple` yields single-item `subitems.json` + full `plan.md` + manifest reasons. +- Dry-run: zero SDK calls / zero edits. + +**End-to-end smoke — *real SDK*, opt-in (on a throwaway branch):** +- A small but non-trivial task runs to completion; all artifacts present; `execution-log.jsonl` maps + every edit to a plan step; gates pass; final exit `0`. Kept separate from the deterministic suite and + **not run in CI unless a cheap-model lane is available**. + +**Add to CI:** `npm run build` + `npm test` (mocked/deterministic only) + format check. Real-SDK e2e +stays manual/opt-in (quota cost) unless a cheap-model lane is available. + +--- + +## 6. Open questions (must resolve, mostly in M0) + +1. Exact TS auto-approve permission option (Python parity: `PermissionHandler.approve_all`). — M0.4 +2. Can built-in file/shell tools be disabled/path-scoped per session? If not, the read-only-checkout + fallback is mandatory and fully specified in `spike/FALLBACK.md`. — **M0.5, design-critical** +3. **Gate command safety:** for the MVP, human approval of the plan block in review mode is the + safety boundary (D12). A sandboxed/allow-listed executor is **deferred** to autopilot — revisit then. +4. **Cost/quota:** the `--concurrency` cap bounds fan-out; a hard token-budget ceiling is **deferred** + (§7) until fan-out actually strains quota. +5. **Phase-5 context size:** plan reads all specs + all research; may need summarization if large. +6. Exact SDK event names / streaming semantics / custom-tool validation surface. — M0.6. +7. Cheap-model CI lane availability for automated e2e (else e2e stays manual, separate from the + deterministic suite). + +--- + +## 7. Out-of-scope observations (noticed, not in this plan) + +> Surfaced for consideration. A few items raised by the plan critique are scheduled (marked ✅ with +> their milestone); the rest, including several deliberately-deferred safety features, remain out of +> scope for the MVP. + +- **Per-run cost/budget ceiling.** Deferred. The `--concurrency` cap bounds fan-out for now; a hard + token/cost ceiling + live readout is future work once spend actually matters. +- **Gate-command sandboxing.** Deferred. MVP relies on human approval of the plan block (D12); a + full allow-listed/jailed executor is needed only for autopilot. +- **Multi-repo / monorepo scoping.** `--allow-dir` exists, but the git-diff guard and working-dir + assume a single repo root; nested/worktree behavior is unspecified. +- **Concurrent-run isolation on shared infra.** Per-run lock + a dirty-worktree warning (M1.5) only; + true sandboxed isolation across runs on shared infra remains out. +- **Artifact retention/telemetry.** `--keep-days` prunes scratch, but there's no aggregate run + history/metrics store to learn which prompts/models perform best over time. +- ✅ **Prompt-template regression testing.** Now **M3.8** (golden-structure tests). Deeper + golden-output/eval integration with `tools/ai-evals` remains a future enhancement. +- **Windows path/shell portability.** `.git/info/exclude`, atomic rename, and gate shell commands + need explicit cross-platform handling; the repo is multi-OS. +- **Secret redaction depth.** Basic pattern redaction is built in from M2.1/M6.4; exhaustive + entropy-based detection and safe-field allowlists remain future work. + +--- + +## 8. Stop/go gate summary + +| Gate | Milestone | Go criterion | +| --- | --- | --- | +| **G0** | M0 | Isolation (nonce), concurrency, permissions, and an enforcement path (native or specified fallback) all confirmed in `FINDINGS.md` (+ `FALLBACK.md` if scoping is unavailable). | +| **G1** | M1 | Atomic/crash-safe I/O, stable hashes, exclusive lock, state round-trip. | +| **G2** | M2 | Phase runs end-to-end with empty git-diff guard + exactly one fresh-session retry on bad output. | +| **G3** | M3 | Templates stable; phases 1–2 produce well-structured artifacts on a sample. | +| **G4** | M4 | Parallel, non-overlapping `research/*` under the cap; single-item skip path works. | +| **G5** | M5 | Full pipeline runs e2e on a branch; failing-gate halts correctly. **(key)** | +| **G6** | M6 | Resume recovers; edit-between-phases flows; dry-run makes no calls. | +| **G7** | M7 | All §9.6 example sessions behave as documented; correct exit codes. | +| **G8** | M8 | CI green; README/CODEOWNERS/index complete. | diff --git a/README.md b/README.md index ca9366729a3..7f0eb5eb80f 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,7 @@ This repository contains useful tools that the Azure SDK team utilizes across th | Package or Intent | Path | Description | Status | | ------------------------------ | ------------------------------------------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Check Enforcer [1] | [Readme](https://github.com/Azure/azure-sdk-actions/blob/main/docs/check-enforcer.md) | Manage GitHub check-runs in a mono-repo. | Enabled via GitHub actions | +| agentic-workflow | [Readme](tools/agentic-workflow/README.md) | Drives a research → plan → implement workflow for AI coding agents, one fresh Copilot SDK session per phase. | Not Yet Enabled | | doc-warden | [Readme](packages/python-packages/doc-warden/README.md) | A tool used to enforce readme standards across Azure SDK Repos. | [![Build Status](https://dev.azure.com/azure-sdk/public/_apis/build/status/108?branchName=main)](https://dev.azure.com/azure-sdk/public/_build/latest?definitionId=108&branchName=main) | | http-fault-injector | [Readme](tools/http-fault-injector/README.md) | HTTP proxy server for testing HTTP clients during "faults" like "connection closed in middle of body". | [![Build Status](https://dev.azure.com/azure-sdk/internal/_apis/build/status/tools/tools%20-%20http-fault-injector?branchName=main)](https://dev.azure.com/azure-sdk/internal/_build/latest?definitionId=2340&branchName=main) | | Maven Plugin for Snippets | [Readme](packages/java-packages/snippet-replacer-maven-plugin/README.md) | A Maven plugin that that updates code snippets referenced from javadoc comments. | Not Yet Enabled | diff --git a/agentic-workflow-design.md b/agentic-workflow-design.md index 8d32afca0fd..db88e8c2ca3 100644 --- a/agentic-workflow-design.md +++ b/agentic-workflow-design.md @@ -79,7 +79,7 @@ any phase's prompt independently. | 3 | classify + split *(skippable, §3.2)* | ✅ | specs + assumptions *(or task + codebase if research skipped)* | `classification.md`, `subitems.json` | | 4 | research-item (loop ×N, parallel) *(skippable, §3.2)* | ✅ each | one sub-item + specs | `research/.md` | | 5 | plan | ✅ | specs + assumptions + all item research *(whatever exists)* | `plan.md` (structured, with checkpoint gates — see §5) | -| 6 | implement | ✅ | `plan.md` | code edits + `execution-log.md` | +| 6 | implement *(staged, §5)* | ✅ per stage | `plan.md`, `execution-log.md`, `handoff.md` | code edits + `execution-log.md` + `handoff.md` (cross-stage memory, §6.3) | `specs/apispec.md` is **conditional**: phase 1 produces it only when the task touches an API surface (REST/RPC/SDK contract, schema, public interface). It captures the relevant existing @@ -233,7 +233,7 @@ tools/agentic-workflow/ ├── orchestrator.ts phase sequencing, checkpoint/resume, gate enforcement ├── phase.ts runPhase: createSession → render prompt → sendAndWait → validate ├── policy.ts per-phase tool/filesystem policy + post-phase git-diff guard (§6.1) - ├── gates.ts parse plan stages/gates; run gate commands; enforce allowed_files (§6.2) + ├── gates.ts parse plan stages/gates; run gate commands; check expected_files scope (§6.2) ├── prompts.ts load + render templates (var substitution) ├── artifacts.ts working-dir mgmt, atomic read/write, hashing, run lock, state file ├── tools.ts custom SDK tools: write_artifact, validate_subitems @@ -260,9 +260,15 @@ await mapWithConcurrency(items, config.concurrency, it => await phase(client, "plan", { reads: ["specs/*","assumptions.md","research/*"], writes: ["plan.md"], validate: planSchema, policy: READ_ONLY }); // Phase 6 is STAGED: each plan stage runs in its own fresh sub-session, gated by the orchestrator. +// Fresh ≠ blank: the orchestrator curates a context pack so no inter-stage knowledge is lost (§6.3). const stages = parsePlanStages("plan.md"); // structured stages + machine-readable gates (§6.2) for (const stage of stages) { - await phase(client, "implement-stage", { stage, reads: ["plan.md"], appends: ["execution-log.md"], updates: ["plan.md"], policy: implementPolicy(stage) }); + const pack = buildContextPack(ctx, stage); // plan.md + prior handoffs + cumulative diff + dep files (§6.3) + await phase(client, "implement-stage", { + stage, contextPack: pack, + reads: ["plan.md", "execution-log.md", "handoff.md"], // durable cross-stage memory + appends: ["execution-log.md", "handoff.md"], // each stage writes a handoff for the next + updates: ["plan.md"], policy: implementPolicy(stage) }); const dev = checkDeviation(stage); // edits outside expected_files must be documented (§5) if (dev.undocumented) { markPaused(state, stage, dev); break; } // missing docs → pause for review const result = runGate(stage.gate); // ORCHESTRATOR runs gate cmds, checks exit codes @@ -344,6 +350,7 @@ Examples of the intent per phase: stages: - id: stage-1 expected_files: ["src/foo.ts", "test/foo.test.ts"] # the plan's best guess at scope (advisory) + context_needed: ["src/bar.ts", "src/types.ts"] # existing files/symbols this stage depends on steps: - { id: "1.1", description: "..." } gate: @@ -353,8 +360,13 @@ Examples of the intent per phase: ``` Each stage names its `expected_files` (the plan's anticipated scope — **advisory, not a hard - wall**, see phase 6) and a `gate` with concrete validation `commands` + expected result. + wall**, see phase 6), the `context_needed` files/symbols it depends on (seeded into the stage's + context pack, §6.3), and a `gate` with concrete validation `commands` + expected result. Free-form prose gates alone are insufficient because they're unenforceable. + **Stage sizing for context safety:** the plan phase must split work into stages that are + **cohesive and loosely coupled** — each independently completable and verifiable by its gate — + to minimize what must cross a session boundary. Tightly-coupled changes that only make sense + together belong in **one** stage, not split across two. - **06-implement (staged).** Implementation is **not** one big session. Each plan *stage* runs in its **own fresh sub-session** (`implement-stage-1 → gate → implement-stage-2 → …`), preserving the clean-context guarantee inside the riskiest phase and bounding context growth. Per stage: @@ -365,6 +377,11 @@ Examples of the intent per phase: - The agent appends to `execution-log.md`. For **every action** the log must (a) **justify the scope** — why it was necessary — and (b) **map it to a concrete step** in `plan.md` (by stage/step id). + - **Stage handoff (cross-stage memory).** Before its gate, each stage appends a concise entry to + `handoff.md` for the *next* stage: what it built, the **new/changed public symbols and files**, + decisions or conventions established, anything intentionally deferred, and known follow-ups. The + **next** stage's prompt is seeded with this (plus the cumulative diff and `context_needed` + files, §6.3) so reasoning/intent — not just code — carries forward across the session boundary. - **Gate enforcement is the orchestrator's job, not the agent's:** after the sub-session ends, the orchestrator runs the stage's `gate.commands`, checks `expected`, records pass/fail + output in the log, and **halts at the first failing gate** rather than continuing. @@ -431,6 +448,30 @@ Gates are **machine-checked by the orchestrator**, never self-reported by the ag - On failure the run **halts at that gate** (autopilot stops too); the user can inspect, fix, and `resume`. This removes reliance on the agent to honestly stop, validate, and report. +### 6.3 Cross-stage context continuity + +Splitting phase 6 into one fresh sub-session per stage bounds context growth and drift, but risks +losing knowledge built up in earlier stages. We treat a fresh session as *clean*, **not blank** — +`buildContextPack(ctx, stage)` curates an explicit context pack for each stage so nothing material +is lost across the boundary: + +- **Code on disk** — the real source written by prior stages *is* the shared state; the new session + reads actual symbols/signatures rather than reconstructing them from memory. +- **`plan.md`** — the durable source of truth, including the running **Plan-changes** log. +- **`handoff.md`** — the prior stages' concise handoff entries (what was built, new/changed public + symbols, decisions/conventions, deferred items) — see §5. +- **Cumulative diff** — a summary of files changed so far in the run, so the stage sees what already + moved. +- **`context_needed`** — the existing files/symbols the plan declared this stage depends on (§5), + pre-opened so the agent doesn't have to rediscover them. + +This is a deliberate trade: a **curated, bounded handoff** over an ever-growing single transcript. +It usually beats one long session, which accretes noise, drifts, and risks context-window limits in +exactly the riskiest phase. Two design rules reinforce it: the plan phase **sizes stages to be +cohesive and loosely coupled** (§5) so little must cross a boundary, and tightly-coupled work stays +in a single stage. If a context pack would have to be enormous for a stage to succeed, that's a +signal the plan split the work at the wrong seam. + ## 7. Configuration `config.ts` maps each phase to a model + options, e.g.: @@ -711,7 +752,8 @@ agentic-workflow run "Add a null guard in src/parse.ts:42" --simple --type bug - + validator tool; parallel phase-4 sessions ordered by `dependsOn`, with a concurrency cap. 6. **M5 — Plan + staged implement + gates (1.5 days):** wire phase 5 (incl. machine-readable `stages:`/`gate:` block + `planSchema`) and the **staged** phase 6 with orchestrator-enforced - gates and `allowed_files`; execution-log + deviation handling; end-to-end run on a sample. + gates, `expected_files` scope checks, and the cross-stage context pack (§6.3); execution-log + + handoff + deviation handling; end-to-end run on a sample. 7. **M6 — Hardening (1 day):** retries/backoff, resume revalidation + locking, dry-run, redaction. 8. **M7 — CLI surface (½ day):** wire commands (`run`/`resume`/`status`/`list`/`approve`/`abort`/ `clean`), the gate-set resolution (presets + `--pause-after`/`--pause-before`/`--auto`), @@ -739,7 +781,7 @@ agentic-workflow run "Add a null guard in src/parse.ts:42" --simple --type bug - - **Context size for phase 5:** plan phase reads all specs + all item research — may need summarization if large. - **Implementation safety:** phase 6 edits real code. Run on a dedicated branch; gates + - `allowed_files` + deviation policy bound scope; human review of `plan.md` recommended. + `expected_files` + deviation policy bound scope; human review of `plan.md` recommended. - **Artifact location:** scratch by default — runs live under a locally-ignored `.agentic-workflow/` root (`.git/info/exclude` + inner `.gitignore`; root-`.gitignore` only via opt-in flag); see §3.1. - **Determinism:** prompt templates are versioned; the template version is stamped into each @@ -753,8 +795,9 @@ agentic-workflow run "Add a null guard in src/parse.ts:42" --simple --type bug - - **Guarantees are orchestrator-enforced, not prompt-only:** per-phase tool/FS policy + git-diff guard (§6.1), machine-readable checkpoint gates the orchestrator runs (§6.2), and resume revalidation + hashing + run lock (§9.4). -- **Phase 6 is staged:** one fresh sub-session per plan stage, each gated and confined to - `allowed_files`. +- **Phase 6 is staged:** one fresh sub-session per plan stage, each gated. `expected_files` is + advisory (documented deviations allowed, §5); cross-stage knowledge is preserved via a curated + context pack — `plan.md` + `handoff.md` + cumulative diff + `context_needed` (§6.3). - Location: new tool at `tools/agentic-workflow/` (this design doc lives at repo root for now). - End state: SDK engine + optional thin skill as the single entrypoint. - CLI: `run` + `resume` core commands. Execution is controlled by a **per-phase gate set** — diff --git a/gao.md b/gao.md new file mode 100644 index 00000000000..697b88d9969 --- /dev/null +++ b/gao.md @@ -0,0 +1,42 @@ +# Plan Critique + +The plan is strong on milestone gating and explicit success proof, but several guarantees are still under-specified enough that implementation could drift or become unsafe. + +## High-priority gaps + +1. **Generated gate commands are the biggest unresolved safety risk.** + The plan notes this in open questions, but it should be resolved before M5, not "before M5.6". A model-generated `plan.md` can currently cause the orchestrator to run arbitrary shell. Define an allowlist, cwd, env, timeout, output limits, interactive-command blocking, and approval semantics before implementing `parsePlanStages`. + +2. **Audit/log ownership is too trusting.** + Phase 6 agents append `execution-log.md`, `handoff.md`, and plan-change entries themselves. That lets the same actor that made an edit also self-report why it was safe. Prefer an orchestrator-owned structured log, e.g. JSONL, where the agent can propose entries but the orchestrator records actual diffs, gates, exit codes, and file changes. + +3. **`manifest.json` ownership is ambiguous.** + M1 makes `manifest.json` part of orchestrator state, but M3 says prompt `01` records decisions in `manifest.json`. Agents should not directly write core state. Better: agents emit decision artifacts; orchestrator validates and updates `manifest.json`. + +4. **The git-diff guard needs a precise definition.** + "`git diff` and untracked files empty" is not enough. `git diff` misses untracked files; ignored files may be hidden; `.agentic-workflow/` is intentionally ignored. Define the exact command/algorithm, exclude only the artifact root, include untracked source files, and compare against a pre-phase snapshot. + +5. **Redaction arrives too late.** + M2 streams events into `execution-log.md`, but redaction is deferred to M6. If SDK output, tool args, shell output, or env leaks appear in logs before M6, the damage is already done. Build log redaction into M2 logging from the start, then harden in M6. + +6. **The read-only checkout / overlay fallback is design-critical but underspecified.** + If native tool scoping is unavailable, the fallback must define worktree creation, artifact handoff, patch transfer, cleanup, symlink behavior, nested repos, ignored files, and how implementation diffs are applied back. This should become a concrete M0 output, not just a yes/no branch. + +## Ambiguities to tighten + +- **Exit codes conflict:** failing gates are described as `10`/`1` in multiple places, while M7 says `10 = paused`, `1 = failure`. Decide whether a failed gate is pause-for-human or hard failure. +- **Plan machine block format:** `stages:` / `gate:` needs exact delimiters and syntax, likely a fenced YAML or JSON block with schema versioning. +- **Approval model:** MVP says mandatory approval before implement, M7 adds `approve`, `resume`, `--yes`, and gate-set overrides. Define where approval is stored and what exactly resumes after approval. +- **Fresh context proof:** planting a fact in session A and asking session B is useful but weak. Use a nonce that is never written to artifacts/repo and define what isolation means relative to SDK memory, external tools, and local files. +- **Resume invalidation:** if an upstream artifact changes, the plan says downstream reads the edited content, but it does not define which completed phases become stale or must re-run. +- **Phase-4 dependencies:** `dependsOn` controls ordering, but it is unclear whether dependent item research receives prior item research as input. +- **Budget ceiling:** cost/quota is listed as open, but fan-out starts in M4. A hard budget should exist before parallel execution. + +## Opportunities + +- Add **schema versions** to every machine artifact: `state.json`, `manifest.json`, `subitems.json`, plan block, handoff, and execution log. +- Use a **typed schema source of truth** such as Zod/TypeBox plus generated JSON Schema to avoid TS types and JSON schemas drifting. +- Make **prompt-template regression tests** part of M3/M8, not future work. Even lightweight golden-structure tests would protect the core IP. +- Add a **repo/branch-level lock or dirty-worktree preflight** before implementation. Per-run locks do not prevent two runs from editing the same branch. +- Separate **mocked deterministic tests** from **real SDK smoke tests**. Gates should not depend on live quota unless explicitly marked opt-in. +- Consider making the MVP "review mode" the default. Fully unattended autopilot should require explicit opt-in once gate-command safety, budgets, and redaction are proven. diff --git a/thinnerplan.plan.md b/thinnerplan.plan.md new file mode 100644 index 00000000000..2f881f3e9e2 --- /dev/null +++ b/thinnerplan.plan.md @@ -0,0 +1,245 @@ +# Thin-Spine Plan — Agentic `research → plan → implement` + +> A deliberately minimal re-cut of `.plan.md`, driven by one constraint: **agent harnesses +> improve rapidly, so write as little bespoke machinery as possible.** Keep the durable value +> (the methodology) in portable plain-text assets, enforce only the guarantees the harness +> can't yet give for free, and lean on the Copilot extension/SDK surface for everything else. +> When the harness grows a capability, we **delete** code — we don't rewrite it. + +--- + +## 0. The one idea + +Split the system into two layers and treat them completely differently: + +| Layer | What it is | Strategy | +| --- | --- | --- | +| **Durable core (the product)** | The 6 phase templates + 2 judge templates + the artifact/phase contract + the machine-readable gate block | Plain markdown + plain data. **Zero framework coupling.** Survives any harness. | +| **Thin spine (the liability)** | Orchestration: spawn fresh session per phase, run gates, capture log, enforce policy | The smallest possible TypeScript, with **all harness calls behind one adapter file**. Lean on the extension/SDK surface; defer everything speculative. | + +If a better harness (or a native "workflow/pipeline" primitive) ships tomorrow, we re-point a +~200-line driver at it and keep ~90% of the value. + +--- + +## 1. What we keep, delegate, and defer + +**Keep (durable, or the only guarantees worth bespoke code):** +- The **6 phase templates** + **2 judge templates** (`prompts/01-research … 06-implement.md`, + `critique.md`, `revise.md`) — the actual IP. +- The **artifact contract** (each phase reads/writes files under `.agentic-workflow//`). +- **Fresh session per phase** — the one thing a skill/single-session can't do (design §2). +- The plan's **machine-readable gate block** (`stages[]` + `gate{commands,expected}`) — the + agent reads it and runs/validates its own stage gates inside the implement session. + +**Delegate to the Copilot extension / SDK surface (don't hand-roll):** +- **Per-phase read-only enforcement** → `onPreToolUse` returning `permissionDecision: "deny"` + (replaces the entire `FALLBACK.md` read-only-checkout machinery and demotes the git-diff + guard to a cheap backstop). +- **Sanctioned artifact writes** → a `write_artifact` custom tool (`skipPermission: true`). +- **Execution log** → `session.on("tool.execution_complete" | "assistant.message" | …)`. +- **Autonomous permissions** → `approveAll`. +- **Transient error retries** → `onErrorOccurred` → `{ errorHandling: "retry" }`. +- **Gate running + validation** → the **agent** runs the stage's `gate.commands` (shell tool) + inside its own implement session and reports pass/fail in `execution-log` + `handoff.md`. We + prefer this low-maintenance path over a code-enforced gate runner; the orchestrator just reads + the agent's reported result to decide whether to continue. +- **Interactive trigger** → a thin `.github/extensions/` front-door (slash command + tool). + +**Defer — add ONLY if the harness still doesn't do it when we actually need it:** +- Content hashing + resume **revalidation** engine (start with "re-run from last completed phase"). +- Run-lock / dirty-worktree coordination (add only when real concurrency appears). +- Redaction **depth** (ship a trivial regex; harness may own this soon). +- The full 7-command CLI + gate-set **resolution engine** (ship `run`/`resume` + `--simple` only). +- Token-budget ceiling, multi-repo scoping, autopilot sandbox, retention/telemetry. +- `FALLBACK.md` read-only-checkout (expected to be obsoleted by `onPreToolUse`). + +--- + +## 2. Architecture (thin spine) + +``` +portable assets/ ← durable IP, harness-agnostic + prompts/01..06.md + critique.md revise.md + validate.ts (a few hand-written checks: subitems + plan block) + +src/ + orchestrator.ts ← sequences phases, reads/writes artifacts, reads agent gate results + harness.ts ← *** the ONLY file that imports @github/copilot-sdk *** + wraps createSession / send / events / hooks / tools + session-options.ts ← shared hooks+tools (onPreToolUse policy, write_artifact, + event→log capture); passed into createSession AND reusable + by the extension front-door + artifacts.ts ← resolve run dir, atomic write, git-ignore (no hashing/lock yet) + gates.ts ← parse plan stages from plan.md (the agent runs the commands) + cli.ts ← `run [--simple] [--no-judge] [--judge-model] | resume [run-id]` (headless, PRIMARY entry) + +.github/extensions/agentic-workflow/ + extension.mjs ← OPTIONAL interactive front-door: registers a slash command + + `run_agentic_workflow` tool that calls the orchestrator +``` + +**Adapter rule (the whole hedge):** harness churn touches **`harness.ts` only**. The +orchestrator talks to phases through a stable internal interface (`runPhase(opts) → +{ artifacts, log }`), never to the SDK directly. + +**Two entry points, one engine:** +- `cli.ts` is the real product — headless, fits the "one command" goal (design §1), TypeScript. +- `extension.mjs` is a thin convenience shim so a user *inside* Copilot CLI can trigger the + same headless run conversationally (and get hot-reload during dev). It contains **no + orchestration logic** — it imports and calls the built orchestrator. + +--- + +## 3. The one enforced guarantee (the only bespoke rigor) + +1. **Fresh context per phase.** Every phase (and every implement-stage / fan-out item) is a + brand-new `createSession()`; retries spawn a *new* session seeded with the prior errors, + never a continued transcript. + +That's it. **Gates are run and validated by the agent itself** (it executes `gate.commands` in +its session and reports the result) — we deliberately prefer this low-maintenance path over a +code-enforced gate runner, accepting that a misreporting agent could pass a failing gate. The +orchestrator simply reads the reported result. Everything else is best-effort and leans on the +harness, in exchange for near-zero maintenance. + +### 3.1 Reflexive critique (LLM-as-judge per artifact) + +After **any** phase's artifact validates, the orchestrator runs a 2-session judge loop — this is +just the **same fresh-session + template primitive applied reflexively**, so it adds capability +without a new infra type. On by default; `--no-judge` disables it. + +1. **Critique (fresh session, *alternate* model).** `createSession({ model: judgeModel })`, + read-only (writes only `critiques/.md` via `write_artifact`). It reads the artifact + and emits **high-signal feedback only** — bugs, gaps, logic/design flaws, contract violations — + **no style nits**. Using a *different* model than the author is the point: it doesn't share the + author's blind spots. +2. **Adjudicate/revise (fresh session, author's model).** Reads the **original artifact + the + critique**, **decides which points to apply** (it's the owner, not a rubber stamp), rewrites the + artifact via `write_artifact`, and logs accepted/rejected points to the execution log. The + revised artifact is then **re-validated** — a bad revision triggers the normal fresh-session + retry. + +**Built-in rubber-duck:** Copilot ships a `rubber-duck` agent tuned for exactly this — high-signal +feedback, explicitly *no* style/formatting comments. **If the SDK can spawn a named built-in agent** +(`createSession({ agent: "rubber-duck", model: judgeModel })` — the "custom agents via SDK" surface, +**verified in T0.5**), use it as the critique session; otherwise the `critique.md` template encodes +the same ethos. Either way, pin the **alternate** model for diversity where the API allows. + +**Cost:** this ~triples sessions per artifact, so it's flag-gated (`--no-judge`) and the judge model +is configurable (`--judge-model`). This is the one place we *add* sessions for quality — justified +because catching a bad spec/plan early is far cheaper than implementing on top of it. + +--- + +## 4. Milestones (4, not 8) + +### T0 — Capability spike *(gates everything; keep it to a day)* +Verify, in a throwaway `spike/`: +- **T0.1** `createSession()` runs headless, streams, stops cleanly; `approveAll` works. +- **T0.2** **Isolation** via single-use nonce (session A reveals it, fresh session B can't). +- **T0.3 (decisive)** **Does `createSession()` accept `hooks` (esp. `onPreToolUse`)?** + - **yes →** hook-based `deny` is our §6.1 enforcement; `FALLBACK.md` is dropped entirely. + - **no →** hooks live only in `joinSession`; fall back to the **post-phase git-diff guard** + as primary enforcement (still cheap, no checkout machinery). +- **T0.4** Concurrency: ≥3 simultaneous sessions for phase-4 fan-out. +- **T0.5** **Can `createSession()` spawn a named built-in agent** (e.g. `agent: "rubber-duck"`) and + **override the model per session**? Decides whether the §3.1 critique uses the built-in + rubber-duck or the `critique.md` template; the alternate-model override is required either way. +- **⛔ G0:** isolation + permissions confirmed, and an enforcement path chosen (hooks *or* + git-diff guard). One short `spike/FINDINGS.md`. No `FALLBACK.md` unless T0.3 = no *and* a + diff guard proves insufficient. + +### T1 — Portable core + minimal substrate *(after G0)* +- **T1.1** The **6 phase templates** + **2 judge templates** (`critique.md`, `revise.md`) in + `prompts/` (the IP; iterate continuously after this). +- **T1.2** A handful of **plain validation functions** (`validate.ts`) for `subitems.json` + + the plan block — `JSON.parse`/`YAML.parse` then check the few required fields and return + readable errors. **No schema library.** Nothing else gets validation until a real format breaks. +- **T1.3** `artifacts.ts`: resolve `.agentic-workflow//`, atomic write, git-ignore via + `.git/info/exclude` + inner `.gitignore`. **No hashing, no run-lock yet.** +- **⛔ G1:** templates render with run vars; validators accept/reject fixtures; artifacts dir is + created and git-ignored without touching tracked `.gitignore`. + +### T2 — Thin spine *(after G1 — the core)* +- **T2.1** `harness.ts`: wrap `createSession` with `session-options.ts` (the shared + `onPreToolUse` policy + `write_artifact` tool + event→`execution-log.jsonl` capture + + `onErrorOccurred` retry). Stable `runPhase()` interface out. +- **T2.2** `orchestrator.ts`: sequence phases 1→6 reading/writing artifacts; fan-out phase 4 + with bounded concurrency; **fresh-session retry** on validation/policy failure. Under + `--simple`, skip phases 1/3/4 and synthesize a single-item `subitems.json` so phases 5–6 + see an identical contract. +- **T2.3** `gates.ts`: parse plan stages from `plan.md`. The **agent runs `gate.commands`** in + its implement session and reports pass/fail in `execution-log`/`handoff.md`; the orchestrator + reads that and stops on a reported failure (no bespoke gate runner). +- **T2.4** Staged implement: one fresh sub-session per stage, `handoff.md` carries forward. +- **T2.5** Judge loop (§3.1): a `judgeArtifact(path, authorModel)` helper — spawn critique (alt + model, or built-in `rubber-duck` per T0.5) → spawn adjudicate/revise (author model) → + re-validate. Wrap every validated artifact; honor `--no-judge` / `--judge-model`. +- **⛔ G2 (key):** a real sample task runs end-to-end on a branch; every edit maps to a plan + step; each validated artifact is critiqued by an alt-model session and revised; the agent runs + its stage gates and a failing gate is reported and halts the run; non-impl phases mutate + nothing outside `.agentic-workflow/` (proven by hook-deny *or* the diff guard). + +### T3 — Entry points + package *(after G2)* +- **T3.1** `cli.ts`, two commands: + - **`run [task] [--simple] [--no-judge] [--judge-model ]`** (default) — drives the headless + pipeline. Prints the **human-readable run-id** it creates (e.g. `20260629-1310-add-auth` = + timestamp + task slug). `--simple` skips the optional phases — research (1), classify (3), and + per-item research (4) — synthesizing a single-item path and running assumptions → plan → + implement. `--no-judge` disables the §3.1 critique loop; `--judge-model` sets the alternate + critique model. + - **`resume [run-id]`** — re-runs from the last completed phase (no hash engine). `run-id` + is **optional**: with one incomplete run, resume it; with several, **refuse and list the + candidates** (id + task + last completed phase) so the user picks; an explicit id resumes + that run. + - Exit codes: `0` done / `10` paused / `1` fail / `2` usage. +- **T3.2 (leverage the extension)** `.github/extensions/agentic-workflow/extension.mjs`: a + slash command + `run_agentic_workflow` tool that invokes the orchestrator from inside an + interactive Copilot CLI session (hot-reloadable). Thin shim, no logic. +- **T3.3** `README.md`, `ci.yml` (`tools - agentic-workflow - ci`), CODEOWNERS row + (`/tools/agentic-workflow/ @JennyPng`), root README index, archive `spike/`. +- **⛔ G3:** `npm run build && npm test` green; `run` completes a sample headlessly; the + extension front-door triggers the same run interactively. **Done.** + +--- + +## 5. How the extension surface is leveraged (summary) + +| Need | Bespoke code in `.plan.md` | Thin-spine: lean on harness | +| --- | --- | --- | +| Read-only non-impl phases | `FALLBACK.md` worktrees + patch-apply + git-diff guard | `onPreToolUse` → `deny` (guard = backstop only) | +| Sanctioned writes | custom `write_artifact` tool | same, `skipPermission: true` | +| Execution log | orchestrator event plumbing | `session.on(...)` events | +| Transient retries | bespoke backoff module (M6.1) | `onErrorOccurred` → `retry` | +| Gate running/validation | orchestrator runs commands + reads exit codes | agent runs them in-session, reports result | +| Artifact critique (judge) | n/a | built-in `rubber-duck` agent if SDK-spawnable, else `critique.md` on an alt model | +| Permissions | resolve exact option (M0.4) | `approveAll` | +| Interactive trigger | n/a | `.github/extensions/` slash command + tool | + +Caveat: hooks on `createSession` are **verified in T0.3**. If unavailable there, the +`onPreToolUse` items degrade to the git-diff guard, and the extension front-door still stands +as the interactive trigger. + +--- + +## 6. Validation (lean) + +- **Unit:** validators accept/reject fixtures; `gates.ts` parses stages from `plan.md`; `artifacts` + atomic write + git-ignore (and non-git no-op); `onPreToolUse` policy denies a write in a + non-impl phase (or diff-guard catches it). +- **Scenario (mocked):** fresh-session retry (fail-once-then-succeed); agent-reported failing + gate halts the run; `resume` continues from last completed phase; judge loop runs + critique→revise on alt model and `--no-judge` skips it. +- **E2E (opt-in, real SDK, throwaway branch):** one small task to completion; log maps every + edit to a plan step; gates pass; exit `0`. Not in CI unless a cheap-model lane exists. +- **CI:** `npm run build` + mocked tests + format check only. + +--- + +## 7. Success proof + +A run is "done" when all phases complete, every gate passed, `execution-log.jsonl` maps each +edit to a `plan.md` step, and exit `0` — **and** the spine stays under a few hundred lines with +the entire SDK footprint isolated in `harness.ts`. The second clause is the point of this cut: +the day the harness ships a native multi-phase primitive, the spine collapses into config. diff --git a/tools/agentic-workflow/.gitignore b/tools/agentic-workflow/.gitignore new file mode 100644 index 00000000000..e684348b168 --- /dev/null +++ b/tools/agentic-workflow/.gitignore @@ -0,0 +1,6 @@ +node_modules/ +dist/ +*.tgz +.agentic-workflow/ +spike/node_modules/ +spike/package-lock.json diff --git a/tools/agentic-workflow/.prettierignore b/tools/agentic-workflow/.prettierignore new file mode 100644 index 00000000000..ddea4133444 --- /dev/null +++ b/tools/agentic-workflow/.prettierignore @@ -0,0 +1,6 @@ +node_modules/ +dist/ +prompts/ +spike/ +*.md +.agentic-workflow/ diff --git a/tools/agentic-workflow/.prettierrc.json b/tools/agentic-workflow/.prettierrc.json new file mode 100644 index 00000000000..092c0c878cc --- /dev/null +++ b/tools/agentic-workflow/.prettierrc.json @@ -0,0 +1,6 @@ +{ + "tabWidth": 4, + "printWidth": 120, + "trailingComma": "all", + "semi": true +} diff --git a/tools/agentic-workflow/EXECUTION-LOG.md b/tools/agentic-workflow/EXECUTION-LOG.md new file mode 100644 index 00000000000..58702d50bc9 --- /dev/null +++ b/tools/agentic-workflow/EXECUTION-LOG.md @@ -0,0 +1,184 @@ +# Agentic Workflow — Implementation Execution Log + +This log records the implementation of `thinnerplan.plan.md` (the thin-spine re-cut of +`.plan.md`). Each entry: **action**, **justification**, **gate validation**, and any +**deviation** from the plan (with reason, for audit). + +Format note: this is the *build* log of the tool itself — distinct from the per-run +`execution-log.jsonl` the tool produces when it runs a workflow. + +--- + +## T0 — Capability spike (gate G0) + +**Action.** Created `spike/`, installed `@github/copilot-sdk@1.0.4`, inspected `dist/*.d.ts` +(authoritative API surface), and ran two live spike scripts (`spike.mjs`, `tool-test.mjs`) with +logged-in-user auth against `claude-haiku-4.5`. + +**Justification.** The plan makes G0 gate everything: isolation + permissions must be confirmed +and an enforcement path chosen before any build-out. Verifying against live SDK + type defs (not +assumptions) follows the repo rule "base conclusions on authoritative sources." + +**Gate validation — G0 = PASS** (see `spike/FINDINGS.md`): +- T0.1 createSession headless + `approveAll` + clean stop — ✅ live. +- T0.2 isolation via single-use nonce — ✅ fresh session replied `NONE`. +- T0.3 `createSession` accepts `hooks.onPreToolUse` and `deny` blocks a tool — ✅ **YES** (live). +- T0.4 ≥3 concurrent sessions — ✅ live. +- T0.5 per-session model override ✅; named built-in-agent spawn ⚠️ not available + (`SessionConfig.agent` references only `customAgents[]`). +- Custom `write_artifact` tool via `defineTool(name,{…,skipPermission:true})` — ✅ live. + +**Deviations from plan.** +- **T0.3 = YES → `FALLBACK.md` dropped entirely** (as the plan's "yes" branch prescribes): + hook-based `deny` is the §6.1 enforcement; git-diff guard demoted to backstop. +- **T0.5 built-in `rubber-duck` spawn unavailable** → judge critique uses the `critique.md` + template on an **alternate model** (the plan's explicit fallback). Required capability + (per-session model override) is confirmed, so judge diversity is preserved. + +--- + +## T1 — Portable core + minimal substrate (gate G1) + +**Action.** Scaffolded the tool package (`package.json`, `tsconfig.json`, ESM, vitest — matching +repo TS conventions: 4-space indent, `"type":"module"`, tsc build). Authored the **6 phase +templates** + **2 judge templates** (`prompts/01-research…06-implement.md`, `critique.md`, +`revise.md`). Wrote the minimal substrate: +- `src/types.ts` — shared contract types (PhaseId, SubItem, Stage, RunState, …). +- `src/artifacts.ts` — run-dir resolution, atomic write, local git-ignore (`.git/info/exclude` + + inner `.gitignore`, never the tracked root `.gitignore`), traversal-guarded `resolveInRunDir`. +- `src/validate.ts` — plain `validateSubitems` + `validatePlan` (no schema lib). +- `src/gates.ts` — `parsePlanStages` extracts/normalizes the ```yaml stages:``` block. +- `src/prompts.ts` — trivial `{{var}}` renderer. + +**Justification.** This is the durable IP + the only substrate that must exist before the spine. +Kept dependency-free (only `yaml`) per the thin-spine "write as little machinery as possible". + +**Gate validation — G1 = PASS:** +- `npx tsc --noEmit` → exit 0 (clean typecheck). +- `npx vitest run` → **22/22 passing** across 4 files: + - templates render with run vars, no unresolved `{{…}}` left, read-only contract preserved; + - validators accept good fixtures and reject empty items / bad enum / non-kebab id / duplicate + id / dangling `dependsOn` / invalid JSON / missing plan sections / missing gate block; + - `parsePlanStages` parses multi-stage blocks with defaults and throws on missing commands; + - artifacts dir is created and ignored via `.git/info/exclude` (idempotent) **without** creating + a tracked `.gitignore`; atomic write/append and traversal rejection verified. + +**Deviations from plan.** None. (Chose `yaml` as the one runtime dep to parse the gate block; +the plan's validators explicitly allow `YAML.parse`.) + +--- + +## T2 — Thin spine (gate G2) + +**Action.** Built the spine, all SDK access isolated in `harness.ts`: +- `src/session-options.ts` — `write_artifact` tool (`skipPermission`), `onPreToolUse` read-only + policy (deny mutating/shell tools), `onErrorOccurred` retry, and redacted JSONL event logging. +- `src/harness.ts` — **the only `@github/copilot-sdk` importer**; wraps `createSession` and + exposes the stable `Harness.runPhase()` seam. +- `src/orchestrator.ts` — sequences phases 1->6, validates each, **fresh-session retry**, phase-4 + fan-out (dependsOn-ordered, bounded concurrency), `--simple` single-item synthesis, the §3.1 + judge loop (`judgeArtifact`: critique alt-model -> revise author-model -> re-validate w/ rollback), + staged implement reading the agent-reported `STAGE_RESULT` to halt on a failing gate. +- `src/state.ts` — atomic `state.json` per-phase status for `resume`. + +**Justification.** Adapter rule: harness churn touches `harness.ts` only; the orchestrator talks +to phases through `runPhase()`, never the SDK. Gates are agent-run/agent-reported per the user's +stated low-maintenance preference. + +**Gate validation — G2 = PASS:** +- Mocked scenario tests (CI-safe): **36/36** across 6 files — fresh-session retry + (fail-once-then-succeed; `plan` ran twice), agent-reported failing gate halts (exit 1), + blocking-assumption pause (exit 10), judge loop runs critique+revise and `--no-judge` skips it, + `--simple` synthesizes a single-item path, full non-simple pipeline with 2-item fan-out, and the + `onPreToolUse` policy denies mutating tools in read-only phases. +- **Live end-to-end** (real SDK, throwaway git repo, `--simple`, judge ON), run via the built CLI: + - exit **0**; real code edit (`src/math.js` gained `export function add`), new `test/` file, + `npm test` green. + - `plan.md` emitted a valid machine-readable `stages:`/`gate:` block. + - **read-only enforcement proven**: 6 `policy_deny` events; no source mutated outside the + implement phase (git diff = only `src/math.js` + untracked `test/`). + - **judge loop ran live**: `critiques/assumptions.md` + `critiques/plan.md`, 2 `judge_complete`. + - **gate run by the agent**: `execution-log.md` records `npm test` exit 0 = PASS; orchestrator + read `STAGE_RESULT: pass`. + - `execution-log.md` maps every edit to a `plan.md` stage/step with scope justification. + +**Deviations from plan.** None of substance. Per gao.md feedback (#2/#5) the structured log is +orchestrator-owned `execution-log.jsonl` with trivial secret redaction built in from the start +(the plan deferred redaction depth but allows a trivial regex now). + +--- + +## T3 — Entry points + packaging (G3) — 2025 + +**Actions taken.** +- `src/cli.ts` — headless front-door. `run ` (flags `--simple`, `--no-judge`, + `--judge-model`, `--run-id`, `--out-root`) and `resume [runId]` (resumes the newest incomplete + run when no id is given). Builds `RunOptions`, constructs `SdkHarness`, calls `runWorkflow`, and + maps `RunResult.exitCode` straight to `process.exit` (0 success, 10 blocking-assumption pause, + 1/2 failure/usage). `src/index.ts` re-exports `runWorkflow`, `SdkHarness`, and the public types + as the tool's programmatic API. +- `.github/extensions/agentic-workflow/extension.mjs` — optional interactive front-door. Thin + shim: registers a `/agentic-workflow` slash command + a `run_agentic_workflow` tool via + `joinSession`, parses the same flags as the CLI, and **dynamically imports the built + `dist/index.js`** to delegate. Holds zero orchestration logic — re-point `TOOL_DIST` and it + still works. +- Packaging: `README.md` (purpose, install/build/test, CLI + extension usage, architecture, where + used), `ci.yml` (eng/pipelines templates), `.gitignore` (node_modules/dist/.agentic-workflow), + `.prettierrc.json` (`tabWidth:4`, `printWidth:120` to match the repo `.editorconfig`) and + `.prettierignore` (keeps prompt templates + spike + markdown pristine). +- Repo wiring: added `/tools/agentic-workflow/ @JennyPng` to `.github/CODEOWNERS` and a row to the + root `README.md` tool index (status **Not Yet Enabled**). +- Archived the spike: removed `spike/tool-test.mjs` scratch driver; kept `spike.mjs` + `FINDINGS.md` + as the G0 evidence (its `node_modules`/lockfile stay gitignored). + +**Justification.** Two entry points, one engine: both `cli.ts` and `extension.mjs` are dumb shims +over `runWorkflow`/`SdkHarness`, so the durable IP (prompts + contract + orchestrator) has a single +home and the SDK stays isolated in `harness.ts`. The extension delegates by importing the built +artifact rather than re-implementing anything, satisfying the "thin spine" goal. + +**Gate validation — G3 = PASS:** +- `npm run build` (tsc) — **ok**, emits `dist/`. +- `npm run format:check` (prettier) — **"All matched files use Prettier code style!"** across the + source/test/json set (after adding `.prettierrc.json` so prettier honors the repo's 4-space style). +- `npm test` (vitest) — **36/36** in 6 files, unchanged after reformat. +- CLI exit-code contract verified: `run` with no task -> **2**, `resume` with no incomplete run -> + **2**, unknown command -> **2**, usage prints. +- Extension verified-by-construction: `node --check extension.mjs` passes; `TOOL_DIST` resolves to + the real `tools/agentic-workflow/dist/index.js`; the imported names (`runWorkflow`, `SdkHarness`) + and call shapes (`new SdkHarness({workingDirectory})`, `RunResult.{message,exitCode,runDir}`, + `harness.stop()`) match the `index.ts` exports exactly. + +**Deviations from plan.** +- **Added `.prettierrc.json` + `.prettierignore`** (not itemized in the plan). Reason: the plan's G3 + requires `format:check` to pass, but prettier's 2-space default contradicts the repo `.editorconfig` + (4-space TS). The config pins prettier to the existing house style instead of mass-reformatting to + 2-space; the ignore file keeps the prompt templates byte-for-byte (their whitespace is part of the + contract) and excludes the spike. Net effect matches the plan's intent (clean `format:check`) + without altering durable IP. +- **Live interactive trigger of the extension** is verified by construction + syntax/import checks + rather than a live `/agentic-workflow` invocation, because that requires a hosted interactive + Copilot CLI session. The delegated engine itself was already proven live end-to-end in G2, so the + only unexercised surface is the thin `joinSession` registration, whose import path and call shapes + are statically confirmed above. + +--- + +## Design/implementation gap — noted for audit + +`agentic-workflow-design.md` carries edits that introduce **§6.3 "cross-stage context continuity"** +— a richer staged-implement model with a `handoff.md` cross-stage memory artifact, a +`buildContextPack(ctx, stage)` curation step, and a per-stage `context_needed` field in the plan +schema. **This is a forward-looking design refinement and is intentionally NOT implemented in the +thin spine.** + +What the implemented thin spine actually does for the staged implement phase: each plan stage runs +in its own fresh sub-session, the orchestrator validates artifacts on disk and reads the +agent-reported `STAGE_RESULT` to halt at the first failing gate. There is **no** `handoff.md`, +`buildContextPack`, or `context_needed` plumbing — cross-stage state is carried by the code on disk ++ `plan.md` + `execution-log.md` only. + +Decision (confirmed with the user): keep the design-doc edits as a **forward-looking note** rather +than reverting them or expanding the thinnerplan scope to build the handoff/context-pack machinery. +Anyone reconciling the doc against the code should treat §6.3 (and the `handoff.md` / `context_needed` +/ `buildContextPack` references in §3, §5, the M5 milestone, and the summary) as **planned, not yet +built**. diff --git a/tools/agentic-workflow/README.md b/tools/agentic-workflow/README.md new file mode 100644 index 00000000000..ba124cb7741 --- /dev/null +++ b/tools/agentic-workflow/README.md @@ -0,0 +1,127 @@ +# agentic-workflow + +A single command that drives a disciplined **research → plan → implement** workflow for AI coding +agents, running **one fresh, isolated [Copilot SDK](https://github.com/github/copilot-sdk) session +per phase** and handing off between phases through **durable artifacts on disk**. + +The hard guarantee — *one clean context per phase* — is what a single skill or one long session +can't give you. Each phase reasons without bias-bleed from the previous one, and every run is +inspectable and resumable because every phase reads/writes plain files. + +> This is a **thin-spine** implementation: the durable value (prompt templates + the artifact +> contract) is plain markdown/data with zero framework coupling, and **all** Copilot SDK calls are +> isolated in a single adapter (`src/harness.ts`). When the agent harness grows a native +> multi-phase primitive, the spine collapses into config instead of being rewritten. + +## How it works + +| # | Phase | Fresh session | Reads | Writes | +| --- | --- | --- | --- | --- | +| 1 | research *(skipped by `--simple`)* | ✅ | the codebase | `specs/*.md`, `manifest.json` | +| 2 | assumptions | ✅ | specs (or task+code) | `assumptions.md` | +| 3 | classify *(skipped by `--simple`)* | ✅ | specs + assumptions | `classification.md`, `subitems.json` | +| 4 | research-item ×N *(skipped by `--simple`)* | ✅ each | one sub-item + specs | `research/.md` | +| 5 | plan | ✅ | everything above | `plan.md` (+ machine-readable gate block) | +| 6 | implement *(staged)* | ✅ per stage | `plan.md`, `handoff.md` | code edits + `execution-log.md` + `handoff.md` | + +After **any** phase's artifact validates, an optional **reflexive judge loop** runs (on by default): +a *critique* session on an **alternate model** emits high-signal feedback only, then an +*adjudicate/revise* session on the author's model decides which points to apply and rewrites the +artifact, which is then re-validated. Disable with `--no-judge`. + +### Guarantees (and where they come from) + +- **Fresh context per phase / retry** — every phase, fan-out item, judge session, and retry is a + brand-new `createSession()`. This is the one piece of bespoke rigor. +- **Read-only non-impl phases** — enforced by an `onPreToolUse` hook that *denies* edit/shell tools + in phases 1–5 (verified live; see `spike/FINDINGS.md`). A post-phase git-diff check is a backstop. +- **Sanctioned writes** — phases persist artifacts only through a `write_artifact` custom tool + (path-traversal guarded). +- **Gates** — the plan embeds a machine-readable `stages:`/`gate:` block. The implement agent runs + each stage's `gate.commands` in-session and reports `STAGE_RESULT: pass|fail`; the orchestrator + halts at the first reported failure. (Low-maintenance by design — see the design doc.) +- **Structured log** — `execution-log.jsonl` is orchestrator-owned with trivial secret redaction. + +## Prerequisites + +- Node.js `^20.19.0` or `>=22.12.0`. +- Authenticated GitHub Copilot (the SDK uses your logged-in user by default). + +## Install & build + +```bash +cd tools/agentic-workflow +npm install +npm run build +npm link # exposes the `agentic-workflow` command on your PATH +``` + +`npm link` registers the package's `bin` so you can call `agentic-workflow` from anywhere. (Use +`npm unlink -g @azure-tools/agentic-workflow` to remove it.) + +## Usage + +```bash +# Headless (primary entry point) — once linked: +agentic-workflow run "" [--simple] [--no-judge] [--judge-model ] [--run-id ] [--out ] + +# Resume an interrupted run from its last completed phase +agentic-workflow resume [] +``` + +Without linking you can still invoke it directly with `node dist/cli.js run ""`, via +`npx agentic-workflow run ""` from this directory, or from source during development with +`npm start -- run "" --simple`. + +| Flag | Effect | +| --- | --- | +| `--simple` | Skip research, classify, and per-item research → `assumptions → plan → implement`. | +| `--no-judge` | Disable the reflexive critique/revise judge loop. | +| `--judge-model ` | Alternate model used for the critique session. | +| `--run-id ` | Explicit run id (default: `YYYYMMDD-HHMM-`). | +| `--out ` | Working-dir root (default: `./.agentic-workflow`). | + +**Exit codes:** `0` done · `10` paused (resume / blocking clarification) · `1` failure · `2` usage. + +### Interactive front-door (optional) + +`.github/extensions/agentic-workflow/extension.mjs` registers a `/agentic-workflow` slash command +and a `run_agentic_workflow` tool so you can trigger the same headless run from inside an +interactive Copilot CLI session. It is a thin shim that imports the built orchestrator — no logic. + +## Artifacts & git + +Each run writes to `.agentic-workflow//`. The scratch tree is ignored **without modifying +the tracked root `.gitignore`** — via `.git/info/exclude` plus an inner `.agentic-workflow/.gitignore`. +Code edits from phase 6 are *real* source changes meant to be reviewed/committed (run on a branch). + +## Layout + +``` +prompts/ the durable IP: 6 phase templates + critique.md + revise.md +src/ + harness.ts *** the only file that imports @github/copilot-sdk *** + session-options.ts write_artifact tool + onPreToolUse policy + JSONL logging + orchestrator.ts phase sequencing, fresh-session retry, fan-out, judge loop + gates.ts parse the plan's machine-readable stages block + validate.ts plain validators (subitems.json + plan) + artifacts.ts run-dir, atomic write, local git-ignore + state.ts state.json (resume) + cli.ts headless entry point +spike/ archived T0 capability spike + FINDINGS.md (SDK assumptions, verified live) +``` + +## Develop + +```bash +npm test # vitest (mocked harness — no SDK calls, CI-safe) +npm run format:check +npm run build +``` + +The end-to-end test against the real SDK is opt-in (it spends model requests); see the design doc. + +## Design + +See [`../../agentic-workflow-design.md`](../../agentic-workflow-design.md) (full design) and the +thin-spine plan it was cut down to. diff --git a/tools/agentic-workflow/ci.yml b/tools/agentic-workflow/ci.yml new file mode 100644 index 00000000000..4aeb754240d --- /dev/null +++ b/tools/agentic-workflow/ci.yml @@ -0,0 +1,57 @@ +trigger: + branches: + include: + - main + - feature/* + - release/* + - hotfix/* + paths: + include: + - tools/agentic-workflow + +pr: + branches: + include: + - main + - feature/* + - release/* + - hotfix/* + paths: + include: + - tools/agentic-workflow + +extends: + template: /eng/pipelines/templates/stages/1es-redirect.yml + parameters: + stages: + - stage: BuildAndTest + displayName: Build and Test + variables: + - template: /eng/pipelines/templates/variables/globals.yml + - template: /eng/pipelines/templates/variables/image.yml + - name: NodeVersion + value: "22.x" + jobs: + - job: BuildTest_Linux + displayName: Build and Test (Linux) + pool: + name: $(LINUXPOOL) + image: $(LINUXVMIMAGE) + os: linux + steps: + - task: NodeTool@0 + inputs: + versionSpec: $(NodeVersion) + displayName: Use Node $(NodeVersion) + - script: npm ci + workingDirectory: $(Build.SourcesDirectory)/tools/agentic-workflow + displayName: Install dependencies + - script: npm run build + workingDirectory: $(Build.SourcesDirectory)/tools/agentic-workflow + displayName: Build + - script: npm run format:check + workingDirectory: $(Build.SourcesDirectory)/tools/agentic-workflow + displayName: Format check + - script: npm test + workingDirectory: $(Build.SourcesDirectory)/tools/agentic-workflow + displayName: Test (mocked harness, no SDK calls) diff --git a/tools/agentic-workflow/package-lock.json b/tools/agentic-workflow/package-lock.json new file mode 100644 index 00000000000..ee3c3fb4125 --- /dev/null +++ b/tools/agentic-workflow/package-lock.json @@ -0,0 +1,2315 @@ +{ + "name": "@azure-tools/agentic-workflow", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "@azure-tools/agentic-workflow", + "version": "0.1.0", + "license": "MIT", + "dependencies": { + "@github/copilot-sdk": "^1.0.4", + "yaml": "^2.5.0" + }, + "bin": { + "agentic-workflow": "dist/cli.js" + }, + "devDependencies": { + "@types/node": "^20.19.0", + "prettier": "^3.3.3", + "rimraf": "^6.0.1", + "tsx": "^4.19.0", + "typescript": "^5.6.0", + "vitest": "^2.1.0" + }, + "engines": { + "node": ">=20.19.0" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.1.tgz", + "integrity": "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.1.tgz", + "integrity": "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.1.tgz", + "integrity": "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.1.tgz", + "integrity": "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.1.tgz", + "integrity": "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.1.tgz", + "integrity": "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.1.tgz", + "integrity": "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.1.tgz", + "integrity": "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.1.tgz", + "integrity": "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.1.tgz", + "integrity": "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.1.tgz", + "integrity": "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.1.tgz", + "integrity": "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.1.tgz", + "integrity": "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.1.tgz", + "integrity": "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.1.tgz", + "integrity": "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.1.tgz", + "integrity": "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.1.tgz", + "integrity": "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.1.tgz", + "integrity": "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.1.tgz", + "integrity": "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.1.tgz", + "integrity": "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.1.tgz", + "integrity": "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.1.tgz", + "integrity": "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.1.tgz", + "integrity": "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.1.tgz", + "integrity": "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.1.tgz", + "integrity": "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.1.tgz", + "integrity": "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@github/copilot": { + "version": "1.0.65", + "resolved": "https://registry.npmjs.org/@github/copilot/-/copilot-1.0.65.tgz", + "integrity": "sha512-J1XvLuOiVpiAi/E1MBICBymszCgdGLnZxokosXzGcmcjEVZd+QSDoW/kPRHq6oEyBT9SDASPcjCEZ9Q0rLJllg==", + "license": "SEE LICENSE IN LICENSE.md", + "dependencies": { + "detect-libc": "^2.1.2" + }, + "bin": { + "copilot": "npm-loader.js" + }, + "optionalDependencies": { + "@github/copilot-darwin-arm64": "1.0.65", + "@github/copilot-darwin-x64": "1.0.65", + "@github/copilot-linux-arm64": "1.0.65", + "@github/copilot-linux-x64": "1.0.65", + "@github/copilot-linuxmusl-arm64": "1.0.65", + "@github/copilot-linuxmusl-x64": "1.0.65", + "@github/copilot-win32-arm64": "1.0.65", + "@github/copilot-win32-x64": "1.0.65" + } + }, + "node_modules/@github/copilot-darwin-arm64": { + "version": "1.0.65", + "resolved": "https://registry.npmjs.org/@github/copilot-darwin-arm64/-/copilot-darwin-arm64-1.0.65.tgz", + "integrity": "sha512-NFc4xIstZNiIuAYkurQT5DVtbZjBoZ/z6yt/Ffcom7Y5QGjfpN4BFuekv9k+OADRioxxR99NgmhjbuNPWtQhNQ==", + "cpu": [ + "arm64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "darwin" + ], + "bin": { + "copilot-darwin-arm64": "copilot" + } + }, + "node_modules/@github/copilot-darwin-x64": { + "version": "1.0.65", + "resolved": "https://registry.npmjs.org/@github/copilot-darwin-x64/-/copilot-darwin-x64-1.0.65.tgz", + "integrity": "sha512-0wtV22KmTa12VbqWRRkgvJcBz/oIbszfcIpyDWGc4MzbCVksajQ3TWVQ6c7Sdzj5RifCaYdkHAX2zuIYXYlLoQ==", + "cpu": [ + "x64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "darwin" + ], + "bin": { + "copilot-darwin-x64": "copilot" + } + }, + "node_modules/@github/copilot-linux-arm64": { + "version": "1.0.65", + "resolved": "https://registry.npmjs.org/@github/copilot-linux-arm64/-/copilot-linux-arm64-1.0.65.tgz", + "integrity": "sha512-dOwdy/YbTXQN/+x2v4ZgiDycdRtWElyHxPuA6ail3yJDt0nagwn8OYAA/diBLPMAJuuBXiOZGvvb9fGRuh7Xgg==", + "cpu": [ + "arm64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "linux" + ], + "bin": { + "copilot-linux-arm64": "copilot" + } + }, + "node_modules/@github/copilot-linux-x64": { + "version": "1.0.65", + "resolved": "https://registry.npmjs.org/@github/copilot-linux-x64/-/copilot-linux-x64-1.0.65.tgz", + "integrity": "sha512-al/1a/l/GrpHtygTxt7PZspmv0eHBPdAZ5B31J7Hv/GRdVZM4STCC9dCIOSUFsOX2fhaKD8yLfz4HureSYs23g==", + "cpu": [ + "x64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "linux" + ], + "bin": { + "copilot-linux-x64": "copilot" + } + }, + "node_modules/@github/copilot-linuxmusl-arm64": { + "version": "1.0.65", + "resolved": "https://registry.npmjs.org/@github/copilot-linuxmusl-arm64/-/copilot-linuxmusl-arm64-1.0.65.tgz", + "integrity": "sha512-xccQeJSR45xyoaL7J5mZjtU++dmte+ZCDQkIlrpTn2yuMl2LWriBvorQ1P2MwVnXmIiW/GHi93B+lNtsybA9yw==", + "cpu": [ + "arm64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "linux" + ], + "bin": { + "copilot-linuxmusl-arm64": "copilot" + } + }, + "node_modules/@github/copilot-linuxmusl-x64": { + "version": "1.0.65", + "resolved": "https://registry.npmjs.org/@github/copilot-linuxmusl-x64/-/copilot-linuxmusl-x64-1.0.65.tgz", + "integrity": "sha512-RHPVUaqjSrhKHQ2EpfGKWErnV+R5elGIZaHXPKO10zpSaQD9b/C9u6nLigZnBuT/8sCaJpVrazPMwOYvYA62aw==", + "cpu": [ + "x64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "linux" + ], + "bin": { + "copilot-linuxmusl-x64": "copilot" + } + }, + "node_modules/@github/copilot-sdk": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/@github/copilot-sdk/-/copilot-sdk-1.0.4.tgz", + "integrity": "sha512-c6/gpatP7LAuP9stlc2uXXOrB+TJMFSs3Jrzu9FG8lJJYFGmdzbBvGz2UdL1jww84fp6D7cohEZa4UGq1+iuyA==", + "license": "MIT", + "dependencies": { + "@github/copilot": "^1.0.65", + "vscode-jsonrpc": "^8.2.1", + "zod": "^4.3.6" + }, + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@github/copilot-win32-arm64": { + "version": "1.0.65", + "resolved": "https://registry.npmjs.org/@github/copilot-win32-arm64/-/copilot-win32-arm64-1.0.65.tgz", + "integrity": "sha512-/vSE/t9Wm3eFSWpxlKyn/oL8OAVOB0yFO7ECxhgbtiqNrBd1tgpYh1k7IXBIWa/saxlV1+de6DEmPuQfx3Z0bg==", + "cpu": [ + "arm64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "win32" + ], + "bin": { + "copilot-win32-arm64": "copilot.exe" + } + }, + "node_modules/@github/copilot-win32-x64": { + "version": "1.0.65", + "resolved": "https://registry.npmjs.org/@github/copilot-win32-x64/-/copilot-win32-x64-1.0.65.tgz", + "integrity": "sha512-wjVWXepET+SpFg8z8V43ZiTy6X1OerCb7yu3QZKNNJ3zY9z20goihPXQCDWkiJpGzszNSgfrsiqUzpUsC9qS0A==", + "cpu": [ + "x64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "win32" + ], + "bin": { + "copilot-win32-x64": "copilot.exe" + } + }, + "node_modules/@jridgewell/sourcemap-codec": { + "version": "1.5.5", + "resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.5.5.tgz", + "integrity": "sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==", + "dev": true, + "license": "MIT" + }, + "node_modules/@rollup/rollup-android-arm-eabi": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.62.2.tgz", + "integrity": "sha512-6o7ZLZK+BeenkZCFNDXqpbjw9bD6nuWonvS/lwQJp7NoVVxm6p3qE7qQ5jGuBjiFsgvqjD8mZAU5oWxTmbOeOg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ] + }, + "node_modules/@rollup/rollup-android-arm64": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.62.2.tgz", + "integrity": "sha512-BaH7BllCACHoH1LguOU56UItGfUWjujlO65kS9LAodViaN4bwIKd7oeW/ZHJ/4ljr/7MIiENnNy3HJ0zXv8Zkw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ] + }, + "node_modules/@rollup/rollup-darwin-arm64": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.62.2.tgz", + "integrity": "sha512-v39RCCvj4He82I9sFmk+M1VZ0PLM9sfsLVikjfx2hYBNALhrrOR2D3JjQA6AhlaSOgcR+RzrKY7e1+bT6SUO/A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@rollup/rollup-darwin-x64": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.62.2.tgz", + "integrity": "sha512-yl0y2vq3S3lHeuXhEdss6TWfKW8vkujImO12tn4ZkG/4oghr09LvdYm2RElVjokTQiUvDUGXLGsYeLqUMCKpGA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@rollup/rollup-freebsd-arm64": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.62.2.tgz", + "integrity": "sha512-tT4pvt4qXD+vEoezupCWi+a1F0vvDiksiHc+PxRlYTOH1I6/X4id9jPxTP+Fg+545euaFT1jJVs4CEdHZAU1vw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@rollup/rollup-freebsd-x64": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.62.2.tgz", + "integrity": "sha512-6nU5F2wCW+qvCBhTn1pdIU3bzsIoF7EUwsCDRxilWGprQR6yd508YnH9+OKFCwpfS8pjZqDUmnCAr7exax0XCg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@rollup/rollup-linux-arm-gnueabihf": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.62.2.tgz", + "integrity": "sha512-n1GJHPOvpIfhi3TmrCeh6S6URt9BFCt0KQE3qvexyGCTAKpR4Lg+eWvNZEqu7epxwus/8ElT3hacYEucm49SZg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm-musleabihf": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.62.2.tgz", + "integrity": "sha512-JqgflS8wEB+UXV/vS1RpRbifGBeN4D5lz8D8oOFbFZw4vedvdOgCFAjfBmIMdW3yL10XpQQ0Ambepw6MXrhOnA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm64-gnu": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.62.2.tgz", + "integrity": "sha512-wnFJkogWvN4jm/hQRF2UBaeUmk20j5+DmHvoyWii2b8HJDyvz1MF2OU/6ynXt2KR63rbZLWkFpoytpdc/yBuSA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm64-musl": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.62.2.tgz", + "integrity": "sha512-HVu2bp0zhvJ8xHEV9+UUs7S90VadmBSY3LcIMvozbPo4AuMGDWlz3ymHLHZPX4hR67TKTt8Qp5PJ5RBg/i+RMQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-loong64-gnu": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-loong64-gnu/-/rollup-linux-loong64-gnu-4.62.2.tgz", + "integrity": "sha512-mQqqAV8QaoSgr9I2fKDLY2BAVvmKjWoGiu/cSYQonsLvtqwEn1E4QYfnCOcp5zoEqNhsDYin1s6jx/VJmrxlZg==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-loong64-musl": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-loong64-musl/-/rollup-linux-loong64-musl-4.62.2.tgz", + "integrity": "sha512-IxKLoxCQ2IWi6bT2akyDUBGsOImDKB+sPp4EsTmwFQ/fMwpCKm8uLSSgP/Kx/QYUgKis6SEZ5/Nlhup0DIA0PQ==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-ppc64-gnu": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-ppc64-gnu/-/rollup-linux-ppc64-gnu-4.62.2.tgz", + "integrity": "sha512-Mk5ha2RQSgyFfmYYLkBpPnUk8D8FriBxesO1u9O75X0mHgXL1UQcH5Itl2lurWL2tj0RxV9b9tJgipac0hRY9A==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-ppc64-musl": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-ppc64-musl/-/rollup-linux-ppc64-musl-4.62.2.tgz", + "integrity": "sha512-CjvEnqJL/0/TQ3TXX3OPIJ/kmBellrWd4heXUmHeJlTnmwjKpSJzoehLaL6Xk0ZnMHBu9dZuFADNOrtjF4v+2w==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-riscv64-gnu": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.62.2.tgz", + "integrity": "sha512-1SiZbzwdkaDURsew/tSOrooKiYy7EQGT6m8ufavAi9NEyQb/6VuIxFXAL1fqa4iZe3g4NbNk4P7J32z2tw5Mgg==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-riscv64-musl": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-musl/-/rollup-linux-riscv64-musl-4.62.2.tgz", + "integrity": "sha512-nQts12zJ3NQRoE6uYljOH89v7szzLDvG2JD/vsX+vGXU8w/At1GowTZ5/7qeFQ8m7L55rpR8Okugnuo5bgjy2Q==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-s390x-gnu": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.62.2.tgz", + "integrity": "sha512-E9/ll019jhPIJgpzfZoIkBGhcz+kKNgVWYRY0zr9srBdPPFVpvOKW8VaJKUbeK+eZXyQF9ltME+Kk6affeaPgg==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-x64-gnu": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.62.2.tgz", + "integrity": "sha512-5BqxR/pshjey51iliyzTD5Xi3EN0aLmQ2lZ3lvefVV9c82BvrLo2/6OT55iifpWBufs6kdwWbuOKS841DrmK9A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-x64-musl": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.62.2.tgz", + "integrity": "sha512-uNN83XxQrRAh/w0/pmAfibcwyb6YWt4gP+dpnQKPVJshAloQ785ii8CT8ZCIxkGg9opVsvAlGhFitSm6D1Jjpg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-openbsd-x64": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-openbsd-x64/-/rollup-openbsd-x64-4.62.2.tgz", + "integrity": "sha512-srjEIxSH3LRnJN6THczDHWQplqEMFiAJrTab0msUryh9kwNpkICf3Ea6q6MN/2cZwRFUNx5w+h6Hpi4QuHS6Zg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ] + }, + "node_modules/@rollup/rollup-openharmony-arm64": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-openharmony-arm64/-/rollup-openharmony-arm64-4.62.2.tgz", + "integrity": "sha512-8hOJnxgbyObnCm5AlRA3A931xX19xq80RjVTKgJOvEKWqJruP/Uf12IbAOaDjjEXYRewwHLfmF0YRIdK3OwKWA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ] + }, + "node_modules/@rollup/rollup-win32-arm64-msvc": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.62.2.tgz", + "integrity": "sha512-mmF4AY1i0hG/bLWUctUq59gtmgaSIRa3cu/A3JFRp/sCNEme2bgDEiDS22P9FbnJB8NJNF4jPJiSP5RHQpUTDg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-ia32-msvc": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.62.2.tgz", + "integrity": "sha512-DZgkknc6jhHrk46V25vbAM0zZkyP0nSDkJB8/dRkLTxv470dOmWDqGoEJl/9A0dFfS7yE3REOwNDxpHwSLSt0Q==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-x64-gnu": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-gnu/-/rollup-win32-x64-gnu-4.62.2.tgz", + "integrity": "sha512-T6xr6ucWSFto+VGajA8YH26LdpHRuP4YLHEKAtCWvJDOlnmWcDZVCI2Jmjr+IFHDlt2zRaTAKE4tfjTaWLgJBg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-x64-msvc": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.62.2.tgz", + "integrity": "sha512-BfzEnDJOt9T8M989/lA37EcJgat01wLRnoi5dQf3QzOH7jzpqTAzdDbVfRljVr5r+jzKqpbHeyOfAaXxAd0PAA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@types/estree": { + "version": "1.0.9", + "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.9.tgz", + "integrity": "sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/node": { + "version": "20.19.43", + "resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.43.tgz", + "integrity": "sha512-6oYBAi5ikg4Pl+kGsoYtawUMBT2zZMCvPNF7pVLnHZfd1zf38DRiWn/gT01RYCdUqkv7Fhr+C9ot4/tb+2sVvA==", + "dev": true, + "license": "MIT", + "peer": true, + "dependencies": { + "undici-types": "~6.21.0" + } + }, + "node_modules/@vitest/expect": { + "version": "2.1.9", + "resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-2.1.9.tgz", + "integrity": "sha512-UJCIkTBenHeKT1TTlKMJWy1laZewsRIzYighyYiJKZreqtdxSos/S1t+ktRMQWu2CKqaarrkeszJx1cgC5tGZw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/spy": "2.1.9", + "@vitest/utils": "2.1.9", + "chai": "^5.1.2", + "tinyrainbow": "^1.2.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/mocker": { + "version": "2.1.9", + "resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-2.1.9.tgz", + "integrity": "sha512-tVL6uJgoUdi6icpxmdrn5YNo3g3Dxv+IHJBr0GXHaEdTcw3F+cPKnsXFhli6nO+f/6SDKPHEK1UN+k+TQv0Ehg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/spy": "2.1.9", + "estree-walker": "^3.0.3", + "magic-string": "^0.30.12" + }, + "funding": { + "url": "https://opencollective.com/vitest" + }, + "peerDependencies": { + "msw": "^2.4.9", + "vite": "^5.0.0" + }, + "peerDependenciesMeta": { + "msw": { + "optional": true + }, + "vite": { + "optional": true + } + } + }, + "node_modules/@vitest/pretty-format": { + "version": "2.1.9", + "resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-2.1.9.tgz", + "integrity": "sha512-KhRIdGV2U9HOUzxfiHmY8IFHTdqtOhIzCpd8WRdJiE7D/HUcZVD0EgQCVjm+Q9gkUXWgBvMmTtZgIG48wq7sOQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "tinyrainbow": "^1.2.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/runner": { + "version": "2.1.9", + "resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-2.1.9.tgz", + "integrity": "sha512-ZXSSqTFIrzduD63btIfEyOmNcBmQvgOVsPNPe0jYtESiXkhd8u2erDLnMxmGrDCwHCCHE7hxwRDCT3pt0esT4g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/utils": "2.1.9", + "pathe": "^1.1.2" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/snapshot": { + "version": "2.1.9", + "resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-2.1.9.tgz", + "integrity": "sha512-oBO82rEjsxLNJincVhLhaxxZdEtV0EFHMK5Kmx5sJ6H9L183dHECjiefOAdnqpIgT5eZwT04PoggUnW88vOBNQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/pretty-format": "2.1.9", + "magic-string": "^0.30.12", + "pathe": "^1.1.2" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/spy": { + "version": "2.1.9", + "resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-2.1.9.tgz", + "integrity": "sha512-E1B35FwzXXTs9FHNK6bDszs7mtydNi5MIfUWpceJ8Xbfb1gBMscAnwLbEu+B44ed6W3XjL9/ehLPHR1fkf1KLQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "tinyspy": "^3.0.2" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/@vitest/utils": { + "version": "2.1.9", + "resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-2.1.9.tgz", + "integrity": "sha512-v0psaMSkNJ3A2NMrUEHFRzJtDPFn+/VWZ5WxImB21T9fjucJRmS7xCS3ppEnARb9y11OAzaD+P2Ps+b+BGX5iQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/pretty-format": "2.1.9", + "loupe": "^3.1.2", + "tinyrainbow": "^1.2.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/assertion-error": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz", + "integrity": "sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12" + } + }, + "node_modules/balanced-match": { + "version": "4.0.4", + "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-4.0.4.tgz", + "integrity": "sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA==", + "dev": true, + "license": "MIT", + "engines": { + "node": "18 || 20 || >=22" + } + }, + "node_modules/brace-expansion": { + "version": "5.0.7", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.7.tgz", + "integrity": "sha512-7oFy703dxfY3/NLxC1fh2SUCQ0H9rmAY+5EpDVfXjUTTs+HEwR2nYaqLv+GWcTsumwxPfiz6CzCNkwXwBUwqCA==", + "dev": true, + "license": "MIT", + "dependencies": { + "balanced-match": "^4.0.2" + }, + "engines": { + "node": "18 || 20 || >=22" + } + }, + "node_modules/cac": { + "version": "6.7.14", + "resolved": "https://registry.npmjs.org/cac/-/cac-6.7.14.tgz", + "integrity": "sha512-b6Ilus+c3RrdDk+JhLKUAQfzzgLEPy6wcXqS7f/xe1EETvsDP6GORG7SFuOs6cID5YkqchW/LXZbX5bc8j7ZcQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=8" + } + }, + "node_modules/chai": { + "version": "5.3.3", + "resolved": "https://registry.npmjs.org/chai/-/chai-5.3.3.tgz", + "integrity": "sha512-4zNhdJD/iOjSH0A05ea+Ke6MU5mmpQcbQsSOkgdaUMJ9zTlDTD/GYlwohmIE2u0gaxHYiVHEn1Fw9mZ/ktJWgw==", + "dev": true, + "license": "MIT", + "dependencies": { + "assertion-error": "^2.0.1", + "check-error": "^2.1.1", + "deep-eql": "^5.0.1", + "loupe": "^3.1.0", + "pathval": "^2.0.0" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/check-error": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/check-error/-/check-error-2.1.3.tgz", + "integrity": "sha512-PAJdDJusoxnwm1VwW07VWwUN1sl7smmC3OKggvndJFadxxDRyFJBX/ggnu/KE4kQAB7a3Dp8f/YXC1FlUprWmA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 16" + } + }, + "node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "dev": true, + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "node_modules/deep-eql": { + "version": "5.0.2", + "resolved": "https://registry.npmjs.org/deep-eql/-/deep-eql-5.0.2.tgz", + "integrity": "sha512-h5k/5U50IJJFpzfL6nO9jaaumfjO/f2NjK/oYB2Djzm4p9L+3T9qWpZqZ2hAbLPuuYq9wrU08WQyBTL5GbPk5Q==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6" + } + }, + "node_modules/detect-libc": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", + "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==", + "license": "Apache-2.0", + "engines": { + "node": ">=8" + } + }, + "node_modules/es-module-lexer": { + "version": "1.7.0", + "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-1.7.0.tgz", + "integrity": "sha512-jEQoCwk8hyb2AZziIOLhDqpm5+2ww5uIE6lkO/6jcOCusfk6LhMHpXXfBLXTZ7Ydyt0j4VoUQv6uGNYbdW+kBA==", + "dev": true, + "license": "MIT" + }, + "node_modules/esbuild": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.1.tgz", + "integrity": "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.28.1", + "@esbuild/android-arm": "0.28.1", + "@esbuild/android-arm64": "0.28.1", + "@esbuild/android-x64": "0.28.1", + "@esbuild/darwin-arm64": "0.28.1", + "@esbuild/darwin-x64": "0.28.1", + "@esbuild/freebsd-arm64": "0.28.1", + "@esbuild/freebsd-x64": "0.28.1", + "@esbuild/linux-arm": "0.28.1", + "@esbuild/linux-arm64": "0.28.1", + "@esbuild/linux-ia32": "0.28.1", + "@esbuild/linux-loong64": "0.28.1", + "@esbuild/linux-mips64el": "0.28.1", + "@esbuild/linux-ppc64": "0.28.1", + "@esbuild/linux-riscv64": "0.28.1", + "@esbuild/linux-s390x": "0.28.1", + "@esbuild/linux-x64": "0.28.1", + "@esbuild/netbsd-arm64": "0.28.1", + "@esbuild/netbsd-x64": "0.28.1", + "@esbuild/openbsd-arm64": "0.28.1", + "@esbuild/openbsd-x64": "0.28.1", + "@esbuild/openharmony-arm64": "0.28.1", + "@esbuild/sunos-x64": "0.28.1", + "@esbuild/win32-arm64": "0.28.1", + "@esbuild/win32-ia32": "0.28.1", + "@esbuild/win32-x64": "0.28.1" + } + }, + "node_modules/estree-walker": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-3.0.3.tgz", + "integrity": "sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/estree": "^1.0.0" + } + }, + "node_modules/expect-type": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.4.0.tgz", + "integrity": "sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==", + "dev": true, + "license": "Apache-2.0", + "engines": { + "node": ">=12.0.0" + } + }, + "node_modules/fsevents": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", + "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/glob": { + "version": "13.0.6", + "resolved": "https://registry.npmjs.org/glob/-/glob-13.0.6.tgz", + "integrity": "sha512-Wjlyrolmm8uDpm/ogGyXZXb1Z+Ca2B8NbJwqBVg0axK9GbBeoS7yGV6vjXnYdGm6X53iehEuxxbyiKp8QmN4Vw==", + "dev": true, + "license": "BlueOak-1.0.0", + "dependencies": { + "minimatch": "^10.2.2", + "minipass": "^7.1.3", + "path-scurry": "^2.0.2" + }, + "engines": { + "node": "18 || 20 || >=22" + }, + "funding": { + "url": "https://github.com/sponsors/isaacs" + } + }, + "node_modules/loupe": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/loupe/-/loupe-3.2.1.tgz", + "integrity": "sha512-CdzqowRJCeLU72bHvWqwRBBlLcMEtIvGrlvef74kMnV2AolS9Y8xUv1I0U/MNAWMhBlKIoyuEgoJ0t/bbwHbLQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/lru-cache": { + "version": "11.5.1", + "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.5.1.tgz", + "integrity": "sha512-RPimw/7aMdv2oqRrxKwvZXcPfwBrn/JZ2xYcY9Hus/6LaS3VOAKVWKWgNLCFSiOm1ESXinjsDlidVU7JlnCN2A==", + "dev": true, + "license": "BlueOak-1.0.0", + "engines": { + "node": "20 || >=22" + } + }, + "node_modules/magic-string": { + "version": "0.30.21", + "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz", + "integrity": "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jridgewell/sourcemap-codec": "^1.5.5" + } + }, + "node_modules/minimatch": { + "version": "10.2.5", + "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.2.5.tgz", + "integrity": "sha512-MULkVLfKGYDFYejP07QOurDLLQpcjk7Fw+7jXS2R2czRQzR56yHRveU5NDJEOviH+hETZKSkIk5c+T23GjFUMg==", + "dev": true, + "license": "BlueOak-1.0.0", + "dependencies": { + "brace-expansion": "^5.0.5" + }, + "engines": { + "node": "18 || 20 || >=22" + }, + "funding": { + "url": "https://github.com/sponsors/isaacs" + } + }, + "node_modules/minipass": { + "version": "7.1.3", + "resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz", + "integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==", + "dev": true, + "license": "BlueOak-1.0.0", + "engines": { + "node": ">=16 || 14 >=14.17" + } + }, + "node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "dev": true, + "license": "MIT" + }, + "node_modules/nanoid": { + "version": "3.3.15", + "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.15.tgz", + "integrity": "sha512-y7Wygv/7mEOvxTuEQDB8StXdMRBWf1kR/tlhAzBRUFkB2jfcLOAxO/SHmOO2zgz1pVgK29/kyupn059/bCHdjA==", + "dev": true, + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "bin": { + "nanoid": "bin/nanoid.cjs" + }, + "engines": { + "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1" + } + }, + "node_modules/package-json-from-dist": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/package-json-from-dist/-/package-json-from-dist-1.0.1.tgz", + "integrity": "sha512-UEZIS3/by4OC8vL3P2dTXRETpebLI2NiI5vIrjaD/5UtrkFX/tNbwjTSRAGC/+7CAo2pIcBaRgWmcBBHcsaCIw==", + "dev": true, + "license": "BlueOak-1.0.0" + }, + "node_modules/path-scurry": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/path-scurry/-/path-scurry-2.0.2.tgz", + "integrity": "sha512-3O/iVVsJAPsOnpwWIeD+d6z/7PmqApyQePUtCndjatj/9I5LylHvt5qluFaBT3I5h3r1ejfR056c+FCv+NnNXg==", + "dev": true, + "license": "BlueOak-1.0.0", + "dependencies": { + "lru-cache": "^11.0.0", + "minipass": "^7.1.2" + }, + "engines": { + "node": "18 || 20 || >=22" + }, + "funding": { + "url": "https://github.com/sponsors/isaacs" + } + }, + "node_modules/pathe": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/pathe/-/pathe-1.1.2.tgz", + "integrity": "sha512-whLdWMYL2TwI08hn8/ZqAbrVemu0LNaNNJZX73O6qaIdCTfXutsLhMkjdENX0qhsQ9uIimo4/aQOmXkoon2nDQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/pathval": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/pathval/-/pathval-2.0.1.tgz", + "integrity": "sha512-//nshmD55c46FuFw26xV/xFAaB5HF9Xdap7HJBBnrKdAd6/GxDBaNA1870O79+9ueg61cZLSVc+OaFlfmObYVQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 14.16" + } + }, + "node_modules/picocolors": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz", + "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==", + "dev": true, + "license": "ISC" + }, + "node_modules/postcss": { + "version": "8.5.16", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.16.tgz", + "integrity": "sha512-vuwillviilfKZsg0VGj5R/YwwcHx4SLsIOI/7K6mQkWx+l5cUHTjj5g0AasTBcyXsbfTgrwsUNmVUb5xVwyPwg==", + "dev": true, + "funding": [ + { + "type": "opencollective", + "url": "https://opencollective.com/postcss/" + }, + { + "type": "tidelift", + "url": "https://tidelift.com/funding/github/npm/postcss" + }, + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "dependencies": { + "nanoid": "^3.3.12", + "picocolors": "^1.1.1", + "source-map-js": "^1.2.1" + }, + "engines": { + "node": "^10 || ^12 || >=14" + } + }, + "node_modules/prettier": { + "version": "3.9.3", + "resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.3.tgz", + "integrity": "sha512-HWmu+K+zvHNpaMfSnYeqdqrDbR16cuIXaPx8WoHaviQkDJh1/0BNtOZmHVQI5jc3wXv0H1yXc9wjvFdXh+n3hQ==", + "dev": true, + "license": "MIT", + "bin": { + "prettier": "bin/prettier.cjs" + }, + "engines": { + "node": ">=14" + }, + "funding": { + "url": "https://github.com/prettier/prettier?sponsor=1" + } + }, + "node_modules/rimraf": { + "version": "6.1.3", + "resolved": "https://registry.npmjs.org/rimraf/-/rimraf-6.1.3.tgz", + "integrity": "sha512-LKg+Cr2ZF61fkcaK1UdkH2yEBBKnYjTyWzTJT6KNPcSPaiT7HSdhtMXQuN5wkTX0Xu72KQ1l8S42rlmexS2hSA==", + "dev": true, + "license": "BlueOak-1.0.0", + "dependencies": { + "glob": "^13.0.3", + "package-json-from-dist": "^1.0.1" + }, + "bin": { + "rimraf": "dist/esm/bin.mjs" + }, + "engines": { + "node": "20 || >=22" + }, + "funding": { + "url": "https://github.com/sponsors/isaacs" + } + }, + "node_modules/rollup": { + "version": "4.62.2", + "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.62.2.tgz", + "integrity": "sha512-RFnrW4lhXA3s3eqHDZvN654g8OTjzRfqpIRJYczCGB6HzphckVAi/Qh4tbPUbRuDi7s1Llv8g/NspLkttY3gTA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/estree": "1.0.9" + }, + "bin": { + "rollup": "dist/bin/rollup" + }, + "engines": { + "node": ">=18.0.0", + "npm": ">=8.0.0" + }, + "optionalDependencies": { + "@rollup/rollup-android-arm-eabi": "4.62.2", + "@rollup/rollup-android-arm64": "4.62.2", + "@rollup/rollup-darwin-arm64": "4.62.2", + "@rollup/rollup-darwin-x64": "4.62.2", + "@rollup/rollup-freebsd-arm64": "4.62.2", + "@rollup/rollup-freebsd-x64": "4.62.2", + "@rollup/rollup-linux-arm-gnueabihf": "4.62.2", + "@rollup/rollup-linux-arm-musleabihf": "4.62.2", + "@rollup/rollup-linux-arm64-gnu": "4.62.2", + "@rollup/rollup-linux-arm64-musl": "4.62.2", + "@rollup/rollup-linux-loong64-gnu": "4.62.2", + "@rollup/rollup-linux-loong64-musl": "4.62.2", + "@rollup/rollup-linux-ppc64-gnu": "4.62.2", + "@rollup/rollup-linux-ppc64-musl": "4.62.2", + "@rollup/rollup-linux-riscv64-gnu": "4.62.2", + "@rollup/rollup-linux-riscv64-musl": "4.62.2", + "@rollup/rollup-linux-s390x-gnu": "4.62.2", + "@rollup/rollup-linux-x64-gnu": "4.62.2", + "@rollup/rollup-linux-x64-musl": "4.62.2", + "@rollup/rollup-openbsd-x64": "4.62.2", + "@rollup/rollup-openharmony-arm64": "4.62.2", + "@rollup/rollup-win32-arm64-msvc": "4.62.2", + "@rollup/rollup-win32-ia32-msvc": "4.62.2", + "@rollup/rollup-win32-x64-gnu": "4.62.2", + "@rollup/rollup-win32-x64-msvc": "4.62.2", + "fsevents": "~2.3.2" + } + }, + "node_modules/siginfo": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz", + "integrity": "sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g==", + "dev": true, + "license": "ISC" + }, + "node_modules/source-map-js": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz", + "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==", + "dev": true, + "license": "BSD-3-Clause", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/stackback": { + "version": "0.0.2", + "resolved": "https://registry.npmjs.org/stackback/-/stackback-0.0.2.tgz", + "integrity": "sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==", + "dev": true, + "license": "MIT" + }, + "node_modules/std-env": { + "version": "3.10.0", + "resolved": "https://registry.npmjs.org/std-env/-/std-env-3.10.0.tgz", + "integrity": "sha512-5GS12FdOZNliM5mAOxFRg7Ir0pWz8MdpYm6AY6VPkGpbA7ZzmbzNcBJQ0GPvvyWgcY7QAhCgf9Uy89I03faLkg==", + "dev": true, + "license": "MIT" + }, + "node_modules/tinybench": { + "version": "2.9.0", + "resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz", + "integrity": "sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==", + "dev": true, + "license": "MIT" + }, + "node_modules/tinyexec": { + "version": "0.3.2", + "resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-0.3.2.tgz", + "integrity": "sha512-KQQR9yN7R5+OSwaK0XQoj22pwHoTlgYqmUscPYoknOoWCWfj/5/ABTMRi69FrKU5ffPVh5QcFikpWJI/P1ocHA==", + "dev": true, + "license": "MIT" + }, + "node_modules/tinypool": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/tinypool/-/tinypool-1.1.1.tgz", + "integrity": "sha512-Zba82s87IFq9A9XmjiX5uZA/ARWDrB03OHlq+Vw1fSdt0I+4/Kutwy8BP4Y/y/aORMo61FQ0vIb5j44vSo5Pkg==", + "dev": true, + "license": "MIT", + "engines": { + "node": "^18.0.0 || >=20.0.0" + } + }, + "node_modules/tinyrainbow": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/tinyrainbow/-/tinyrainbow-1.2.0.tgz", + "integrity": "sha512-weEDEq7Z5eTHPDh4xjX789+fHfF+P8boiFB+0vbWzpbnbsEr/GRaohi/uMKxg8RZMXnl1ItAi/IUHWMsjDV7kQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/tinyspy": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/tinyspy/-/tinyspy-3.0.2.tgz", + "integrity": "sha512-n1cw8k1k0x4pgA2+9XrOkFydTerNcJ1zWCO5Nn9scWHTD+5tp8dghT2x1uduQePZTZgd3Tupf+x9BxJjeJi77Q==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/tsx": { + "version": "4.22.4", + "resolved": "https://registry.npmjs.org/tsx/-/tsx-4.22.4.tgz", + "integrity": "sha512-X8EX+XV4QR5xCsrgxaED954zTDfY8KqlDtskKEL0cHhyS/P8b4IFOvGDQpsC9Q1XnLq915wEfwwY/zzskCtmhg==", + "dev": true, + "license": "MIT", + "dependencies": { + "esbuild": "~0.28.0" + }, + "bin": { + "tsx": "dist/cli.mjs" + }, + "engines": { + "node": ">=18.0.0" + }, + "optionalDependencies": { + "fsevents": "~2.3.3" + } + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + }, + "node_modules/undici-types": { + "version": "6.21.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", + "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/vite": { + "version": "5.4.21", + "resolved": "https://registry.npmjs.org/vite/-/vite-5.4.21.tgz", + "integrity": "sha512-o5a9xKjbtuhY6Bi5S3+HvbRERmouabWbyUcpXXUA1u+GNUKoROi9byOJ8M0nHbHYHkYICiMlqxkg1KkYmm25Sw==", + "dev": true, + "license": "MIT", + "dependencies": { + "esbuild": "^0.21.3", + "postcss": "^8.4.43", + "rollup": "^4.20.0" + }, + "bin": { + "vite": "bin/vite.js" + }, + "engines": { + "node": "^18.0.0 || >=20.0.0" + }, + "funding": { + "url": "https://github.com/vitejs/vite?sponsor=1" + }, + "optionalDependencies": { + "fsevents": "~2.3.3" + }, + "peerDependencies": { + "@types/node": "^18.0.0 || >=20.0.0", + "less": "*", + "lightningcss": "^1.21.0", + "sass": "*", + "sass-embedded": "*", + "stylus": "*", + "sugarss": "*", + "terser": "^5.4.0" + }, + "peerDependenciesMeta": { + "@types/node": { + "optional": true + }, + "less": { + "optional": true + }, + "lightningcss": { + "optional": true + }, + "sass": { + "optional": true + }, + "sass-embedded": { + "optional": true + }, + "stylus": { + "optional": true + }, + "sugarss": { + "optional": true + }, + "terser": { + "optional": true + } + } + }, + "node_modules/vite-node": { + "version": "2.1.9", + "resolved": "https://registry.npmjs.org/vite-node/-/vite-node-2.1.9.tgz", + "integrity": "sha512-AM9aQ/IPrW/6ENLQg3AGY4K1N2TGZdR5e4gu/MmmR2xR3Ll1+dib+nook92g4TV3PXVyeyxdWwtaCAiUL0hMxA==", + "dev": true, + "license": "MIT", + "dependencies": { + "cac": "^6.7.14", + "debug": "^4.3.7", + "es-module-lexer": "^1.5.4", + "pathe": "^1.1.2", + "vite": "^5.0.0" + }, + "bin": { + "vite-node": "vite-node.mjs" + }, + "engines": { + "node": "^18.0.0 || >=20.0.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + } + }, + "node_modules/vite/node_modules/@esbuild/aix-ppc64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.21.5.tgz", + "integrity": "sha512-1SDgH6ZSPTlggy1yI6+Dbkiz8xzpHJEVAlF/AM1tHPLsf5STom9rwtjE4hKAF20FfXXNTFqEYXyJNWh1GiZedQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/android-arm": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.21.5.tgz", + "integrity": "sha512-vCPvzSjpPHEi1siZdlvAlsPxXl7WbOVUBBAowWug4rJHb68Ox8KualB+1ocNvT5fjv6wpkX6o/iEpbDrf68zcg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/android-arm64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.21.5.tgz", + "integrity": "sha512-c0uX9VAUBQ7dTDCjq+wdyGLowMdtR/GoC2U5IYk/7D1H1JYC0qseD7+11iMP2mRLN9RcCMRcjC4YMclCzGwS/A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/android-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.21.5.tgz", + "integrity": "sha512-D7aPRUUNHRBwHxzxRvp856rjUHRFW1SdQATKXH2hqA0kAZb1hKmi02OpYRacl0TxIGz/ZmXWlbZgjwWYaCakTA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/darwin-arm64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.21.5.tgz", + "integrity": "sha512-DwqXqZyuk5AiWWf3UfLiRDJ5EDd49zg6O9wclZ7kUMv2WRFr4HKjXp/5t8JZ11QbQfUS6/cRCKGwYhtNAY88kQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/darwin-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.21.5.tgz", + "integrity": "sha512-se/JjF8NlmKVG4kNIuyWMV/22ZaerB+qaSi5MdrXtd6R08kvs2qCN4C09miupktDitvh8jRFflwGFBQcxZRjbw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/freebsd-arm64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.21.5.tgz", + "integrity": "sha512-5JcRxxRDUJLX8JXp/wcBCy3pENnCgBR9bN6JsY4OmhfUtIHe3ZW0mawA7+RDAcMLrMIZaf03NlQiX9DGyB8h4g==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/freebsd-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.21.5.tgz", + "integrity": "sha512-J95kNBj1zkbMXtHVH29bBriQygMXqoVQOQYA+ISs0/2l3T9/kj42ow2mpqerRBxDJnmkUDCaQT/dfNXWX/ZZCQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/linux-arm": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.21.5.tgz", + "integrity": "sha512-bPb5AHZtbeNGjCKVZ9UGqGwo8EUu4cLq68E95A53KlxAPRmUyYv2D6F0uUI65XisGOL1hBP5mTronbgo+0bFcA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/linux-arm64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.21.5.tgz", + "integrity": "sha512-ibKvmyYzKsBeX8d8I7MH/TMfWDXBF3db4qM6sy+7re0YXya+K1cem3on9XgdT2EQGMu4hQyZhan7TeQ8XkGp4Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/linux-ia32": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.21.5.tgz", + "integrity": "sha512-YvjXDqLRqPDl2dvRODYmmhz4rPeVKYvppfGYKSNGdyZkA01046pLWyRKKI3ax8fbJoK5QbxblURkwK/MWY18Tg==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/linux-loong64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.21.5.tgz", + "integrity": "sha512-uHf1BmMG8qEvzdrzAqg2SIG/02+4/DHB6a9Kbya0XDvwDEKCoC8ZRWI5JJvNdUjtciBGFQ5PuBlpEOXQj+JQSg==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/linux-mips64el": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.21.5.tgz", + "integrity": "sha512-IajOmO+KJK23bj52dFSNCMsz1QP1DqM6cwLUv3W1QwyxkyIWecfafnI555fvSGqEKwjMXVLokcV5ygHW5b3Jbg==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/linux-ppc64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.21.5.tgz", + "integrity": "sha512-1hHV/Z4OEfMwpLO8rp7CvlhBDnjsC3CttJXIhBi+5Aj5r+MBvy4egg7wCbe//hSsT+RvDAG7s81tAvpL2XAE4w==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/linux-riscv64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.21.5.tgz", + "integrity": "sha512-2HdXDMd9GMgTGrPWnJzP2ALSokE/0O5HhTUvWIbD3YdjME8JwvSCnNGBnTThKGEB91OZhzrJ4qIIxk/SBmyDDA==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/linux-s390x": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.21.5.tgz", + "integrity": "sha512-zus5sxzqBJD3eXxwvjN1yQkRepANgxE9lgOW2qLnmr8ikMTphkjgXu1HR01K4FJg8h1kEEDAqDcZQtbrRnB41A==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/linux-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.21.5.tgz", + "integrity": "sha512-1rYdTpyv03iycF1+BhzrzQJCdOuAOtaqHTWJZCWvijKD2N5Xu0TtVC8/+1faWqcP9iBCWOmjmhoH94dH82BxPQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/netbsd-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.21.5.tgz", + "integrity": "sha512-Woi2MXzXjMULccIwMnLciyZH4nCIMpWQAs049KEeMvOcNADVxo0UBIQPfSmxB3CWKedngg7sWZdLvLczpe0tLg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/openbsd-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.21.5.tgz", + "integrity": "sha512-HLNNw99xsvx12lFBUwoT8EVCsSvRNDVxNpjZ7bPn947b8gJPzeHWyNVhFsaerc0n3TsbOINvRP2byTZ5LKezow==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/sunos-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.21.5.tgz", + "integrity": "sha512-6+gjmFpfy0BHU5Tpptkuh8+uw3mnrvgs+dSPQXQOv3ekbordwnzTVEb4qnIvQcYXq6gzkyTnoZ9dZG+D4garKg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/win32-arm64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.21.5.tgz", + "integrity": "sha512-Z0gOTd75VvXqyq7nsl93zwahcTROgqvuAcYDUr+vOv8uHhNSKROyU961kgtCD1e95IqPKSQKH7tBTslnS3tA8A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/win32-ia32": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.21.5.tgz", + "integrity": "sha512-SWXFF1CL2RVNMaVs+BBClwtfZSvDgtL//G/smwAc5oVK/UPu2Gu9tIaRgFmYFFKrmg3SyAjSrElf0TiJ1v8fYA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/@esbuild/win32-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.21.5.tgz", + "integrity": "sha512-tQd/1efJuzPC6rCFwEvLtci/xNFcTZknmXs98FYDfGE4wP9ClFV98nyKrzJKVPMhdDnjzLhdUyMX4PsQAPjwIw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/vite/node_modules/esbuild": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.21.5.tgz", + "integrity": "sha512-mg3OPMV4hXywwpoDxu3Qda5xCKQi+vCTZq8S9J/EpkhB2HzKXq4SNFZE3+NK93JYxc8VMSep+lOUSC/RVKaBqw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=12" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.21.5", + "@esbuild/android-arm": "0.21.5", + "@esbuild/android-arm64": "0.21.5", + "@esbuild/android-x64": "0.21.5", + "@esbuild/darwin-arm64": "0.21.5", + "@esbuild/darwin-x64": "0.21.5", + "@esbuild/freebsd-arm64": "0.21.5", + "@esbuild/freebsd-x64": "0.21.5", + "@esbuild/linux-arm": "0.21.5", + "@esbuild/linux-arm64": "0.21.5", + "@esbuild/linux-ia32": "0.21.5", + "@esbuild/linux-loong64": "0.21.5", + "@esbuild/linux-mips64el": "0.21.5", + "@esbuild/linux-ppc64": "0.21.5", + "@esbuild/linux-riscv64": "0.21.5", + "@esbuild/linux-s390x": "0.21.5", + "@esbuild/linux-x64": "0.21.5", + "@esbuild/netbsd-x64": "0.21.5", + "@esbuild/openbsd-x64": "0.21.5", + "@esbuild/sunos-x64": "0.21.5", + "@esbuild/win32-arm64": "0.21.5", + "@esbuild/win32-ia32": "0.21.5", + "@esbuild/win32-x64": "0.21.5" + } + }, + "node_modules/vitest": { + "version": "2.1.9", + "resolved": "https://registry.npmjs.org/vitest/-/vitest-2.1.9.tgz", + "integrity": "sha512-MSmPM9REYqDGBI8439mA4mWhV5sKmDlBKWIYbA3lRb2PTHACE0mgKwA8yQ2xq9vxDTuk4iPrECBAEW2aoFXY0Q==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vitest/expect": "2.1.9", + "@vitest/mocker": "2.1.9", + "@vitest/pretty-format": "^2.1.9", + "@vitest/runner": "2.1.9", + "@vitest/snapshot": "2.1.9", + "@vitest/spy": "2.1.9", + "@vitest/utils": "2.1.9", + "chai": "^5.1.2", + "debug": "^4.3.7", + "expect-type": "^1.1.0", + "magic-string": "^0.30.12", + "pathe": "^1.1.2", + "std-env": "^3.8.0", + "tinybench": "^2.9.0", + "tinyexec": "^0.3.1", + "tinypool": "^1.0.1", + "tinyrainbow": "^1.2.0", + "vite": "^5.0.0", + "vite-node": "2.1.9", + "why-is-node-running": "^2.3.0" + }, + "bin": { + "vitest": "vitest.mjs" + }, + "engines": { + "node": "^18.0.0 || >=20.0.0" + }, + "funding": { + "url": "https://opencollective.com/vitest" + }, + "peerDependencies": { + "@edge-runtime/vm": "*", + "@types/node": "^18.0.0 || >=20.0.0", + "@vitest/browser": "2.1.9", + "@vitest/ui": "2.1.9", + "happy-dom": "*", + "jsdom": "*" + }, + "peerDependenciesMeta": { + "@edge-runtime/vm": { + "optional": true + }, + "@types/node": { + "optional": true + }, + "@vitest/browser": { + "optional": true + }, + "@vitest/ui": { + "optional": true + }, + "happy-dom": { + "optional": true + }, + "jsdom": { + "optional": true + } + } + }, + "node_modules/vscode-jsonrpc": { + "version": "8.2.1", + "resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-8.2.1.tgz", + "integrity": "sha512-kdjOSJ2lLIn7r1rtrMbbNCHjyMPfRnowdKjBQ+mGq6NAW5QY2bEZC/khaC5OR8svbbjvLEaIXkOq45e2X9BIbQ==", + "license": "MIT", + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/why-is-node-running": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/why-is-node-running/-/why-is-node-running-2.3.0.tgz", + "integrity": "sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==", + "dev": true, + "license": "MIT", + "dependencies": { + "siginfo": "^2.0.0", + "stackback": "0.0.2" + }, + "bin": { + "why-is-node-running": "cli.js" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/yaml": { + "version": "2.9.0", + "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz", + "integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==", + "license": "ISC", + "bin": { + "yaml": "bin.mjs" + }, + "engines": { + "node": ">= 14.6" + }, + "funding": { + "url": "https://github.com/sponsors/eemeli" + } + }, + "node_modules/zod": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/zod/-/zod-4.4.3.tgz", + "integrity": "sha512-ytENFjIJFl2UwYglde2jchW2Hwm4GJFLDiSXWdTrJQBIN9Fcyp7n4DhxJEiWNAJMV1/BqWfW/kkg71UDcHJyTQ==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + } + } +} diff --git a/tools/agentic-workflow/package.json b/tools/agentic-workflow/package.json new file mode 100644 index 00000000000..45422072064 --- /dev/null +++ b/tools/agentic-workflow/package.json @@ -0,0 +1,48 @@ +{ + "name": "@azure-tools/agentic-workflow", + "version": "0.1.0", + "description": "Agentic research -> plan -> implement workflow: one fresh Copilot SDK session per phase, artifacts on disk, orchestrator-gated.", + "main": "dist/index.js", + "homepage": "https://github.com/Azure/azure-sdk-tools/tree/main/tools/agentic-workflow#readme", + "repository": { + "type": "git", + "url": "git+https://github.com/Azure/azure-sdk-tools.git" + }, + "bugs": { + "url": "https://github.com/Azure/azure-sdk-tools/issues" + }, + "type": "module", + "engines": { + "node": ">=20.19.0" + }, + "bin": { + "agentic-workflow": "dist/cli.js" + }, + "files": [ + "dist", + "prompts" + ], + "scripts": { + "build": "npm run clean && tsc", + "clean": "rimraf ./dist", + "start": "tsx src/cli.ts", + "test": "vitest run", + "test:watch": "vitest", + "format": "prettier . --write", + "format:check": "prettier . --check" + }, + "author": "Microsoft Corporation", + "license": "MIT", + "dependencies": { + "@github/copilot-sdk": "^1.0.4", + "yaml": "^2.5.0" + }, + "devDependencies": { + "@types/node": "^20.19.0", + "prettier": "^3.3.3", + "rimraf": "^6.0.1", + "tsx": "^4.19.0", + "typescript": "^5.6.0", + "vitest": "^2.1.0" + } +} diff --git a/tools/agentic-workflow/prompts/01-research.md b/tools/agentic-workflow/prompts/01-research.md new file mode 100644 index 00000000000..f616d12c8b8 --- /dev/null +++ b/tools/agentic-workflow/prompts/01-research.md @@ -0,0 +1,31 @@ +# Phase 1 — Research (read-only) + +You are in the **research** phase of an automated `research → plan → implement` workflow. +Your job is to produce factual specs of the **current** code relevant to the task. You are +**read-only**: do not modify any source file, do not run shell commands, do not design or +propose changes. Just understand and document what exists today. + +## Task +{{task}} + +## Inputs +- The codebase (read freely). + +## Outputs — write each via the `write_artifact` tool (the ONLY way to persist files) +1. `specs/architecture.md` — the components, modules, data flow, and key entry points relevant + to the task. Cite concrete file paths and symbols. No proposals. +2. `specs/functional.md` — the current behavior relevant to the task: what the code does today, + the user-visible/contract behavior, and the edge cases it already handles. +3. `specs/apispec.md` — **conditional.** Only if the task touches an API surface (REST/RPC/SDK + contract, schema, public interface). Capture the relevant existing API definition + (endpoints/operations, request/response shapes, contracts). +4. `manifest.json` — record the apispec decision explicitly so downstream phases are unambiguous: + ```json + { "apispec": { "required": false, "reason": "" } } + ``` + +## Constraints +- Read-only. Editing source or running code is a hard failure for this phase. +- Cite real file paths/symbols; do not invent. +- Stay scoped to what the task needs — do not document the whole repo. +- When you have written all required artifacts, end your turn. diff --git a/tools/agentic-workflow/prompts/02-assumptions.md b/tools/agentic-workflow/prompts/02-assumptions.md new file mode 100644 index 00000000000..db0d54e072f --- /dev/null +++ b/tools/agentic-workflow/prompts/02-assumptions.md @@ -0,0 +1,31 @@ +# Phase 2 — Assumptions (read-only) + +You are in the **assumptions** phase. Enumerate the baseline assumptions, unknowns, and risks the +later planning/implementation will rely on. You are **read-only**. + +## Task +{{task}} + +## Inputs +- `specs/architecture.md`, `specs/functional.md`, and `specs/apispec.md` if present. +- {{researchNote}} + +## Output — via the `write_artifact` tool +`assumptions.md` — one assumption per line, each with a short **rationale** and a **confidence** +(`high` / `medium` / `low`). + +### Blocking clarifications +If any assumption is **low-confidence AND affects correctness, security, or API behavior**, mark +it explicitly with a leading `blocking: true` token on that line, e.g.: + +``` +- blocking: true | We assume tokens are validated upstream | rationale: no validation found in scope | confidence: low +``` + +The orchestrator pauses the run and asks the human when any `blocking: true` assumption exists, +rather than inventing an answer. Use this sparingly and only when it genuinely gates correctness. + +## Constraints +- Read-only; no source edits, no shell. +- Be concrete and tied to the specs/codebase, not generic. +- End your turn once `assumptions.md` is written. diff --git a/tools/agentic-workflow/prompts/03-classify.md b/tools/agentic-workflow/prompts/03-classify.md new file mode 100644 index 00000000000..ba416e7a682 --- /dev/null +++ b/tools/agentic-workflow/prompts/03-classify.md @@ -0,0 +1,42 @@ +# Phase 3 — Classify & split (read-only) + +You are in the **classify** phase. Classify the task and split it into **independent, +non-overlapping** sub-items. You are **read-only**. + +## Task +{{task}} + +## Inputs +- `specs/*` and `assumptions.md` if present. {{researchNote}} + +## Outputs — via the `write_artifact` tool +1. `classification.md` — human-readable: the overall classification + (`feature` / `bug` / `refactor` / `mixed`) and the reasoning for the split. +2. `subitems.json` — machine-readable, matching this shape exactly: + ```json + { + "task": "string — original task description", + "classification": "feature | bug | refactor | mixed", + "items": [ + { + "id": "kebab-case-id", + "type": "feature | bug | refactor", + "title": "short title", + "description": "what this sub-item covers", + "rationale": "why it is a separate item", + "dependsOn": ["other-item-id"], + "expectedFilesOrAreas": ["src/api/**"], + "acceptanceCriteria": ["..."], + "nonGoals": ["..."], + "overlapRisk": "low | medium | high" + } + ] + } + ``` + +## Constraints +- Aim for sub-items that are **independent and non-overlapping**; populate `dependsOn` and + `overlapRisk` honestly so later phases can order/parallelize safely. +- `id` values are unique, kebab-case. `items` must be non-empty. +- Read-only; no source edits, no shell. +- End your turn once both artifacts are written. diff --git a/tools/agentic-workflow/prompts/04-research-item.md b/tools/agentic-workflow/prompts/04-research-item.md new file mode 100644 index 00000000000..7006013a36a --- /dev/null +++ b/tools/agentic-workflow/prompts/04-research-item.md @@ -0,0 +1,26 @@ +# Phase 4 — Research a sub-item (read-only) + +You are in the **research-item** phase. Do deep, isolated research on a **single** sub-item, +using the specs as context. You are **read-only**. + +## Original task +{{task}} + +## This sub-item +```json +{{item}} +``` + +## Inputs +- `specs/*` if present. {{researchNote}} +- Prior sibling research notes this item depends on: {{dependsOnNote}} + +## Output — via the `write_artifact` tool +`research/{{itemId}}.md` — a focused research note for **this item only**: the exact files and +symbols it will touch, the current behavior in that area, constraints, edge cases, and any +sequencing concerns relative to sibling items. Cite real code locations. + +## Constraints +- Stay scoped to **this** sub-item; do not research the whole task. +- Read-only; no source edits, no shell. +- End your turn once the note is written. diff --git a/tools/agentic-workflow/prompts/05-plan.md b/tools/agentic-workflow/prompts/05-plan.md new file mode 100644 index 00000000000..a11cfb36aa3 --- /dev/null +++ b/tools/agentic-workflow/prompts/05-plan.md @@ -0,0 +1,63 @@ +# Phase 5 — Plan (read-only) + +You are in the **plan** phase. Produce a highly detailed, structured implementation plan. This is +the highest-leverage artifact in the run. You are **read-only** — you plan, you do not implement. + +## Task +{{task}} + +## Inputs +- `specs/*`, `assumptions.md`, and all `research/*.md` notes that exist. {{researchNote}} + +## Output — via the `write_artifact` tool +`plan.md`, containing these sections **in order**: + +0. **Research reconciliation** — reconcile the independent phase-4 notes: overlaps, + contradictions, duplicated work, and the single coherent strategy chosen. (If there is only + one sub-item, say so; do not invent overlaps.) +1. **Decisions and rationale** — the choices made and why. +2. **End-to-end approach** — the overall strategy, and explicitly how the success criterion will + be *proved*. +3. **Step-by-step implementation plan** — ordered, concrete, file-by-file steps, grouped into + stages (each stage ends in a checkpoint gate). +4. **Stop/go gates** — explicit points where work pauses for validation. +5. **Validation plan** — tests to run, tests to add, observability checks. +6. **Rollout strategy**. +7. **Rollback plan**. +8. **Risks and mitigations**. +9. **Definition of done** — concrete, checkable completion criteria. +10. **Open questions**. +11. **Out-of-scope observations**. +12. **Plan changes** — leave empty (`_none yet_`); phase 6 appends here when implementation + deviates from the plan. + +## Machine-readable gate block (REQUIRED) + +In addition to the prose, embed **exactly one** fenced `yaml` block with a top-level `stages:` +key. The orchestrator parses this; the implement agent runs each stage's `gate.commands` and +reports pass/fail. Shape: + +```yaml +stages: + - id: stage-1 + expected_files: ["src/foo.ts", "test/foo.test.ts"] # anticipated scope (advisory) + context_needed: ["src/bar.ts", "src/types.ts"] # existing files this stage depends on + steps: + - { id: "1.1", description: "..." } + gate: + id: gate-1 + commands: ["npm test -- foo"] # the agent runs these in-session + expected: exit_code_0 +``` + +### Stage sizing +Split work into **cohesive, loosely-coupled** stages — each independently completable and +verifiable by its gate — to minimize what crosses a session boundary. Tightly-coupled changes +that only make sense together belong in **one** stage. Every stage MUST have a `gate` with at +least one command. + +## Constraints +- Read-only; no source edits, no shell. +- The gate block must be valid YAML and parse into at least one stage, each with an `id` and a + `gate.commands` array. +- End your turn once `plan.md` is written. diff --git a/tools/agentic-workflow/prompts/06-implement.md b/tools/agentic-workflow/prompts/06-implement.md new file mode 100644 index 00000000000..c5b9a27fae5 --- /dev/null +++ b/tools/agentic-workflow/prompts/06-implement.md @@ -0,0 +1,52 @@ +# Phase 6 — Implement a single stage + +You are implementing **one stage** of the plan in a fresh session. You **may edit source code and +run shell commands** for this stage only. Earlier stages already ran; their code is on disk. + +## Task +{{task}} + +## This stage +```yaml +{{stage}} +``` + +## Context pack (clean ≠ blank — everything you need has been gathered for you) +- **`plan.md`** is the source of truth (read it, including the running "Plan changes" log). +- **Prior handoffs** from earlier stages: +{{handoff}} +- **Cumulative diff so far** (files already changed in this run): +{{cumulativeDiff}} +- **`context_needed`** files this stage depends on: {{contextNeeded}} + +## What to do +1. Implement **this stage's steps**, editing the real source files. `expected_files` is the + anticipated scope — **advisory, not a wall**. You may edit beyond it when correctness requires + it, **provided you document the deviation** (see below). +2. **Run this stage's `gate.commands`** yourself (shell tool) and confirm they meet `expected` + (e.g. exit code 0). If a gate fails, fix the code and re-run until it passes or you are + genuinely blocked. +3. Append to **`execution-log.md`** (via `write_artifact`, appending) — for **every** action: + (a) **justify the scope** (why it was necessary) and (b) **map it to a concrete step** in + `plan.md` by stage/step id. Record each gate command, its exit code, and pass/fail. +4. Append a concise entry to **`handoff.md`** (via `write_artifact`, appending) for the *next* + stage: what you built, the **new/changed public symbols and files**, decisions/conventions + established, anything deferred, and known follow-ups. + +## Deviation policy (documented, not blocked) +Deviating from the plan is **allowed** when it is the correct response to a gap. The requirement +is transparency: +- Append a **"Plan changes"** entry to `plan.md` describing what changed and why. +- The `execution-log.md` entry for the deviating action justifies its scope and maps it to the + originating step / plan-change entry. +- **Larger deviations** — a change to architecture, public API, or overall test strategy — must + be called out prominently at the top of your handoff so the orchestrator/human can review; + if the divergence invalidates the plan, say so explicitly. +- Silent, undocumented scope expansion is **not** allowed. + +## Report at the end of your turn +End with a short, explicit status line the orchestrator reads: +- `STAGE_RESULT: pass` if every gate command met `expected`, or +- `STAGE_RESULT: fail — ` otherwise. + +{{priorErrors}} diff --git a/tools/agentic-workflow/prompts/critique.md b/tools/agentic-workflow/prompts/critique.md new file mode 100644 index 00000000000..e1951355e91 --- /dev/null +++ b/tools/agentic-workflow/prompts/critique.md @@ -0,0 +1,26 @@ +# Judge — Critique (read-only, alternate model) + +You are a **critic** running in a fresh, isolated session on a **different model** than the author, +so you do not share the author's blind spots. Your job is to find real problems in one artifact — +**high-signal feedback only**. + +## Artifact under review +`{{artifactPath}}` (read it). + +## Original task (for context) +{{task}} + +## What to report +Only substantive issues that would make the downstream work wrong, unsafe, or incomplete: +- **Bugs / logic errors** in the reasoning or proposed approach. +- **Gaps** — missing cases, missing inputs/outputs, unhandled risks. +- **Contract violations** — the artifact does not satisfy its phase's required format/sections. +- **Design flaws** — choices that will cause rework or break a stated constraint. + +**Do NOT** comment on style, wording, formatting, or trivia. If the artifact is sound, say so and +keep it short — do not invent problems. + +## Output — via the `write_artifact` tool +`critiques/{{artifactName}}.md` — a terse list. For each point: a one-line description, the +**severity** (`blocker` / `should-fix` / `optional`), and the specific location/section it refers +to. End your turn once written. diff --git a/tools/agentic-workflow/prompts/revise.md b/tools/agentic-workflow/prompts/revise.md new file mode 100644 index 00000000000..2913b7f890f --- /dev/null +++ b/tools/agentic-workflow/prompts/revise.md @@ -0,0 +1,30 @@ +# Judge — Adjudicate & revise (author's model) + +You are the **owner** of an artifact deciding which critique points to apply. You are not a rubber +stamp: apply the points that genuinely improve correctness/completeness, and **reject** points you +disagree with — but you must justify each rejection. + +## Artifact +`{{artifactPath}}` (read it — this is your original work). + +## Critique +`{{critiquePath}}` (read it). + +## Original task (for context) +{{task}} + +## What to do +1. For **each** critique point, decide **accept** or **reject** with a one-line reason. +2. Rewrite the artifact via the `write_artifact` tool at the **same path** `{{artifactPath}}`, + incorporating every accepted point. Preserve the phase's required format/sections exactly — the + revised artifact is re-validated, and a malformed revision triggers a fresh-session retry. +3. End your turn with an explicit adjudication summary the orchestrator logs: + ``` + ADJUDICATION: + - : accepted — + - : rejected — + ``` + +## Constraints +- Only write the artifact itself (read-only otherwise; no source edits, no shell). +- Do not regress correct content while applying critique points. diff --git a/tools/agentic-workflow/spike/FINDINGS.md b/tools/agentic-workflow/spike/FINDINGS.md new file mode 100644 index 00000000000..0442ac7ce16 --- /dev/null +++ b/tools/agentic-workflow/spike/FINDINGS.md @@ -0,0 +1,39 @@ +# T0 Capability Spike — Findings (G0) + +Verified **live** against `@github/copilot-sdk@1.0.4` (Node 25, logged-in user auth) via +`spike.mjs` + `tool-test.mjs`. Model used: `claude-haiku-4.5`. + +| ID | Capability | Result | Evidence | +| --- | --- | --- | --- | +| T0.1 | `createSession()` headless, streams, stops cleanly; `approveAll` | ✅ | sessions created/answered/disconnected; `client.stop()` clean | +| T0.2 | **Isolation** via single-use nonce | ✅ | nonce `ISO-…` told to session A; fresh session B replied `NONE` (no bleed) | +| **T0.3** | **`createSession()` accepts `hooks` (`onPreToolUse`)** | ✅ **YES** | `hooks.onPreToolUse` returning `{permissionDecision:"deny"}` blocked a shell call; agent reported it could not run the command | +| T0.4 | ≥3 concurrent sessions (phase-4 fan-out) | ✅ | 3 simultaneous sessions each returned correct distinct answers | +| T0.5 | Spawn **named built-in** agent (e.g. `rubber-duck`) + per-session model override | ⚠️ Partial | per-session `model` override ✅; but `SessionConfig.agent` **only references `customAgents[]`** you define — no direct built-in-agent spawn by name | + +Custom tool: `defineTool(name, { parameters, handler, skipPermission:true })` works; the agent +called `write_artifact` and the handler fired. `defineTool`'s name is the **first positional arg**. + +## Decisions gated by this spike + +1. **Enforcement path = hooks (`onPreToolUse` → `deny`).** T0.3 is YES, so read-only enforcement + for non-impl phases is a hook that denies edit/shell tools. **`FALLBACK.md` read-only-checkout + machinery is dropped entirely.** The post-phase git-diff guard remains as a cheap backstop only. +2. **Judge critique uses the `critique.md` template on an *alternate model*** (not a built-in + `rubber-duck` spawn), because `agent` can't name a built-in agent. Per-session model override — + the part actually required for judge diversity — is confirmed working. +3. **`write_artifact`** is a `defineTool` custom tool with `skipPermission: true`. +4. **Autonomous runs** use `onPermissionRequest: approveAll`. + +## API shape confirmed (from `dist/*.d.ts`, authoritative) + +- `client.createSession(config: SessionConfig)`; `SessionConfig extends SessionConfigBase`. +- `SessionConfigBase.model?: string`; `SessionConfig.hooks?: SessionHooks` + (`onPreToolUse`, `onPostToolUse`, `onPostToolUseFailure`, `onErrorOccurred`). +- `onErrorOccurred` returns `{ errorHandling?: "retry" | "skip" | "abort" }` → transient retries. +- `onPreToolUse` returns `{ permissionDecision: "allow"|"deny"|"ask", permissionDecisionReason? }`. +- `Tool.skipPermission?: boolean`; `approveAll: PermissionHandler`. +- Events: `assistant.message`, `session.idle` (+ typed `tool.execution_complete`, etc.). +- Session lifecycle: `session.send({prompt})`, `session.disconnect()`, `client.stop()`. + +**G0 = PASS.** Isolation + permissions confirmed; enforcement path chosen (hooks). No `FALLBACK.md`. diff --git a/tools/agentic-workflow/spike/package.json b/tools/agentic-workflow/spike/package.json new file mode 100644 index 00000000000..8bbc8756056 --- /dev/null +++ b/tools/agentic-workflow/spike/package.json @@ -0,0 +1,16 @@ +{ + "name": "spike", + "version": "1.0.0", + "description": "", + "main": "index.js", + "scripts": { + "test": "echo \"Error: no test specified\" && exit 1" + }, + "keywords": [], + "author": "", + "license": "ISC", + "type": "commonjs", + "dependencies": { + "@github/copilot-sdk": "^1.0.4" + } +} diff --git a/tools/agentic-workflow/spike/spike.mjs b/tools/agentic-workflow/spike/spike.mjs new file mode 100644 index 00000000000..3fa38add2ed --- /dev/null +++ b/tools/agentic-workflow/spike/spike.mjs @@ -0,0 +1,81 @@ +// T0 capability spike — verifies the SDK assumptions the whole design rests on. +// Run: node spike.mjs (writes findings to stdout; auth via logged-in user) +import { CopilotClient, approveAll, defineTool } from "@github/copilot-sdk"; + +const MODEL = process.env.SPIKE_MODEL || "claude-haiku-4.5"; +const results = {}; + +function ask(session, prompt, timeoutMs = 60000) { + return new Promise(async (resolve, reject) => { + let text = ""; + const timer = setTimeout(() => reject(new Error("timeout")), timeoutMs); + session.on("assistant.message", (e) => { text += e.data.content ?? ""; }); + session.on("session.idle", () => { clearTimeout(timer); resolve(text); }); + await session.send({ prompt }); + }); +} + +const client = new CopilotClient(); +await client.start(); +try { + // T0.1 + T0.2: isolation via single-use nonce + const nonce = "ISO-" + Math.random().toString(36).slice(2, 10).toUpperCase(); + const a = await client.createSession({ model: MODEL, onPermissionRequest: approveAll }); + await ask(a, `Remember this exact token, I will quiz you later: ${nonce}. Reply only "ok".`); + const b = await client.createSession({ model: MODEL, onPermissionRequest: approveAll }); + const bAns = await ask(b, `Earlier in THIS conversation I gave you a token starting with "ISO-". Output it verbatim. If you have never seen it, output exactly NONE.`); + results.isolation = !bAns.includes(nonce); + results.isolationDetail = { nonce, fresh_session_reply: bAns.trim().slice(0, 120) }; + await a.disconnect(); + await b.disconnect(); + + // T0.3: hooks (onPreToolUse deny) accepted by createSession + let denyFired = false; + const h = await client.createSession({ + model: MODEL, + onPermissionRequest: approveAll, + hooks: { + onPreToolUse: async (inv) => { + if ((inv?.toolName || "").includes("shell") || (inv?.toolName || "").includes("bash")) { + denyFired = true; + return { permissionDecision: "deny", permissionDecisionReason: "spike: read-only phase" }; + } + return { permissionDecision: "allow" }; + }, + }, + }); + const hAns = await ask(h, `Run the shell command "echo hello-from-shell" and tell me the output.`); + results.hooks_accepted = true; // createSession did not throw on hooks + results.hook_deny_fired = denyFired; + results.hook_detail = hAns.trim().slice(0, 160); + await h.disconnect(); + + // T0.4: concurrency — 3 simultaneous sessions + const conc = await Promise.all([1, 2, 3].map(async (n) => { + const s = await client.createSession({ model: MODEL, onPermissionRequest: approveAll }); + const ans = await ask(s, `Reply with only the number ${n * 7}.`); + await s.disconnect(); + return ans.includes(String(n * 7)); + })); + results.concurrency = conc.every(Boolean); + + // T0.5: per-session model override + custom tool with skipPermission + let toolCalled = false; + const tool = defineTool({ + name: "write_artifact", + description: "Spike artifact writer", + parameters: { type: "object", properties: { content: { type: "string" } }, required: ["content"] }, + skipPermission: true, + handler: async ({ content }) => { toolCalled = true; return { content: [{ type: "text", text: "written" }] }; }, + }); + const t = await client.createSession({ model: MODEL, onPermissionRequest: approveAll, tools: [tool] }); + await ask(t, `Call the write_artifact tool with content "hello". Then reply "done".`); + results.custom_tool = toolCalled; + results.model_override = true; // session created with explicit model + await t.disconnect(); +} catch (e) { + results.error = String(e && e.stack ? e.stack : e); +} finally { + await client.stop(); +} +console.log("SPIKE_RESULTS " + JSON.stringify(results, null, 2)); diff --git a/tools/agentic-workflow/src/artifacts.ts b/tools/agentic-workflow/src/artifacts.ts new file mode 100644 index 00000000000..6196c562d50 --- /dev/null +++ b/tools/agentic-workflow/src/artifacts.ts @@ -0,0 +1,142 @@ +/** + * Artifact storage: resolve the run directory, write atomically, and keep the scratch tree out + * of git WITHOUT mutating any tracked file. + * + * Deliberately minimal (thinnerplan T1.3): no content hashing, no run lock yet — those are + * deferred until a real format/concurrency need appears. + */ +import { spawnSync } from "node:child_process"; +import { randomBytes } from "node:crypto"; +import * as fs from "node:fs"; +import * as path from "node:path"; + +export const ARTIFACT_ROOT = ".agentic-workflow"; + +/** Resolve the git repo root, or fall back to cwd when not in a git repo. */ +export function repoRoot(cwd: string = process.cwd()): string { + const r = spawnSync("git", ["rev-parse", "--show-toplevel"], { cwd, encoding: "utf8" }); + if (r.status === 0 && r.stdout.trim()) { + return r.stdout.trim(); + } + return cwd; +} + +export function isGitRepo(cwd: string = process.cwd()): boolean { + const r = spawnSync("git", ["rev-parse", "--is-inside-work-tree"], { cwd, encoding: "utf8" }); + return r.status === 0 && r.stdout.trim() === "true"; +} + +/** Slugify a task into a filesystem-safe fragment. */ +export function slugify(task: string, max = 32): string { + const slug = task + .toLowerCase() + .replace(/[^a-z0-9]+/g, "-") + .replace(/^-+|-+$/g, "") + .slice(0, max) + .replace(/-+$/g, ""); + return slug || "task"; +} + +/** Human-readable run id: `YYYYMMDD-HHMM-`. */ +export function makeRunId(task: string, now: Date = new Date()): string { + const pad = (n: number) => String(n).padStart(2, "0"); + const ts = + `${now.getFullYear()}${pad(now.getMonth() + 1)}${pad(now.getDate())}` + + `-${pad(now.getHours())}${pad(now.getMinutes())}`; + return `${ts}-${slugify(task)}`; +} + +export interface RunDir { + /** Absolute path to the run directory. */ + dir: string; + /** Absolute path to the artifact root (the locally-ignored scratch tree). */ + root: string; +} + +/** + * Resolve and create the run directory, applying local git-ignore without touching tracked files. + * When `outRoot` is provided and lives outside the default scratch root, ignoring is skipped (the + * user is explicitly choosing where artifacts land). + */ +export function ensureRunDir(runId: string, opts: { cwd?: string; outRoot?: string } = {}): RunDir { + const cwd = opts.cwd ?? process.cwd(); + const base = repoRoot(cwd); + const usingDefaultRoot = !opts.outRoot; + const root = path.resolve(base, opts.outRoot ?? ARTIFACT_ROOT); + const dir = path.join(root, runId); + fs.mkdirSync(dir, { recursive: true }); + + if (usingDefaultRoot) { + applyLocalIgnore(base, root); + } + return { dir, root }; +} + +/** + * Ignore the scratch tree two ways, neither of which edits the tracked root `.gitignore`: + * 1. `.git/info/exclude` — a local, untracked ignore file. + * 2. an inner `.agentic-workflow/.gitignore` (`*` + `!.gitignore`) that travels with the dir. + * Both no-op if already present or if this is not a git repo. + */ +function applyLocalIgnore(base: string, root: string): void { + const innerGitignore = path.join(root, ".gitignore"); + if (!fs.existsSync(innerGitignore)) { + fs.mkdirSync(root, { recursive: true }); + fs.writeFileSync(innerGitignore, "*\n!.gitignore\n", "utf8"); + } + + if (!isGitRepo(base)) { + return; + } + const excludePath = path.join(base, ".git", "info", "exclude"); + const rel = `${path.relative(base, root)}/`; + try { + let contents = fs.existsSync(excludePath) ? fs.readFileSync(excludePath, "utf8") : ""; + const lines = contents.split(/\r?\n/).map((l) => l.trim()); + if (!lines.includes(rel)) { + if (contents.length && !contents.endsWith("\n")) { + contents += "\n"; + } + contents += `${rel}\n`; + fs.mkdirSync(path.dirname(excludePath), { recursive: true }); + fs.writeFileSync(excludePath, contents, "utf8"); + } + } catch { + // Local ignore is best-effort; the inner .gitignore already covers the tree. + } +} + +/** Write a file atomically (temp file + rename) within the run dir, creating parents. */ +export function atomicWrite(filePath: string, content: string): void { + fs.mkdirSync(path.dirname(filePath), { recursive: true }); + const tmp = `${filePath}.${randomBytes(6).toString("hex")}.tmp`; + fs.writeFileSync(tmp, content, "utf8"); + fs.renameSync(tmp, filePath); +} + +/** + * Resolve a relative artifact path inside the run dir, rejecting traversal/absolute escapes. + * This is the guard behind the `write_artifact` custom tool. + */ +export function resolveInRunDir(runDir: string, relPath: string): string { + const normalized = path.normalize(relPath); + if (path.isAbsolute(normalized) || normalized.startsWith("..")) { + throw new Error(`write_artifact: path escapes run dir: ${relPath}`); + } + const abs = path.resolve(runDir, normalized); + const rel = path.relative(runDir, abs); + if (rel.startsWith("..") || path.isAbsolute(rel)) { + throw new Error(`write_artifact: path escapes run dir: ${relPath}`); + } + return abs; +} + +/** Append text to an artifact (used for execution-log.md / handoff.md). */ +export function appendArtifact(filePath: string, content: string): void { + fs.mkdirSync(path.dirname(filePath), { recursive: true }); + fs.appendFileSync(filePath, content, "utf8"); +} + +export function readIfExists(filePath: string): string | undefined { + return fs.existsSync(filePath) ? fs.readFileSync(filePath, "utf8") : undefined; +} diff --git a/tools/agentic-workflow/src/cli.ts b/tools/agentic-workflow/src/cli.ts new file mode 100644 index 00000000000..f7587fde8e8 --- /dev/null +++ b/tools/agentic-workflow/src/cli.ts @@ -0,0 +1,164 @@ +#!/usr/bin/env node +/** + * cli.ts — the primary, headless entry point (thinnerplan T3.1). + * + * Commands: + * run [task] [--simple] [--no-judge] [--judge-model ] [--out ] [--run-id ] + * resume [run-id] [--out ] + * + * Exit codes: 0 done / 10 paused / 1 fail / 2 usage. + */ +import * as fs from "node:fs"; +import * as path from "node:path"; +import { ARTIFACT_ROOT, repoRoot } from "./artifacts.js"; +import { SdkHarness } from "./harness.js"; +import { runWorkflow, type RunResult } from "./orchestrator.js"; +import { loadState } from "./state.js"; +import type { RunOptions } from "./types.js"; + +interface ParsedArgs { + command: string; + positional: string[]; + flags: Record; +} + +function parseArgs(argv: string[]): ParsedArgs { + const [command = "run", ...rest] = argv; + const positional: string[] = []; + const flags: Record = {}; + for (let i = 0; i < rest.length; i++) { + const a = rest[i]; + if (a.startsWith("--")) { + const key = a.slice(2); + const next = rest[i + 1]; + if (next !== undefined && !next.startsWith("--")) { + flags[key] = next; + i++; + } else { + flags[key] = true; + } + } else { + positional.push(a); + } + } + return { command, positional, flags }; +} + +const USAGE = `agentic-workflow — research -> plan -> implement, one fresh session per phase + +Usage: + agentic-workflow run "" [--simple] [--no-judge] [--judge-model ] [--out ] [--run-id ] + agentic-workflow resume [] [--out ] + +Options: + --simple Skip research, classify, per-item research (assumptions -> plan -> implement). + --no-judge Disable the reflexive critique/revise judge loop (on by default). + --judge-model Alternate model used for the critique session. + --out Working-dir root (default: ./.agentic-workflow). + --run-id Explicit run id (default: timestamp + task slug). + +Exit codes: 0 done, 10 paused (resume/clarify), 1 failure, 2 usage.`; + +function findResumableRun(outRoot: string): string[] { + if (!fs.existsSync(outRoot)) { + return []; + } + const incomplete: string[] = []; + for (const entry of fs.readdirSync(outRoot, { withFileTypes: true })) { + if (!entry.isDirectory()) continue; + const state = loadState(path.join(outRoot, entry.name)); + if (state && Object.values(state.phases).some((s) => s !== "completed")) { + incomplete.push(entry.name); + } + } + return incomplete; +} + +async function main(): Promise { + const { command, positional, flags } = parseArgs(process.argv.slice(2)); + + if (command === "help" || flags.help) { + console.log(USAGE); + return 0; + } + + const outRoot = typeof flags.out === "string" ? flags.out : undefined; + const resolvedRoot = path.resolve(repoRoot(), outRoot ?? ARTIFACT_ROOT); + + let opts: RunOptions; + if (command === "run") { + const task = positional.join(" ").trim(); + if (!task) { + console.error("error: `run` requires a task description.\n\n" + USAGE); + return 2; + } + opts = { + task, + simple: flags.simple === true, + judge: flags["no-judge"] !== true, + judgeModel: typeof flags["judge-model"] === "string" ? flags["judge-model"] : undefined, + outRoot, + runId: typeof flags["run-id"] === "string" ? flags["run-id"] : undefined, + }; + } else if (command === "resume") { + let runId = positional[0]; + if (!runId) { + const candidates = findResumableRun(resolvedRoot); + if (candidates.length === 0) { + console.error("error: no incomplete runs to resume under " + resolvedRoot); + return 2; + } + if (candidates.length > 1) { + console.error("error: multiple incomplete runs — pass an explicit run-id:"); + for (const c of candidates) { + const st = loadState(path.join(resolvedRoot, c)); + const last = st + ? Object.entries(st.phases) + .filter(([, v]) => v === "completed") + .map(([k]) => k) + .pop() + : "?"; + console.error(` ${c} (task: ${st?.task ?? "?"}, last completed: ${last ?? "none"})`); + } + return 2; + } + runId = candidates[0]; + } + const state = loadState(path.join(resolvedRoot, runId)); + if (!state) { + console.error(`error: no run state found for "${runId}" under ${resolvedRoot}`); + return 2; + } + opts = { + task: state.task, + simple: state.simple, + judge: state.judge, + judgeModel: state.judgeModel, + outRoot, + runId, + }; + } else { + console.error(`error: unknown command "${command}".\n\n` + USAGE); + return 2; + } + + const harness = new SdkHarness({ workingDirectory: process.cwd() }); + let result: RunResult; + try { + result = await runWorkflow(opts, harness); + } finally { + await harness.stop(); + } + + console.log(`\n${result.message}`); + console.log(`run-id: ${result.runId}`); + console.log(`artifacts: ${result.runDir}`); + return result.exitCode; +} + +main() + .then((code) => process.exit(code)) + .catch((err) => { + console.error(err); + process.exit(1); + }); diff --git a/tools/agentic-workflow/src/gates.ts b/tools/agentic-workflow/src/gates.ts new file mode 100644 index 00000000000..dc3a343039a --- /dev/null +++ b/tools/agentic-workflow/src/gates.ts @@ -0,0 +1,83 @@ +/** + * Parse the machine-readable `stages:` gate block from a generated `plan.md`. + * + * The orchestrator does NOT run the gate commands (thinnerplan §3): the implement agent runs them + * in-session and reports pass/fail. This module only *extracts and shapes* the stages so the + * orchestrator knows how many stages exist, their ids, expected files, and the commands the agent + * is expected to have run. + */ +import { parse as parseYaml } from "yaml"; +import type { Stage } from "./types.js"; + +/** Extract the first fenced ```yaml block that contains a top-level `stages:` key. */ +export function extractGateBlock(planMarkdown: string): string | undefined { + const fenceRe = /```ya?ml\s*\n([\s\S]*?)```/g; + let m: RegExpExecArray | null; + while ((m = fenceRe.exec(planMarkdown)) !== null) { + const body = m[1]; + if (/^\s*stages\s*:/m.test(body)) { + return body; + } + } + return undefined; +} + +/** + * Parse plan stages. Throws a readable error if the block is missing or malformed so the plan + * phase can be retried in a fresh session. + */ +export function parsePlanStages(planMarkdown: string): Stage[] { + const block = extractGateBlock(planMarkdown); + if (!block) { + throw new Error("plan.md has no machine-readable ```yaml stages: block"); + } + let doc: unknown; + try { + doc = parseYaml(block); + } catch (e) { + throw new Error(`plan.md gate block is not valid YAML: ${(e as Error).message}`); + } + const stagesRaw = (doc as { stages?: unknown })?.stages; + if (!Array.isArray(stagesRaw) || stagesRaw.length === 0) { + throw new Error("plan.md gate block must contain a non-empty `stages` array"); + } + + return stagesRaw.map((raw, i) => normalizeStage(raw, i)); +} + +function normalizeStage(raw: unknown, index: number): Stage { + const s = (raw ?? {}) as Record; + const id = typeof s.id === "string" && s.id.trim() ? s.id.trim() : `stage-${index + 1}`; + const gateRaw = (s.gate ?? {}) as Record; + const commands = Array.isArray(gateRaw.commands) + ? gateRaw.commands.filter((c): c is string => typeof c === "string") + : []; + if (commands.length === 0) { + throw new Error(`stage "${id}" has no gate.commands`); + } + const steps = Array.isArray(s.steps) + ? s.steps.map((st) => { + const so = (st ?? {}) as Record; + return { + id: typeof so.id === "string" ? so.id : "", + description: typeof so.description === "string" ? so.description : "", + }; + }) + : []; + + return { + id, + expected_files: toStringArray(s.expected_files), + context_needed: toStringArray(s.context_needed), + steps, + gate: { + id: typeof gateRaw.id === "string" ? gateRaw.id : `gate-${index + 1}`, + commands, + expected: typeof gateRaw.expected === "string" ? gateRaw.expected : "exit_code_0", + }, + }; +} + +function toStringArray(v: unknown): string[] { + return Array.isArray(v) ? v.filter((x): x is string => typeof x === "string") : []; +} diff --git a/tools/agentic-workflow/src/harness.ts b/tools/agentic-workflow/src/harness.ts new file mode 100644 index 00000000000..9d5fa0e14aa --- /dev/null +++ b/tools/agentic-workflow/src/harness.ts @@ -0,0 +1,108 @@ +/** + * harness.ts — *** the ONLY file that imports @github/copilot-sdk *** + * + * It wraps `createSession` with the shared session options (write_artifact tool, onPreToolUse + * read-only policy, onErrorOccurred retry, event -> execution-log.jsonl capture) and exposes a + * single stable internal interface, `Harness.runPhase`, that the orchestrator talks to. Harness + * churn is contained here: re-point this file at a new SDK/primitive and the orchestrator is + * unchanged. + */ +import { CopilotClient, approveAll } from "@github/copilot-sdk"; +import type { PhaseRunResult } from "./types.js"; +import { + ensureLog, + logEvent, + makeErrorHook, + makePreToolUseHook, + makeWriteArtifactTool, + redact, +} from "./session-options.js"; + +export interface PhaseRequest { + /** Stable label for the log (e.g. "research", "plan", "implement:stage-1", "critique"). */ + label: string; + prompt: string; + runDir: string; + /** Path to the run's execution-log.jsonl. */ + logPath: string; + /** Read-only phases deny mutating/shell tools. Implement stages are NOT read-only. */ + readOnly: boolean; + model?: string; + /** Hard ceiling on a single phase session (ms). */ + timeoutMs?: number; +} + +/** The stable seam between orchestrator and the SDK. Test doubles implement this. */ +export interface Harness { + runPhase(req: PhaseRequest): Promise; + stop(): Promise; +} + +const DEFAULT_TIMEOUT_MS = 15 * 60 * 1000; + +export class SdkHarness implements Harness { + private client: CopilotClient | undefined; + + constructor(private readonly defaults: { workingDirectory?: string } = {}) {} + + private async getClient(): Promise { + if (!this.client) { + this.client = new CopilotClient({ workingDirectory: this.defaults.workingDirectory }); + await this.client.start(); + } + return this.client; + } + + async runPhase(req: PhaseRequest): Promise { + ensureLog(req.logPath); + const client = await this.getClient(); + logEvent(req.logPath, { kind: "phase_start", label: req.label, readOnly: req.readOnly, model: req.model }); + + const session = await client.createSession({ + model: req.model, + onPermissionRequest: approveAll, + tools: [makeWriteArtifactTool(req.runDir, req.logPath)], + hooks: { + onPreToolUse: makePreToolUseHook({ readOnly: req.readOnly, logPath: req.logPath }), + onErrorOccurred: makeErrorHook(req.logPath), + }, + }); + + let finalText = ""; + let toolCalls = 0; + session.on("assistant.message", (e) => { + finalText += e.data?.content ?? ""; + }); + session.on("tool.execution_complete", (e) => { + toolCalls += 1; + logEvent(req.logPath, { kind: "tool_complete", success: e.data?.success, toolCallId: e.data?.toolCallId }); + }); + + const timeoutMs = req.timeoutMs ?? DEFAULT_TIMEOUT_MS; + try { + await new Promise((resolve, reject) => { + const timer = setTimeout(() => reject(new Error(`phase "${req.label}" timed out`)), timeoutMs); + session.on("session.idle", () => { + clearTimeout(timer); + resolve(); + }); + session.on("session.error", (e) => { + logEvent(req.logPath, { kind: "session_error", detail: redact(JSON.stringify(e.data ?? {})) }); + }); + session.send({ prompt: req.prompt }).catch(reject); + }); + } finally { + await session.disconnect().catch(() => {}); + } + + logEvent(req.logPath, { kind: "phase_end", label: req.label, toolCalls }); + return { artifacts: [], finalText }; + } + + async stop(): Promise { + if (this.client) { + await this.client.stop().catch(() => {}); + this.client = undefined; + } + } +} diff --git a/tools/agentic-workflow/src/index.ts b/tools/agentic-workflow/src/index.ts new file mode 100644 index 00000000000..59e29dbc019 --- /dev/null +++ b/tools/agentic-workflow/src/index.ts @@ -0,0 +1,7 @@ +/** + * Public entry: programmatic API for the agentic workflow. + * The CLI (`cli.ts`) and the extension front-door both call `runWorkflow`. + */ +export { runWorkflow, type RunResult } from "./orchestrator.js"; +export { SdkHarness, type Harness, type PhaseRequest } from "./harness.js"; +export type { RunOptions, RunState } from "./types.js"; diff --git a/tools/agentic-workflow/src/orchestrator.ts b/tools/agentic-workflow/src/orchestrator.ts new file mode 100644 index 00000000000..e1aaaae3838 --- /dev/null +++ b/tools/agentic-workflow/src/orchestrator.ts @@ -0,0 +1,385 @@ +/** + * orchestrator.ts — sequences phases 1->6 through the stable `Harness` seam, reading/writing + * artifacts, validating each, retrying with a FRESH session on failure (the one enforced + * guarantee), fanning out phase 4, and running the reflexive judge loop on validated artifacts. + * + * It never imports the SDK — only the `Harness` interface — so harness churn cannot reach here. + */ +import { spawnSync } from "node:child_process"; +import * as fs from "node:fs"; +import * as path from "node:path"; +import { ensureRunDir, makeRunId, readIfExists } from "./artifacts.js"; +import { parsePlanStages } from "./gates.js"; +import type { Harness } from "./harness.js"; +import { renderTemplate, type PromptName } from "./prompts.js"; +import { ensureLog, logEvent } from "./session-options.js"; +import { initState, loadState, PHASE_ORDER, saveState, setPhase } from "./state.js"; +import type { RunOptions, RunState, Stage, SubItems, ValidationResult } from "./types.js"; +import { validatePlan, validateSubitemsJson } from "./validate.js"; + +export interface RunResult { + /** 0 done, 10 paused, 1 fail, 2 usage. */ + exitCode: number; + runId: string; + runDir: string; + message: string; +} + +/** Per-phase model defaults (overridable). Cheap where possible, strong where it matters. */ +const PHASE_MODELS: Record = { + research: "claude-sonnet-4.5", + assumptions: undefined, + classify: "claude-haiku-4.5", + "research-item": "claude-sonnet-4.5", + plan: "claude-sonnet-4.5", + implement: "claude-sonnet-4.5", + critique: "claude-haiku-4.5", +}; + +export async function runWorkflow(opts: RunOptions, harness: Harness): Promise { + const runId = opts.runId ?? makeRunId(opts.task); + const { dir: runDir } = ensureRunDir(runId, { outRoot: opts.outRoot }); + const logPath = path.join(runDir, "execution-log.jsonl"); + ensureLog(logPath); + + const simple = !!opts.simple; + const judge = opts.judge !== false; + const maxRetries = opts.maxRetries ?? 2; + const concurrency = opts.concurrency ?? 3; + + let state = loadState(runDir); + if (!state) { + state = initState(runDir, { runId, task: opts.task, simple, judge, judgeModel: opts.judgeModel }); + } + + const done = (phase: string) => state!.phases[phase] === "completed"; + const researchNote = simple + ? "Research was skipped (--simple); read the task description and codebase directly." + : ""; + + /** Run one phase: render template -> harness.runPhase -> validate; retry FRESH on failure. */ + const runPhase = async ( + phase: string, + template: PromptName, + vars: Record, + validate: () => ValidationResult, + readOnly: boolean, + label = phase, + ): Promise => { + setPhase(runDir, state!, phase, "in_progress"); + let priorErrors = ""; + for (let attempt = 0; attempt <= maxRetries; attempt++) { + const prompt = renderTemplate(template, { ...vars, priorErrors }); + await harness.runPhase({ + label: attempt === 0 ? label : `${label} (retry ${attempt})`, + prompt, + runDir, + logPath, + readOnly, + model: resolveModel(phase, opts.judgeModel, opts), + }); + const result = validate(); + if (result.ok) { + setPhase(runDir, state!, phase, "completed"); + return true; + } + priorErrors = + `\n## Your previous attempt failed validation. Fix exactly these and rewrite the artifact(s):\n` + + result.errors.map((e) => `- ${e}`).join("\n") + + `\n`; + logEvent(logPath, { kind: "validation_failed", phase, attempt, errors: result.errors }); + } + setPhase(runDir, state!, phase, "failed"); + return false; + }; + + const exists = (rel: string) => fs.existsSync(path.join(runDir, rel)); + const fail = (message: string): RunResult => ({ exitCode: 1, runId, runDir, message }); + const paused = (message: string): RunResult => ({ exitCode: 10, runId, runDir, message }); + + // ---- Phase 1: research (skippable) ---- + if (!simple && !done("research")) { + const ok = await runPhase( + "research", + "01-research", + { task: opts.task }, + () => + exists("specs/architecture.md") && exists("specs/functional.md") && exists("manifest.json") + ? { ok: true, errors: [] } + : { ok: false, errors: ["expected specs/architecture.md, specs/functional.md, manifest.json"] }, + true, + ); + if (!ok) return fail("research phase failed validation after retries"); + if (judge) + await judgeArtifact(harness, runDir, logPath, "specs/architecture.md", opts, () => ({ + ok: true, + errors: [], + })); + } else if (simple) { + setPhase(runDir, state, "research", "completed"); + } + + // ---- Phase 2: assumptions ---- + if (!done("assumptions")) { + const ok = await runPhase( + "assumptions", + "02-assumptions", + { task: opts.task, researchNote }, + () => + exists("assumptions.md") + ? { ok: true, errors: [] } + : { ok: false, errors: ["assumptions.md not written"] }, + true, + ); + if (!ok) return fail("assumptions phase failed validation after retries"); + if (judge) + await judgeArtifact(harness, runDir, logPath, "assumptions.md", opts, () => ({ ok: true, errors: [] })); + + // Blocking-clarification stop (fires regardless of mode). + const assumptions = readIfExists(path.join(runDir, "assumptions.md")) ?? ""; + if (/blocking:\s*true/i.test(assumptions)) { + logEvent(logPath, { kind: "blocking_clarification", phase: "assumptions" }); + return paused("A blocking assumption needs human clarification before continuing. See assumptions.md."); + } + } + + // ---- Phase 3: classify (skippable -> synthesize single-item subitems.json) ---- + if (!done("classify")) { + if (simple) { + const synthesized: SubItems = { + task: opts.task, + classification: "feature", + items: [ + { + id: "main", + type: "feature", + title: opts.task.slice(0, 60), + description: opts.task, + dependsOn: [], + overlapRisk: "low", + }, + ], + }; + fs.writeFileSync(path.join(runDir, "subitems.json"), JSON.stringify(synthesized, null, 2)); + setPhase(runDir, state, "classify", "completed"); + } else { + const ok = await runPhase( + "classify", + "03-classify", + { task: opts.task, researchNote }, + () => validateSubitemsJson(readIfExists(path.join(runDir, "subitems.json")) ?? ""), + true, + ); + if (!ok) return fail("classify phase failed validation after retries"); + } + } + + const subitems = JSON.parse(readIfExists(path.join(runDir, "subitems.json")) ?? "{}") as SubItems; + + // ---- Phase 4: research-item fan-out (skippable) ---- + if (!simple && !done("research-item")) { + setPhase(runDir, state, "research-item", "in_progress"); + const okAll = await fanOutResearch(harness, runDir, logPath, opts, subitems, concurrency, maxRetries); + if (!okAll) return fail("one or more research-item phases failed"); + setPhase(runDir, state, "research-item", "completed"); + } else if (simple) { + setPhase(runDir, state, "research-item", "completed"); + } + + // ---- Phase 5: plan ---- + if (!done("plan")) { + const ok = await runPhase( + "plan", + "05-plan", + { task: opts.task, researchNote }, + () => validatePlan(readIfExists(path.join(runDir, "plan.md")) ?? ""), + true, + ); + if (!ok) return fail("plan phase failed validation after retries"); + if (judge) { + await judgeArtifact(harness, runDir, logPath, "plan.md", opts, () => + validatePlan(readIfExists(path.join(runDir, "plan.md")) ?? ""), + ); + } + } + + // ---- Phase 6: staged implement ---- + if (!done("implement")) { + setPhase(runDir, state, "implement", "in_progress"); + let stages: Stage[]; + try { + stages = parsePlanStages(readIfExists(path.join(runDir, "plan.md")) ?? ""); + } catch (e) { + return fail(`cannot parse plan stages: ${(e as Error).message}`); + } + const handoffPath = path.join(runDir, "handoff.md"); + for (const stage of stages) { + const handoff = readIfExists(handoffPath) ?? "_(none yet)_"; + const cumulativeDiff = gitDiffStat(runDir); + const res = await harness.runPhase({ + label: `implement:${stage.id}`, + prompt: renderTemplate("06-implement", { + task: opts.task, + stage: JSON.stringify(stage, null, 2), + handoff, + cumulativeDiff, + contextNeeded: stage.context_needed.join(", "), + priorErrors: "", + }), + runDir, + logPath, + readOnly: false, + model: resolveModel("implement", opts.judgeModel, opts), + }); + // The agent runs gate.commands in-session and reports the result (low-maintenance path). + const reportedPass = /STAGE_RESULT:\s*pass/i.test(res.finalText); + logEvent(logPath, { kind: "stage_result", stage: stage.id, reportedPass }); + if (!reportedPass) { + setPhase(runDir, state, "implement", "failed"); + return fail(`stage ${stage.id} reported a failing gate; halting. See execution-log.jsonl.`); + } + } + setPhase(runDir, state, "implement", "completed"); + } + + saveState(runDir, state); + return { exitCode: 0, runId, runDir, message: `Run ${runId} complete.` }; +} + +function resolveModel(phase: string, _judgeModel: string | undefined, opts: RunOptions): string | undefined { + void opts; + return PHASE_MODELS[phase]; +} + +/** + * Reflexive judge loop (§3.1): critique on an ALTERNATE model (read-only), then adjudicate/revise + * on the author's model, then re-validate. A bad revision is left for the normal phase retry path + * (we do not regress: revision only replaces the artifact if it still validates). + */ +export async function judgeArtifact( + harness: Harness, + runDir: string, + logPath: string, + artifactRel: string, + opts: RunOptions, + revalidate: () => ValidationResult, +): Promise { + const artifactName = path.basename(artifactRel).replace(/\.[^.]+$/, ""); + const critiqueRel = `critiques/${artifactName}.md`; + const judgeModel = opts.judgeModel ?? PHASE_MODELS.critique; + + await harness.runPhase({ + label: `critique:${artifactName}`, + prompt: renderTemplate("critique", { task: opts.task, artifactPath: artifactRel, artifactName }), + runDir, + logPath, + readOnly: true, + model: judgeModel, + }); + if (!fs.existsSync(path.join(runDir, critiqueRel))) { + logEvent(logPath, { kind: "judge_skipped", artifact: artifactRel, reason: "no critique produced" }); + return; + } + + const before = readIfExists(path.join(runDir, artifactRel)) ?? ""; + await harness.runPhase({ + label: `revise:${artifactName}`, + prompt: renderTemplate("revise", { + task: opts.task, + artifactPath: artifactRel, + critiquePath: critiqueRel, + }), + runDir, + logPath, + readOnly: true, + model: PHASE_MODELS.plan, + }); + + // Don't accept a revision that broke validation: roll back to the pre-revision content. + const result = revalidate(); + if (!result.ok) { + fs.writeFileSync(path.join(runDir, artifactRel), before); + logEvent(logPath, { kind: "revision_rolled_back", artifact: artifactRel, errors: result.errors }); + } else { + logEvent(logPath, { kind: "judge_complete", artifact: artifactRel }); + } +} + +/** Fan out phase-4 research, honoring dependsOn ordering with bounded concurrency. */ +async function fanOutResearch( + harness: Harness, + runDir: string, + logPath: string, + opts: RunOptions, + subitems: SubItems, + concurrency: number, + maxRetries: number, +): Promise { + const items = subitems.items; + const completed = new Set(); + const failed = new Set(); + const research = (id: string) => path.join(runDir, "research", `${id}.md`); + + while (completed.size + failed.size < items.length) { + const ready = items.filter( + (it) => + !completed.has(it.id) && + !failed.has(it.id) && + it.dependsOn.every((d) => completed.has(d) || !items.some((x) => x.id === d)), + ); + if (ready.length === 0) { + // remaining items are blocked by failed deps + items.filter((it) => !completed.has(it.id)).forEach((it) => failed.add(it.id)); + break; + } + const batch = ready.slice(0, concurrency); + const results = await Promise.all( + batch.map(async (it) => { + const dependsOnNote = it.dependsOn.length + ? it.dependsOn.map((d) => `research/${d}.md`).join(", ") + : "none"; + let priorErrors = ""; + for (let attempt = 0; attempt <= maxRetries; attempt++) { + await harness.runPhase({ + label: `research-item:${it.id}${attempt ? ` (retry ${attempt})` : ""}`, + prompt: renderTemplate("04-research-item", { + task: opts.task, + item: JSON.stringify(it, null, 2), + itemId: it.id, + dependsOnNote, + researchNote: "", + priorErrors, + }), + runDir, + logPath, + readOnly: true, + model: PHASE_MODELS["research-item"], + }); + if (fs.existsSync(research(it.id))) { + return { id: it.id, ok: true }; + } + priorErrors = `\nYour previous attempt did not write research/${it.id}.md. Write it now.\n`; + } + return { id: it.id, ok: false }; + }), + ); + for (const r of results) { + if (r.ok) completed.add(r.id); + else failed.add(r.id); + } + } + return failed.size === 0; +} + +/** Cumulative diff summary for the implement context pack (best-effort; empty when not a repo). */ +function gitDiffStat(cwd: string): string { + try { + const r = spawnSync("git", ["diff", "--stat"], { cwd, encoding: "utf8" }); + return r.status === 0 ? r.stdout.trim() || "(no changes yet)" : "(diff unavailable)"; + } catch { + return "(diff unavailable)"; + } +} + +export { PHASE_ORDER }; +export type { RunState }; diff --git a/tools/agentic-workflow/src/prompts.ts b/tools/agentic-workflow/src/prompts.ts new file mode 100644 index 00000000000..2461c346dd2 --- /dev/null +++ b/tools/agentic-workflow/src/prompts.ts @@ -0,0 +1,34 @@ +/** + * Load + render the durable prompt templates with run variables. Trivial `{{var}}` substitution — + * no template engine, by design. + */ +import * as fs from "node:fs"; +import { fileURLToPath } from "node:url"; +import * as path from "node:path"; + +const here = path.dirname(fileURLToPath(import.meta.url)); +// Templates ship next to the package (../prompts relative to dist/ or src/). +const PROMPTS_DIR = path.resolve(here, "..", "prompts"); + +export type PromptName = + | "01-research" + | "02-assumptions" + | "03-classify" + | "04-research-item" + | "05-plan" + | "06-implement" + | "critique" + | "revise"; + +export function loadTemplate(name: PromptName, dir: string = PROMPTS_DIR): string { + return fs.readFileSync(path.join(dir, `${name}.md`), "utf8"); +} + +/** Replace every `{{key}}` with vars[key]; unknown keys render as an empty string. */ +export function render(template: string, vars: Record): string { + return template.replace(/\{\{\s*([a-zA-Z0-9_]+)\s*\}\}/g, (_, key: string) => vars[key] ?? ""); +} + +export function renderTemplate(name: PromptName, vars: Record, dir?: string): string { + return render(loadTemplate(name, dir), vars); +} diff --git a/tools/agentic-workflow/src/session-options.ts b/tools/agentic-workflow/src/session-options.ts new file mode 100644 index 00000000000..123afd2ca66 --- /dev/null +++ b/tools/agentic-workflow/src/session-options.ts @@ -0,0 +1,129 @@ +/** + * Shared session wiring: the `write_artifact` custom tool, the `onPreToolUse` read-only policy, + * the `onErrorOccurred` retry hook, and event -> `execution-log.jsonl` capture (with trivial + * secret redaction baked in from the start). + * + * This module builds plain config fragments; it does NOT import the SDK directly so it can be + * unit-tested. `harness.ts` is the only file that imports `@github/copilot-sdk` and stitches + * these fragments into a real session. + */ +import { defineTool, type Tool } from "@github/copilot-sdk"; +import * as fs from "node:fs"; +import { appendArtifact, atomicWrite, resolveInRunDir } from "./artifacts.js"; + +/** Tool-name fragments that mutate state or execute code. Denied in read-only phases. */ +const MUTATING_TOOL_RE = + /(shell|bash|process|execute|run_command|str_replace|apply_patch|create_file|write_file|edit_file|delete|remove|^edit$|^create$|^write$)/i; + +/** Trivial redaction: mask obvious secret-looking tokens before they hit the log. */ +export function redact(text: string): string { + if (!text) { + return text; + } + return text + .replace(/\bgh[pousr]_[A-Za-z0-9]{20,}\b/g, "[REDACTED_GH_TOKEN]") + .replace(/\bsk-[A-Za-z0-9]{20,}\b/g, "[REDACTED_API_KEY]") + .replace(/\bAKIA[0-9A-Z]{16}\b/g, "[REDACTED_AWS_KEY]") + .replace(/\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b/g, "[REDACTED_JWT]") + .replace(/(?<=(?:password|secret|token|api[_-]?key)["'\s:=]{1,4})(?!\[REDACTED)[^\s"']{8,}/gi, "[REDACTED]"); +} + +/** Append one structured, redacted JSONL record to the run's execution log. */ +export function logEvent(logPath: string, record: Record): void { + const line = redact(JSON.stringify({ ts: new Date().toISOString(), ...record })) + "\n"; + appendArtifact(logPath, line); +} + +/** + * Build the `write_artifact` tool bound to a run dir. It is the only sanctioned write path for + * read-only phases (path-traversal guarded). Supports append mode for `execution-log.md` / + * `handoff.md` in the implement phase. + */ +export function makeWriteArtifactTool(runDir: string, logPath: string): Tool { + return defineTool("write_artifact", { + description: + "Persist a workflow artifact to the run directory. This is the ONLY way to write " + + "specs/notes/plan/log files. `path` is relative to the run dir.", + parameters: { + type: "object", + properties: { + path: { type: "string", description: "Relative path within the run dir, e.g. specs/architecture.md" }, + content: { type: "string", description: "Full file content (or text to append when append=true)" }, + append: { type: "boolean", description: "Append instead of overwrite (for logs/handoff)" }, + }, + required: ["path", "content"], + }, + skipPermission: true, + handler: (args: unknown) => { + const { + path: rel, + content, + append, + } = (args ?? {}) as { + path?: string; + content?: string; + append?: boolean; + }; + if (typeof rel !== "string" || typeof content !== "string") { + return { + content: [{ type: "text", text: "write_artifact: `path` and `content` are required strings" }], + }; + } + let abs: string; + try { + abs = resolveInRunDir(runDir, rel); + } catch (e) { + return { content: [{ type: "text", text: (e as Error).message }] }; + } + if (append) { + appendArtifact(abs, content.endsWith("\n") ? content : content + "\n"); + } else { + atomicWrite(abs, content); + } + logEvent(logPath, { kind: "artifact_write", path: rel, append: !!append, bytes: content.length }); + return { content: [{ type: "text", text: `wrote ${rel} (${content.length} bytes)` }] }; + }, + }); +} + +export interface PolicyOptions { + /** Read-only phases deny mutating/shell tools via onPreToolUse. */ + readOnly: boolean; + logPath: string; +} + +/** Build the `onPreToolUse` hook enforcing the per-phase policy (deny in read-only phases). */ +export function makePreToolUseHook(opts: PolicyOptions) { + return (input: { toolName: string; toolArgs?: unknown }) => { + const name = input.toolName ?? ""; + if (name === "write_artifact") { + return { permissionDecision: "allow" as const }; + } + if (opts.readOnly && MUTATING_TOOL_RE.test(name)) { + logEvent(opts.logPath, { kind: "policy_deny", tool: name, reason: "read-only phase" }); + return { + permissionDecision: "deny" as const, + permissionDecisionReason: + "This phase is read-only. Use the write_artifact tool to persist artifacts; " + + "source edits and shell are not permitted here.", + }; + } + return { permissionDecision: "allow" as const }; + }; +} + +/** Build the `onErrorOccurred` hook: retry transient model/tool errors, abort on the rest. */ +export function makeErrorHook(logPath: string) { + return (input: { errorContext?: string; error?: unknown }) => { + const transient = input.errorContext === "model_call" || input.errorContext === "tool_execution"; + logEvent(logPath, { kind: "error", context: input.errorContext, transient }); + return { errorHandling: (transient ? "retry" : "abort") as "retry" | "abort" }; + }; +} + +/** Ensure a log file exists so appends start from a known place. */ +export function ensureLog(logPath: string): void { + if (!fs.existsSync(logPath)) { + atomicWrite(logPath, ""); + } +} diff --git a/tools/agentic-workflow/src/state.ts b/tools/agentic-workflow/src/state.ts new file mode 100644 index 00000000000..99c3b1af450 --- /dev/null +++ b/tools/agentic-workflow/src/state.ts @@ -0,0 +1,53 @@ +/** + * Run state persistence. `state.json` is written atomically after every phase so `resume` can + * continue from the last completed phase. No content-hash revalidation engine yet (deferred per + * thinnerplan §1) — resume re-runs from the last completed phase. + */ +import * as fs from "node:fs"; +import * as path from "node:path"; +import { atomicWrite } from "./artifacts.js"; +import type { PhaseStatus, RunState } from "./types.js"; + +export const PHASE_ORDER = ["research", "assumptions", "classify", "research-item", "plan", "implement"] as const; + +export function statePath(runDir: string): string { + return path.join(runDir, "state.json"); +} + +export function loadState(runDir: string): RunState | undefined { + const p = statePath(runDir); + if (!fs.existsSync(p)) { + return undefined; + } + try { + return JSON.parse(fs.readFileSync(p, "utf8")) as RunState; + } catch { + return undefined; + } +} + +export function initState( + runDir: string, + init: Omit, +): RunState { + const now = new Date().toISOString(); + const state: RunState = { + schemaVersion: 1, + phases: Object.fromEntries(PHASE_ORDER.map((p) => [p, "pending"])) as Record, + createdAt: now, + updatedAt: now, + ...init, + }; + saveState(runDir, state); + return state; +} + +export function saveState(runDir: string, state: RunState): void { + state.updatedAt = new Date().toISOString(); + atomicWrite(statePath(runDir), JSON.stringify(state, null, 2)); +} + +export function setPhase(runDir: string, state: RunState, phase: string, status: PhaseStatus): void { + state.phases[phase] = status; + saveState(runDir, state); +} diff --git a/tools/agentic-workflow/src/types.ts b/tools/agentic-workflow/src/types.ts new file mode 100644 index 00000000000..25881733605 --- /dev/null +++ b/tools/agentic-workflow/src/types.ts @@ -0,0 +1,80 @@ +/** + * Shared types for the agentic workflow thin spine. + * Kept dependency-free and small on purpose — the durable contract lives here. + */ + +export type PhaseId = "research" | "assumptions" | "classify" | "research-item" | "plan" | "implement"; + +export type PhaseStatus = "pending" | "in_progress" | "failed" | "completed"; + +export interface SubItem { + id: string; + type: "feature" | "bug" | "refactor"; + title: string; + description: string; + rationale?: string; + dependsOn: string[]; + expectedFilesOrAreas?: string[]; + acceptanceCriteria?: string[]; + nonGoals?: string[]; + overlapRisk: "low" | "medium" | "high"; +} + +export interface SubItems { + task: string; + classification: "feature" | "bug" | "refactor" | "mixed"; + items: SubItem[]; +} + +export interface GateSpec { + id: string; + commands: string[]; + expected: string; +} + +export interface Stage { + id: string; + expected_files: string[]; + context_needed: string[]; + steps: { id: string; description: string }[]; + gate: GateSpec; +} + +export interface ValidationResult { + ok: boolean; + errors: string[]; +} + +/** Persisted run state (state.json), written atomically after each phase. */ +export interface RunState { + schemaVersion: 1; + runId: string; + task: string; + simple: boolean; + judge: boolean; + judgeModel?: string; + phases: Record; + items?: Record; + createdAt: string; + updatedAt: string; +} + +/** Result of running a single phase through the harness adapter. */ +export interface PhaseRunResult { + artifacts: string[]; + /** Final assistant text, used to read agent-reported gate/stage results. */ + finalText: string; +} + +export interface RunOptions { + task: string; + runId?: string; + simple?: boolean; + judge?: boolean; + judgeModel?: string; + outRoot?: string; + concurrency?: number; + maxRetries?: number; + /** Inject a harness for testing; defaults to the real SDK harness. */ + dryRun?: boolean; +} diff --git a/tools/agentic-workflow/src/validate.ts b/tools/agentic-workflow/src/validate.ts new file mode 100644 index 00000000000..1f9642c8419 --- /dev/null +++ b/tools/agentic-workflow/src/validate.ts @@ -0,0 +1,98 @@ +/** + * A handful of plain validation functions (thinnerplan T1.2). No schema library: parse, check the + * few required fields, return readable errors. Only `subitems.json` and the plan block get + * validation until a real format breaks. + */ +import { parsePlanStages } from "./gates.js"; +import type { SubItems, ValidationResult } from "./types.js"; + +const CLASSIFICATIONS = ["feature", "bug", "refactor", "mixed"]; +const ITEM_TYPES = ["feature", "bug", "refactor"]; +const OVERLAP = ["low", "medium", "high"]; +const ID_RE = /^[a-z0-9]+(-[a-z0-9]+)*$/; + +/** Validate the parsed object shape of `subitems.json`. */ +export function validateSubitems(input: unknown): ValidationResult { + const errors: string[] = []; + const obj = input as Partial | null; + + if (!obj || typeof obj !== "object") { + return { ok: false, errors: ["subitems.json: not an object"] }; + } + if (typeof obj.task !== "string" || !obj.task.trim()) { + errors.push("subitems.json: `task` must be a non-empty string"); + } + if (!obj.classification || !CLASSIFICATIONS.includes(obj.classification)) { + errors.push(`subitems.json: \`classification\` must be one of ${CLASSIFICATIONS.join(", ")}`); + } + if (!Array.isArray(obj.items) || obj.items.length === 0) { + errors.push("subitems.json: `items` must be a non-empty array"); + return { ok: errors.length === 0, errors }; + } + + const seen = new Set(); + const ids = new Set(obj.items.map((it) => it?.id).filter((x): x is string => typeof x === "string")); + obj.items.forEach((it, i) => { + const where = `items[${i}]`; + if (!it || typeof it !== "object") { + errors.push(`${where}: not an object`); + return; + } + if (typeof it.id !== "string" || !ID_RE.test(it.id)) { + errors.push(`${where}: \`id\` must be kebab-case`); + } else if (seen.has(it.id)) { + errors.push(`${where}: duplicate id "${it.id}"`); + } else { + seen.add(it.id); + } + if (!it.type || !ITEM_TYPES.includes(it.type)) { + errors.push(`${where}: \`type\` must be one of ${ITEM_TYPES.join(", ")}`); + } + if (typeof it.title !== "string" || !it.title.trim()) { + errors.push(`${where}: \`title\` must be a non-empty string`); + } + if (!Array.isArray(it.dependsOn)) { + errors.push(`${where}: \`dependsOn\` must be an array`); + } else { + for (const dep of it.dependsOn) { + if (typeof dep !== "string" || !ids.has(dep)) { + errors.push(`${where}: dependsOn references unknown item "${dep}"`); + } + } + } + if (!it.overlapRisk || !OVERLAP.includes(it.overlapRisk)) { + errors.push(`${where}: \`overlapRisk\` must be one of ${OVERLAP.join(", ")}`); + } + }); + + return { ok: errors.length === 0, errors }; +} + +/** Parse + validate `subitems.json` text. */ +export function validateSubitemsJson(text: string): ValidationResult { + let parsed: unknown; + try { + parsed = JSON.parse(text); + } catch (e) { + return { ok: false, errors: [`subitems.json: invalid JSON — ${(e as Error).message}`] }; + } + return validateSubitems(parsed); +} + +const REQUIRED_PLAN_SECTIONS = ["Decisions and rationale", "Step-by-step implementation plan", "Definition of done"]; + +/** Validate a generated `plan.md`: required prose sections + a parseable gate block. */ +export function validatePlan(planMarkdown: string): ValidationResult { + const errors: string[] = []; + for (const section of REQUIRED_PLAN_SECTIONS) { + if (!planMarkdown.toLowerCase().includes(section.toLowerCase())) { + errors.push(`plan.md: missing required section "${section}"`); + } + } + try { + parsePlanStages(planMarkdown); + } catch (e) { + errors.push((e as Error).message); + } + return { ok: errors.length === 0, errors }; +} diff --git a/tools/agentic-workflow/test/artifacts.test.ts b/tools/agentic-workflow/test/artifacts.test.ts new file mode 100644 index 00000000000..39548fc91ac --- /dev/null +++ b/tools/agentic-workflow/test/artifacts.test.ts @@ -0,0 +1,73 @@ +import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import { spawnSync } from "node:child_process"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import { ensureRunDir, atomicWrite, resolveInRunDir, makeRunId, slugify, appendArtifact } from "../src/artifacts.js"; + +let tmp: string; + +beforeEach(() => { + tmp = fs.mkdtempSync(path.join(os.tmpdir(), "aw-test-")); +}); +afterEach(() => { + fs.rmSync(tmp, { recursive: true, force: true }); +}); + +describe("slugify / makeRunId", () => { + it("slugifies a task", () => { + expect(slugify("Add Rate Limiting!! to API")).toBe("add-rate-limiting-to-api"); + }); + it("builds a timestamped run id", () => { + const id = makeRunId("Fix bug", new Date(2026, 5, 29, 13, 10)); + expect(id).toBe("20260629-1310-fix-bug"); + }); +}); + +describe("ensureRunDir (non-git)", () => { + it("creates the run dir and an inner .gitignore, no crash without git", () => { + const { dir, root } = ensureRunDir("run-1", { cwd: tmp }); + expect(fs.existsSync(dir)).toBe(true); + const inner = fs.readFileSync(path.join(root, ".gitignore"), "utf8"); + expect(inner).toContain("*"); + expect(inner).toContain("!.gitignore"); + }); +}); + +describe("ensureRunDir (git)", () => { + it("adds the scratch root to .git/info/exclude without touching tracked .gitignore", () => { + spawnSync("git", ["init", "-q"], { cwd: tmp }); + const { root } = ensureRunDir("run-1", { cwd: tmp }); + const exclude = fs.readFileSync(path.join(tmp, ".git", "info", "exclude"), "utf8"); + expect(exclude).toContain(".agentic-workflow/"); + // tracked root .gitignore is NOT created/modified + expect(fs.existsSync(path.join(tmp, ".gitignore"))).toBe(false); + // idempotent + ensureRunDir("run-2", { cwd: tmp }); + const exclude2 = fs.readFileSync(path.join(tmp, ".git", "info", "exclude"), "utf8"); + expect(exclude2.match(/\.agentic-workflow\//g)).toHaveLength(1); + void root; + }); +}); + +describe("atomicWrite / appendArtifact", () => { + it("writes and appends", () => { + const f = path.join(tmp, "a", "b.md"); + atomicWrite(f, "hello"); + expect(fs.readFileSync(f, "utf8")).toBe("hello"); + appendArtifact(f, "\nmore"); + expect(fs.readFileSync(f, "utf8")).toBe("hello\nmore"); + }); +}); + +describe("resolveInRunDir", () => { + it("resolves a safe relative path", () => { + const p = resolveInRunDir(tmp, "specs/architecture.md"); + expect(p).toBe(path.join(tmp, "specs", "architecture.md")); + }); + it("rejects traversal and absolute paths", () => { + expect(() => resolveInRunDir(tmp, "../escape.md")).toThrow(/escapes/); + expect(() => resolveInRunDir(tmp, "/etc/passwd")).toThrow(/escapes/); + expect(() => resolveInRunDir(tmp, "a/../../b")).toThrow(/escapes/); + }); +}); diff --git a/tools/agentic-workflow/test/gates.test.ts b/tools/agentic-workflow/test/gates.test.ts new file mode 100644 index 00000000000..acf7b2ee63f --- /dev/null +++ b/tools/agentic-workflow/test/gates.test.ts @@ -0,0 +1,48 @@ +import { describe, it, expect } from "vitest"; +import { parsePlanStages, extractGateBlock } from "../src/gates.js"; + +const plan = ` +intro +\`\`\`yaml +stages: + - id: stage-1 + expected_files: ["src/a.ts", "test/a.test.ts"] + context_needed: ["src/types.ts"] + steps: + - { id: "1.1", description: "edit a" } + gate: + id: gate-1 + commands: ["npm test -- a", "npm run lint"] + expected: exit_code_0 + - id: stage-2 + gate: + commands: ["npm test -- b"] +\`\`\` +`; + +describe("gates", () => { + it("extracts the yaml stages block", () => { + expect(extractGateBlock(plan)).toContain("stages:"); + }); + + it("parses stages with defaults filled in", () => { + const stages = parsePlanStages(plan); + expect(stages).toHaveLength(2); + expect(stages[0].id).toBe("stage-1"); + expect(stages[0].gate.commands).toEqual(["npm test -- a", "npm run lint"]); + expect(stages[0].expected_files).toEqual(["src/a.ts", "test/a.test.ts"]); + // stage-2 omitted optional fields -> defaults + expect(stages[1].gate.id).toBe("gate-2"); + expect(stages[1].gate.expected).toBe("exit_code_0"); + expect(stages[1].expected_files).toEqual([]); + }); + + it("throws when no gate block exists", () => { + expect(() => parsePlanStages("no block")).toThrow(/no machine-readable/); + }); + + it("throws when a stage has no commands", () => { + const bad = "```yaml\nstages:\n - id: s1\n gate:\n commands: []\n```"; + expect(() => parsePlanStages(bad)).toThrow(/no gate.commands/); + }); +}); diff --git a/tools/agentic-workflow/test/orchestrator.test.ts b/tools/agentic-workflow/test/orchestrator.test.ts new file mode 100644 index 00000000000..dde2efd0723 --- /dev/null +++ b/tools/agentic-workflow/test/orchestrator.test.ts @@ -0,0 +1,227 @@ +import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import { runWorkflow } from "../src/orchestrator.js"; +import type { Harness, PhaseRequest } from "../src/harness.js"; +import type { PhaseRunResult } from "../src/types.js"; + +const VALID_PLAN = `# Plan +## 0. Research reconciliation +single item. +## 1. Decisions and rationale +because. +## 3. Step-by-step implementation plan +do it. +## 9. Definition of done +done when tests pass. + +\`\`\`yaml +stages: + - id: stage-1 + expected_files: ["src/a.ts"] + context_needed: [] + steps: + - { id: "1.1", description: "x" } + gate: + id: gate-1 + commands: ["npm test"] + expected: exit_code_0 +\`\`\` +`; + +function write(runDir: string, rel: string, content: string) { + const p = path.join(runDir, rel); + fs.mkdirSync(path.dirname(p), { recursive: true }); + fs.writeFileSync(p, content); +} + +/** Programmable fake harness: a responder decides what each phase "writes" and returns. */ +class FakeHarness implements Harness { + calls: { label: string; readOnly: boolean }[] = []; + counts: Record = {}; + constructor(private responder: (req: PhaseRequest, n: number) => string) {} + async runPhase(req: PhaseRequest): Promise { + const key = req.label.replace(/\s*\(retry \d+\)/, "").split(":")[0]; + this.counts[key] = (this.counts[key] ?? 0) + 1; + this.calls.push({ label: req.label, readOnly: req.readOnly }); + const finalText = this.responder(req, this.counts[key]); + return { artifacts: [], finalText }; + } + async stop() {} +} + +let tmp: string; +beforeEach(() => { + tmp = fs.mkdtempSync(path.join(os.tmpdir(), "aw-orch-")); +}); +afterEach(() => { + fs.rmSync(tmp, { recursive: true, force: true }); +}); + +const baseOpts = (over: object = {}) => ({ task: "demo task", outRoot: tmp, judge: false, ...over }); + +describe("orchestrator — simple run", () => { + it("runs assumptions -> plan -> implement and completes (exit 0)", async () => { + const h = new FakeHarness((req) => { + const rd = req.runDir; + if (req.label.startsWith("assumptions")) write(rd, "assumptions.md", "- ok | high"); + if (req.label.startsWith("plan")) write(rd, "plan.md", VALID_PLAN); + if (req.label.startsWith("implement")) return "STAGE_RESULT: pass"; + return ""; + }); + const res = await runWorkflow(baseOpts({ simple: true, runId: "r1" }), h); + expect(res.exitCode).toBe(0); + // research/classify/research-item skipped, not sent to harness + expect(h.counts["research"]).toBeUndefined(); + expect(h.counts["classify"]).toBeUndefined(); + // synthesized subitems.json exists + expect(fs.existsSync(path.join(res.runDir, "subitems.json"))).toBe(true); + const state = JSON.parse(fs.readFileSync(path.join(res.runDir, "state.json"), "utf8")); + expect(state.phases.implement).toBe("completed"); + }); +}); + +describe("orchestrator — fresh-session retry", () => { + it("retries the plan phase in a fresh session after a validation failure", async () => { + const h = new FakeHarness((req, n) => { + const rd = req.runDir; + if (req.label.startsWith("assumptions")) write(rd, "assumptions.md", "- ok | high"); + if (req.label.startsWith("plan")) { + // first attempt writes an invalid plan (no gate block), second writes valid + write(rd, "plan.md", n === 1 ? "## Decisions\nbad" : VALID_PLAN); + } + if (req.label.startsWith("implement")) return "STAGE_RESULT: pass"; + return ""; + }); + const res = await runWorkflow(baseOpts({ simple: true }), h); + expect(res.exitCode).toBe(0); + expect(h.counts["plan"]).toBe(2); // failed once, retried fresh + }); +}); + +describe("orchestrator — failing gate halts", () => { + it("halts (exit 1) when the agent reports a failing stage gate", async () => { + const h = new FakeHarness((req) => { + const rd = req.runDir; + if (req.label.startsWith("assumptions")) write(rd, "assumptions.md", "- ok | high"); + if (req.label.startsWith("plan")) write(rd, "plan.md", VALID_PLAN); + if (req.label.startsWith("implement")) return "STAGE_RESULT: fail — tests red"; + return ""; + }); + const res = await runWorkflow(baseOpts({ simple: true }), h); + expect(res.exitCode).toBe(1); + const state = JSON.parse(fs.readFileSync(path.join(res.runDir, "state.json"), "utf8")); + expect(state.phases.implement).toBe("failed"); + }); +}); + +describe("orchestrator — blocking clarification", () => { + it("pauses (exit 10) on a blocking assumption", async () => { + const h = new FakeHarness((req) => { + if (req.label.startsWith("assumptions")) + write(req.runDir, "assumptions.md", "- blocking: true | unsure about auth | low"); + return ""; + }); + const res = await runWorkflow(baseOpts({ simple: true }), h); + expect(res.exitCode).toBe(10); + }); +}); + +describe("orchestrator — judge loop", () => { + it("runs critique + revise on validated artifacts when judge is on", async () => { + const h = new FakeHarness((req) => { + const rd = req.runDir; + if (req.label.startsWith("assumptions")) write(rd, "assumptions.md", "- ok | high"); + if (req.label.startsWith("plan")) write(rd, "plan.md", VALID_PLAN); + if (req.label.startsWith("critique")) + write(rd, `critiques/${req.label.split(":")[1]}.md`, "- looks fine | optional"); + if (req.label.startsWith("revise")) { + /* keep artifact valid: rewrite plan identically */ + if (req.label.includes("plan")) write(rd, "plan.md", VALID_PLAN); + } + if (req.label.startsWith("implement")) return "STAGE_RESULT: pass"; + return ""; + }); + const res = await runWorkflow(baseOpts({ simple: true, judge: true }), h); + expect(res.exitCode).toBe(0); + expect(h.counts["critique"]).toBeGreaterThanOrEqual(1); + expect(h.counts["revise"]).toBeGreaterThanOrEqual(1); + // critique sessions are read-only + expect(h.calls.filter((c) => c.label.startsWith("critique")).every((c) => c.readOnly)).toBe(true); + }); + + it("skips the judge loop with judge:false", async () => { + const h = new FakeHarness((req) => { + const rd = req.runDir; + if (req.label.startsWith("assumptions")) write(rd, "assumptions.md", "- ok | high"); + if (req.label.startsWith("plan")) write(rd, "plan.md", VALID_PLAN); + if (req.label.startsWith("implement")) return "STAGE_RESULT: pass"; + return ""; + }); + const res = await runWorkflow(baseOpts({ simple: true, judge: false }), h); + expect(res.exitCode).toBe(0); + expect(h.counts["critique"]).toBeUndefined(); + }); +}); + +describe("orchestrator — resume", () => { + it("skips completed phases on resume", async () => { + // First run: implement fails, so plan/assumptions complete but implement does not. + const failing = new FakeHarness((req) => { + const rd = req.runDir; + if (req.label.startsWith("assumptions")) write(rd, "assumptions.md", "- ok | high"); + if (req.label.startsWith("plan")) write(rd, "plan.md", VALID_PLAN); + if (req.label.startsWith("implement")) return "STAGE_RESULT: fail"; + return ""; + }); + const first = await runWorkflow(baseOpts({ simple: true, runId: "resume-run" }), failing); + expect(first.exitCode).toBe(1); + + // Resume with same runId: assumptions/plan already completed -> not re-sent. + const passing = new FakeHarness((req) => { + if (req.label.startsWith("implement")) return "STAGE_RESULT: pass"; + return ""; + }); + const second = await runWorkflow(baseOpts({ simple: true, runId: "resume-run" }), passing); + expect(second.exitCode).toBe(0); + expect(passing.counts["assumptions"]).toBeUndefined(); + expect(passing.counts["plan"]).toBeUndefined(); + expect(passing.counts["implement"]).toBe(1); + }); +}); + +describe("orchestrator — full pipeline (non-simple) with fan-out", () => { + it("runs research -> classify -> research-item xN -> plan -> implement", async () => { + const subitems = { + task: "demo", + classification: "feature", + items: [ + { id: "a", type: "feature", title: "A", description: "a", dependsOn: [], overlapRisk: "low" }, + { id: "b", type: "feature", title: "B", description: "b", dependsOn: ["a"], overlapRisk: "low" }, + ], + }; + const h = new FakeHarness((req) => { + const rd = req.runDir; + if (req.label.startsWith("research") && !req.label.startsWith("research-item")) { + write(rd, "specs/architecture.md", "arch"); + write(rd, "specs/functional.md", "func"); + write(rd, "manifest.json", JSON.stringify({ apispec: { required: false, reason: "n/a" } })); + } + if (req.label.startsWith("classify")) write(rd, "subitems.json", JSON.stringify(subitems)); + if (req.label.startsWith("assumptions")) write(rd, "assumptions.md", "- ok | high"); + if (req.label.startsWith("research-item")) { + const id = req.label.split(":")[1].replace(/\s.*/, ""); + write(rd, `research/${id}.md`, `note ${id}`); + } + if (req.label.startsWith("plan")) write(rd, "plan.md", VALID_PLAN); + if (req.label.startsWith("implement")) return "STAGE_RESULT: pass"; + return ""; + }); + const res = await runWorkflow(baseOpts({ judge: false, runId: "full" }), h); + expect(res.exitCode).toBe(0); + expect(fs.existsSync(path.join(res.runDir, "research", "a.md"))).toBe(true); + expect(fs.existsSync(path.join(res.runDir, "research", "b.md"))).toBe(true); + expect(h.counts["research-item"]).toBe(2); + }); +}); diff --git a/tools/agentic-workflow/test/prompts.test.ts b/tools/agentic-workflow/test/prompts.test.ts new file mode 100644 index 00000000000..84144c0f3c2 --- /dev/null +++ b/tools/agentic-workflow/test/prompts.test.ts @@ -0,0 +1,43 @@ +import { describe, it, expect } from "vitest"; +import { render, loadTemplate, renderTemplate, type PromptName } from "../src/prompts.js"; + +const ALL: PromptName[] = [ + "01-research", + "02-assumptions", + "03-classify", + "04-research-item", + "05-plan", + "06-implement", + "critique", + "revise", +]; + +describe("prompt rendering", () => { + it("substitutes vars and blanks unknown keys", () => { + expect(render("Task: {{task}} / {{missing}}", { task: "X" })).toBe("Task: X / "); + }); + + it("all 8 templates load and render run vars", () => { + for (const name of ALL) { + const raw = loadTemplate(name); + expect(raw.length).toBeGreaterThan(0); + const out = renderTemplate(name, { + task: "demo task", + item: "{}", + itemId: "main", + stage: "id: stage-1", + artifactPath: "plan.md", + artifactName: "plan", + critiquePath: "critiques/plan.md", + }); + // no unresolved placeholders remain + expect(out).not.toMatch(/\{\{\s*[a-zA-Z0-9_]+\s*\}\}/); + } + }); + + it("phase 1 keeps its read-only contract language", () => { + const out = renderTemplate("01-research", { task: "t" }); + expect(out.toLowerCase()).toContain("read-only"); + expect(out).toContain("write_artifact"); + }); +}); diff --git a/tools/agentic-workflow/test/session-options.test.ts b/tools/agentic-workflow/test/session-options.test.ts new file mode 100644 index 00000000000..fda12777a94 --- /dev/null +++ b/tools/agentic-workflow/test/session-options.test.ts @@ -0,0 +1,63 @@ +import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import { makePreToolUseHook, redact, logEvent, makeWriteArtifactTool } from "../src/session-options.js"; + +let tmp: string; +beforeEach(() => { + tmp = fs.mkdtempSync(path.join(os.tmpdir(), "aw-so-")); +}); +afterEach(() => { + fs.rmSync(tmp, { recursive: true, force: true }); +}); + +describe("onPreToolUse policy", () => { + const hook = (readOnly: boolean) => makePreToolUseHook({ readOnly, logPath: path.join(tmp, "log.jsonl") }); + + it("denies shell/edit tools in a read-only phase", () => { + const h = hook(true); + for (const tool of ["shell", "bash", "str_replace", "create_file", "edit_file", "delete_path"]) { + expect(h({ toolName: tool }).permissionDecision).toBe("deny"); + } + }); + + it("allows read tools and write_artifact in a read-only phase", () => { + const h = hook(true); + for (const tool of ["view", "grep", "glob", "read_file", "write_artifact"]) { + expect(h({ toolName: tool }).permissionDecision).toBe("allow"); + } + }); + + it("allows mutating tools in an implement (non-read-only) phase", () => { + const h = hook(false); + expect(h({ toolName: "shell" }).permissionDecision).toBe("allow"); + expect(h({ toolName: "str_replace" }).permissionDecision).toBe("allow"); + }); +}); + +describe("redaction", () => { + it("masks common secret shapes", () => { + expect(redact("token ghp_ABCDEFGHIJKLMNOPQRSTUVWXYZ012345")).toContain("[REDACTED_GH_TOKEN]"); + expect(redact("key sk-ABCDEFGHIJKLMNOPQRSTUV")).toContain("[REDACTED_API_KEY]"); + expect(redact('password: "supersecretvalue"')).toContain("[REDACTED]"); + }); + it("logEvent writes redacted JSONL", () => { + const log = path.join(tmp, "log.jsonl"); + logEvent(log, { kind: "x", token: "ghp_ABCDEFGHIJKLMNOPQRSTUVWXYZ012345" }); + const line = fs.readFileSync(log, "utf8"); + expect(line).toContain("[REDACTED_GH_TOKEN]"); + expect(JSON.parse(line)).toMatchObject({ kind: "x" }); + }); +}); + +describe("write_artifact tool", () => { + it("writes within the run dir and rejects traversal", async () => { + const tool = makeWriteArtifactTool(tmp, path.join(tmp, "log.jsonl")); + const handler = tool.handler as (a: unknown) => { content: { text: string }[] }; + handler({ path: "specs/a.md", content: "hi" }); + expect(fs.readFileSync(path.join(tmp, "specs", "a.md"), "utf8")).toBe("hi"); + const bad = handler({ path: "../escape.md", content: "x" }); + expect(bad.content[0].text).toContain("escapes run dir"); + }); +}); diff --git a/tools/agentic-workflow/test/validate.test.ts b/tools/agentic-workflow/test/validate.test.ts new file mode 100644 index 00000000000..6286e0bc600 --- /dev/null +++ b/tools/agentic-workflow/test/validate.test.ts @@ -0,0 +1,102 @@ +import { describe, it, expect } from "vitest"; +import { validateSubitems, validateSubitemsJson, validatePlan } from "../src/validate.js"; + +const goodItem = { + id: "main", + type: "feature", + title: "Do the thing", + description: "covers the thing", + dependsOn: [], + overlapRisk: "low", +}; + +describe("validateSubitems", () => { + it("accepts a well-formed object", () => { + const r = validateSubitems({ task: "t", classification: "feature", items: [goodItem] }); + expect(r.ok).toBe(true); + expect(r.errors).toEqual([]); + }); + + it("rejects empty items", () => { + const r = validateSubitems({ task: "t", classification: "feature", items: [] }); + expect(r.ok).toBe(false); + expect(r.errors.join()).toContain("non-empty array"); + }); + + it("rejects bad classification and non-kebab id", () => { + const r = validateSubitems({ + task: "t", + classification: "nope", + items: [{ ...goodItem, id: "Bad_ID" }], + }); + expect(r.ok).toBe(false); + expect(r.errors.join()).toContain("classification"); + expect(r.errors.join()).toContain("kebab-case"); + }); + + it("rejects duplicate ids and dangling dependsOn", () => { + const r = validateSubitems({ + task: "t", + classification: "mixed", + items: [ + { ...goodItem, id: "a", dependsOn: ["ghost"] }, + { ...goodItem, id: "a" }, + ], + }); + expect(r.ok).toBe(false); + expect(r.errors.join()).toContain("duplicate id"); + expect(r.errors.join()).toContain("unknown item"); + }); + + it("validateSubitemsJson reports invalid JSON", () => { + const r = validateSubitemsJson("{ not json"); + expect(r.ok).toBe(false); + expect(r.errors.join()).toContain("invalid JSON"); + }); +}); + +const planWithGate = ` +# Plan +## 0. Research reconciliation +single item. +## 1. Decisions and rationale +because. +## 3. Step-by-step implementation plan +do it. +## 9. Definition of done +done when tests pass. + +\`\`\`yaml +stages: + - id: stage-1 + expected_files: ["src/a.ts"] + context_needed: [] + steps: + - { id: "1.1", description: "x" } + gate: + id: gate-1 + commands: ["npm test"] + expected: exit_code_0 +\`\`\` +`; + +describe("validatePlan", () => { + it("accepts a plan with required sections and a gate block", () => { + const r = validatePlan(planWithGate); + expect(r.ok).toBe(true); + }); + + it("rejects a plan missing the gate block", () => { + const r = validatePlan( + "## Decisions and rationale\n## Step-by-step implementation plan\n## Definition of done\n", + ); + expect(r.ok).toBe(false); + expect(r.errors.join()).toContain("stages"); + }); + + it("rejects a plan missing required sections", () => { + const r = validatePlan("no sections here"); + expect(r.ok).toBe(false); + expect(r.errors.join()).toContain("Definition of done"); + }); +}); diff --git a/tools/agentic-workflow/tsconfig.json b/tools/agentic-workflow/tsconfig.json new file mode 100644 index 00000000000..51218393d03 --- /dev/null +++ b/tools/agentic-workflow/tsconfig.json @@ -0,0 +1,19 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "lib": ["ES2022"], + "outDir": "./dist", + "rootDir": "./src", + "declaration": true, + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true, + "resolveJsonModule": true, + "sourceMap": true + }, + "include": ["src/**/*.ts"], + "exclude": ["node_modules", "dist", "spike", "test"] +} From ec0a146e8d537157901ba92346b06fbbf453a57b Mon Sep 17 00:00:00 2001 From: jennypng <63012604+JennyPng@users.noreply.github.com> Date: Mon, 29 Jun 2026 14:13:53 -0700 Subject: [PATCH 03/21] cleanup --- .plan.md | 479 ---------------------- gao.md | 42 -- thinnerplan.plan.md | 245 ----------- tools/agentic-workflow/.gitignore | 2 - tools/agentic-workflow/.prettierignore | 1 - tools/agentic-workflow/README.md | 3 +- tools/agentic-workflow/spike/FINDINGS.md | 39 -- tools/agentic-workflow/spike/package.json | 16 - tools/agentic-workflow/spike/spike.mjs | 81 ---- 9 files changed, 1 insertion(+), 907 deletions(-) delete mode 100644 .plan.md delete mode 100644 gao.md delete mode 100644 thinnerplan.plan.md delete mode 100644 tools/agentic-workflow/spike/FINDINGS.md delete mode 100644 tools/agentic-workflow/spike/package.json delete mode 100644 tools/agentic-workflow/spike/spike.mjs diff --git a/.plan.md b/.plan.md deleted file mode 100644 index 6a59fc3c653..00000000000 --- a/.plan.md +++ /dev/null @@ -1,479 +0,0 @@ -# Implementation Plan — Agentic `research → plan → implement` Workflow - -> Derived from `agentic-workflow-design.md`. This is a **planning artifact only** — no code is -> implemented here. It translates the design into an ordered, gated build with explicit -> decisions, success proof, validation, and stop/go checkpoints. - ---- - -## 0. Decisions & rationale (locked for this build) - -| # | Decision | Rationale | -| --- | --- | --- | -| D1 | Build as a self-contained TS tool at `tools/agentic-workflow/` | Matches repo convention (`/tools//` with README/tests/CI); design §10/§13. | -| D2 | TypeScript + `@github/copilot-sdk` | Richest SDK surface, bundles CLI, ESM matches existing TS tools (`js-sdk-release-tools`). | -| D3 | Toolchain: `tsc` build, `vitest --run` tests, `prettier` format, ESM (`"type":"module"`) | Mirror `tools/js-sdk-release-tools` exactly to inherit known-good CI patterns. | -| D4 | **M0 capability spike gates everything** | The whole architecture (fresh-context isolation, tool scoping, permissions) rests on unverified SDK behavior. Build nothing further until proven. | -| D5 | Ship the **MVP cut first**, then widen to full CLI | Design §11 explicitly recommends de-risking gate/policy enforcement before the rich CLI surface. | -| D6 | Orchestrator-enforced guarantees, never prompt-only | Policy (§6.1), gates (§6.2), resume revalidation (§9.4) are code-checked. Prompts can't be trusted to self-police. | -| D7 | Fresh session per phase **and** per retry; staged phase-6 | Clean-context guarantee is the product's core value (design §13). | -| D8 | Artifacts scratch-by-default under locally-ignored `.agentic-workflow/` | No tracked-file mutation before phase 6 runs (§3.1). | -| D9 | **Orchestrator records the real diff, gate exit codes, and changed files; agents supply rationale only** | The execution log's *facts* (what changed, what gates returned) come from the orchestrator, not the agent's self-report. Keeps the audit trustworthy without an elaborate two-party ceremony. | -| D10 | **Validate machine artifacts with Zod at read/write boundaries** | `subitems.json` and the plan block are checked against Zod schemas in code; malformed output → fresh-session retry. No `schemaVersion` field and no generated JSON Schema until a format actually breaks (KISS). | -| D11 | **A failing gate pauses for human review (exit `10`) by default; hard failure (exit `1`) only under `--auto`/autopilot for that boundary or on retry exhaustion** | Resolves the `10`/`1` conflict consistently across §2, M2, M5, M7 (gao ambiguity: exit codes). | -| D12 | **Gate-command safety for v1 = human approval of the plan block in review mode** | The default review gate means a human reads and approves `plan.md` (including its gate commands) before implement runs. A full sandboxed/allow-listed executor is deferred until autopilot (D13). | -| D13 | **Review mode is the default; full autopilot (incl. gate sandbox + budget caps) is explicit opt-in** | Unattended phase 6 requires `--autopilot` and the extra safety machinery; the MVP doesn't pay for it. | - -**Build philosophy:** each milestone ends at a **stop/go gate** with a concrete, observable proof. -Milestones are ordered by **dependency, not duration** — an agent moves quickly, so the only thing -that throttles progress is whether the prerequisite gate is green. Do not start a milestone until -its predecessor gate passes. Where the dependency graph allows, work is parallelizable across -agents (noted per milestone — e.g. the six prompt templates in M3, and per-item research in M4). - -**Parallelization map (independent once their gate is green):** -- After **G1**: the M2 modules — `tools.ts`, `policy.ts`, and the `phase()` happy path — can be - authored concurrently, then integrated. -- After **G2**: the six M3 templates are independent files; author them in parallel. -- After **G3**: M4's schema (`subitems.schema.json`) and the fan-out runner are independent. -- M8 packaging items (README, `ci.yml`, CODEOWNERS, root index) are mutually independent. - -The critical path is strictly serial: **G0 → G1 → G2 → G3 → G4 → G5**, because each builds on the -prior's runtime contract. M6/M7 layer onto the M5 core and can interleave once G5 is green. - ---- - -## 1. End-to-end approach - -1. **Prove the platform (M0).** A throwaway spike answers the design-critical SDK questions. Its - findings either confirm the §6.1 enforcement model or trigger the **read-only-checkout fallback**. -2. **Lay the durable substrate (M1).** Working-dir, artifact I/O, hashing, run lock, `state.json`, - `manifest.json`, types, config. Everything else reads/writes through this. -3. **Build the phase engine (M2).** `phase()` = createSession → render → sendAndWait → validate → - policy/git-diff guard → fresh-session retry → checkpoint, with event streaming to the log. -4. **Author the IP (M3).** The six prompt templates — the highest-leverage, most-iterated asset. -5. **Add fan-out (M4)** and **plan + staged implement + gates (M5)** — the workflow's spine. -6. **Harden (M6)** resume/locking/dry-run/redaction, then **expose the CLI (M7)** and **package (M8).** - -Each phase communicates **only** through files in `.agentic-workflow//`, so any phase can be -re-run, inspected, or improved independently. - ---- - -## 2. How success will be proved - -The system's success criterion is: *one command drives research → … → implement, producing -reviewable, gated artifacts, with each phase isolated and each guarantee enforced by code.* This is -proved by layered, observable checks: - -| Claim | Proof | -| --- | --- | -| **Fresh context per phase** | M0 spike: a **single-use nonce** (random token never written to any artifact, log, or the repo) is revealed only inside session A; a fresh session B is asked for it and cannot produce it. "Isolation" is defined relative to SDK memory, external tools, and on-disk files — none may carry the nonce. Unit assertion on session isolation. | -| **Phases communicate only via files** | Post-phase git-diff guard (algorithm in M2.4) returns empty for phases 1–5 on a real run; integration test asserts no source mutation. | -| **Gates are orchestrator-run, not self-reported** | Inject a deliberately failing gate command → in default/review mode the run **pauses** at that stage with exit code `10`; under `--auto` for that boundary it **fails** with exit `1` (D11). The orchestrator runs the gate and records the captured exit code. | -| **Schema contracts hold** | `subitems.json` and the `plan.md` machine block validate against their Zod schemas (D10); malformed agent output triggers a fresh-session retry, proven by a fixture test. | -| **Audit records are trustworthy** | The execution log records the **actual diff, gate commands, and exit codes from the orchestrator** (D9); the agent only adds rationale. | -| **Resume is safe** | Kill a run mid-phase, `resume` re-runs the `in_progress` phase from scratch; edit an artifact between phases → new hash recorded, downstream phases re-run reading the edited content (M6.2). | -| **Skips preserve the contract** | `--simple` run produces a single-item `subitems.json` and a full structured `plan.md`; `manifest.json` records the skip reasons. | -| **End-to-end** | A real sample task runs to completion on a branch, produces all artifacts + an `execution-log.jsonl` mapping every edit to a plan step, and passes its own gates. | - -A run is "done" when: all phases `completed` in `state.json`, every gate passed, `execution-log.jsonl` -maps each action to a `plan.md` step, and exit code `0`. - ---- - -## 3. Step-by-step plan (by milestone) - -> Format per step — **What / Where / Why / Expected outcome.** Gates marked **⛔ STOP/GO**. - -### M0 — Capability spike *(blocks all other milestones)* - -- **M0.1 Stand up a minimal SDK program.** - - *What:* a single throwaway script that creates a `CopilotClient`, opens a session, sends a - prompt, streams deltas, and stops. - - *Where:* `tools/agentic-workflow/spike/` (deleted or archived before M8). - - *Why:* confirm the SDK installs, authenticates, and runs in this environment at all. - - *Expected:* a streamed assistant reply printed to stdout; clean `client.stop()`. - -- **M0.2 Verify session isolation.** - - *What:* generate a **single-use nonce** (random token), reveal it only inside session A, then in a *new* session B ask for it. The nonce is never written to any artifact, log, or the repo. - - *Why:* the entire design depends on `createSession()` giving a clean context; "isolation" must hold against SDK memory, external tools, and on-disk files. - - *Expected:* B cannot produce the nonce. Result documented verbatim in spike notes (without echoing the nonce itself). - -- **M0.3 Verify concurrency.** Open ≥3 sessions from one client simultaneously; confirm all - progress independently (needed for phase-4 fan-out). *Expected:* no cross-talk, no deadlock. - -- **M0.4 Resolve permissions hook.** Find the **exact TS option** to auto-approve tool/file - permissions (Python = `PermissionHandler.approve_all`). *Expected:* an autonomous session runs - without interactive prompts. - -- **M0.5 Resolve tool scoping (design-critical).** Determine whether built-in file/shell tools can - be **disabled or path-scoped per session**. *Expected:* a definitive yes/no. - - **yes →** §6.1 read-only policy uses native scoping. - - **no →** adopt the **read-only-checkout/overlay fallback** for non-impl phases. This must become a - **concrete M0 design output (`spike/FALLBACK.md`)**, not a yes/no branch, specifying: worktree/checkout - creation per phase, artifact handoff into the scoped tree, how implementation diffs are transferred - back to the real working dir (patch apply), cleanup, symlink handling, nested-repo behavior, and - treatment of ignored files. **This decision and its mechanism feed M2's policy design.** - -- **M0.5a (deferred to autopilot).** A standalone gate-command sandbox (allowlist/scrubbed env/ - no-TTY executor) is **not built for the MVP** — review-mode human approval of the plan block is the - v1 safety boundary (D12). Revisit when `--autopilot` is added. - -- **M0.6 Catalog the surface.** Record exact event names, streaming behavior, working-dir behavior, - custom-tool (`defineTool`) schema/validation support, model override, token/cost usage events, - and that **code editing is actually available** via the runtime. - -- **⛔ STOP/GO GATE G0:** A written `spike/FINDINGS.md` answers every §11-M0 / §12 question with a - concrete result, and (if scoping is unavailable) `spike/FALLBACK.md` exists. **Go** only if isolation - (M0.2), concurrency (M0.3), permissions (M0.4), and an enforcement path (M0.5, native *or* specified - fallback) are all confirmed. **No-go →** revise the design's enforcement model before proceeding (do - not build on assumptions). - ---- - -### M1 — Artifact contract + working dir *(after G0)* - -- **M1.1 Project scaffold.** - - *What:* `package.json` (ESM, scripts: `build`/`test`/`format` mirroring `js-sdk-release-tools`), - `tsconfig.json`, `vitest.config.ts`, tool-local `.gitignore`. - - *Where:* `tools/agentic-workflow/`. - - *Why:* inherit the repo's known-good TS tool conventions (D3). - - *Expected:* `npm install && npm run build` succeeds on an empty `src/index.ts`. - -- **M1.2 Shared types + schemas.** `src/types.ts` — `Phase`, `SubItem`, `RunState`, `Stage`, `Gate`, - `Manifest`, `Policy`, `GateSet`. Define machine artifacts with **Zod** and validate against those - schemas at boundaries (D10). No `schemaVersion` fields or generated JSON Schema files. *Why:* single - source of truth consumed by every module. - -- **M1.3 Working-dir + ignore management.** `src/artifacts.ts` — resolve root (`--out` → env → - config → default `.agentic-workflow/`), create `/`, write `.git/info/exclude` entry + - inner `.agentic-workflow/.gitignore` (`*` / `!.gitignore`); both no-op if present or non-git - (§3.1). *Expected:* artifacts dir exists and is git-ignored without touching tracked `.gitignore`. - -- **M1.4 Atomic I/O + hashing.** Atomic write (temp + rename), read, content-hash helpers in - `artifacts.ts`. *Why:* resume revalidation and crash-safety depend on atomicity + hashes. - -- **M1.5 Run lock + dirty-worktree preflight.** `run.lock` acquire/release around state mutation - (§9.4); a second concurrent run/resume on the same run-id fails fast. Plus a lightweight preflight: - warn (don't hard-block) if the worktree is dirty outside `.agentic-workflow/`, and record the branch - + base commit for the git-diff guard. *Why:* crash-safety and a clean base, without speculative - cross-run branch coordination. *Expected:* concurrent same-run-id access fails fast; a dirty worktree - warns. - -- **M1.6 Checkpoint state + manifest (orchestrator-written).** `state.json` (per-phase status, retry - counts, template versions, artifact hashes, resolved gate set) and `manifest.json` (apispec/research/ - classify decisions), **both written only by the orchestrator** from agent-supplied decision - artifacts (D9). Write atomically. *Why:* the spine of resume and skip-vs-missing disambiguation. - -- **M1.7 Config.** `src/config.ts` — per-phase model map, retry budget, concurrency cap, working-dir - root, all overridable. *Expected:* defaults load; `--model plan=…` style overrides parse. - -- **⛔ STOP/GO GATE G1:** Unit tests prove atomic write survives a simulated crash, hashes are - stable, the lock is exclusive, and a fresh `state.json` round-trips. **Go** when green. - ---- - -### M2 — `phase()` core + policy *(after G1)* - -- **M2.1 `phase()` happy path.** `src/phase.ts` — `createSession({model, tools, onPermissionRequest, - policy})` → render prompt → `sendAndWait` → stream events into an **orchestrator-owned, redacted - structured log** (`execution-log.jsonl`). *Where:* `phase.ts` + `prompts.ts` (template load/render - with var substitution) + `log.ts`. The orchestrator records the facts (events, tool calls, exit - codes, file changes); **redaction runs inline at write time** (secret/token patterns scrubbed before - anything touches disk — built in here, hardened in M6.4, not deferred) (gao crit. #5). *Expected:* a - stub phase produces a declared artifact via `write_artifact`, with no raw secrets in the log. - -- **M2.2 Validation pipeline.** exists → parses → schema-valid, per phase. *Why:* gate output - quality before the orchestrator advances. *Expected:* a malformed artifact is rejected. - -- **M2.3 Custom tools.** `src/tools.ts` — `write_artifact(path, content)` (path normalization + - traversal rejection) and `validate_subitems(json)`. *Why:* the only sanctioned write path for - phases 1–5; lets a phase self-correct in-turn (§6). - -- **M2.4 Policy enforcement.** `src/policy.ts` — apply the M0-chosen mechanism (native scoping *or* - read-only checkout per `spike/FALLBACK.md`) + a **post-phase git-diff guard** for non-impl phases: - 1. Record HEAD + tracked/untracked state **before** the phase (the M1.5 base). - 2. After the phase, compute changes via `git status --porcelain` so **untracked source files are - included** (plain `git diff` misses them). - 3. **Exclude only the artifact root** (`.agentic-workflow/`). - 4. Any other add/modify/delete vs. the pre-phase snapshot → **fail the phase** (§6.1). - *Expected:* a phase that creates or edits a tracked or untracked source file is caught and failed; - edits confined to `.agentic-workflow/` pass. - -- **M2.5 Fresh-session retry.** On validation/policy failure, spawn a **brand-new** session (never - continue the transcript), seeded with original inputs **plus** the validator/policy errors, up to - `--max-retries`. *Why:* preserves clean-context guarantee under retry (D7). *Expected:* a fixture - that fails once then succeeds proves the retry path; exhausting retries is a **hard failure → exit - `1`** (D11). - -- **M2.6 Checkpoint write.** Persist phase status + artifact hashes atomically after each phase. - -- **⛔ STOP/GO GATE G2:** With a stub prompt, a phase runs end-to-end: session → artifact → - validate → git-diff guard empty → checkpoint; an injected bad output triggers exactly one - fresh-session retry. **Go** when proven on a real (cheap-model) call. - ---- - -### M3 — Prompt templates *(after G2; templates parallelizable across agents)* - -- **M3.1–M3.6 Author the six templates** in `prompts/`: `01-research.md`, `02-assumptions.md`, - `03-classify.md`, `04-research-item.md`, `05-plan.md`, `06-implement.md`. - - *What:* each with Role/goal, exact input paths, exact output path(s)+sections, constraints - (stay in scope, don't implement during research, cite code locations). - - *Why:* the IP of the system; isolating them as versioned files lets each be iterated alone. - - *Key contract points to encode:* - - `01` conditionally writes `specs/apispec.md` and emits an apispec-decision note that the - **orchestrator** reads and records in `manifest.json` (D9). Branches on `manifest.research.skipped`. - - `02` flags `blocking: true` for low-confidence assumptions affecting correctness/security/API. - - `03` emits schema-valid `subitems.json` (deps/overlap/expectedFilesOrAreas) + `classification.md`. - - `05` produces the ordered prose sections **and** the machine-readable plan block: a single fenced - ` ```yaml agentic-plan ` block carrying `stages[]` and per-stage `gate{commands,expected}`, - validated against its Zod schema (D10); sizes stages cohesive + loosely coupled. - - `06` writes `execution-log` rationale + `handoff.md`; the orchestrator separately records the - actual diff and gate exit codes (D9). Deviations are logged as `plan.md` "Plan changes" entries. - - *Expected:* each template renders with run vars and, on a sample task, yields a well-formed - artifact. - -- **M3.7 Template versioning.** Stamp a template version into each artifact + `state.json` (§12 - determinism). *Expected:* version visible in produced artifacts. - -- **M3.8 Prompt-template regression tests.** Lightweight **golden-structure tests** assert each - rendered template contains its required sections and exact input/output paths, so prompt edits can't - silently drop the core contract (gao opp. 3). *Expected:* a structural change to any template fails - its golden test. - -- **⛔ STOP/GO GATE G3:** On one real sample task, phases 1→2 produce sensible `specs/*` + - `assumptions.md` (with at least the structure correct). **Go** when templates are stable enough to - build fan-out/plan on top. (Prompt iteration continues throughout later milestones.) - ---- - -### M4 — Classification + parallel fan-out *(after G3)* - -- **M4.1 `subitems.json` schema.** `schema/subitems` Zod schema per the §3 sketch (id/type/deps/ - overlapRisk/expectedFilesOrAreas/acceptanceCriteria/nonGoals). Wire into `validate_subitems`. The - `--concurrency` cap bounds fan-out cost; a hard token-budget ceiling is deferred (see §7). - -- **M4.2 Classify phase + skip synthesis.** Run phase 3; when `--skip-classify`, synthesize a - single-item `subitems.json` (`id:"main"`, type from `--type` or `feature`, `dependsOn:[]`, - `overlapRisk:"low"`) and record the skip in `manifest.json`. *Expected:* downstream contract - identical whether classify ran or was skipped. - -- **M4.3 Ordered parallel fan-out.** `mapWithConcurrency` over items, **fresh session each**, - ordered by `dependsOn`, bounded by `--concurrency`; each item write-scoped to `research/.md`. - **Dependency input contract:** `dependsOn` controls ordering *and* feeds context — when item B - depends on item A, B's research session receives A's completed `research/A.md` as read-only input - (independent items do not). *Why:* phase-4 deep research, safely parallel, without losing upstream - findings. *Expected:* N notes produced; a dependent item's note references its prerequisite; an item - failure doesn't corrupt siblings; per-item status tracked in `state.json`. - -- **M4.4 Skip-research collapse.** With `--skip-research=all`, phase 4 is omitted; `=specs` keeps it. - Single-item + skip-research → phase 4 is a no-op. - -- **⛔ STOP/GO GATE G4:** A multi-item sample produces independent, non-overlapping `research/*.md` - in parallel under the concurrency cap; a single-item skip path also works. **Go** when green. - ---- - -### M5 — Plan + staged implement + gates *(after G4 — the core)* - -- **M5.1 `planSchema` + plan phase.** Validate the embedded plan block (the fenced `yaml agentic-plan` - block, M3 format) against its Zod schema; phase 5 reads all available specs/assumptions/research and - writes the full structured `plan.md` (research reconciliation first, through to plan-changes log). - *Expected:* `plan.md` parses and the machine block validates. - -- **M5.2 Stage parser.** `src/gates.ts` — `parsePlanStages("plan.md")` → ordered stages with - `expected_files`, `context_needed`, steps, gate commands + expected. *Expected:* round-trips the - sample plan's stages. (Gate commands run only after the plan block is human-approved in review mode, - D12.) - -- **M5.3 Context pack builder.** `buildContextPack(ctx, stage)` — `plan.md` + prior `handoff.md` + - cumulative diff + `context_needed` files (§6.3). *Why:* fresh ≠ blank; no inter-stage knowledge - loss. *Expected:* pack contains exactly the curated inputs, bounded in size. - -- **M5.4 Staged implement loop.** For each stage: fresh sub-session with the pack → agent edits and - writes `execution-log` rationale + `handoff.md`; the **orchestrator** records the actual diff and - changed files (D9). *Expected:* each stage runs isolated; handoff carries forward. - -- **M5.5 Deviation detection.** Orchestrator diffs edits vs `expected_files`; out-of-scope edits are - **allowed but require** a matching plan-changes entry + execution-log justification. Undocumented - out-of-scope edit → **pause for review** (the missing docs are the failure, not the edit, §5/§6.1). - -- **M5.6 Orchestrator-run gates.** After each stage, the orchestrator (not the agent) runs - `gate.commands`, checks `expected` (e.g. `exit_code_0`), records exit code + output in the - execution log, and **stops at the first failing gate** (§6.2): in default/review mode it - **pauses → exit `10`**; under `--auto` for that boundary it **fails → exit `1`** (D11). *Expected:* - a deliberately failing gate pauses (review) or fails (auto) the run at that stage. - -- **M5.7 Orchestrator wiring.** `src/orchestrator.ts` + `src/index.ts` — sequence phases per the - §4 pseudocode; load checkpoint; acquire lock. - -- **⛔ STOP/GO GATE G5 (key milestone):** A **real sample task runs end-to-end** on a branch: - specs → assumptions → classify → research → plan → staged implement; every edit maps to a plan - step; gates run by the orchestrator; an injected failing gate halts correctly. **Go** when the - full happy path + the failing-gate path are both demonstrated. - ---- - -### M6 — Hardening *(after G5)* - -- **M6.1 Retry/backoff** on transient SDK/auth errors (distinct from validation retries). -- **M6.2 Resume revalidation.** On `resume`, re-hash + re-validate inputs; `in_progress`/`failed` - phases re-run from scratch in a fresh session. When an artifact's hash changed between runs, - **re-run the phases after it in the fixed pipeline order** (`research → assumptions/classify → - per-item research → plan → implement`) reading the edited content — no per-artifact dependency-graph - bookkeeping. New hashes are recorded and re-checked before downstream reads (§9.4). Phase-4 resumes - per item (completed + matching-hash items skipped). -- **M6.3 Dry-run.** Render prompts + planned sessions/artifacts for every phase with **no SDK calls - or edits** (orthogonal to gating). -- **M6.4 Redaction.** A basic secret/token regex scrubs log/artifact writes (built inline in M2.1); - M6 adds tests proving injected secrets in SDK output, tool args, and shell output don't reach disk. - *Why:* present from the first log write, not deferred — but kept to simple patterns, not exhaustive - detection. -- **⛔ STOP/GO GATE G6:** Kill a run mid-phase → `resume` recovers correctly; edit-between-phases - flows downstream; `--dry-run` makes zero calls. **Go** when proven. - ---- - -### M7 — CLI surface *(after G6)* - -- **M7.1 Commands.** `run` (default), `resume`, `status`, `list`, `approve`, `abort`, `clean` - over the same checkpoint state (§9). **Approval model:** an approval is an orchestrator-recorded - entry in `state.json` (which boundary, by whom/when, the approved artifact hash); `approve` records - it and `resume` continues from the paused boundary — re-validating that the approved artifact's hash - is unchanged before proceeding (gao ambiguity: approval model). -- **M7.2 Gate-set resolution (review is the default).** Default preset is `--review` (pause at every - phase boundary); `--autopilot` is **explicit opt-in** (D13). Apply `--pause-after`/`--pause-before`/ - `--auto` left-to-right (incl. `implement:*` / `implement:`) → persist resolved gate set in - `state.json`; `status` prints it. -- **M7.3 Skip/preset flags.** `--simple`, `--skip-research[=all|specs]`, `--skip-classify`, - `--no-skip-*` negations, `--type`, `--reason`. -- **M7.4 Safety stops.** Blocking-clarification (phase-2 `blocking:true`), undocumented out-of-scope - edit, and failing gate — fire **regardless** of gate set; `--yes` suppresses only the clarification. - A failing gate **pauses (exit `10`)** unless that boundary is `--auto`, where it **fails (exit `1`)** - (D11). -- **M7.5 Exit codes.** `0` complete / `10` paused (incl. pending approval, paused gate) / `1` failure - (incl. `--auto` gate failure, retry exhaustion) / `2` bad usage (§9.5). -- **⛔ STOP/GO GATE G7:** The §9.6 example sessions all behave as documented (autopilot, - stage-gated review, `--pause-before implement`, `--dry-run`, `--simple`). **Go** when green. - ---- - -### M8 — Packaging *(after G7)* - -- **M8.1** `README.md` (usage + design summary) and tests covering the success-proof scenarios in §2. -- **M8.2** `ci.yml` from `eng/pipelines/templates`; pipeline names `tools - agentic-workflow - ci` - (public) / `tools - agentic-workflow` (internal). -- **M8.3** Add `/tools/agentic-workflow/ @JennyPng` to `.github/CODEOWNERS`; add a row to root - `README.md` tool index. -- **M8.4** Optional thin `SKILL.md` front-door that only invokes the tool. -- **M8.5** Remove/archive the M0 `spike/`. -- **⛔ STOP/GO GATE G8:** CI green; README accurate; CODEOWNERS + index updated. **Done.** - ---- - -## 4. MVP cut (land first, per design §11) - -Before the full CLI, ship: research → (assumptions + classify) → optional per-item research → -plan → **mandatory human approval** → staged implement with orchestrator-enforced gates. The MVP is -**review-mode** (D13): the single hard-coded "approve before implement" gate stands in for M7's -gate-set engine, and the approval is recorded in `state.json` so `resume` can continue from it. -Gate-command safety for the MVP is simply that a human approves the plan block (including its gates) -before implement runs (D12) — no separate sandbox or budget machinery. -**Defer:** `--only`, `--from`, `approve`-separate-from-`resume`, root-`.gitignore` mutation, the gate -sandbox, the token-budget ceiling, and fully unattended phase 6 (autopilot). This sequences as -**M0 → M1 → M2 → M3 → M4(core) → M5**. - ---- - -## 5. Validation plan - -**Unit tests (vitest, mirroring `js-sdk-release-tools`):** -- `artifacts`: atomic write/crash-safety, hashing stability, lock exclusivity, dirty-worktree warning, - gitignore/exclude writing (and non-git no-op). -- `schema`: `subitems.json` and the plan block (Zod) accept valid fixtures, reject malformed ones with - useful errors. -- `gates`: `parsePlanStages` round-trips a sample plan; gate runner captures exit codes correctly. -- `policy`: the git-diff guard flags tracked + untracked source mutations but ignores - `.agentic-workflow/`; `write_artifact` rejects path traversal. -- `gate-set`: preset + override resolution produces the expected boundary set (table-driven), with - **review as default**. -- `manifest`: skip flags record the right `skipped`/`apispec` entries; only the orchestrator writes - `manifest.json`. -- `log`: redaction scrubs injected secrets from SDK output, tool args, and shell output. -- `prompts`: golden-structure regression tests for the six templates (M3.8). - -**Integration / scenario tests — *mocked, deterministic* (no live quota):** -- Session isolation assertion via nonce (carry-over of the M0 spike). -- Fresh-session retry: fail-once-then-succeed fixture; retry-exhaustion → exit `1`. -- Failing-gate halt: injected failing command → **pause (exit `10`) in review, fail (exit `1`) under - `--auto`**; log records it. -- Resume: kill mid-phase → recover; edit-between-phases → new hash + later pipeline phases re-run - reading the edited file. -- Skip paths: `--simple` yields single-item `subitems.json` + full `plan.md` + manifest reasons. -- Dry-run: zero SDK calls / zero edits. - -**End-to-end smoke — *real SDK*, opt-in (on a throwaway branch):** -- A small but non-trivial task runs to completion; all artifacts present; `execution-log.jsonl` maps - every edit to a plan step; gates pass; final exit `0`. Kept separate from the deterministic suite and - **not run in CI unless a cheap-model lane is available**. - -**Add to CI:** `npm run build` + `npm test` (mocked/deterministic only) + format check. Real-SDK e2e -stays manual/opt-in (quota cost) unless a cheap-model lane is available. - ---- - -## 6. Open questions (must resolve, mostly in M0) - -1. Exact TS auto-approve permission option (Python parity: `PermissionHandler.approve_all`). — M0.4 -2. Can built-in file/shell tools be disabled/path-scoped per session? If not, the read-only-checkout - fallback is mandatory and fully specified in `spike/FALLBACK.md`. — **M0.5, design-critical** -3. **Gate command safety:** for the MVP, human approval of the plan block in review mode is the - safety boundary (D12). A sandboxed/allow-listed executor is **deferred** to autopilot — revisit then. -4. **Cost/quota:** the `--concurrency` cap bounds fan-out; a hard token-budget ceiling is **deferred** - (§7) until fan-out actually strains quota. -5. **Phase-5 context size:** plan reads all specs + all research; may need summarization if large. -6. Exact SDK event names / streaming semantics / custom-tool validation surface. — M0.6. -7. Cheap-model CI lane availability for automated e2e (else e2e stays manual, separate from the - deterministic suite). - ---- - -## 7. Out-of-scope observations (noticed, not in this plan) - -> Surfaced for consideration. A few items raised by the plan critique are scheduled (marked ✅ with -> their milestone); the rest, including several deliberately-deferred safety features, remain out of -> scope for the MVP. - -- **Per-run cost/budget ceiling.** Deferred. The `--concurrency` cap bounds fan-out for now; a hard - token/cost ceiling + live readout is future work once spend actually matters. -- **Gate-command sandboxing.** Deferred. MVP relies on human approval of the plan block (D12); a - full allow-listed/jailed executor is needed only for autopilot. -- **Multi-repo / monorepo scoping.** `--allow-dir` exists, but the git-diff guard and working-dir - assume a single repo root; nested/worktree behavior is unspecified. -- **Concurrent-run isolation on shared infra.** Per-run lock + a dirty-worktree warning (M1.5) only; - true sandboxed isolation across runs on shared infra remains out. -- **Artifact retention/telemetry.** `--keep-days` prunes scratch, but there's no aggregate run - history/metrics store to learn which prompts/models perform best over time. -- ✅ **Prompt-template regression testing.** Now **M3.8** (golden-structure tests). Deeper - golden-output/eval integration with `tools/ai-evals` remains a future enhancement. -- **Windows path/shell portability.** `.git/info/exclude`, atomic rename, and gate shell commands - need explicit cross-platform handling; the repo is multi-OS. -- **Secret redaction depth.** Basic pattern redaction is built in from M2.1/M6.4; exhaustive - entropy-based detection and safe-field allowlists remain future work. - ---- - -## 8. Stop/go gate summary - -| Gate | Milestone | Go criterion | -| --- | --- | --- | -| **G0** | M0 | Isolation (nonce), concurrency, permissions, and an enforcement path (native or specified fallback) all confirmed in `FINDINGS.md` (+ `FALLBACK.md` if scoping is unavailable). | -| **G1** | M1 | Atomic/crash-safe I/O, stable hashes, exclusive lock, state round-trip. | -| **G2** | M2 | Phase runs end-to-end with empty git-diff guard + exactly one fresh-session retry on bad output. | -| **G3** | M3 | Templates stable; phases 1–2 produce well-structured artifacts on a sample. | -| **G4** | M4 | Parallel, non-overlapping `research/*` under the cap; single-item skip path works. | -| **G5** | M5 | Full pipeline runs e2e on a branch; failing-gate halts correctly. **(key)** | -| **G6** | M6 | Resume recovers; edit-between-phases flows; dry-run makes no calls. | -| **G7** | M7 | All §9.6 example sessions behave as documented; correct exit codes. | -| **G8** | M8 | CI green; README/CODEOWNERS/index complete. | diff --git a/gao.md b/gao.md deleted file mode 100644 index 697b88d9969..00000000000 --- a/gao.md +++ /dev/null @@ -1,42 +0,0 @@ -# Plan Critique - -The plan is strong on milestone gating and explicit success proof, but several guarantees are still under-specified enough that implementation could drift or become unsafe. - -## High-priority gaps - -1. **Generated gate commands are the biggest unresolved safety risk.** - The plan notes this in open questions, but it should be resolved before M5, not "before M5.6". A model-generated `plan.md` can currently cause the orchestrator to run arbitrary shell. Define an allowlist, cwd, env, timeout, output limits, interactive-command blocking, and approval semantics before implementing `parsePlanStages`. - -2. **Audit/log ownership is too trusting.** - Phase 6 agents append `execution-log.md`, `handoff.md`, and plan-change entries themselves. That lets the same actor that made an edit also self-report why it was safe. Prefer an orchestrator-owned structured log, e.g. JSONL, where the agent can propose entries but the orchestrator records actual diffs, gates, exit codes, and file changes. - -3. **`manifest.json` ownership is ambiguous.** - M1 makes `manifest.json` part of orchestrator state, but M3 says prompt `01` records decisions in `manifest.json`. Agents should not directly write core state. Better: agents emit decision artifacts; orchestrator validates and updates `manifest.json`. - -4. **The git-diff guard needs a precise definition.** - "`git diff` and untracked files empty" is not enough. `git diff` misses untracked files; ignored files may be hidden; `.agentic-workflow/` is intentionally ignored. Define the exact command/algorithm, exclude only the artifact root, include untracked source files, and compare against a pre-phase snapshot. - -5. **Redaction arrives too late.** - M2 streams events into `execution-log.md`, but redaction is deferred to M6. If SDK output, tool args, shell output, or env leaks appear in logs before M6, the damage is already done. Build log redaction into M2 logging from the start, then harden in M6. - -6. **The read-only checkout / overlay fallback is design-critical but underspecified.** - If native tool scoping is unavailable, the fallback must define worktree creation, artifact handoff, patch transfer, cleanup, symlink behavior, nested repos, ignored files, and how implementation diffs are applied back. This should become a concrete M0 output, not just a yes/no branch. - -## Ambiguities to tighten - -- **Exit codes conflict:** failing gates are described as `10`/`1` in multiple places, while M7 says `10 = paused`, `1 = failure`. Decide whether a failed gate is pause-for-human or hard failure. -- **Plan machine block format:** `stages:` / `gate:` needs exact delimiters and syntax, likely a fenced YAML or JSON block with schema versioning. -- **Approval model:** MVP says mandatory approval before implement, M7 adds `approve`, `resume`, `--yes`, and gate-set overrides. Define where approval is stored and what exactly resumes after approval. -- **Fresh context proof:** planting a fact in session A and asking session B is useful but weak. Use a nonce that is never written to artifacts/repo and define what isolation means relative to SDK memory, external tools, and local files. -- **Resume invalidation:** if an upstream artifact changes, the plan says downstream reads the edited content, but it does not define which completed phases become stale or must re-run. -- **Phase-4 dependencies:** `dependsOn` controls ordering, but it is unclear whether dependent item research receives prior item research as input. -- **Budget ceiling:** cost/quota is listed as open, but fan-out starts in M4. A hard budget should exist before parallel execution. - -## Opportunities - -- Add **schema versions** to every machine artifact: `state.json`, `manifest.json`, `subitems.json`, plan block, handoff, and execution log. -- Use a **typed schema source of truth** such as Zod/TypeBox plus generated JSON Schema to avoid TS types and JSON schemas drifting. -- Make **prompt-template regression tests** part of M3/M8, not future work. Even lightweight golden-structure tests would protect the core IP. -- Add a **repo/branch-level lock or dirty-worktree preflight** before implementation. Per-run locks do not prevent two runs from editing the same branch. -- Separate **mocked deterministic tests** from **real SDK smoke tests**. Gates should not depend on live quota unless explicitly marked opt-in. -- Consider making the MVP "review mode" the default. Fully unattended autopilot should require explicit opt-in once gate-command safety, budgets, and redaction are proven. diff --git a/thinnerplan.plan.md b/thinnerplan.plan.md deleted file mode 100644 index 2f881f3e9e2..00000000000 --- a/thinnerplan.plan.md +++ /dev/null @@ -1,245 +0,0 @@ -# Thin-Spine Plan — Agentic `research → plan → implement` - -> A deliberately minimal re-cut of `.plan.md`, driven by one constraint: **agent harnesses -> improve rapidly, so write as little bespoke machinery as possible.** Keep the durable value -> (the methodology) in portable plain-text assets, enforce only the guarantees the harness -> can't yet give for free, and lean on the Copilot extension/SDK surface for everything else. -> When the harness grows a capability, we **delete** code — we don't rewrite it. - ---- - -## 0. The one idea - -Split the system into two layers and treat them completely differently: - -| Layer | What it is | Strategy | -| --- | --- | --- | -| **Durable core (the product)** | The 6 phase templates + 2 judge templates + the artifact/phase contract + the machine-readable gate block | Plain markdown + plain data. **Zero framework coupling.** Survives any harness. | -| **Thin spine (the liability)** | Orchestration: spawn fresh session per phase, run gates, capture log, enforce policy | The smallest possible TypeScript, with **all harness calls behind one adapter file**. Lean on the extension/SDK surface; defer everything speculative. | - -If a better harness (or a native "workflow/pipeline" primitive) ships tomorrow, we re-point a -~200-line driver at it and keep ~90% of the value. - ---- - -## 1. What we keep, delegate, and defer - -**Keep (durable, or the only guarantees worth bespoke code):** -- The **6 phase templates** + **2 judge templates** (`prompts/01-research … 06-implement.md`, - `critique.md`, `revise.md`) — the actual IP. -- The **artifact contract** (each phase reads/writes files under `.agentic-workflow//`). -- **Fresh session per phase** — the one thing a skill/single-session can't do (design §2). -- The plan's **machine-readable gate block** (`stages[]` + `gate{commands,expected}`) — the - agent reads it and runs/validates its own stage gates inside the implement session. - -**Delegate to the Copilot extension / SDK surface (don't hand-roll):** -- **Per-phase read-only enforcement** → `onPreToolUse` returning `permissionDecision: "deny"` - (replaces the entire `FALLBACK.md` read-only-checkout machinery and demotes the git-diff - guard to a cheap backstop). -- **Sanctioned artifact writes** → a `write_artifact` custom tool (`skipPermission: true`). -- **Execution log** → `session.on("tool.execution_complete" | "assistant.message" | …)`. -- **Autonomous permissions** → `approveAll`. -- **Transient error retries** → `onErrorOccurred` → `{ errorHandling: "retry" }`. -- **Gate running + validation** → the **agent** runs the stage's `gate.commands` (shell tool) - inside its own implement session and reports pass/fail in `execution-log` + `handoff.md`. We - prefer this low-maintenance path over a code-enforced gate runner; the orchestrator just reads - the agent's reported result to decide whether to continue. -- **Interactive trigger** → a thin `.github/extensions/` front-door (slash command + tool). - -**Defer — add ONLY if the harness still doesn't do it when we actually need it:** -- Content hashing + resume **revalidation** engine (start with "re-run from last completed phase"). -- Run-lock / dirty-worktree coordination (add only when real concurrency appears). -- Redaction **depth** (ship a trivial regex; harness may own this soon). -- The full 7-command CLI + gate-set **resolution engine** (ship `run`/`resume` + `--simple` only). -- Token-budget ceiling, multi-repo scoping, autopilot sandbox, retention/telemetry. -- `FALLBACK.md` read-only-checkout (expected to be obsoleted by `onPreToolUse`). - ---- - -## 2. Architecture (thin spine) - -``` -portable assets/ ← durable IP, harness-agnostic - prompts/01..06.md + critique.md revise.md - validate.ts (a few hand-written checks: subitems + plan block) - -src/ - orchestrator.ts ← sequences phases, reads/writes artifacts, reads agent gate results - harness.ts ← *** the ONLY file that imports @github/copilot-sdk *** - wraps createSession / send / events / hooks / tools - session-options.ts ← shared hooks+tools (onPreToolUse policy, write_artifact, - event→log capture); passed into createSession AND reusable - by the extension front-door - artifacts.ts ← resolve run dir, atomic write, git-ignore (no hashing/lock yet) - gates.ts ← parse plan stages from plan.md (the agent runs the commands) - cli.ts ← `run [--simple] [--no-judge] [--judge-model] | resume [run-id]` (headless, PRIMARY entry) - -.github/extensions/agentic-workflow/ - extension.mjs ← OPTIONAL interactive front-door: registers a slash command - + `run_agentic_workflow` tool that calls the orchestrator -``` - -**Adapter rule (the whole hedge):** harness churn touches **`harness.ts` only**. The -orchestrator talks to phases through a stable internal interface (`runPhase(opts) → -{ artifacts, log }`), never to the SDK directly. - -**Two entry points, one engine:** -- `cli.ts` is the real product — headless, fits the "one command" goal (design §1), TypeScript. -- `extension.mjs` is a thin convenience shim so a user *inside* Copilot CLI can trigger the - same headless run conversationally (and get hot-reload during dev). It contains **no - orchestration logic** — it imports and calls the built orchestrator. - ---- - -## 3. The one enforced guarantee (the only bespoke rigor) - -1. **Fresh context per phase.** Every phase (and every implement-stage / fan-out item) is a - brand-new `createSession()`; retries spawn a *new* session seeded with the prior errors, - never a continued transcript. - -That's it. **Gates are run and validated by the agent itself** (it executes `gate.commands` in -its session and reports the result) — we deliberately prefer this low-maintenance path over a -code-enforced gate runner, accepting that a misreporting agent could pass a failing gate. The -orchestrator simply reads the reported result. Everything else is best-effort and leans on the -harness, in exchange for near-zero maintenance. - -### 3.1 Reflexive critique (LLM-as-judge per artifact) - -After **any** phase's artifact validates, the orchestrator runs a 2-session judge loop — this is -just the **same fresh-session + template primitive applied reflexively**, so it adds capability -without a new infra type. On by default; `--no-judge` disables it. - -1. **Critique (fresh session, *alternate* model).** `createSession({ model: judgeModel })`, - read-only (writes only `critiques/.md` via `write_artifact`). It reads the artifact - and emits **high-signal feedback only** — bugs, gaps, logic/design flaws, contract violations — - **no style nits**. Using a *different* model than the author is the point: it doesn't share the - author's blind spots. -2. **Adjudicate/revise (fresh session, author's model).** Reads the **original artifact + the - critique**, **decides which points to apply** (it's the owner, not a rubber stamp), rewrites the - artifact via `write_artifact`, and logs accepted/rejected points to the execution log. The - revised artifact is then **re-validated** — a bad revision triggers the normal fresh-session - retry. - -**Built-in rubber-duck:** Copilot ships a `rubber-duck` agent tuned for exactly this — high-signal -feedback, explicitly *no* style/formatting comments. **If the SDK can spawn a named built-in agent** -(`createSession({ agent: "rubber-duck", model: judgeModel })` — the "custom agents via SDK" surface, -**verified in T0.5**), use it as the critique session; otherwise the `critique.md` template encodes -the same ethos. Either way, pin the **alternate** model for diversity where the API allows. - -**Cost:** this ~triples sessions per artifact, so it's flag-gated (`--no-judge`) and the judge model -is configurable (`--judge-model`). This is the one place we *add* sessions for quality — justified -because catching a bad spec/plan early is far cheaper than implementing on top of it. - ---- - -## 4. Milestones (4, not 8) - -### T0 — Capability spike *(gates everything; keep it to a day)* -Verify, in a throwaway `spike/`: -- **T0.1** `createSession()` runs headless, streams, stops cleanly; `approveAll` works. -- **T0.2** **Isolation** via single-use nonce (session A reveals it, fresh session B can't). -- **T0.3 (decisive)** **Does `createSession()` accept `hooks` (esp. `onPreToolUse`)?** - - **yes →** hook-based `deny` is our §6.1 enforcement; `FALLBACK.md` is dropped entirely. - - **no →** hooks live only in `joinSession`; fall back to the **post-phase git-diff guard** - as primary enforcement (still cheap, no checkout machinery). -- **T0.4** Concurrency: ≥3 simultaneous sessions for phase-4 fan-out. -- **T0.5** **Can `createSession()` spawn a named built-in agent** (e.g. `agent: "rubber-duck"`) and - **override the model per session**? Decides whether the §3.1 critique uses the built-in - rubber-duck or the `critique.md` template; the alternate-model override is required either way. -- **⛔ G0:** isolation + permissions confirmed, and an enforcement path chosen (hooks *or* - git-diff guard). One short `spike/FINDINGS.md`. No `FALLBACK.md` unless T0.3 = no *and* a - diff guard proves insufficient. - -### T1 — Portable core + minimal substrate *(after G0)* -- **T1.1** The **6 phase templates** + **2 judge templates** (`critique.md`, `revise.md`) in - `prompts/` (the IP; iterate continuously after this). -- **T1.2** A handful of **plain validation functions** (`validate.ts`) for `subitems.json` + - the plan block — `JSON.parse`/`YAML.parse` then check the few required fields and return - readable errors. **No schema library.** Nothing else gets validation until a real format breaks. -- **T1.3** `artifacts.ts`: resolve `.agentic-workflow//`, atomic write, git-ignore via - `.git/info/exclude` + inner `.gitignore`. **No hashing, no run-lock yet.** -- **⛔ G1:** templates render with run vars; validators accept/reject fixtures; artifacts dir is - created and git-ignored without touching tracked `.gitignore`. - -### T2 — Thin spine *(after G1 — the core)* -- **T2.1** `harness.ts`: wrap `createSession` with `session-options.ts` (the shared - `onPreToolUse` policy + `write_artifact` tool + event→`execution-log.jsonl` capture + - `onErrorOccurred` retry). Stable `runPhase()` interface out. -- **T2.2** `orchestrator.ts`: sequence phases 1→6 reading/writing artifacts; fan-out phase 4 - with bounded concurrency; **fresh-session retry** on validation/policy failure. Under - `--simple`, skip phases 1/3/4 and synthesize a single-item `subitems.json` so phases 5–6 - see an identical contract. -- **T2.3** `gates.ts`: parse plan stages from `plan.md`. The **agent runs `gate.commands`** in - its implement session and reports pass/fail in `execution-log`/`handoff.md`; the orchestrator - reads that and stops on a reported failure (no bespoke gate runner). -- **T2.4** Staged implement: one fresh sub-session per stage, `handoff.md` carries forward. -- **T2.5** Judge loop (§3.1): a `judgeArtifact(path, authorModel)` helper — spawn critique (alt - model, or built-in `rubber-duck` per T0.5) → spawn adjudicate/revise (author model) → - re-validate. Wrap every validated artifact; honor `--no-judge` / `--judge-model`. -- **⛔ G2 (key):** a real sample task runs end-to-end on a branch; every edit maps to a plan - step; each validated artifact is critiqued by an alt-model session and revised; the agent runs - its stage gates and a failing gate is reported and halts the run; non-impl phases mutate - nothing outside `.agentic-workflow/` (proven by hook-deny *or* the diff guard). - -### T3 — Entry points + package *(after G2)* -- **T3.1** `cli.ts`, two commands: - - **`run [task] [--simple] [--no-judge] [--judge-model ]`** (default) — drives the headless - pipeline. Prints the **human-readable run-id** it creates (e.g. `20260629-1310-add-auth` = - timestamp + task slug). `--simple` skips the optional phases — research (1), classify (3), and - per-item research (4) — synthesizing a single-item path and running assumptions → plan → - implement. `--no-judge` disables the §3.1 critique loop; `--judge-model` sets the alternate - critique model. - - **`resume [run-id]`** — re-runs from the last completed phase (no hash engine). `run-id` - is **optional**: with one incomplete run, resume it; with several, **refuse and list the - candidates** (id + task + last completed phase) so the user picks; an explicit id resumes - that run. - - Exit codes: `0` done / `10` paused / `1` fail / `2` usage. -- **T3.2 (leverage the extension)** `.github/extensions/agentic-workflow/extension.mjs`: a - slash command + `run_agentic_workflow` tool that invokes the orchestrator from inside an - interactive Copilot CLI session (hot-reloadable). Thin shim, no logic. -- **T3.3** `README.md`, `ci.yml` (`tools - agentic-workflow - ci`), CODEOWNERS row - (`/tools/agentic-workflow/ @JennyPng`), root README index, archive `spike/`. -- **⛔ G3:** `npm run build && npm test` green; `run` completes a sample headlessly; the - extension front-door triggers the same run interactively. **Done.** - ---- - -## 5. How the extension surface is leveraged (summary) - -| Need | Bespoke code in `.plan.md` | Thin-spine: lean on harness | -| --- | --- | --- | -| Read-only non-impl phases | `FALLBACK.md` worktrees + patch-apply + git-diff guard | `onPreToolUse` → `deny` (guard = backstop only) | -| Sanctioned writes | custom `write_artifact` tool | same, `skipPermission: true` | -| Execution log | orchestrator event plumbing | `session.on(...)` events | -| Transient retries | bespoke backoff module (M6.1) | `onErrorOccurred` → `retry` | -| Gate running/validation | orchestrator runs commands + reads exit codes | agent runs them in-session, reports result | -| Artifact critique (judge) | n/a | built-in `rubber-duck` agent if SDK-spawnable, else `critique.md` on an alt model | -| Permissions | resolve exact option (M0.4) | `approveAll` | -| Interactive trigger | n/a | `.github/extensions/` slash command + tool | - -Caveat: hooks on `createSession` are **verified in T0.3**. If unavailable there, the -`onPreToolUse` items degrade to the git-diff guard, and the extension front-door still stands -as the interactive trigger. - ---- - -## 6. Validation (lean) - -- **Unit:** validators accept/reject fixtures; `gates.ts` parses stages from `plan.md`; `artifacts` - atomic write + git-ignore (and non-git no-op); `onPreToolUse` policy denies a write in a - non-impl phase (or diff-guard catches it). -- **Scenario (mocked):** fresh-session retry (fail-once-then-succeed); agent-reported failing - gate halts the run; `resume` continues from last completed phase; judge loop runs - critique→revise on alt model and `--no-judge` skips it. -- **E2E (opt-in, real SDK, throwaway branch):** one small task to completion; log maps every - edit to a plan step; gates pass; exit `0`. Not in CI unless a cheap-model lane exists. -- **CI:** `npm run build` + mocked tests + format check only. - ---- - -## 7. Success proof - -A run is "done" when all phases complete, every gate passed, `execution-log.jsonl` maps each -edit to a `plan.md` step, and exit `0` — **and** the spine stays under a few hundred lines with -the entire SDK footprint isolated in `harness.ts`. The second clause is the point of this cut: -the day the harness ships a native multi-phase primitive, the spine collapses into config. diff --git a/tools/agentic-workflow/.gitignore b/tools/agentic-workflow/.gitignore index e684348b168..c3d6ff562a7 100644 --- a/tools/agentic-workflow/.gitignore +++ b/tools/agentic-workflow/.gitignore @@ -2,5 +2,3 @@ node_modules/ dist/ *.tgz .agentic-workflow/ -spike/node_modules/ -spike/package-lock.json diff --git a/tools/agentic-workflow/.prettierignore b/tools/agentic-workflow/.prettierignore index ddea4133444..d3864901fea 100644 --- a/tools/agentic-workflow/.prettierignore +++ b/tools/agentic-workflow/.prettierignore @@ -1,6 +1,5 @@ node_modules/ dist/ prompts/ -spike/ *.md .agentic-workflow/ diff --git a/tools/agentic-workflow/README.md b/tools/agentic-workflow/README.md index ba124cb7741..d9e53acd382 100644 --- a/tools/agentic-workflow/README.md +++ b/tools/agentic-workflow/README.md @@ -34,7 +34,7 @@ artifact, which is then re-validated. Disable with `--no-judge`. - **Fresh context per phase / retry** — every phase, fan-out item, judge session, and retry is a brand-new `createSession()`. This is the one piece of bespoke rigor. - **Read-only non-impl phases** — enforced by an `onPreToolUse` hook that *denies* edit/shell tools - in phases 1–5 (verified live; see `spike/FINDINGS.md`). A post-phase git-diff check is a backstop. + in phases 1–5 (verified live during the capability spike). A post-phase git-diff check is a backstop. - **Sanctioned writes** — phases persist artifacts only through a `write_artifact` custom tool (path-traversal guarded). - **Gates** — the plan embeds a machine-readable `stages:`/`gate:` block. The implement agent runs @@ -108,7 +108,6 @@ src/ artifacts.ts run-dir, atomic write, local git-ignore state.ts state.json (resume) cli.ts headless entry point -spike/ archived T0 capability spike + FINDINGS.md (SDK assumptions, verified live) ``` ## Develop diff --git a/tools/agentic-workflow/spike/FINDINGS.md b/tools/agentic-workflow/spike/FINDINGS.md deleted file mode 100644 index 0442ac7ce16..00000000000 --- a/tools/agentic-workflow/spike/FINDINGS.md +++ /dev/null @@ -1,39 +0,0 @@ -# T0 Capability Spike — Findings (G0) - -Verified **live** against `@github/copilot-sdk@1.0.4` (Node 25, logged-in user auth) via -`spike.mjs` + `tool-test.mjs`. Model used: `claude-haiku-4.5`. - -| ID | Capability | Result | Evidence | -| --- | --- | --- | --- | -| T0.1 | `createSession()` headless, streams, stops cleanly; `approveAll` | ✅ | sessions created/answered/disconnected; `client.stop()` clean | -| T0.2 | **Isolation** via single-use nonce | ✅ | nonce `ISO-…` told to session A; fresh session B replied `NONE` (no bleed) | -| **T0.3** | **`createSession()` accepts `hooks` (`onPreToolUse`)** | ✅ **YES** | `hooks.onPreToolUse` returning `{permissionDecision:"deny"}` blocked a shell call; agent reported it could not run the command | -| T0.4 | ≥3 concurrent sessions (phase-4 fan-out) | ✅ | 3 simultaneous sessions each returned correct distinct answers | -| T0.5 | Spawn **named built-in** agent (e.g. `rubber-duck`) + per-session model override | ⚠️ Partial | per-session `model` override ✅; but `SessionConfig.agent` **only references `customAgents[]`** you define — no direct built-in-agent spawn by name | - -Custom tool: `defineTool(name, { parameters, handler, skipPermission:true })` works; the agent -called `write_artifact` and the handler fired. `defineTool`'s name is the **first positional arg**. - -## Decisions gated by this spike - -1. **Enforcement path = hooks (`onPreToolUse` → `deny`).** T0.3 is YES, so read-only enforcement - for non-impl phases is a hook that denies edit/shell tools. **`FALLBACK.md` read-only-checkout - machinery is dropped entirely.** The post-phase git-diff guard remains as a cheap backstop only. -2. **Judge critique uses the `critique.md` template on an *alternate model*** (not a built-in - `rubber-duck` spawn), because `agent` can't name a built-in agent. Per-session model override — - the part actually required for judge diversity — is confirmed working. -3. **`write_artifact`** is a `defineTool` custom tool with `skipPermission: true`. -4. **Autonomous runs** use `onPermissionRequest: approveAll`. - -## API shape confirmed (from `dist/*.d.ts`, authoritative) - -- `client.createSession(config: SessionConfig)`; `SessionConfig extends SessionConfigBase`. -- `SessionConfigBase.model?: string`; `SessionConfig.hooks?: SessionHooks` - (`onPreToolUse`, `onPostToolUse`, `onPostToolUseFailure`, `onErrorOccurred`). -- `onErrorOccurred` returns `{ errorHandling?: "retry" | "skip" | "abort" }` → transient retries. -- `onPreToolUse` returns `{ permissionDecision: "allow"|"deny"|"ask", permissionDecisionReason? }`. -- `Tool.skipPermission?: boolean`; `approveAll: PermissionHandler`. -- Events: `assistant.message`, `session.idle` (+ typed `tool.execution_complete`, etc.). -- Session lifecycle: `session.send({prompt})`, `session.disconnect()`, `client.stop()`. - -**G0 = PASS.** Isolation + permissions confirmed; enforcement path chosen (hooks). No `FALLBACK.md`. diff --git a/tools/agentic-workflow/spike/package.json b/tools/agentic-workflow/spike/package.json deleted file mode 100644 index 8bbc8756056..00000000000 --- a/tools/agentic-workflow/spike/package.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "name": "spike", - "version": "1.0.0", - "description": "", - "main": "index.js", - "scripts": { - "test": "echo \"Error: no test specified\" && exit 1" - }, - "keywords": [], - "author": "", - "license": "ISC", - "type": "commonjs", - "dependencies": { - "@github/copilot-sdk": "^1.0.4" - } -} diff --git a/tools/agentic-workflow/spike/spike.mjs b/tools/agentic-workflow/spike/spike.mjs deleted file mode 100644 index 3fa38add2ed..00000000000 --- a/tools/agentic-workflow/spike/spike.mjs +++ /dev/null @@ -1,81 +0,0 @@ -// T0 capability spike — verifies the SDK assumptions the whole design rests on. -// Run: node spike.mjs (writes findings to stdout; auth via logged-in user) -import { CopilotClient, approveAll, defineTool } from "@github/copilot-sdk"; - -const MODEL = process.env.SPIKE_MODEL || "claude-haiku-4.5"; -const results = {}; - -function ask(session, prompt, timeoutMs = 60000) { - return new Promise(async (resolve, reject) => { - let text = ""; - const timer = setTimeout(() => reject(new Error("timeout")), timeoutMs); - session.on("assistant.message", (e) => { text += e.data.content ?? ""; }); - session.on("session.idle", () => { clearTimeout(timer); resolve(text); }); - await session.send({ prompt }); - }); -} - -const client = new CopilotClient(); -await client.start(); -try { - // T0.1 + T0.2: isolation via single-use nonce - const nonce = "ISO-" + Math.random().toString(36).slice(2, 10).toUpperCase(); - const a = await client.createSession({ model: MODEL, onPermissionRequest: approveAll }); - await ask(a, `Remember this exact token, I will quiz you later: ${nonce}. Reply only "ok".`); - const b = await client.createSession({ model: MODEL, onPermissionRequest: approveAll }); - const bAns = await ask(b, `Earlier in THIS conversation I gave you a token starting with "ISO-". Output it verbatim. If you have never seen it, output exactly NONE.`); - results.isolation = !bAns.includes(nonce); - results.isolationDetail = { nonce, fresh_session_reply: bAns.trim().slice(0, 120) }; - await a.disconnect(); - await b.disconnect(); - - // T0.3: hooks (onPreToolUse deny) accepted by createSession - let denyFired = false; - const h = await client.createSession({ - model: MODEL, - onPermissionRequest: approveAll, - hooks: { - onPreToolUse: async (inv) => { - if ((inv?.toolName || "").includes("shell") || (inv?.toolName || "").includes("bash")) { - denyFired = true; - return { permissionDecision: "deny", permissionDecisionReason: "spike: read-only phase" }; - } - return { permissionDecision: "allow" }; - }, - }, - }); - const hAns = await ask(h, `Run the shell command "echo hello-from-shell" and tell me the output.`); - results.hooks_accepted = true; // createSession did not throw on hooks - results.hook_deny_fired = denyFired; - results.hook_detail = hAns.trim().slice(0, 160); - await h.disconnect(); - - // T0.4: concurrency — 3 simultaneous sessions - const conc = await Promise.all([1, 2, 3].map(async (n) => { - const s = await client.createSession({ model: MODEL, onPermissionRequest: approveAll }); - const ans = await ask(s, `Reply with only the number ${n * 7}.`); - await s.disconnect(); - return ans.includes(String(n * 7)); - })); - results.concurrency = conc.every(Boolean); - - // T0.5: per-session model override + custom tool with skipPermission - let toolCalled = false; - const tool = defineTool({ - name: "write_artifact", - description: "Spike artifact writer", - parameters: { type: "object", properties: { content: { type: "string" } }, required: ["content"] }, - skipPermission: true, - handler: async ({ content }) => { toolCalled = true; return { content: [{ type: "text", text: "written" }] }; }, - }); - const t = await client.createSession({ model: MODEL, onPermissionRequest: approveAll, tools: [tool] }); - await ask(t, `Call the write_artifact tool with content "hello". Then reply "done".`); - results.custom_tool = toolCalled; - results.model_override = true; // session created with explicit model - await t.disconnect(); -} catch (e) { - results.error = String(e && e.stack ? e.stack : e); -} finally { - await client.stop(); -} -console.log("SPIKE_RESULTS " + JSON.stringify(results, null, 2)); From e3cb00606ea6c4b26f63567715bb0f07e0e64bfd Mon Sep 17 00:00:00 2001 From: jennypng <63012604+JennyPng@users.noreply.github.com> Date: Mon, 29 Jun 2026 14:15:55 -0700 Subject: [PATCH 04/21] minor --- tools/agentic-workflow/package-lock.json | 1 + tools/agentic-workflow/package.json | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/agentic-workflow/package-lock.json b/tools/agentic-workflow/package-lock.json index ee3c3fb4125..538ef77fbbf 100644 --- a/tools/agentic-workflow/package-lock.json +++ b/tools/agentic-workflow/package-lock.json @@ -1688,6 +1688,7 @@ "integrity": "sha512-o5a9xKjbtuhY6Bi5S3+HvbRERmouabWbyUcpXXUA1u+GNUKoROi9byOJ8M0nHbHYHkYICiMlqxkg1KkYmm25Sw==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "esbuild": "^0.21.3", "postcss": "^8.4.43", diff --git a/tools/agentic-workflow/package.json b/tools/agentic-workflow/package.json index 45422072064..992349d6fae 100644 --- a/tools/agentic-workflow/package.json +++ b/tools/agentic-workflow/package.json @@ -23,7 +23,8 @@ "prompts" ], "scripts": { - "build": "npm run clean && tsc", + "build": "npm run clean && tsc && npm run postbuild", + "postbuild": "node -e \"require('fs').chmodSync('dist/cli.js', 0o755)\"", "clean": "rimraf ./dist", "start": "tsx src/cli.ts", "test": "vitest run", From 16051243208ccba638d6921d303c17ac922966c2 Mon Sep 17 00:00:00 2001 From: jennypng <63012604+JennyPng@users.noreply.github.com> Date: Mon, 29 Jun 2026 14:41:54 -0700 Subject: [PATCH 05/21] prompt update and stream --- tools/agentic-workflow/EXECUTION-LOG.md | 27 +++++++++ tools/agentic-workflow/README.md | 6 ++ .../prompts/02-assumptions.md | 60 ++++++++++++++++++- tools/agentic-workflow/prompts/03-classify.md | 2 +- .../agentic-workflow/prompts/06-implement.md | 58 ++++++++++++++---- tools/agentic-workflow/prompts/critique.md | 19 +++++- tools/agentic-workflow/src/cli.ts | 8 ++- tools/agentic-workflow/src/harness.ts | 35 ++++++++++- 8 files changed, 192 insertions(+), 23 deletions(-) diff --git a/tools/agentic-workflow/EXECUTION-LOG.md b/tools/agentic-workflow/EXECUTION-LOG.md index 58702d50bc9..dd3ccba9fd0 100644 --- a/tools/agentic-workflow/EXECUTION-LOG.md +++ b/tools/agentic-workflow/EXECUTION-LOG.md @@ -182,3 +182,30 @@ than reverting them or expanding the thinnerplan scope to build the handoff/cont Anyone reconciling the doc against the code should treat §6.3 (and the `handoff.md` / `context_needed` / `buildContextPack` references in §3, §5, the M5 milestone, and the summary) as **planned, not yet built**. + +--- + +## Post-T3 enhancement — live session streaming + +**Action.** Surfaced the live Copilot session to the console so a run is observable while it +executes. The SDK exposes `streaming: true` on `createSession` (emitting ephemeral +`assistant.message_delta` with `deltaContent`) plus `tool.execution_start` (which, unlike +`tool.execution_complete`, carries a clean `toolName`). `SdkHarness` now takes a `stream` flag; when +set it enables `streaming`, prints a `=== (model: …) ===` header, streams assistant +`deltaContent` inline, prints `[tool] ` on each tool start, and a `[done] — N tool +call(s)` footer — **all to stderr**, leaving stdout for the final machine-readable summary. The CLI +defaults streaming **on** and adds `--quiet` to disable. As a side benefit, `tool_start` (with +`toolName`) is now also recorded in `execution-log.jsonl`, filling the gap where +`tool.execution_complete` lacked a tool name. + +**Justification.** Kept inside the adapter (`harness.ts`) — the orchestrator is untouched because +`stream` is a harness-construction concern, preserving the "harness churn stays in harness.ts" seam. +Streaming goes to stderr so `2>/dev/null` still yields clean, pipeable stdout. + +**Gate validation:** +- `tsc` clean, `npm run build` ok, `format:check` clean, **36/36** tests pass (FakeHarness unaffected + by the `SdkHarness` constructor change). +- **Live E2E** in a throwaway repo (`--simple --no-judge`): exit **0**; stderr streamed all three + phases live (`assumptions`/`plan`/`implement:stage-1`) with per-tool lines, the model's prose, and + the `STAGE_RESULT: pass`; the real code edit landed (`farewell()` added to `src/greet.js`); the + final summary appeared on **stdout** only. diff --git a/tools/agentic-workflow/README.md b/tools/agentic-workflow/README.md index d9e53acd382..a3f0edc13aa 100644 --- a/tools/agentic-workflow/README.md +++ b/tools/agentic-workflow/README.md @@ -80,6 +80,12 @@ Without linking you can still invoke it directly with `node dist/cli.js run "` | Alternate model used for the critique session. | | `--run-id ` | Explicit run id (default: `YYYYMMDD-HHMM-`). | | `--out ` | Working-dir root (default: `./.agentic-workflow`). | +| `--quiet` | Suppress live session streaming (streaming to stderr is **on by default**). | + +**Live progress:** by default the tool streams each phase's assistant output and tool activity to +**stderr** as it happens (`=== ===` headers, `[tool] ` lines, the model's text, and a +`[done] — N tool call(s)` footer). The final run summary goes to **stdout**, so you can pipe +results cleanly (`agentic-workflow run "…" 2>/dev/null`) or silence streaming with `--quiet`. **Exit codes:** `0` done · `10` paused (resume / blocking clarification) · `1` failure · `2` usage. diff --git a/tools/agentic-workflow/prompts/02-assumptions.md b/tools/agentic-workflow/prompts/02-assumptions.md index db0d54e072f..5cde6bbba4a 100644 --- a/tools/agentic-workflow/prompts/02-assumptions.md +++ b/tools/agentic-workflow/prompts/02-assumptions.md @@ -1,7 +1,8 @@ # Phase 2 — Assumptions (read-only) -You are in the **assumptions** phase. Enumerate the baseline assumptions, unknowns, and risks the -later planning/implementation will rely on. You are **read-only**. +You are in the **assumptions** phase. Surface every assumption I might be making about +this work, the codebase, and the constraints. Do not propose a plan, do +not propose code changes, and do not classify the work yet. ## Task {{task}} @@ -14,6 +15,61 @@ later planning/implementation will rely on. You are **read-only**. `assumptions.md` — one assumption per line, each with a short **rationale** and a **confidence** (`high` / `medium` / `low`). +Cover at least the following categories: + +1. Assumptions about the work item + - What the work item is asking for. + - What it is not asking for. + - Acceptance criteria, whether stated or inferred. + - Priority, deadline, and known stakeholders. + +2. Assumptions about the codebase + - Which areas, modules, or services are likely affected. + - Which areas are out of scope. + - Existing patterns, conventions, and idioms in the relevant area. + +3. Assumptions about behavior + - What the current behavior is. + - What the expected behavior is after the work is complete. + - What behavior must remain unchanged. + +4. Assumptions about constraints + - Build, test, and deployment constraints. + - Performance, reliability, security, and compliance constraints. + - Compatibility constraints with consumers, callers, or other services. + +5. Assumptions about validation + - How I will know the work is done. + - What evidence I will need to show my reviewer. + - What tests already exist that protect the affected area. + +6. Open questions and unknowns + - Things I cannot determine from the work item or codebase alone. + - Things that require a decision from me or my manager. + +Scope discipline: +- Every assumption you list must be tied to this specific work item. Do + not list assumptions about unrelated parts of the codebase, even if + you notice issues there. +- This is a large, older codebase. You will see refactor opportunities, + dead code, outdated patterns, and other smells that have nothing to + do with this work item. Do not weave them into the assumptions. + Capture them in a separate "Out-of-scope observations" section at the + end of assumptions.md so they are not lost, but keep them clearly + marked as out of scope. +- For each assumption, include a one-line justification that ties it to + the work item, an acceptance criterion, or a specific area the work + item touches. If you cannot justify it against the work item, do not + include it. + +Use clear +headings, short bullet points, and call out each assumption explicitly. +For open questions, list them at the end so I can resolve them before +moving on. + +Keep the language simple and +straightforward. + ### Blocking clarifications If any assumption is **low-confidence AND affects correctness, security, or API behavior**, mark it explicitly with a leading `blocking: true` token on that line, e.g.: diff --git a/tools/agentic-workflow/prompts/03-classify.md b/tools/agentic-workflow/prompts/03-classify.md index ba416e7a682..40c88455567 100644 --- a/tools/agentic-workflow/prompts/03-classify.md +++ b/tools/agentic-workflow/prompts/03-classify.md @@ -1,6 +1,6 @@ # Phase 3 — Classify & split (read-only) -You are in the **classify** phase. Classify the task and split it into **independent, +You are in the **classify** phase. Classify the task as either refactor, new feature, or bug. If the work spans multiple buckets, split it into **independent, non-overlapping** sub-items. You are **read-only**. ## Task diff --git a/tools/agentic-workflow/prompts/06-implement.md b/tools/agentic-workflow/prompts/06-implement.md index c5b9a27fae5..fa04912722f 100644 --- a/tools/agentic-workflow/prompts/06-implement.md +++ b/tools/agentic-workflow/prompts/06-implement.md @@ -1,7 +1,9 @@ # Phase 6 — Implement a single stage -You are implementing **one stage** of the plan in a fresh session. You **may edit source code and -run shell commands** for this stage only. Earlier stages already ran; their code is on disk. +You are in **IMPLEMENT mode**, implementing **one stage** of the approved plan in a fresh session. +You **may edit source code and run shell commands** for this stage only. Earlier stages already +ran; their code is on disk. Implement the approved plan **exactly as written**. Keep the language +in your logs simple. No emojis. ## Task {{task}} @@ -13,6 +15,7 @@ run shell commands** for this stage only. Earlier stages already ran; their code ## Context pack (clean ≠ blank — everything you need has been gathered for you) - **`plan.md`** is the source of truth (read it, including the running "Plan changes" log). +- **`assumptions.md`** is your context document — read it before you start. - **Prior handoffs** from earlier stages: {{handoff}} - **Cumulative diff so far** (files already changed in this run): @@ -20,23 +23,52 @@ run shell commands** for this stage only. Earlier stages already ran; their code - **`context_needed`** files this stage depends on: {{contextNeeded}} ## What to do -1. Implement **this stage's steps**, editing the real source files. `expected_files` is the - anticipated scope — **advisory, not a wall**. You may edit beyond it when correctness requires - it, **provided you document the deviation** (see below). -2. **Run this stage's `gate.commands`** yourself (shell tool) and confirm they meet `expected` +1. Implement **this stage's steps**, editing the real source files. Follow the plan step-by-step. + `expected_files` is the anticipated scope — **advisory, not a wall**. You may edit beyond it + when correctness requires it, **provided you document the deviation** (see below). +2. Make **small, reviewable changes.** Group related edits, and for each group summarize *what* + changed, *which plan step* it satisfies, and *what you verified*. +3. **Run this stage's `gate.commands`** yourself (shell tool) and confirm they meet `expected` (e.g. exit code 0). If a gate fails, fix the code and re-run until it passes or you are genuinely blocked. -3. Append to **`execution-log.md`** (via `write_artifact`, appending) — for **every** action: - (a) **justify the scope** (why it was necessary) and (b) **map it to a concrete step** in - `plan.md` by stage/step id. Record each gate command, its exit code, and pass/fail. -4. Append a concise entry to **`handoff.md`** (via `write_artifact`, appending) for the *next* +4. Append to **`execution-log.md`** (via `write_artifact`, appending) — see "Execution log" below. +5. Append a concise entry to **`handoff.md`** (via `write_artifact`, appending) for the *next* stage: what you built, the **new/changed public symbols and files**, decisions/conventions established, anything deferred, and known follow-ups. +## Scope discipline (stay strictly within the plan) +This may be a large, older codebase. You **will** see refactor opportunities, outdated patterns, +dead code, and smells in the files you touch. **Do not act on any of them.** If a fix is genuinely +required to make a plan step work, **STOP** and follow the deviation policy below instead of doing +it silently. +- **Observability:** add or update observability **exactly as the plan defines** — no more. +- **Dependencies:** do **not** introduce dependencies beyond what the plan and your team's package + policy allow. Use only feeds/registries your team approves. +- **Tests:** do **not** write tests in this step unless a plan step explicitly calls for it. + (Running the stage's existing `gate.commands` is required and is not the same as authoring tests.) +- **Secrets/PII:** never include secrets or PII in code, logs, or artifacts. +- **Commits:** do **not** run `git commit`. Stage your changes and let the build be verified first. +- Every change must tie to a specific plan step. If a change **cannot** be tied to a plan step, + **do not make it** — record it under "Out-of-scope observations" in the execution log instead. + +## Execution log (`execution-log.md`) +For **every** change group, record: +- a **one-line scope justification** naming the concrete `plan.md` step (by stage/step id) it + satisfies — *(a)* why it was necessary and *(b)* what it maps to; +- each **gate command**, its exit code, and pass/fail. +Also maintain these sections in the log: +- a **Test results** section (placeholder, to be filled later if tests are out of scope here); +- a **Plan changes** section (mirrors any deviation entries you add to `plan.md`); +- a **Verification evidence** section (gate output; placeholder where evidence comes later); +- an **Out-of-scope observations** section at the bottom for anything you noticed but correctly + did *not* act on; +- a **PR description draft**: reference the work item / task id and summarize **scope, changes, + rollout, rollback, and monitoring signals**. + ## Deviation policy (documented, not blocked) -Deviating from the plan is **allowed** when it is the correct response to a gap. The requirement -is transparency: -- Append a **"Plan changes"** entry to `plan.md` describing what changed and why. +Deviating from the plan is **allowed** when it is the correct response to a gap. If the plan is +incomplete or incorrect: **(1) STOP. (2)** append a **"Plan changes"** entry to `plan.md` +describing the deviation and rationale. **(3)** continue only after that update is recorded. - The `execution-log.md` entry for the deviating action justifies its scope and maps it to the originating step / plan-change entry. - **Larger deviations** — a change to architecture, public API, or overall test strategy — must diff --git a/tools/agentic-workflow/prompts/critique.md b/tools/agentic-workflow/prompts/critique.md index e1951355e91..7b6df0d8a7d 100644 --- a/tools/agentic-workflow/prompts/critique.md +++ b/tools/agentic-workflow/prompts/critique.md @@ -1,8 +1,6 @@ # Judge — Critique (read-only, alternate model) -You are a **critic** running in a fresh, isolated session on a **different model** than the author, -so you do not share the author's blind spots. Your job is to find real problems in one artifact — -**high-signal feedback only**. +You must critique the input document written by a different AI model. Identify gaps and opportunities. ## Artifact under review `{{artifactPath}}` (read it). @@ -17,6 +15,21 @@ Only substantive issues that would make the downstream work wrong, unsafe, or in - **Contract violations** — the artifact does not satisfy its phase's required format/sections. - **Design flaws** — choices that will cause rework or break a stated constraint. +Rubric: +1. Specificity -- Is the assumption falsifiable? Could a reader point at a + file, line, behavior, or work-item field that would prove it wrong? +2. Scope -- Does the assumption name the module, file, endpoint, or + boundary it applies to, or is it written in the abstract? +3. Coverage gap -- Are any of the six required categories (work item, + codebase, behavior, constraints, validation, open questions) thin, + missing, or treated as boilerplate? +4. Platitude -- Flag any assumption that would be true of almost any + software project ("must be backward compatible", "should have good + performance", "code must be maintainable"). These are not useful + assumptions; they are restated values. +5. Open-question candidate -- Flag any item written as an assertion that + is actually unknown and should be moved to the open questions list. + **Do NOT** comment on style, wording, formatting, or trivia. If the artifact is sound, say so and keep it short — do not invent problems. diff --git a/tools/agentic-workflow/src/cli.ts b/tools/agentic-workflow/src/cli.ts index f7587fde8e8..7dc986c28e3 100644 --- a/tools/agentic-workflow/src/cli.ts +++ b/tools/agentic-workflow/src/cli.ts @@ -47,8 +47,8 @@ function parseArgs(argv: string[]): ParsedArgs { const USAGE = `agentic-workflow — research -> plan -> implement, one fresh session per phase Usage: - agentic-workflow run "" [--simple] [--no-judge] [--judge-model ] [--out ] [--run-id ] - agentic-workflow resume [] [--out ] + agentic-workflow run "" [--simple] [--no-judge] [--judge-model ] [--out ] [--run-id ] [--quiet] + agentic-workflow resume [] [--out ] [--quiet] Options: --simple Skip research, classify, per-item research (assumptions -> plan -> implement). @@ -56,6 +56,7 @@ Options: --judge-model Alternate model used for the critique session. --out Working-dir root (default: ./.agentic-workflow). --run-id Explicit run id (default: timestamp + task slug). + --quiet Suppress live session streaming to stderr (streaming is on by default). Exit codes: 0 done, 10 paused (resume/clarify), 1 failure, 2 usage.`; @@ -142,7 +143,8 @@ async function main(): Promise { return 2; } - const harness = new SdkHarness({ workingDirectory: process.cwd() }); + const stream = flags.quiet !== true; + const harness = new SdkHarness({ workingDirectory: process.cwd(), stream }); let result: RunResult; try { result = await runWorkflow(opts, harness); diff --git a/tools/agentic-workflow/src/harness.ts b/tools/agentic-workflow/src/harness.ts index 9d5fa0e14aa..d8c406c152d 100644 --- a/tools/agentic-workflow/src/harness.ts +++ b/tools/agentic-workflow/src/harness.ts @@ -43,7 +43,7 @@ const DEFAULT_TIMEOUT_MS = 15 * 60 * 1000; export class SdkHarness implements Harness { private client: CopilotClient | undefined; - constructor(private readonly defaults: { workingDirectory?: string } = {}) {} + constructor(private readonly defaults: { workingDirectory?: string; stream?: boolean } = {}) {} private async getClient(): Promise { if (!this.client) { @@ -58,8 +58,15 @@ export class SdkHarness implements Harness { const client = await this.getClient(); logEvent(req.logPath, { kind: "phase_start", label: req.label, readOnly: req.readOnly, model: req.model }); + const streaming = this.defaults.stream === true; + const write = (s: string) => process.stderr.write(s); + if (streaming) { + write(`\n=== ${req.label}${req.model ? ` (model: ${req.model})` : ""} ===\n`); + } + const session = await client.createSession({ model: req.model, + streaming, onPermissionRequest: approveAll, tools: [makeWriteArtifactTool(req.runDir, req.logPath)], hooks: { @@ -70,9 +77,29 @@ export class SdkHarness implements Harness { let finalText = ""; let toolCalls = 0; + let midLine = false; session.on("assistant.message", (e) => { finalText += e.data?.content ?? ""; }); + if (streaming) { + session.on("assistant.message_delta", (e) => { + const chunk = e.data?.deltaContent ?? ""; + if (chunk) { + write(chunk); + midLine = !chunk.endsWith("\n"); + } + }); + session.on("tool.execution_start", (e) => { + if (midLine) { + write("\n"); + midLine = false; + } + write(` [tool] ${e.data?.toolName ?? "?"}\n`); + }); + } + session.on("tool.execution_start", (e) => { + logEvent(req.logPath, { kind: "tool_start", toolName: e.data?.toolName, toolCallId: e.data?.toolCallId }); + }); session.on("tool.execution_complete", (e) => { toolCalls += 1; logEvent(req.logPath, { kind: "tool_complete", success: e.data?.success, toolCallId: e.data?.toolCallId }); @@ -95,6 +122,12 @@ export class SdkHarness implements Harness { await session.disconnect().catch(() => {}); } + if (streaming) { + if (midLine) { + write("\n"); + } + write(` [done] ${req.label} — ${toolCalls} tool call(s)\n`); + } logEvent(req.logPath, { kind: "phase_end", label: req.label, toolCalls }); return { artifacts: [], finalText }; } From d1a5677d2b94e78d6d129ec11ec1b814ad060e5a Mon Sep 17 00:00:00 2001 From: jennypng <63012604+JennyPng@users.noreply.github.com> Date: Tue, 30 Jun 2026 16:10:22 -0700 Subject: [PATCH 06/21] initial dump of cli extension --- .github/CODEOWNERS | 2 +- .../extensions/agentic-workflow/.gitignore | 2 + .../agentic-workflow/EXECUTION-LOG.md | 162 ++ .github/extensions/agentic-workflow/README.md | 116 + .../extensions/agentic-workflow/extension.mjs | 517 +++- .../agentic-workflow/package-lock.json | 209 ++ .../extensions/agentic-workflow/package.json | 16 + .../agentic-workflow/prompts/01-research.md | 56 + .../prompts/02-assumptions.md | 23 +- .../agentic-workflow/prompts/03-classify.md | 20 +- .../prompts/04-research-item.md | 42 + .../agentic-workflow/prompts/05-plan.md | 66 + .../agentic-workflow/prompts/06-implement.md | 67 +- .../agentic-workflow/prompts/critique.md | 51 + .../agentic-workflow/test/smoke.test.mjs | 68 + README.md | 2 +- cli-extension-design.md | 141 + cli-extension-impl.md | 220 ++ tools/agentic-workflow/.gitignore | 4 - tools/agentic-workflow/.prettierignore | 5 - tools/agentic-workflow/.prettierrc.json | 6 - tools/agentic-workflow/EXECUTION-LOG.md | 211 -- tools/agentic-workflow/README.md | 132 - tools/agentic-workflow/ci.yml | 57 - tools/agentic-workflow/package-lock.json | 2316 ----------------- tools/agentic-workflow/package.json | 49 - tools/agentic-workflow/prompts/01-research.md | 31 - .../prompts/04-research-item.md | 26 - tools/agentic-workflow/prompts/05-plan.md | 63 - tools/agentic-workflow/prompts/critique.md | 39 - tools/agentic-workflow/prompts/revise.md | 30 - tools/agentic-workflow/src/artifacts.ts | 142 - tools/agentic-workflow/src/cli.ts | 166 -- tools/agentic-workflow/src/gates.ts | 83 - tools/agentic-workflow/src/harness.ts | 141 - tools/agentic-workflow/src/index.ts | 7 - tools/agentic-workflow/src/orchestrator.ts | 385 --- tools/agentic-workflow/src/prompts.ts | 34 - tools/agentic-workflow/src/session-options.ts | 129 - tools/agentic-workflow/src/state.ts | 53 - tools/agentic-workflow/src/types.ts | 80 - tools/agentic-workflow/src/validate.ts | 98 - tools/agentic-workflow/test/artifacts.test.ts | 73 - tools/agentic-workflow/test/gates.test.ts | 48 - .../test/orchestrator.test.ts | 227 -- tools/agentic-workflow/test/prompts.test.ts | 43 - .../test/session-options.test.ts | 63 - tools/agentic-workflow/test/validate.test.ts | 102 - tools/agentic-workflow/tsconfig.json | 19 - 49 files changed, 1664 insertions(+), 4978 deletions(-) create mode 100644 .github/extensions/agentic-workflow/.gitignore create mode 100644 .github/extensions/agentic-workflow/EXECUTION-LOG.md create mode 100644 .github/extensions/agentic-workflow/README.md create mode 100644 .github/extensions/agentic-workflow/package-lock.json create mode 100644 .github/extensions/agentic-workflow/package.json create mode 100644 .github/extensions/agentic-workflow/prompts/01-research.md rename {tools => .github/extensions}/agentic-workflow/prompts/02-assumptions.md (77%) rename {tools => .github/extensions}/agentic-workflow/prompts/03-classify.md (61%) create mode 100644 .github/extensions/agentic-workflow/prompts/04-research-item.md create mode 100644 .github/extensions/agentic-workflow/prompts/05-plan.md rename {tools => .github/extensions}/agentic-workflow/prompts/06-implement.md (55%) create mode 100644 .github/extensions/agentic-workflow/prompts/critique.md create mode 100644 .github/extensions/agentic-workflow/test/smoke.test.mjs create mode 100644 cli-extension-design.md create mode 100644 cli-extension-impl.md delete mode 100644 tools/agentic-workflow/.gitignore delete mode 100644 tools/agentic-workflow/.prettierignore delete mode 100644 tools/agentic-workflow/.prettierrc.json delete mode 100644 tools/agentic-workflow/EXECUTION-LOG.md delete mode 100644 tools/agentic-workflow/README.md delete mode 100644 tools/agentic-workflow/ci.yml delete mode 100644 tools/agentic-workflow/package-lock.json delete mode 100644 tools/agentic-workflow/package.json delete mode 100644 tools/agentic-workflow/prompts/01-research.md delete mode 100644 tools/agentic-workflow/prompts/04-research-item.md delete mode 100644 tools/agentic-workflow/prompts/05-plan.md delete mode 100644 tools/agentic-workflow/prompts/critique.md delete mode 100644 tools/agentic-workflow/prompts/revise.md delete mode 100644 tools/agentic-workflow/src/artifacts.ts delete mode 100644 tools/agentic-workflow/src/cli.ts delete mode 100644 tools/agentic-workflow/src/gates.ts delete mode 100644 tools/agentic-workflow/src/harness.ts delete mode 100644 tools/agentic-workflow/src/index.ts delete mode 100644 tools/agentic-workflow/src/orchestrator.ts delete mode 100644 tools/agentic-workflow/src/prompts.ts delete mode 100644 tools/agentic-workflow/src/session-options.ts delete mode 100644 tools/agentic-workflow/src/state.ts delete mode 100644 tools/agentic-workflow/src/types.ts delete mode 100644 tools/agentic-workflow/src/validate.ts delete mode 100644 tools/agentic-workflow/test/artifacts.test.ts delete mode 100644 tools/agentic-workflow/test/gates.test.ts delete mode 100644 tools/agentic-workflow/test/orchestrator.test.ts delete mode 100644 tools/agentic-workflow/test/prompts.test.ts delete mode 100644 tools/agentic-workflow/test/session-options.test.ts delete mode 100644 tools/agentic-workflow/test/validate.test.ts delete mode 100644 tools/agentic-workflow/tsconfig.json diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 350afe67be7..c2b129c6489 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -65,7 +65,7 @@ # Azure SDK Tools MCP ##################### /eng/common/instructions/azsdk-tools/ @maririos @praveenkuttappan @MrJustinB -/tools/agentic-workflow/ @JennyPng +/.github/extensions/agentic-workflow/ @JennyPng # PRLabel: %azsdk-cli /tools/ai-evals/azsdk-mcp/ @jeo02 @smw-ms @praveenkuttappan @maririos # PRLabel: %azsdk-cli diff --git a/.github/extensions/agentic-workflow/.gitignore b/.github/extensions/agentic-workflow/.gitignore new file mode 100644 index 00000000000..9b03e82f80d --- /dev/null +++ b/.github/extensions/agentic-workflow/.gitignore @@ -0,0 +1,2 @@ +node_modules/ +.aw/ diff --git a/.github/extensions/agentic-workflow/EXECUTION-LOG.md b/.github/extensions/agentic-workflow/EXECUTION-LOG.md new file mode 100644 index 00000000000..ad1760d38cd --- /dev/null +++ b/.github/extensions/agentic-workflow/EXECUTION-LOG.md @@ -0,0 +1,162 @@ +# Execution Log — CLI-Extension Phased Workflow + +Implementation of `cli-extension-impl.md`. Records every action taken, the justification, and any +deviation from the plan. The plan's design principle (lean on the agent, minimize deterministic +code) was followed throughout: the result is `extension.mjs` + `prompts/` with no `lib/` of +validators, gate parsers, artifact tools, or a state machine. + +## Step 0 — Prerequisites + +- Created `.github/extensions/agentic-workflow/package.json` (`type: module`, dependency + `@github/copilot-sdk ^1.0.4`). +- `npm install` → installed SDK v1.0.4 (confirmed `node_modules/@github/copilot-sdk/package.json` + version 1.0.4). +- Verified import: `node -e "import('@github/copilot-sdk/extension')…"` → `ok`. +- **Verification of SDK surface (v1.0.4 .d.ts):** confirmed every API the plan/design name before + using it — `joinSession` (extension.d.ts), `CustomAgentConfig {name,model,tools:string[]|null, + prompt}` (types.d.ts:1157), `DefaultAgentConfig.excludedTools` (1208), `customAgents`/`defaultAgent` + /`infiniteSessions` on the join config (1626/1633/InfiniteSessionConfig), `session.rpc.agent.select + ({name})` (rpc.d.ts), `session.sendAndWait(prompt|MessageOptions,timeout)` returning + `AssistantMessageEvent` with text at `.data.content` (session.d.ts:126, session-events.d.ts:2634), + `session.ui.{confirm,select,input,elicitation}` gated on `session.capabilities.ui?.elicitation` + (types.d.ts SessionUiApi), `session.setModel` (session.d.ts:268), `session.rpc.commands.enqueue + ({command})` (rpc.d.ts:14553), `session.abort()` (254), `session.log` (280), and + `commands: CommandDefinition[]` with `handler(ctx)` + raw `ctx.args` (types.d.ts:425,1424). + +## Step 1 — Prompt templates (durable IP) + +Created `prompts/{01-research,02-assumptions,03-classify,04-research-item,05-plan,06-implement, +critique}.md`, carried over from the prototype's reasoning content (the durable IP) with the +plan's three strips applied: + +1. `write_artifact`/`read_artifact` custom-tool references + run-context preamble → "write/read this + file under the run directory (`{{runDir}}`) with your normal file tools". +2. The machine-readable `stages:`/`gate:` YAML block the orchestrator parsed → "define your stages + **in prose** with explicit gate commands and **run the gates yourself**" (05-plan.md, 06-implement + reads prose stages). +3. Fresh-isolated-session / external-validator wording → the `PHASE_RESULT: pass|fail|needs_input` + self-report. Blocking assumptions now surface as `needs_input` instead of an orchestrator-parsed + `blocking: true` pause. + +Kept `{{task}}` and added `{{runDir}}`/`{{priorErrors}}` placeholders (verified each template uses +only those three — supplied by `dispatch`). Each template now self-checks its output; 06-implement +additionally runs the gate commands. + +**Deviation D1 — research-item is one agent over all sub-items, not N dispatches.** The prototype +dispatched phase 4 once per sub-item with `{{item}}`/`{{itemId}}` placeholders. To stay agent-first +and keep `dispatch` uniform (only `{{task}}/{{runDir}}/{{priorErrors}}`), `04-research-item.md` +instructs the single `aw-research-item` agent to read `subitems.json` itself and produce +`research/.md` for every item in dependency order. Lower-maintenance, no per-item orchestration +code. (Aligns with @JennyPng's "lean on the agent" preference.) + +**Deviation D2 — implement is one agent over all stages, not one dispatch per stage.** The prototype +ran a fresh session per stage with `{{stage}}`/`{{handoff}}`/`{{cumulativeDiff}}` injected. Here the +single `aw-implement` agent reads the whole prose plan and works stage-by-stage, running each stage's +gates before advancing. This removes the stage state machine and the YAML gate parser entirely — the +explicit goal of the plan ("no state machine, run gates yourself"). + +**Deviation D3 — `/aw-judge` revises via the author phase, no separate `revise.md`.** The prototype +had a `revise.md` adjudication prompt. The impl plan's `/aw-judge` spec is "run critique, then re-run +the author phase with the critique as `priorErrors`", so the author phase's own template handles the +revision. `revise.md` was intentionally dropped. + +## Steps 2–5 — `extension.mjs` + +Single file: a phase registry (`PHASES` + `CRITIQUE`) carrying per-phase model, scoped tools, +template, and the sentinel artifact for completion detection; `dispatch`; `autoRun`; thin command +handlers; and the `joinSession` config. + +- **Step 2 (agents + guardrail).** Built `customAgents` (one per phase incl. critique) with scoped + `tools` and a stable role `prompt`, plus `defaultAgent.excludedTools: + ["edit","create","delete","write"]` and `infiniteSessions.enabled`. Read phases get + `["view","glob","grep"]` (research also `bash` for `gh`/`git`/`curl`); implement gets `tools:null` + (all). Tool names match this CLI host's tools. + + **Deviation D4 — models repinned at runtime via `setModel`, not pinned in `customAgents[].model`.** + The plan's table pins a model per agent and `/aw-model` is supposed to repin live. But + `customAgents` is fixed at join time, and a pinned `customAgents[].model` would *win* over + `session.setModel`, making `/aw-model` a no-op without an `/extensions reload`. To make `/aw-model` + genuinely work (the design's "repin models" goal) I omit `customAgents[].model`, store each phase's + model in the registry, and apply it via `session.setModel(phase.model)` immediately before + `agent.select` in `dispatch`. Agents with no pinned model fall back to the session model + (CustomAgentConfig.model docs), so this is clean. `/aw-model` mutates `phase.model` for the next + dispatch. Net: same per-phase pinned-model behavior, but live-repinnable. + + **Deviation D5 — agent `prompt` is a short role line; full filled template delivered per dispatch.** + The plan put "the phase prompt" in `customAgents[].prompt`. But the templates contain + `{{task}}/{{runDir}}` unknown at join time, so a statically-pinned template would leak literal + `{{…}}` into agent context. Instead each agent's pinned `prompt` is a concise role/identity line + and the fully-substituted phase instructions are sent per-dispatch via `sendAndWait`. The prompt + files remain the durable IP; only their delivery moment changed. Avoids double-injection and + placeholder leakage. + +- **Step 3 (dispatch + commands).** Run dir = `/.aw//` (repo-agnostic, derived from + `process.cwd()`); active run tracked in memory; next phase derived by `fs.existsSync` checks on + each phase's sentinel artifact (no `state.json`). `dispatch(phase,{priorErrors})` selects the + agent, repins model, substitutes the three placeholders, `sendAndWait`s, and parses the + `PHASE_RESULT` sentinel with one regex (`parsePhaseResult`). Thin handlers for `/aw-start`, + `/aw-research`…`/aw-implement`, `/aw-continue [n]`, `/aw-judge`, `/aw-redo`, `/aw-model`, + `/aw-status` (artifacts + `git diff --stat`, degrades on non-git). + + **Minor D6 — templates embed `{{task}}`, so `dispatch` does not also prepend the raw task.** The + plan sketch wrote `prompt: task + "\n\n" + template`. Since every template already has a `## Task + {{task}}` section, prepending would duplicate it; `dispatch` just substitutes. + +- **Step 4 (auto-runner + elicitation).** `autoRun({from,to,unattended,pauseAt})` walks the phase + order; per phase branches on `PHASE_RESULT`: `pass`→advance; `needs_input`→`askHuman` (UI + `input` when `capabilities.ui?.elicitation` && not unattended, else a safe-default answer); + `fail`→retry up to `MAX_RETRIES=2` with the agent's reason as `priorErrors`, then `resolveFailure` + (UI `select` retry/skip/abort, or abort when unattended/no-UI). `/aw-pause` sets a cooperative flag + checked each boundary; `pauseAt` honored as a breakpoint. `from` defaults to the next incomplete + phase, `to` defaults to `implement`. + +- **Step 5 (context hygiene).** `/aw-compact` enqueues `/compact` via + `session.rpc.commands.enqueue`; `dispatch` auto-enqueues `/compact` before the `implement` phase; + `infiniteSessions:{enabled:true}` set in the join config. All state on disk → resume by re-deriving + the next phase from existing files. + +- **Removed the old shim.** The prior `extension.mjs` (a thin front-door delegating to the built + prototype `dist/`) was deleted and fully replaced. + +## Step 6 — Replace prototype, document + +- Deleted `tools/agentic-workflow/` (prototype). Verified no remaining active references in + `.github/`, `eng/`, or `README.md`. +- `README.md` index row for `agentic-workflow` repointed to + `.github/extensions/agentic-workflow/README.md` with an updated description. +- `.github/CODEOWNERS`: `/tools/agentic-workflow/ @JennyPng` → + `/.github/extensions/agentic-workflow/ @JennyPng`. +- Wrote `.github/extensions/agentic-workflow/README.md` (purpose, layout, phase/agent table, + install, command list, manual/auto modes, `PHASE_RESULT` convention, develop/test). +- Added `.gitignore` (`node_modules/`, `.aw/`) and a `test` npm script. +- **Light test (optional in plan):** `test/smoke.test.mjs` (node:test) covers the only non-trivial + pure logic — `parsePhaseResult`, `parseKv`, `isTruthy`, `slugify` — plus the `joinConfig` shape + (seven agents, tool scoping, default-agent guardrail, command surface). Guarded the top-level + `joinSession` behind `AW_SKIP_JOIN` so the module imports cleanly under test without a live host. + + Left the older `agentic-workflow-design.md` (the *prototype's* historical design doc) untouched — + out of scope for this implementation; only `cli-extension-design.md`/`cli-extension-impl.md` drive + this work. + +## Validation performed + +- `node --check extension.mjs` → clean. +- `npm test` → 8/8 pass (pure logic + join-config shape). +- `npm ci` → lockfile consistent, 0 vulnerabilities. +- SDK import smoke (`@github/copilot-sdk/extension`) → ok. + +**Not run here:** the live `/extensions info agentic-workflow` and a real end-to-end `/aw-start → +… → /aw-implement` run require the interactive Copilot CLI host, which isn't available in this +environment. The join config is verified against the v1.0.4 type definitions and the +`joinConfig`-shape test; these are the Step 1–6 "Verify" items achievable offline. The remaining +live verifications are the first manual dogfood per the plan's Rollout step 1 (spike on a throwaway +branch). + +## What is code vs delegated to the agent (as built) + +Irreducible code: `customAgents` registry (scoped tools + role) + runtime `setModel` repin; +`defaultAgent.excludedTools` guardrail; thin `commands` + `parseKv`/`isTruthy` arg parsing; +`dispatch` + `parsePhaseResult` (one regex); `autoRun` loop + stop branching; `session.ui` +elicitation calls. Delegated to the agent: artifact correctness/self-check, running gate commands, +stage breakdown/sequencing, reading/writing artifacts, classification + research fan-out, judging +critiques. diff --git a/.github/extensions/agentic-workflow/README.md b/.github/extensions/agentic-workflow/README.md new file mode 100644 index 00000000000..e45445102b2 --- /dev/null +++ b/.github/extensions/agentic-workflow/README.md @@ -0,0 +1,116 @@ +# Agentic Workflow — Copilot CLI extension + +A self-contained [Copilot CLI](https://github.com/github/copilot-cli) extension that drives a +disciplined **research → plan → implement** workflow inside your interactive session. Each phase +runs as its own sub-agent with a **pinned model**, **scoped tools**, and a **phase prompt**, handing +off to the next phase through **artifacts on disk**. You stay in the loop — advance phases manually, +inspect artifacts between steps, repin models, answer decisions via dialogs — or let the auto-runner +chain phases unattended. + +## Design + +Agent-first: the extension is *config + thin command handlers + a tiny dispatch loop*. There is no +`lib/` of validators, gate parsers, or a state machine. Each phase agent self-reports with a single +sentinel line the loop reads: + +``` +PHASE_RESULT: pass | fail | needs_input +``` + +(optionally followed by a short reason). That one convention replaces deterministic validators and +gate parsers — the agent owns correctness, runs its own gate commands, and reads/writes artifacts +with normal file tools. + +### Layout + +``` +.github/extensions/agentic-workflow/ + extension.mjs # joinSession: per-phase sub-agents + commands + auto-runner loop + package.json # @github/copilot-sdk dependency + prompts/ # one .md per phase (+ critique) — the durable IP + test/ # smoke test for arg parsing + PHASE_RESULT parsing + README.md +``` + +### Phases + +| Phase | Agent | Model (default) | Tools | Writes | +| --- | --- | --- | --- | --- | +| research | `aw-research` | `claude-sonnet-4.5` | read + shell | `specs/*.md`, `manifest.json` | +| assumptions | `aw-assumptions` | session default | read | `assumptions.md` | +| classify | `aw-classify` | `claude-haiku-4.5` | read | `subitems.json`, `classification.md` | +| research-item | `aw-research-item` | `claude-sonnet-4.5` | read | `research/.md` | +| plan | `aw-plan` | `claude-sonnet-4.5` | read | `plan.md` | +| implement | `aw-implement` | `claude-sonnet-4.5` | **all** | code edits + `execution-log.md`, `handoff.md` | +| critique | `aw-critique` | `claude-haiku-4.5` | read | `critiques/.md` | + +The default agent has mutating tools (`edit`/`create`/`delete`/`write`) excluded, so only the +`implement` phase can change source. + +State lives entirely in a per-run directory: `/.aw//`. The workflow is inspectable, +resumable, and survives an extension reload — the next phase is derived from which artifact files +already exist, so re-running `/aw-start ` resumes where you left off. + +## Install + +The Copilot CLI **auto-discovers** extensions — it scans `.github/extensions/` (project) and +`~/.copilot/extensions/` (user) for subdirectories containing an `extension.mjs`. There is no +`/extensions` command; discovery is automatic. This extension ships in the repo under +`.github/extensions/agentic-workflow/`. Install its dependencies once: + +```bash +cd .github/extensions/agentic-workflow +npm install +``` + +Then make the CLI (re)discover it and confirm it loaded: + +``` +/clear # extensions are (re)loaded on /clear; or restart the CLI entirely +/env # lists loaded extensions, agents, and commands — confirm agentic-workflow is there +``` + +> If the extension was added while the CLI was already running, a full **restart** of the CLI is the +> most reliable way to pick it up (the in-session `/clear` reload also works once it's been +> discovered). + +To use it across every repo you open, copy this folder (with its installed `node_modules`) into the +user extensions directory `~/.copilot/extensions/agentic-workflow/`, then `/clear` or restart. + +## Commands + +| Command | Behavior | +| --- | --- | +| `/aw-start [simple]` | Init the run dir and run the first phase. `simple` = short flow (research → assumptions → plan → implement). | +| `/aw-run [from:

] [to:

] [unattended:true] [pause-at:

]` | Auto-run a range of phases. | +| `/aw-continue [n]` | Run the next phase (or next `n`), then stop. | +| `/aw-pause` | Stop the auto-runner at the next phase boundary. | +| `/aw-research` … `/aw-implement` | Run one specific phase. | +| `/aw-judge ` | Critique an artifact, then re-run its author phase to revise it. | +| `/aw-redo ` | Re-run a phase with steering notes. | +| `/aw-model ` | Repin a phase's model for its next run. | +| `/aw-status` | Show phase checklist, run-dir artifacts, and `git diff --stat`. | +| `/aw-compact` | Reclaim context by queuing `/compact`. | + +## Execution modes + +- **Manual** — `/aw-` or `/aw-continue`; stops after every phase. +- **Ranged auto** — `/aw-run from:assumptions to:plan`; stops at the boundary or a stop condition. +- **Full auto** — `/aw-run` (or `unattended:true`); runs to `implement`, halting only at gates, + `needs_input`, or hard failure. + +At a `needs_input` stop (interactive, not `unattended`) the runner shows a `session.ui` dialog to +collect the answer and feeds it back into the next attempt. On `fail` it retries up to twice with the +agent's reason as feedback, then asks retry/skip/abort. With `unattended:true` (or no UI capability) +it auto-resolves with safe defaults and only halts on hard failure. + +## Develop / test + +```bash +npm test # node --test: smoke test for arg parsing + PHASE_RESULT parsing +``` + +The durable IP is the per-phase prompt in `prompts/` plus the on-disk artifact contract; tuning +prompts is usually the only thing that needs iteration. Keep prompts repo-agnostic — let the agent +discover repo specifics at runtime; repo-specific guidance belongs in that repo's own `.github/` +instructions, not here. diff --git a/.github/extensions/agentic-workflow/extension.mjs b/.github/extensions/agentic-workflow/extension.mjs index 92e2eb8f623..647390a83a8 100644 --- a/.github/extensions/agentic-workflow/extension.mjs +++ b/.github/extensions/agentic-workflow/extension.mjs @@ -1,91 +1,460 @@ -// Optional interactive front-door for the agentic workflow (thinnerplan T3.2). +// Agentic-workflow Copilot CLI extension. // -// Thin shim ONLY: it registers a `/agentic-workflow` slash command + a `run_agentic_workflow` -// tool inside an interactive Copilot CLI session, and delegates to the *built* orchestrator. It -// contains no orchestration logic of its own — re-point the import and it still works. +// Self-contained, agent-first driver for a phased research -> plan -> implement workflow. +// One joinSession() registers a custom sub-agent per phase (scoped tools + phase prompt); models +// are repinnable at runtime via session.setModel before each dispatch. State lives entirely in a +// per-run directory on disk (/.aw//) so the workflow is inspectable, resumable, and +// survives an extension reload. The agent self-reports each phase with a sentinel line: +// PHASE_RESULT: pass | fail | needs_input (+ optional reason) +// which the dispatch loop and auto-runner read instead of any deterministic validator. +// +// Protocol hygiene: .mjs only; diagnostics go to stderr; user-facing messages go to session.log. + import { joinSession } from "@github/copilot-sdk/extension"; import { fileURLToPath } from "node:url"; import * as path from "node:path"; +import * as fs from "node:fs"; +import { execSync } from "node:child_process"; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const PROMPTS_DIR = path.join(HERE, "prompts"); +const PHASE_TIMEOUT_MS = 30 * 60 * 1000; // generous wait; does not abort in-flight agent work +const MAX_RETRIES = 2; + +// --- Phase registry: the per-phase config (default model + scoped tools + prompt template). ------ +// Models default here but are mutable at runtime (see /aw-model). Tool scoping + prompt are bound +// into customAgents at join time. `artifact` is the sentinel file/dir that marks the phase complete. +// `simple` phases form the abbreviated flow used by `/aw-start simple`. +const READ_TOOLS = ["view", "glob", "grep"]; +const READ_SHELL_TOOLS = ["view", "glob", "grep", "bash"]; + +const PHASES = [ + { id: "research", agent: "aw-research", template: "01-research.md", model: "claude-sonnet-4.5", tools: READ_SHELL_TOOLS, artifact: "specs/architecture.md", simple: true }, + { id: "assumptions", agent: "aw-assumptions", template: "02-assumptions.md", model: null, tools: READ_TOOLS, artifact: "assumptions.md", simple: true }, + { id: "classify", agent: "aw-classify", template: "03-classify.md", model: "claude-haiku-4.5", tools: READ_TOOLS, artifact: "subitems.json", simple: false }, + { id: "research-item", agent: "aw-research-item", template: "04-research-item.md", model: "claude-sonnet-4.5", tools: READ_TOOLS, artifact: "research", simple: false }, + { id: "plan", agent: "aw-plan", template: "05-plan.md", model: "claude-sonnet-4.5", tools: READ_TOOLS, artifact: "plan.md", simple: true }, + { id: "implement", agent: "aw-implement", template: "06-implement.md", model: "claude-sonnet-4.5", tools: null, artifact: "execution-log.md", simple: true }, +]; + +const CRITIQUE = { id: "critique", agent: "aw-critique", template: "critique.md", model: "claude-haiku-4.5", tools: READ_TOOLS }; + +// Map an artifact filename to the phase that authors it (for /aw-judge). +const ARTIFACT_TO_PHASE = { + "architecture.md": "research", + "functional.md": "research", + "apispec.md": "research", + "assumptions.md": "assumptions", + "subitems.json": "classify", + "classification.md": "classify", + "plan.md": "plan", +}; + +// --- Module-scoped run state (in memory; the run dir on disk is the source of truth). ------------ +let session; +let activeTask = ""; +let activeRunDir = ""; +let activeSimple = false; +let pauseRequested = false; + +const log = (msg) => session?.log(`agentic-workflow: ${msg}`).catch(() => {}); +const diag = (msg) => process.stderr.write(`[agentic-workflow] ${msg}\n`); + +// --- Run-dir + phase helpers --------------------------------------------------------------------- +function slugify(task) { + const s = task.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-+|-+$/g, "").slice(0, 40); + return s || "run"; +} +export { slugify }; + +function readTemplate(name) { + return fs.readFileSync(path.join(PROMPTS_DIR, name), "utf8"); +} + +function phaseOrder(simple) { + return simple ? PHASES.filter((p) => p.simple) : PHASES; +} + +function phaseComplete(runDir, phase) { + const target = path.join(runDir, phase.artifact); + if (!fs.existsSync(target)) return false; + // research-item's artifact is a directory: complete only when it holds at least one note. + try { + const st = fs.statSync(target); + if (st.isDirectory()) return fs.readdirSync(target).length > 0; + } catch { + return false; + } + return true; +} + +function nextPhase(runDir, simple) { + return phaseOrder(simple).find((p) => !phaseComplete(runDir, p)) || null; +} + +function phaseById(id) { + return PHASES.find((p) => p.id === id) || null; +} + +// Parse the agent's self-reported sentinel line. Returns { result, reason }. +export function parsePhaseResult(text) { + const m = (text ?? "").match(/PHASE_RESULT:\s*(pass|fail|needs_input)\b[^\S\r\n]*[—:-]?[^\S\r\n]*(.*)/i); + const result = (m?.[1] ?? "fail").toLowerCase(); + const reason = (m?.[2] ?? (m ? "" : "no PHASE_RESULT sentinel found")).trim(); + return { result, reason }; +} + +// --- Core dispatch: select agent, repin model, send phase prompt, parse PHASE_RESULT. ------------ +async function dispatch(phase, { priorErrors = "" } = {}) { + if (!activeRunDir) { + await log("no active run. Start one with /aw-start ."); + return { result: "fail", reason: "no active run", text: "" }; + } + // Automatic context check before the expensive implement phase. + if (phase.id === "implement") { + try { + await session.rpc.commands.enqueue({ command: "/compact" }); + } catch (e) { + diag(`pre-implement compact skipped: ${e?.message ?? e}`); + } + } + try { + if (phase.model) await session.setModel(phase.model); + } catch (e) { + diag(`setModel(${phase.model}) failed, using session default: ${e?.message ?? e}`); + } + await session.rpc.agent.select({ name: phase.agent }); + + const priorBlock = priorErrors ? `## Prior feedback to address\n${priorErrors}\n` : ""; + const prompt = readTemplate(phase.template) + .replaceAll("{{task}}", activeTask) + .replaceAll("{{runDir}}", activeRunDir) + .replaceAll("{{priorErrors}}", priorBlock); + + await log(`running ${phase.id}${phase.model ? ` (${phase.model})` : ""}…`); + const ev = await session.sendAndWait({ prompt }, PHASE_TIMEOUT_MS); + const text = ev?.data?.content ?? ""; + const { result, reason } = parsePhaseResult(text); + diag(`phase ${phase.id} -> ${result}${reason ? ` (${reason})` : ""}`); + return { result, reason, text }; +} + +// --- Human interaction (elicitation), gated on capability + unattended. -------------------------- +function uiAvailable() { + return Boolean(session?.capabilities?.ui?.elicitation); +} + +async function askHuman(question, unattended) { + if (unattended || !uiAvailable()) { + return "No human is available (unattended run). Proceed with your most reasonable assumption and document it explicitly."; + } + const answer = await session.ui.input(`Needs input: ${question}`); + return answer; // null if the user cancels +} + +async function resolveFailure(phase, reason, unattended) { + if (unattended || !uiAvailable()) return "abort"; + const choice = await session.ui.select(`Phase ${phase.id} failed: ${reason || "unknown"}. What now?`, ["retry", "skip", "abort"]); + return choice ?? "abort"; +} + +// --- Auto-runner: walk phases from -> to, branching on PHASE_RESULT. ------------------------------ +async function autoRun({ from, to, unattended = false, pauseAt } = {}) { + if (!activeRunDir) { + await log("no active run. Start one with /aw-start ."); + return; + } + pauseRequested = false; + const order = phaseOrder(activeSimple); + let startIdx = from ? order.findIndex((p) => p.id === from) : -1; + if (startIdx < 0) { + const next = nextPhase(activeRunDir, activeSimple); + startIdx = next ? order.findIndex((p) => p.id === next.id) : order.length; + } + const toIdx = to ? order.findIndex((p) => p.id === to) : order.length - 1; + if (toIdx < 0) { + await log(`unknown 'to' phase: ${to}`); + return; + } + + for (let i = startIdx; i <= toIdx; i++) { + if (pauseRequested) { + await log(`paused before ${order[i].id}.`); + return; + } + const phase = order[i]; + if (phaseComplete(activeRunDir, phase)) { + diag(`skipping completed phase ${phase.id}`); + continue; + } + + let priorErrors = ""; + let retries = 0; + // Re-dispatch this phase until it passes, is skipped, or the run aborts. + for (;;) { + const res = await dispatch(phase, { priorErrors }); + if (res.result === "pass") break; + + if (res.result === "needs_input") { + const answer = await askHuman(res.reason || `${phase.id} needs a decision`, unattended); + if (answer === null) { + await log(`stopped: ${phase.id} needs input and the request was cancelled.`); + return; + } + priorErrors = answer; + retries = 0; + continue; + } + + // fail + retries += 1; + if (retries > MAX_RETRIES) { + const decision = await resolveFailure(phase, res.reason, unattended); + if (decision === "abort") { + await log(`aborted at ${phase.id} after ${MAX_RETRIES} retries: ${res.reason || "failed"}.`); + return; + } + if (decision === "skip") { + await log(`skipping ${phase.id} (manual override).`); + break; + } + retries = 0; // retry + } + priorErrors = res.reason || "The previous attempt did not pass. Fix the issues and try again."; + } + + if (pauseAt && phase.id === pauseAt) { + await log(`paused at breakpoint ${pauseAt}.`); + return; + } + } + await log(`auto-run reached ${order[toIdx].id}.`); +} + +// --- Tiny arg parsing (key:value pairs + free text). --------------------------------------------- +function parseKv(argstr) { + const kv = {}; + const rest = []; + for (const tok of (argstr ?? "").trim().split(/\s+/).filter(Boolean)) { + const m = tok.match(/^([a-z-]+):(.*)$/i); + if (m) kv[m[1].toLowerCase()] = m[2]; + else rest.push(tok); + } + return { kv, rest }; +} + +function isTruthy(v) { + return v === "" || /^(true|1|yes|on)$/i.test(v ?? ""); +} +export { parseKv, isTruthy }; + +// --- Commands ------------------------------------------------------------------------------------ +async function cmdStart(argstr) { + const { kv, rest } = parseKv(argstr); + let simple = isTruthy(kv.simple) || rest.some((t) => /^(--?)?simple$/i.test(t)); + const task = rest.filter((t) => !/^(--?)?simple$/i.test(t)).join(" ").trim() || kv.task || ""; + if (!task) { + await log("provide a task, e.g. /aw-start Add CSV export, or /aw-start Fix bug simple"); + return; + } + activeTask = task; + activeSimple = simple; + activeRunDir = path.join(process.cwd(), ".aw", slugify(task)); + fs.mkdirSync(activeRunDir, { recursive: true }); + fs.writeFileSync(path.join(activeRunDir, "task.txt"), task + "\n"); + await log(`run dir: ${path.relative(process.cwd(), activeRunDir) || activeRunDir}${simple ? " (simple flow)" : ""}`); + const next = nextPhase(activeRunDir, activeSimple); + if (!next) { + await log("all phases already complete for this task. Use /aw-status or /aw-redo ."); + return; + } + await dispatch(next); +} -const here = path.dirname(fileURLToPath(import.meta.url)); -// Resolve the built tool relative to this file: repo/.github/extensions/agentic-workflow -> tool. -const TOOL_DIST = path.resolve(here, "..", "..", "..", "tools", "agentic-workflow", "dist", "index.js"); - -let sessionRef; - -/** Parse the free-form arg string into RunOptions for the orchestrator. */ -function parseArgs(argstr) { - const tokens = (argstr ?? "").trim().match(/(?:[^\s"]+|"[^"]*")+/g) ?? []; - const opts = { task: "", judge: true }; - const taskWords = []; - for (let i = 0; i < tokens.length; i++) { - const t = tokens[i].replace(/^"|"$/g, ""); - if (t === "--simple") opts.simple = true; - else if (t === "--no-judge") opts.judge = false; - else if (t === "--judge-model") opts.judgeModel = tokens[++i]?.replace(/^"|"$/g, ""); - else if (t === "--run-id") opts.runId = tokens[++i]?.replace(/^"|"$/g, ""); - else taskWords.push(t); - } - opts.task = taskWords.join(" "); - return opts; -} - -async function execute(argstr) { - const session = sessionRef; - const opts = parseArgs(argstr); - if (!opts.task) { - await session?.log("agentic-workflow: provide a task, e.g. /agentic-workflow Add CSV export --simple"); +async function cmdPhase(id, argstr) { + const phase = phaseById(id); + if (!phase) { + await log(`unknown phase: ${id}`); + return; + } + const priorErrors = (argstr ?? "").trim(); + await dispatch(phase, { priorErrors }); +} + +async function cmdContinue(argstr) { + const n = Math.max(1, parseInt((argstr ?? "").trim(), 10) || 1); + for (let k = 0; k < n; k++) { + const next = nextPhase(activeRunDir, activeSimple); + if (!next) { + await log("no remaining phases."); + return; + } + const res = await dispatch(next); + if (res.result !== "pass") { + await log(`stopped at ${next.id} (${res.result}${res.reason ? `: ${res.reason}` : ""}).`); + return; + } + } +} + +async function cmdRun(argstr) { + const { kv } = parseKv(argstr); + await autoRun({ + from: kv.from, + to: kv.to, + unattended: isTruthy(kv.unattended), + pauseAt: kv["pause-at"] || kv.pauseat, + }); +} + +async function cmdPause() { + pauseRequested = true; + await log("pause requested; the auto-runner will stop at the next phase boundary."); +} + +async function cmdJudge(argstr) { + const artifact = (argstr ?? "").trim(); + if (!artifact) { + await log("usage: /aw-judge , e.g. /aw-judge plan.md"); + return; + } + const base = path.basename(artifact); + const phaseId = ARTIFACT_TO_PHASE[base]; + const reviewInstr = `Review the artifact at \`${artifact}\` (relative to the run dir). Read it, then write your critique to \`critiques/${base}.md\`.`; + const crit = await dispatch(CRITIQUE, { priorErrors: reviewInstr }); + if (crit.result !== "pass") { + await log(`critique did not complete (${crit.result}).`); + return; + } + const author = phaseId ? phaseById(phaseId) : null; + if (!author) { + await log(`critique written to critiques/${base}.md (no author phase mapped for ${base}; not auto-revising).`); return; } - let runWorkflow, SdkHarness; + await dispatch(author, { + priorErrors: `A critique of your previous \`${artifact}\` is at \`critiques/${base}.md\`. Read it and revise \`${artifact}\` to address every blocker/should-fix point you agree with; justify any you reject.`, + }); +} + +async function cmdRedo(argstr) { + const { rest } = parseKv(argstr); + const id = rest.shift(); + const feedback = rest.join(" ").trim(); + if (!id) { + await log("usage: /aw-redo "); + return; + } + await cmdPhase(id, feedback); +} + +async function cmdModel(argstr) { + const { rest } = parseKv(argstr); + const id = rest.shift(); + const model = rest.shift(); + if (!id || !model) { + await log("usage: /aw-model "); + return; + } + const phase = phaseById(id) || (id === "critique" ? CRITIQUE : null); + if (!phase) { + await log(`unknown phase: ${id}`); + return; + } + phase.model = model; // applied via session.setModel on the next dispatch of this phase + await log(`${id} will use model ${model} on its next run.`); +} + +function walkArtifacts(dir, base = dir, out = []) { + let entries = []; try { - ({ runWorkflow, SdkHarness } = await import(TOOL_DIST)); + entries = fs.readdirSync(dir, { withFileTypes: true }); } catch { - await session?.log( - `agentic-workflow: build the tool first (cd tools/agentic-workflow && npm run build). Looked for ${TOOL_DIST}`, - ); + return out; + } + for (const e of entries) { + const full = path.join(dir, e.name); + if (e.isDirectory()) walkArtifacts(full, base, out); + else out.push(path.relative(base, full)); + } + return out; +} + +async function cmdStatus() { + if (!activeRunDir) { + await log("no active run. Start one with /aw-start ."); return; } - await session?.log(`agentic-workflow: starting run for "${opts.task}"${opts.simple ? " (--simple)" : ""}…`); - const harness = new SdkHarness({ workingDirectory: process.cwd() }); + const order = phaseOrder(activeSimple); + const phaseLines = order.map((p) => ` ${phaseComplete(activeRunDir, p) ? "[x]" : "[ ]"} ${p.id}`).join("\n"); + const artifacts = walkArtifacts(activeRunDir).filter((f) => f !== "task.txt").sort(); + let diffStat = ""; try { - const result = await runWorkflow(opts, harness); - await session?.log( - `agentic-workflow: ${result.message} (exit ${result.exitCode}) — artifacts at ${result.runDir}`, - ); - } catch (err) { - await session?.log(`agentic-workflow: run failed — ${err?.message ?? err}`); - } finally { - await harness.stop(); + diffStat = execSync("git diff --stat", { cwd: process.cwd(), encoding: "utf8", stdio: ["ignore", "pipe", "ignore"] }).trim(); + } catch { + diffStat = "(not a git repo or git unavailable)"; } + await log( + `status for "${activeTask}"\nphases:\n${phaseLines}\nartifacts (${artifacts.length}):\n ${artifacts.join("\n ") || "(none yet)"}\ngit diff --stat:\n${diffStat || "(no changes)"}`, + ); } -sessionRef = await joinSession({ +async function cmdCompact() { + try { + await session.rpc.commands.enqueue({ command: "/compact" }); + await log("queued /compact to reclaim context."); + } catch (e) { + await log(`compact failed: ${e?.message ?? e}`); + } +} + +// --- Register the session: per-phase sub-agents + commands. -------------------------------------- +// The agent's pinned identity = role + scoped tools. The full, parameterized phase instructions +// (the durable IP in prompts/) are delivered per-dispatch via sendAndWait, since the templates +// depend on the task/run dir which are unknown at join time. Keeping a short role prompt here +// avoids injecting unfilled {{task}}/{{runDir}} placeholders into the agent context. +const customAgents = [...PHASES, CRITIQUE].map((p) => ({ + name: p.agent, + displayName: p.agent, + description: `Agentic-workflow ${p.id} phase`, + tools: p.tools, // null => all tools (implement); read/read-shell scoping otherwise + prompt: + `You are the **${p.id}** phase of an automated research → plan → implement workflow. ` + + `Follow the detailed phase instructions delivered in each message exactly, honor your tool ` + + `restrictions, persist artifacts under the run directory you are given, and end your turn ` + + `with the single sentinel line \`PHASE_RESULT: pass | fail | needs_input\` (plus an optional reason).`, +})); + +const cmd = (name, description, handler) => ({ name, description, handler: (ctx) => Promise.resolve(handler(ctx)).catch((e) => diag(`/${name} error: ${e?.stack ?? e}`)) }); + +// The full join configuration, built once so it can be inspected by tests (the seven agents, the +// command surface, and the read-only guardrail) without standing up a live session host. +export const joinConfig = { + customAgents, + // The one cheap guardrail: hide mutating tools from the default agent so only the implement + // sub-agent (tools: null) can change source. Read-only phases scope tools explicitly above. + defaultAgent: { excludedTools: ["edit", "create", "delete", "write"] }, + infiniteSessions: { enabled: true }, commands: [ - { - name: "agentic-workflow", - description: "Run the research -> plan -> implement workflow on a task.", - handler: (ctx) => execute(ctx.args), - }, + cmd("aw-start", "Start a workflow run on a task (append 'simple' for the short flow).", (c) => cmdStart(c.args)), + cmd("aw-run", "Auto-run phases: from: to: unattended:true pause-at:.", (c) => cmdRun(c.args)), + cmd("aw-continue", "Run the next phase (or next N).", (c) => cmdContinue(c.args)), + cmd("aw-pause", "Stop the auto-runner at the next phase boundary.", () => cmdPause()), + cmd("aw-research", "Run the research phase.", (c) => cmdPhase("research", c.args)), + cmd("aw-assumptions", "Run the assumptions phase.", (c) => cmdPhase("assumptions", c.args)), + cmd("aw-classify", "Run the classify phase.", (c) => cmdPhase("classify", c.args)), + cmd("aw-research-item", "Run the per-sub-item research phase.", (c) => cmdPhase("research-item", c.args)), + cmd("aw-plan", "Run the plan phase.", (c) => cmdPhase("plan", c.args)), + cmd("aw-implement", "Run the implement phase.", (c) => cmdPhase("implement", c.args)), + cmd("aw-judge", "Critique an artifact, then revise it: /aw-judge .", (c) => cmdJudge(c.args)), + cmd("aw-redo", "Re-run a phase with steering notes: /aw-redo .", (c) => cmdRedo(c.args)), + cmd("aw-model", "Repin a phase's model: /aw-model .", (c) => cmdModel(c.args)), + cmd("aw-status", "Show phase state, artifacts, and git diff --stat.", () => cmdStatus()), + cmd("aw-compact", "Reclaim context by queuing /compact.", () => cmdCompact()), ], - tools: [ - { - name: "run_agentic_workflow", - description: - "Run the agentic research -> plan -> implement workflow headlessly on a task " + - "description. Accepts the same flags as the CLI (--simple, --no-judge, --judge-model).", - parameters: { - type: "object", - properties: { - task: { type: "string", description: "The task description" }, - args: { type: "string", description: "Optional flags, e.g. --simple --no-judge" }, - }, - required: ["task"], - }, - handler: async (a) => { - await execute(`${a?.task ?? ""} ${a?.args ?? ""}`); - return { content: [{ type: "text", text: "agentic-workflow run dispatched" }] }; - }, - }, - ], -}); +}; + +// Guard so the pure helpers / config can be imported by tests without a live session host. +if (!process.env.AW_SKIP_JOIN) { + session = await joinSession(joinConfig); + diag(`loaded with ${customAgents.length} phase agents; run dir base ${path.join(process.cwd(), ".aw")}`); +} diff --git a/.github/extensions/agentic-workflow/package-lock.json b/.github/extensions/agentic-workflow/package-lock.json new file mode 100644 index 00000000000..1fc28fc85b9 --- /dev/null +++ b/.github/extensions/agentic-workflow/package-lock.json @@ -0,0 +1,209 @@ +{ + "name": "agentic-workflow-extension", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "agentic-workflow-extension", + "version": "0.1.0", + "dependencies": { + "@github/copilot-sdk": "^1.0.4" + }, + "engines": { + "node": ">=20.19.0" + } + }, + "node_modules/@github/copilot": { + "version": "1.0.66", + "resolved": "https://registry.npmjs.org/@github/copilot/-/copilot-1.0.66.tgz", + "integrity": "sha512-m3+3FLSgum90xN4+eiwnLvdrDvM+oZzur5DfhOH88duNDKBcLQvKQY9fG/I1l1t8a1iBwjpgtRpsBwykE8k3Zw==", + "license": "SEE LICENSE IN LICENSE.md", + "dependencies": { + "detect-libc": "^2.1.2" + }, + "bin": { + "copilot": "npm-loader.js" + }, + "optionalDependencies": { + "@github/copilot-darwin-arm64": "1.0.66", + "@github/copilot-darwin-x64": "1.0.66", + "@github/copilot-linux-arm64": "1.0.66", + "@github/copilot-linux-x64": "1.0.66", + "@github/copilot-linuxmusl-arm64": "1.0.66", + "@github/copilot-linuxmusl-x64": "1.0.66", + "@github/copilot-win32-arm64": "1.0.66", + "@github/copilot-win32-x64": "1.0.66" + } + }, + "node_modules/@github/copilot-darwin-arm64": { + "version": "1.0.66", + "resolved": "https://registry.npmjs.org/@github/copilot-darwin-arm64/-/copilot-darwin-arm64-1.0.66.tgz", + "integrity": "sha512-cJPXE2rWSjR+B8GRBUUd0k9PM4euWRUe3xgHoJqi9o/jJjtRYn6DZMrmFt9xgjoYWf0WZOyrlDgedqO1V+zDAg==", + "cpu": [ + "arm64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "darwin" + ], + "bin": { + "copilot-darwin-arm64": "copilot" + } + }, + "node_modules/@github/copilot-darwin-x64": { + "version": "1.0.66", + "resolved": "https://registry.npmjs.org/@github/copilot-darwin-x64/-/copilot-darwin-x64-1.0.66.tgz", + "integrity": "sha512-44mpx2ZcRFHDx4B9xlrL5OQyTgaD/Hn+bAkeStXgcG8UkkfYSsRtLhnaxqUEQrtIEiVQrw++XWvUO0AscRrX+w==", + "cpu": [ + "x64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "darwin" + ], + "bin": { + "copilot-darwin-x64": "copilot" + } + }, + "node_modules/@github/copilot-linux-arm64": { + "version": "1.0.66", + "resolved": "https://registry.npmjs.org/@github/copilot-linux-arm64/-/copilot-linux-arm64-1.0.66.tgz", + "integrity": "sha512-uXtTs/rYjk6kacNs/T0s/lxn0JBvAgu78pBoZeWpU5APhICkPy9kC+lNAzLYoZujVVDOHT05IoeifHppFpQ8+w==", + "cpu": [ + "arm64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "linux" + ], + "bin": { + "copilot-linux-arm64": "copilot" + } + }, + "node_modules/@github/copilot-linux-x64": { + "version": "1.0.66", + "resolved": "https://registry.npmjs.org/@github/copilot-linux-x64/-/copilot-linux-x64-1.0.66.tgz", + "integrity": "sha512-tXn3OuJCx/YEDNgYg8mdOGSFiIjmLJtTEyZ/VoEA86ffUIPxrunc0wnapEFk2zOW1unwdJeBuVIkzlB3RS1/eA==", + "cpu": [ + "x64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "linux" + ], + "bin": { + "copilot-linux-x64": "copilot" + } + }, + "node_modules/@github/copilot-linuxmusl-arm64": { + "version": "1.0.66", + "resolved": "https://registry.npmjs.org/@github/copilot-linuxmusl-arm64/-/copilot-linuxmusl-arm64-1.0.66.tgz", + "integrity": "sha512-sHRag7W5CG0kbbX3j9v9cUmIafk/0N8MGGr2knvPeIHtxwZQYYjx397gT1nN6xagLWt5mvchkYybfQFCyCBaxg==", + "cpu": [ + "arm64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "linux" + ], + "bin": { + "copilot-linuxmusl-arm64": "copilot" + } + }, + "node_modules/@github/copilot-linuxmusl-x64": { + "version": "1.0.66", + "resolved": "https://registry.npmjs.org/@github/copilot-linuxmusl-x64/-/copilot-linuxmusl-x64-1.0.66.tgz", + "integrity": "sha512-bdIgHOaVZlvsF/4awzMxsby6T+4k7aWe9HZr+sr+qU8tuG19jwi/1LXGB6tKdlFeFgY78yX0lR+ywByVJc5loA==", + "cpu": [ + "x64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "linux" + ], + "bin": { + "copilot-linuxmusl-x64": "copilot" + } + }, + "node_modules/@github/copilot-sdk": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/@github/copilot-sdk/-/copilot-sdk-1.0.4.tgz", + "integrity": "sha512-c6/gpatP7LAuP9stlc2uXXOrB+TJMFSs3Jrzu9FG8lJJYFGmdzbBvGz2UdL1jww84fp6D7cohEZa4UGq1+iuyA==", + "license": "MIT", + "dependencies": { + "@github/copilot": "^1.0.65", + "vscode-jsonrpc": "^8.2.1", + "zod": "^4.3.6" + }, + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@github/copilot-win32-arm64": { + "version": "1.0.66", + "resolved": "https://registry.npmjs.org/@github/copilot-win32-arm64/-/copilot-win32-arm64-1.0.66.tgz", + "integrity": "sha512-T7FGONCVWIPjjAxp22cu4WKqNogq56FknHGAvj7Ryn5ZoanFAR3vXXlXDsYnDKLBcshjRYGxocl2UnmRTMxgvg==", + "cpu": [ + "arm64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "win32" + ], + "bin": { + "copilot-win32-arm64": "copilot.exe" + } + }, + "node_modules/@github/copilot-win32-x64": { + "version": "1.0.66", + "resolved": "https://registry.npmjs.org/@github/copilot-win32-x64/-/copilot-win32-x64-1.0.66.tgz", + "integrity": "sha512-eroxRUSJZOJCk0luLyX6A1qqGIWs8p4w0EjZFhCzvdFvJ0abIovGyt3R/gN9DeyJM8Qs7ROPGvqevUlXh6DhCg==", + "cpu": [ + "x64" + ], + "license": "SEE LICENSE IN LICENSE.md", + "optional": true, + "os": [ + "win32" + ], + "bin": { + "copilot-win32-x64": "copilot.exe" + } + }, + "node_modules/detect-libc": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", + "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==", + "license": "Apache-2.0", + "engines": { + "node": ">=8" + } + }, + "node_modules/vscode-jsonrpc": { + "version": "8.2.1", + "resolved": "https://registry.npmjs.org/vscode-jsonrpc/-/vscode-jsonrpc-8.2.1.tgz", + "integrity": "sha512-kdjOSJ2lLIn7r1rtrMbbNCHjyMPfRnowdKjBQ+mGq6NAW5QY2bEZC/khaC5OR8svbbjvLEaIXkOq45e2X9BIbQ==", + "license": "MIT", + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/zod": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/zod/-/zod-4.4.3.tgz", + "integrity": "sha512-ytENFjIJFl2UwYglde2jchW2Hwm4GJFLDiSXWdTrJQBIN9Fcyp7n4DhxJEiWNAJMV1/BqWfW/kkg71UDcHJyTQ==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + } + } +} diff --git a/.github/extensions/agentic-workflow/package.json b/.github/extensions/agentic-workflow/package.json new file mode 100644 index 00000000000..8873f96d1a3 --- /dev/null +++ b/.github/extensions/agentic-workflow/package.json @@ -0,0 +1,16 @@ +{ + "name": "agentic-workflow-extension", + "version": "0.1.0", + "description": "Copilot CLI extension driving a phased research -> plan -> implement workflow with per-phase pinned-model sub-agents and on-disk artifacts.", + "private": true, + "type": "module", + "engines": { + "node": ">=20.19.0" + }, + "scripts": { + "test": "node --test" + }, + "dependencies": { + "@github/copilot-sdk": "^1.0.4" + } +} diff --git a/.github/extensions/agentic-workflow/prompts/01-research.md b/.github/extensions/agentic-workflow/prompts/01-research.md new file mode 100644 index 00000000000..cca722a29b3 --- /dev/null +++ b/.github/extensions/agentic-workflow/prompts/01-research.md @@ -0,0 +1,56 @@ +# Phase 1 — Research (read-only) + +You are in the **research** phase of an automated `research → plan → implement` workflow. +Your job is to produce factual specs of the **current** code relevant to the task. You are +**read-only**: do not modify, create, or delete source files, and do not design or propose +changes. You **may** run read-only shell commands (see Inputs) to gather context. Just understand +and document what exists today. + +## Task +{{task}} + +## Run directory +Your workflow run directory is `{{runDir}}`. Write your artifacts there using your **normal file +tools** (create/edit/write), and read prior artifacts from there the same way. All artifact paths +below are **relative to the run directory**, not your current working directory (which is the +target code repo — use normal file tools there for reading SOURCE CODE only). + +## Inputs +- The codebase (read freely). +- **Read-only shell is available** for gathering external context: use `gh issue view --repo + /`, `gh pr view`, `gh api repos///issues/`, `git log`, or `curl` to + pull in a referenced issue/PR/spec. Prefer these over cloning other repositories — do **not** clone + large external repos into the working tree. + +## Constraints +- Read-only: do not edit, create, or delete tracked **source** files. (Writing your artifacts under + the run directory above is expected and is not a source edit.) +- Cite real file paths/symbols; do not invent. +- Stay scoped to what the task needs — do not document the whole repo. +- When you have written all required artifacts, end your turn. + +## Outputs — write each file under the run directory with your normal file tools +1. `specs/architecture.md` — the components, modules, data flow, and key entry points relevant + to the task. Cite concrete file paths and symbols. No proposals. +2. `specs/functional.md` — the current behavior relevant to the task: what the code does today, + the user-visible/contract behavior, and the edge cases it already handles. +3. `specs/apispec.md` — **conditional.** Only if the task touches an API surface (REST/RPC/SDK + contract, schema, public interface). Capture the relevant existing API definition + (endpoints/operations, request/response shapes, contracts). +4. `manifest.json` — record the apispec decision explicitly so downstream phases are unambiguous: + ```json + { "apispec": { "required": false, "reason": "" } } + ``` + +## Self-check +Before finishing, confirm `specs/architecture.md`, `specs/functional.md`, and `manifest.json` exist +in the run directory, every cited path is real, and (if the task touches an API) `specs/apispec.md` +exists. If anything is missing or wrong, fix it before reporting. + +## Report at the end of your turn +End with exactly one status line the runner reads: +- `PHASE_RESULT: pass` if all required artifacts are written and self-check passed, +- `PHASE_RESULT: fail — ` if you could not complete the artifacts, or +- `PHASE_RESULT: needs_input — ` if a blocking question prevents factual research. + +{{priorErrors}} diff --git a/tools/agentic-workflow/prompts/02-assumptions.md b/.github/extensions/agentic-workflow/prompts/02-assumptions.md similarity index 77% rename from tools/agentic-workflow/prompts/02-assumptions.md rename to .github/extensions/agentic-workflow/prompts/02-assumptions.md index 5cde6bbba4a..e4871fc6ac4 100644 --- a/tools/agentic-workflow/prompts/02-assumptions.md +++ b/.github/extensions/agentic-workflow/prompts/02-assumptions.md @@ -7,11 +7,15 @@ not propose code changes, and do not classify the work yet. ## Task {{task}} +## Run directory +Your workflow run directory is `{{runDir}}`. Read prior artifacts from there and write your output +there using your **normal file tools**. Artifact paths below are relative to the run directory. + ## Inputs -- `specs/architecture.md`, `specs/functional.md`, and `specs/apispec.md` if present. -- {{researchNote}} +- `specs/architecture.md`, `specs/functional.md`, and `specs/apispec.md` if present (read them from + the run directory). -## Output — via the `write_artifact` tool +## Output — write under the run directory with your normal file tools `assumptions.md` — one assumption per line, each with a short **rationale** and a **confidence** (`high` / `medium` / `low`). @@ -78,10 +82,19 @@ it explicitly with a leading `blocking: true` token on that line, e.g.: - blocking: true | We assume tokens are validated upstream | rationale: no validation found in scope | confidence: low ``` -The orchestrator pauses the run and asks the human when any `blocking: true` assumption exists, -rather than inventing an answer. Use this sparingly and only when it genuinely gates correctness. +When any `blocking: true` assumption exists, do **not** invent an answer: report `needs_input` +(see below) with the blocking question so the human can resolve it. Use this sparingly and only +when it genuinely gates correctness. ## Constraints - Read-only; no source edits, no shell. - Be concrete and tied to the specs/codebase, not generic. - End your turn once `assumptions.md` is written. + +## Report at the end of your turn +End with exactly one status line the runner reads: +- `PHASE_RESULT: pass` if `assumptions.md` is written with no blocking clarification, +- `PHASE_RESULT: needs_input — ` if a `blocking: true` assumption needs a human decision, +- `PHASE_RESULT: fail — ` if you could not produce the artifact. + +{{priorErrors}} diff --git a/tools/agentic-workflow/prompts/03-classify.md b/.github/extensions/agentic-workflow/prompts/03-classify.md similarity index 61% rename from tools/agentic-workflow/prompts/03-classify.md rename to .github/extensions/agentic-workflow/prompts/03-classify.md index 40c88455567..dd45abc1f19 100644 --- a/tools/agentic-workflow/prompts/03-classify.md +++ b/.github/extensions/agentic-workflow/prompts/03-classify.md @@ -6,10 +6,14 @@ non-overlapping** sub-items. You are **read-only**. ## Task {{task}} +## Run directory +Your workflow run directory is `{{runDir}}`. Read prior artifacts from there and write your output +there using your **normal file tools**. Artifact paths below are relative to the run directory. + ## Inputs -- `specs/*` and `assumptions.md` if present. {{researchNote}} +- `specs/*` and `assumptions.md` if present (read them from the run directory). -## Outputs — via the `write_artifact` tool +## Outputs — write under the run directory with your normal file tools 1. `classification.md` — human-readable: the overall classification (`feature` / `bug` / `refactor` / `mixed`) and the reasoning for the split. 2. `subitems.json` — machine-readable, matching this shape exactly: @@ -40,3 +44,15 @@ non-overlapping** sub-items. You are **read-only**. - `id` values are unique, kebab-case. `items` must be non-empty. - Read-only; no source edits, no shell. - End your turn once both artifacts are written. + +## Self-check +Confirm `subitems.json` is valid JSON, `items` is non-empty, every `id` is unique kebab-case, and +every `dependsOn` references an existing `id`. Fix any issue before reporting. + +## Report at the end of your turn +End with exactly one status line the runner reads: +- `PHASE_RESULT: pass` if both artifacts are written and `subitems.json` passes self-check, +- `PHASE_RESULT: fail — ` otherwise, +- `PHASE_RESULT: needs_input — ` if classification is genuinely blocked on a decision. + +{{priorErrors}} diff --git a/.github/extensions/agentic-workflow/prompts/04-research-item.md b/.github/extensions/agentic-workflow/prompts/04-research-item.md new file mode 100644 index 00000000000..ccc825cdfdd --- /dev/null +++ b/.github/extensions/agentic-workflow/prompts/04-research-item.md @@ -0,0 +1,42 @@ +# Phase 4 — Research the sub-items (read-only) + +You are in the **research-item** phase. Do deep, isolated research on **each** sub-item produced by +the classify phase, using the specs as context. You are **read-only**. + +## Original task +{{task}} + +## Run directory +Your workflow run directory is `{{runDir}}`. Read prior artifacts from there and write your output +there using your **normal file tools**. Artifact paths below are relative to the run directory. + +## Inputs +- `subitems.json` (read it from the run directory) — the list of sub-items to research. +- `specs/*` and `assumptions.md` if present. + +## Work +Read `subitems.json`. For **every** item in `items`, produce a focused research note. Process items +in dependency order (an item's `dependsOn` siblings first) so each note can reference the prior +notes it depends on. + +## Output — write one note per sub-item under the run directory +For each item with id ``, write `research/.md`: the exact files and symbols it will touch, +the current behavior in that area, constraints, edge cases, and any sequencing concerns relative to +sibling items. Cite real code locations. Keep each note scoped to **its** sub-item only. + +## Constraints +- Cover every item in `subitems.json` — do not skip any. +- Read-only; no source edits, no shell. +- End your turn once all notes are written. + +## Self-check +Confirm there is one `research/.md` for every `id` in `subitems.json`. Fix gaps before +reporting. + +## Report at the end of your turn +End with exactly one status line the runner reads: +- `PHASE_RESULT: pass` if a note exists for every sub-item, +- `PHASE_RESULT: fail — ` if you could not complete every note, +- `PHASE_RESULT: needs_input — ` if research is blocked on a decision. + +{{priorErrors}} diff --git a/.github/extensions/agentic-workflow/prompts/05-plan.md b/.github/extensions/agentic-workflow/prompts/05-plan.md new file mode 100644 index 00000000000..dcc08456946 --- /dev/null +++ b/.github/extensions/agentic-workflow/prompts/05-plan.md @@ -0,0 +1,66 @@ +# Phase 5 — Plan (read-only) + +You are in the **plan** phase. Produce a highly detailed, structured implementation plan. This is +the highest-leverage artifact in the run. You are **read-only** — you plan, you do not implement. + +## Task +{{task}} + +## Run directory +Your workflow run directory is `{{runDir}}`. Read prior artifacts from there and write your output +there using your **normal file tools**. Artifact paths below are relative to the run directory. + +## Inputs +- `specs/*`, `assumptions.md`, and all `research/*.md` notes that exist (read from the run dir). + +## Output — write `plan.md` under the run directory with your normal file tools +`plan.md`, containing these sections **in order**: + +0. **Research reconciliation** — reconcile the independent phase-4 notes: overlaps, + contradictions, duplicated work, and the single coherent strategy chosen. (If there is only + one sub-item, say so; do not invent overlaps.) +1. **Decisions and rationale** — the choices made and why. +2. **End-to-end approach** — the overall strategy, and explicitly how the success criterion will + be *proved*. +3. **Step-by-step implementation plan** — ordered, concrete, file-by-file steps, grouped into + **stages**. Define each stage **in prose** with: + - a stage id and short title, + - the ordered steps (each with a step id and description), + - the anticipated files/scope (advisory), + - existing files the stage depends on for context, + - a **gate**: one or more concrete shell commands the implement phase will run to verify the + stage (e.g. `npm test -- foo`), and the expected result (typically exit code 0). +4. **Stop/go gates** — explicit points where work pauses for validation. +5. **Validation plan** — tests to run, tests to add, observability checks. +6. **Rollout strategy**. +7. **Rollback plan**. +8. **Risks and mitigations**. +9. **Definition of done** — concrete, checkable completion criteria. +10. **Open questions**. +11. **Out-of-scope observations**. +12. **Plan changes** — leave empty (`_none yet_`); phase 6 appends here when implementation + deviates from the plan. + +## Stage sizing +Split work into **cohesive, loosely-coupled** stages — each independently completable and +verifiable by its gate. Tightly-coupled changes that only make sense together belong in **one** +stage. Every stage MUST have a gate with at least one concrete command. You define and own the +stage breakdown in prose — there is no machine-readable block to emit and no external validator; +the implement phase reads your prose plan and runs the gate commands you specify. + +## Constraints +- Read-only; no source edits, no shell. +- Every stage has an id and at least one gate command. +- End your turn once `plan.md` is written. + +## Self-check +Confirm `plan.md` exists with all sections 0–12 present, at least one stage defined, and every +stage carries an explicit gate command. Fix gaps before reporting. + +## Report at the end of your turn +End with exactly one status line the runner reads: +- `PHASE_RESULT: pass` if `plan.md` is complete and self-check passed, +- `PHASE_RESULT: fail — ` otherwise, +- `PHASE_RESULT: needs_input — ` if planning is blocked on a decision. + +{{priorErrors}} diff --git a/tools/agentic-workflow/prompts/06-implement.md b/.github/extensions/agentic-workflow/prompts/06-implement.md similarity index 55% rename from tools/agentic-workflow/prompts/06-implement.md rename to .github/extensions/agentic-workflow/prompts/06-implement.md index fa04912722f..fb67249a80b 100644 --- a/tools/agentic-workflow/prompts/06-implement.md +++ b/.github/extensions/agentic-workflow/prompts/06-implement.md @@ -1,40 +1,40 @@ -# Phase 6 — Implement a single stage +# Phase 6 — Implement the plan -You are in **IMPLEMENT mode**, implementing **one stage** of the approved plan in a fresh session. -You **may edit source code and run shell commands** for this stage only. Earlier stages already -ran; their code is on disk. Implement the approved plan **exactly as written**. Keep the language -in your logs simple. No emojis. +You are in **IMPLEMENT mode**, implementing the approved plan. You **may edit source code and run +shell commands**. Implement the approved plan **exactly as written**. Keep the language in your +logs simple. No emojis. ## Task {{task}} -## This stage -```yaml -{{stage}} -``` +## Run directory +Your workflow run directory is `{{runDir}}`. Read the plan and prior artifacts from there, and +append your logs there, using your **normal file tools**. Artifact paths below are relative to the +run directory. **Source code** edits go in the working directory (the target repo) with the same +file tools. -## Context pack (clean ≠ blank — everything you need has been gathered for you) -- **`plan.md`** is the source of truth (read it, including the running "Plan changes" log). +## Context pack (read these first) +- **`plan.md`** is the source of truth (read it, including the running "Plan changes" log). It + defines the stages and each stage's gate commands. - **`assumptions.md`** is your context document — read it before you start. -- **Prior handoffs** from earlier stages: -{{handoff}} -- **Cumulative diff so far** (files already changed in this run): -{{cumulativeDiff}} -- **`context_needed`** files this stage depends on: {{contextNeeded}} +- Any `research/*.md` notes for areas you are touching. ## What to do -1. Implement **this stage's steps**, editing the real source files. Follow the plan step-by-step. - `expected_files` is the anticipated scope — **advisory, not a wall**. You may edit beyond it +Work through the plan **stage by stage, in order**. For each stage: +1. Implement **that stage's steps**, editing the real source files. Follow the plan step-by-step. + The anticipated files are the advisory scope — **advisory, not a wall**. You may edit beyond it when correctness requires it, **provided you document the deviation** (see below). 2. Make **small, reviewable changes.** Group related edits, and for each group summarize *what* changed, *which plan step* it satisfies, and *what you verified*. -3. **Run this stage's `gate.commands`** yourself (shell tool) and confirm they meet `expected` - (e.g. exit code 0). If a gate fails, fix the code and re-run until it passes or you are +3. **Run that stage's gate commands** yourself (shell tool) and confirm they meet the expected + result (e.g. exit code 0). If a gate fails, fix the code and re-run until it passes or you are genuinely blocked. -4. Append to **`execution-log.md`** (via `write_artifact`, appending) — see "Execution log" below. -5. Append a concise entry to **`handoff.md`** (via `write_artifact`, appending) for the *next* - stage: what you built, the **new/changed public symbols and files**, decisions/conventions - established, anything deferred, and known follow-ups. +4. Append to **`execution-log.md`** (under the run dir) — see "Execution log" below. +5. Append a concise entry to **`handoff.md`** (under the run dir) for the *next* stage: what you + built, the **new/changed public symbols and files**, decisions/conventions established, anything + deferred, and known follow-ups. + +Only advance to the next stage after the current stage's gates pass. ## Scope discipline (stay strictly within the plan) This may be a large, older codebase. You **will** see refactor opportunities, outdated patterns, @@ -44,8 +44,8 @@ it silently. - **Observability:** add or update observability **exactly as the plan defines** — no more. - **Dependencies:** do **not** introduce dependencies beyond what the plan and your team's package policy allow. Use only feeds/registries your team approves. -- **Tests:** do **not** write tests in this step unless a plan step explicitly calls for it. - (Running the stage's existing `gate.commands` is required and is not the same as authoring tests.) +- **Tests:** do **not** write tests unless a plan step explicitly calls for it. (Running a stage's + existing gate commands is required and is not the same as authoring tests.) - **Secrets/PII:** never include secrets or PII in code, logs, or artifacts. - **Commits:** do **not** run `git commit`. Stage your changes and let the build be verified first. - Every change must tie to a specific plan step. If a change **cannot** be tied to a plan step, @@ -57,9 +57,9 @@ For **every** change group, record: satisfies — *(a)* why it was necessary and *(b)* what it maps to; - each **gate command**, its exit code, and pass/fail. Also maintain these sections in the log: -- a **Test results** section (placeholder, to be filled later if tests are out of scope here); +- a **Test results** section; - a **Plan changes** section (mirrors any deviation entries you add to `plan.md`); -- a **Verification evidence** section (gate output; placeholder where evidence comes later); +- a **Verification evidence** section (gate output); - an **Out-of-scope observations** section at the bottom for anything you noticed but correctly did *not* act on; - a **PR description draft**: reference the work item / task id and summarize **scope, changes, @@ -72,13 +72,14 @@ describing the deviation and rationale. **(3)** continue only after that update - The `execution-log.md` entry for the deviating action justifies its scope and maps it to the originating step / plan-change entry. - **Larger deviations** — a change to architecture, public API, or overall test strategy — must - be called out prominently at the top of your handoff so the orchestrator/human can review; - if the divergence invalidates the plan, say so explicitly. + be called out prominently in your final report so the human can review; if the divergence + invalidates the plan, say so explicitly. - Silent, undocumented scope expansion is **not** allowed. ## Report at the end of your turn -End with a short, explicit status line the orchestrator reads: -- `STAGE_RESULT: pass` if every gate command met `expected`, or -- `STAGE_RESULT: fail — ` otherwise. +End with exactly one status line the runner reads: +- `PHASE_RESULT: pass` if **every** stage's gate commands met their expected result, +- `PHASE_RESULT: fail — ` if any gate could not be made to pass, +- `PHASE_RESULT: needs_input — ` if implementation is blocked on a human decision. {{priorErrors}} diff --git a/.github/extensions/agentic-workflow/prompts/critique.md b/.github/extensions/agentic-workflow/prompts/critique.md new file mode 100644 index 00000000000..002da033cf2 --- /dev/null +++ b/.github/extensions/agentic-workflow/prompts/critique.md @@ -0,0 +1,51 @@ +# Judge — Critique (read-only, alternate model) + +You must critique an artifact written by a different AI model. Identify gaps and opportunities. + +## Original task (for context) +{{task}} + +## Run directory +Your workflow run directory is `{{runDir}}`. Read the artifact under review from there and write +your critique there using your **normal file tools**. Artifact paths are relative to the run dir. + +## Artifact under review +The specific artifact to review is named in the instructions appended at the end of this prompt +(see "Review target"). Read that artifact from the run directory before critiquing. + +## What to report +Only substantive issues that would make the downstream work wrong, unsafe, or incomplete: +- **Bugs / logic errors** in the reasoning or proposed approach. +- **Gaps** — missing cases, missing inputs/outputs, unhandled risks. +- **Contract violations** — the artifact does not satisfy its phase's required format/sections. +- **Design flaws** — choices that will cause rework or break a stated constraint. + +Rubric: +1. Specificity -- Is the claim falsifiable? Could a reader point at a + file, line, behavior, or work-item field that would prove it wrong? +2. Scope -- Does the claim name the module, file, endpoint, or + boundary it applies to, or is it written in the abstract? +3. Coverage gap -- Are any required categories/sections thin, missing, or + treated as boilerplate? +4. Platitude -- Flag anything that would be true of almost any + software project ("must be backward compatible", "should have good + performance", "code must be maintainable"). These are not useful; they + are restated values. +5. Open-question candidate -- Flag any item written as an assertion that + is actually unknown and should be moved to an open questions list. + +**Do NOT** comment on style, wording, formatting, or trivia. If the artifact is sound, say so and +keep it short — do not invent problems. + +## Output — write under the run directory with your normal file tools +`critiques/.md` — a terse list. For each point: a one-line description, the +**severity** (`blocker` / `should-fix` / `optional`), and the specific location/section it refers +to. End your turn once written. + +## Report at the end of your turn +End with exactly one status line the runner reads: +- `PHASE_RESULT: pass` once the critique file is written (even if it concludes the artifact is sound), +- `PHASE_RESULT: fail — ` if you could not produce the critique. + +## Review target +{{priorErrors}} diff --git a/.github/extensions/agentic-workflow/test/smoke.test.mjs b/.github/extensions/agentic-workflow/test/smoke.test.mjs new file mode 100644 index 00000000000..7bbc59f3ff0 --- /dev/null +++ b/.github/extensions/agentic-workflow/test/smoke.test.mjs @@ -0,0 +1,68 @@ +// Smoke test for the only non-trivial pure logic in extension.mjs: arg parsing and the +// PHASE_RESULT sentinel parser. Run: npm test (sets AW_SKIP_JOIN so importing the module does +// not try to join a live session host). +import { test } from "node:test"; +import assert from "node:assert/strict"; + +process.env.AW_SKIP_JOIN = "1"; +const { parsePhaseResult, parseKv, isTruthy, slugify } = await import("../extension.mjs"); + +test("parsePhaseResult reads pass/fail/needs_input and reason", () => { + assert.equal(parsePhaseResult("work...\nPHASE_RESULT: pass").result, "pass"); + assert.equal(parsePhaseResult("PHASE_RESULT: fail — build broke").result, "fail"); + assert.equal(parsePhaseResult("PHASE_RESULT: fail — build broke").reason, "build broke"); + assert.equal(parsePhaseResult("PHASE_RESULT: needs_input - which db?").result, "needs_input"); + assert.equal(parsePhaseResult("PHASE_RESULT: needs_input - which db?").reason, "which db?"); + assert.equal(parsePhaseResult("PHASE_RESULT:pass").result, "pass"); +}); + +test("parsePhaseResult defaults to fail when sentinel missing", () => { + const r = parsePhaseResult("the agent forgot the sentinel"); + assert.equal(r.result, "fail"); + assert.match(r.reason, /no PHASE_RESULT sentinel/); +}); + +test("parseKv splits key:value pairs from free text", () => { + const { kv, rest } = parseKv("from:research to:plan unattended:true Add CSV export"); + assert.equal(kv.from, "research"); + assert.equal(kv.to, "plan"); + assert.equal(kv.unattended, "true"); + assert.equal(rest.join(" "), "Add CSV export"); +}); + +test("isTruthy treats bare flag, true/1/yes/on as true", () => { + for (const v of ["", "true", "1", "yes", "on", "TRUE"]) assert.equal(isTruthy(v), true, v); + for (const v of ["false", "0", "no", undefined]) assert.equal(isTruthy(v), false, String(v)); +}); + +test("slugify produces a safe run-dir name", () => { + assert.equal(slugify("Add CSV export!"), "add-csv-export"); + assert.equal(slugify(" "), "run"); + assert.equal(slugify("a".repeat(80)).length, 40); +}); + +const { joinConfig } = await import("../extension.mjs"); + +test("joinConfig registers the seven phase agents", () => { + const names = joinConfig.customAgents.map((a) => a.name).sort(); + assert.deepEqual(names, [ + "aw-assumptions", "aw-classify", "aw-critique", "aw-implement", "aw-plan", "aw-research", "aw-research-item", + ].sort()); +}); + +test("joinConfig scopes tools: implement has all, read phases are restricted", () => { + const byName = Object.fromEntries(joinConfig.customAgents.map((a) => [a.name, a])); + assert.equal(byName["aw-implement"].tools, null); // all tools + assert.ok(!byName["aw-research"].tools.includes("edit")); + assert.ok(byName["aw-research"].tools.includes("bash")); // research gets read + shell + assert.ok(!byName["aw-plan"].tools.includes("bash")); // pure read phase, no shell +}); + +test("joinConfig guards the default agent and registers the command surface", () => { + assert.deepEqual(joinConfig.defaultAgent.excludedTools, ["edit", "create", "delete", "write"]); + assert.equal(joinConfig.infiniteSessions.enabled, true); + const cmds = joinConfig.commands.map((c) => c.name); + for (const expected of ["aw-start", "aw-run", "aw-continue", "aw-pause", "aw-judge", "aw-redo", "aw-model", "aw-status", "aw-compact", "aw-implement"]) { + assert.ok(cmds.includes(expected), `missing /${expected}`); + } +}); diff --git a/README.md b/README.md index 7f0eb5eb80f..61a9d39639a 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ This repository contains useful tools that the Azure SDK team utilizes across th | Package or Intent | Path | Description | Status | | ------------------------------ | ------------------------------------------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Check Enforcer [1] | [Readme](https://github.com/Azure/azure-sdk-actions/blob/main/docs/check-enforcer.md) | Manage GitHub check-runs in a mono-repo. | Enabled via GitHub actions | -| agentic-workflow | [Readme](tools/agentic-workflow/README.md) | Drives a research → plan → implement workflow for AI coding agents, one fresh Copilot SDK session per phase. | Not Yet Enabled | +| agentic-workflow | [Readme](.github/extensions/agentic-workflow/README.md) | Copilot CLI extension driving a research → plan → implement workflow with per-phase pinned-model sub-agents and on-disk artifacts. | Not Yet Enabled | | doc-warden | [Readme](packages/python-packages/doc-warden/README.md) | A tool used to enforce readme standards across Azure SDK Repos. | [![Build Status](https://dev.azure.com/azure-sdk/public/_apis/build/status/108?branchName=main)](https://dev.azure.com/azure-sdk/public/_build/latest?definitionId=108&branchName=main) | | http-fault-injector | [Readme](tools/http-fault-injector/README.md) | HTTP proxy server for testing HTTP clients during "faults" like "connection closed in middle of body". | [![Build Status](https://dev.azure.com/azure-sdk/internal/_apis/build/status/tools/tools%20-%20http-fault-injector?branchName=main)](https://dev.azure.com/azure-sdk/internal/_build/latest?definitionId=2340&branchName=main) | | Maven Plugin for Snippets | [Readme](packages/java-packages/snippet-replacer-maven-plugin/README.md) | A Maven plugin that that updates code snippets referenced from javadoc comments. | Not Yet Enabled | diff --git a/cli-extension-design.md b/cli-extension-design.md new file mode 100644 index 00000000000..8ef9e2cf97a --- /dev/null +++ b/cli-extension-design.md @@ -0,0 +1,141 @@ +# CLI-Extension Design: Phased Agentic Workflow + +A design for running a disciplined **research → plan → implement** workflow as a single **Copilot CLI +extension** that lives inside the user's interactive session — giving the user fine-grained control +over each phase *and* an opt-in auto-runner that chains phases unattended. + +## 1. Goal + +Drive a multi-phase coding workflow where each phase runs with its own **pinned model**, **scoped +tools**, and **prompt**, handing off to the next phase through **artifacts on disk**. Unlike a +headless runner, the user stays in the loop: they advance phases manually, inspect artifacts between +steps, repin models, and answer decisions via dialogs — or let it auto-run when they want hands-off +progress. + +The workflow phases: + +| Phase | Reads | Writes | Mutates code? | +| --- | --- | --- | --- | +| `research` | the codebase | `specs/*.md` | no | +| `assumptions` | specs + task | `assumptions.md` | no | +| `classify` | specs | `subitems.json` | no | +| `research-item` ×N | one sub-item | `research/.md` | no | +| `plan` | everything above | `plan.md` (+ stage/gate block) | no | +| `implement` (staged) | `plan.md`, `handoff.md` | code edits + `execution-log` | **yes** | + +The durable IP is the **prompt template per phase** plus the **on-disk artifact contract**. The +extension is just the driver. + +## 2. Core idea: one extension, sub-agents per phase + +The extension calls `joinSession()` once and registers one **custom agent per phase**. Dispatching a +sub-agent runs it in **its own context window** — the parent session only gets the summarized result +back — so each phase reasons in isolation without a separate process or session. This replaces a +bespoke orchestrator with native SDK primitives: + +- **Pinned model per phase** — `customAgents[].model`. +- **Tool scoping** — `customAgents[].tools`; non-implementation phases simply don't list + `edit`/`create`/`write`. +- **Read-only enforcement** — `defaultAgent.excludedTools` hides mutating tools from the default + agent so only the `implement` agent can change source. +- **Phase prompt** — `customAgents[].prompt`. + +State and handoff live entirely in a per-run directory on disk, so the workflow is inspectable, +resumable, and survives an extension reload. + +## 3. SDK surface used + +Verified against the installed `@github/copilot-sdk` **v1.0.4** type definitions. + +| Need | API | Citation | +| --- | --- | --- | +| Join the live session | `joinSession(config)` | `extension.d.ts` | +| Per-phase agent + pinned model | `customAgents: CustomAgentConfig[]` (`.model`, `.tools`, `.prompt`) | `types.d.ts:1157,1197,1626` | +| Delegation-only (mutating) tools | `defaultAgent.excludedTools` | `types.d.ts:1208` | +| Select/switch active agent | `session.rpc.agent.select({ name })` | `rpc.d.ts:2537` | +| Slash commands | `commands: CommandDefinition[]` + `handler(ctx)` | `types.d.ts:425,1424` | +| Structured dialogs | `session.ui.{confirm,select,input,elicitation}` (gate on `session.capabilities.ui?.elicitation`) | `types.d.ts:614` | +| Switch model mid-session | `session.setModel(model, opts)` | `session.d.ts:268` | +| Reclaim context | enqueue `/compact`; or `infiniteSessions` config | `rpc.d.ts:14553`, `types.d.ts:1671` | +| Artifact tools | custom `tools` (`read_artifact` marked `skipPermission`) | `types.d.ts` | + +**Version note:** newer SDK releases rename slash commands to `slashCommands` with +`action(session, params)` (typed params) and adjust the elicitation result shape. On v1.0.4 use +`commands`/`handler(ctx)` and parse `ctx.args` yourself. If you upgrade, confirm the +`slashCommands`/elicitation types in `node_modules/@github/copilot-sdk/dist/*.d.ts` first. + +## 4. Command surface + +Each phase is a **user-triggered** command. Between commands the user can inspect artifacts, chat, +edit files, or repin models. + +| Command | Params | Behavior | +| --- | --- | --- | +| `/aw-start` | `task`, `simple?` | init run-dir; dispatch the first phase | +| `/aw-run` | `from?`, `to?`, `unattended?`, `pause-at?` | **auto-run** a range of phases (§5) | +| `/aw-continue` | `n?` | single-step: run the next phase (or next `n`), then pause | +| `/aw-pause` | — | stop the auto-runner at the next phase boundary | +| `/aw-plan` / `/aw-implement` / … | — | run one specific phase | +| `/aw-judge` | `artifact` | on-demand critique→revise on an alternate-model agent | +| `/aw-redo` | `phase`, `feedback` | re-run a phase with steering notes | +| `/aw-model` | `phase`, `model` | repin a phase's model | +| `/aw-status` | — | print phase state + artifacts + `git diff --stat` | + +On v1.0.4, params are parsed from `ctx.args`. Post-upgrade they become typed `slashCommands` params. + +## 5. Execution modes: manual ↔ auto + +Control is a spectrum; the same per-phase agents back all modes. + +| Mode | How | Stops | +| --- | --- | --- | +| **Manual** | `/aw-` or `/aw-continue` | after every phase | +| **Ranged auto** | `/aw-run from:assumptions to:plan` | at the boundary or a stop condition | +| **Full auto** | `/aw-run` (or `unattended:true`) | at stop conditions (only on hard failure when `unattended`) | + +The **auto-runner** is a small async function in the extension (no separate process). It walks the +phase order from `from` (default: next incomplete) toward `to` (default: `implement`); for each phase +it dispatches the sub-agent, validates the artifact, auto-retries up to N on validation failure, then +checks **stop conditions** before advancing: + +1. breakpoint / `to` boundary reached, +2. `assumptions.md` flags a blocking clarification, +3. a stage gate reported failure, +4. retries exhausted, +5. `/aw-pause` was requested. + +At a stop, the runner yields to the human: interactively it shows a `session.ui` dialog (retry / skip +/ abort a gate; resolve a blocking assumption; pick which critique points to apply) and resumes per +the answer. With `unattended:true` it auto-resolves with safe defaults and only halts on hard +failure. + +The loop is cooperative — it checks the pause flag at each boundary, so a long auto-run stops cleanly +without killing the session; `session.abort()` cancels an in-flight phase. Because all state is in the +run-dir, the user can switch between auto and manual at any boundary +(`/aw-run` → inspect → `/aw-continue` → `/aw-run to:implement`). + +`/aw-run` with no params = "advance to the end, pausing only at gates and blocking assumptions" — +autonomous where safe, interactive where judgment is needed. + +## 6. Trade-offs and gotchas + +- **Not zero-shared-lineage.** Sub-agents isolate working context, but the parent accumulates each + phase's handoff summary. Fine for almost all workflows; use `/compact` or `infiniteSessions` for + long runs. +- **Single foreground session.** `joinSession` binds to one session — don't use `/new` to reset + phases; `/clear` reloads the extension and wipes in-memory state (so keep all state on disk). +- **One hooks owner.** If multiple extensions register hooks, only the last-loaded fires — + consolidate hooks into this one extension. +- **Protocol hygiene.** `.mjs` only; log to **stderr** (stdout corrupts JSON-RPC); never + `session.send()` synchronously from a prompt hook. + +## 7. Layout + +``` +.github/extensions/agentic-workflow/ + extension.mjs # joinSession: customAgents + commands + artifact tools + auto-runner + prompts/ # one prompt template per phase (the durable IP) +``` + +A single file plus the prompt templates. No orchestrator, no state machine — the SDK's sub-agents, +slash commands, and elicitation dialogs do the work. diff --git a/cli-extension-impl.md b/cli-extension-impl.md new file mode 100644 index 00000000000..0e85f247f30 --- /dev/null +++ b/cli-extension-impl.md @@ -0,0 +1,220 @@ +# Implementation Plan: CLI-Extension Phased Workflow + +Step-by-step plan to build the design in `cli-extension-design.md` as a **self-contained, agent-first** +Copilot CLI extension that **replaces** the `tools/agentic-workflow` prototype. + +Design principle: **lean on the agent, minimize deterministic code.** Validation, gate-running, stage +breakdown, and artifact I/O are delegated to the phase agents. The extension is essentially *config + +thin command handlers + a tiny dispatch loop* — no `lib/` of validators/parsers, no build step +(extensions are `.mjs`). + +Target layout: + +``` +.github/extensions/agentic-workflow/ + extension.mjs # joinSession: agents + commands + auto-runner loop + package.json # @github/copilot-sdk dependency + prompts/ # one .md per phase (+ critique) — the durable IP + README.md +``` + +The agent reports status by ending each phase with a sentinel line the loop reads: + +``` +PHASE_RESULT: pass | fail | needs_input +``` + +(optionally followed by a short reason). That single convention replaces deterministic validators and +gate parsers. + +## Step 0 — Prerequisites + +- Create `.github/extensions/agentic-workflow/package.json` (`{ "type": "module" }` + + `@github/copilot-sdk` dependency); `npm install` there. +- **SDK version:** plan targets **v1.0.4** (`commands` + `handler(ctx)`, `session.ui.*`). If you pin a + newer release, re-check `slashCommands`/elicitation types in `node_modules/.../dist/*.d.ts` and + adjust Steps 3–4. +- **Verify:** `node -e "import('@github/copilot-sdk/extension').then(()=>console.error('ok'))"`. + +## Step 1 — Prompt templates (the durable IP) + +- **Files:** `prompts/{01-research,02-assumptions,03-classify,04-research-item,05-plan,06-implement, + critique}.md`. +- **Source:** **start from the prototype's `tools/agentic-workflow/prompts/*.md`** — that reasoning + content (what each phase investigates, produces, and hands off) is the durable IP and should be + reused. **Strip only the deterministic-handoff scaffolding** that no longer applies: + - references to the `write_artifact`/`read_artifact` custom tools and the run-context preamble → + replace with "write/read this file in the run dir with your normal file tools"; + - the requirement to emit a **machine-readable `stages:`/`gate:` block** for the orchestrator to + parse → replace with "define your stages in prose and **run the gate commands yourself**"; + - any wording assuming a fresh isolated session or external validator → replace with the + `PHASE_RESULT` self-report below. +- **Work:** carry over each phase's instructions, apply the strip above, keep `{{task}}`/ + `{{priorErrors}}` placeholders, and have each template tell the agent to: + - write its output to the run dir (a path the extension passes in, e.g. `.aw//plan.md`) using + normal file tools, and read prior artifacts the same way; + - **self-check** its own output and the `06-implement` template additionally **run the gate commands + it defined in the plan**; + - end with `PHASE_RESULT: pass|fail|needs_input` (+ reason). Use `needs_input` for blocking + questions. +- **No deterministic validator** backs these — the agent owns correctness. +- **Verify:** each template's placeholders are supplied by its caller (Step 4); each instructs the + sentinel line. + +## Step 2 — Register the session (agents = the real value) + +- **Files:** `extension.mjs`. +- **Work:** `await joinSession({...})` with: + - `customAgents`: one per phase — **pinned model + scoped tools + the phase prompt** (the whole + reason to be an extension): + + | agent | model | tools | template | + | --- | --- | --- | --- | + | `aw-research` | `claude-sonnet-4.5` | read/shell | `01-research` | + | `aw-assumptions` | default | read | `02-assumptions` | + | `aw-classify` | `claude-haiku-4.5` | read | `03-classify` | + | `aw-research-item` | `claude-sonnet-4.5` | read | `04-research-item` | + | `aw-plan` | `claude-sonnet-4.5` | read | `05-plan` | + | `aw-implement` | `claude-sonnet-4.5` | **all** (incl. edit/create/shell) | `06-implement` | + | `aw-critique` | `claude-haiku-4.5` | read | `critique` | + - `defaultAgent: { excludedTools: ["edit","create","delete","write"] }` — the one cheap guardrail + so non-implement phases can't mutate source (config, not code). Optional; drop if you want zero + enforcement. + - keep a module-scoped `session` ref; **log to stderr only**. +- **Verify:** `/extensions info agentic-workflow` lists the seven agents. + +## Step 3 — Dispatch loop + thin command handlers + +- **Files:** `extension.mjs`. +- **Work:** + - **Run dir / current phase** — on `/aw-start`, derive a run dir from the task; track the active run + dir in memory. Determine the next phase by which artifact files already exist in the run dir (a + few `fs.existsSync` checks) — no state machine, no `state.json`. + - **`runPhase(phaseId, { priorErrors })`** — `session.rpc.agent.select({ name })`, then + `await session.sendAndWait({ prompt: task + "\n\n" + readFile(prompts/