[docs] Plan progressive tool disclosure for the playground build kit

docs/design/agent-workflows/projects/progressive-tool-disclosure/README.md

@@ -0,0 +1,42 @@
+# Progressive tool disclosure (playground build kit)
+
+Status: PLANNING — design workspace only, no implementation.
+Date: 2026-07-20
+
+The playground advertises ~13 platform-op tool schemas to the model on every turn, before the
+model has done anything — ~15K prompt tokens on a bare "hi", plus every extra always-on tool is
+a "wander target" that derails runs. Skills already avoid this (on disk, only name+description
+in the prompt, body loaded on demand). This project applies the same idea to the platform ops:
+advertise a small **discovery meta-toolset** and load a full op schema only when the model asks.
+The lazy layer lives at the runner's `advertisedToolSpecs()` seam, so it is harness-agnostic
+(Pi + Claude) and touches no committed agent config.
+
+## Decisions (locked)
+
+- **Seam = runner advertisement layer.** Intercept the advertised projection at
+  `services/runner/src/tools/public-spec.ts` (`advertisedToolSpecs`), consumed at exactly two
+  sites (`pi-assets.ts`, `environment.ts`). Execution + permission stay below it, unchanged.
+- **Skills are out of scope.** Already progressive.
+- **Op-catalog contents / overlay-cleanup is out of scope.** This work makes *any* op set cheap
+  to carry; it does not decide which ops belong.
+- **Playground overlay only.** No change to any saved/committed agent.
+- **Schema diet is complementary, not exclusive.** Shrinking the embedded agent-template schema
+  is worth doing regardless and lands as its own slice.
+- **No commits during planning.** Implementation happens later on its own branch.
+
+## Deliverables
+
+- [context.md](context.md) — problem, scope, non-goals, product language, success criteria.
+- [research.md](research.md) — how the current advertise/execute path works, with `file:line`,
+  and the numbered seams the plan pins.
+- [design.md](design.md) — the meta-toolset, the execution + permission path, alternatives.
+- [plan.md](plan.md) — the sliced implementation plan, each slice with an exit check.
+- [status.md](status.md) — living source of truth: locked decisions, open questions, next action.
+
+## Intended outcome
+
+A playground author opens an agent and types a message. The model sees two small platform tools
+(`agenta_ops` to list what it can do, `agenta_op` to fetch a schema and act) instead of a wall
+of op schemas. A no-op turn costs a low-hundreds token constant instead of ~15K, and stays flat
+as the catalog grows. Every op still runs with its exact self-targeting binding and approval
+gate. Builds are cheaper and, per the internal-tools review, more reliable.

docs/design/agent-workflows/projects/progressive-tool-disclosure/context.md

@@ -0,0 +1,75 @@
+# Context
+
+## Problem
+
+Open a playground agent that shows "Tools: None" and type "hi". The turn costs ~15K prompt
+tokens. Nothing the author did explains it: the cost is the playground **build kit**, which
+injects ~13 platform-op tool schemas into the agent template and advertises every one of them to
+the model on every turn.
+
+Two costs, not one:
+
+- **Tokens.** Measured (tiktoken `o200k_base`, 2026-07-17): the ops dominate. `test_run` (~6.5K)
+  and `commit_revision` (~5.8K) each embed the full ~5.5K-token agent-template delta schema;
+  `query_spans` ~1.3K; all ops together ~15.4K. Skills are NOT the cost (a ~68-token
+  announcement only). This is a ~5x tax on every turn, paid before the model is useful.
+- **Reliability.** The internal-tools review
+  (`../builder-agent-reliability/tools-review/part-2-internal-tools.md`) found the same tools are
+  a *double* cost: "each unused tool is context cost plus a wander target (the capstone showed
+  extra visible tools derail runs)." Fewer advertised tools is a correctness win, not just a bill.
+
+It scales the wrong way: adding a catalog op is a one-line data change that ships to every
+playground agent unconditionally, so every op we add makes every turn heavier.
+
+Skills already solved this exact shape — on disk, only name+description in the prompt, body
+loaded on demand. The platform ops never got the same treatment.
+
+## Scope (this delivery)
+
+- The **runner advertisement layer** for playground platform ops: `advertisedToolSpecs()` in
+  `services/runner/src/tools/public-spec.ts`, consumed by the Pi and Claude delivery paths.
+- A **discovery meta-toolset** (`agenta_ops` + `agenta_op`) that replaces the always-advertised
+  op schemas: list ops cheaply, fetch one schema on demand, execute against the private spec.
+- A **schema diet** for the two heaviest op schemas (`commit_revision`, `test_run`) —
+  independently valuable, folded in as its own slice.
+- A **token + reliability baseline** so before/after is measured, not asserted.
+- A **one-line nudge** in the always-loaded `build-an-agent` skill so the model uses the
+  meta-toolset.
+
+## Out of scope for the first delivery
+
+- **Skills.** Already progressive; untouched.
+- **Which ops belong in the overlay** (the build-kit-tools-cleanup debate). Orthogonal — this
+  work makes any op set cheap, which lowers the pressure to prune.
+- **External tool discovery (`discover_tools`).** That discovers *Composio* tools to wire into an
+  agent; it stays as-is and is itself one of the ops we disclose.
+- **User / gateway / code / client tools.** The POC targets platform ops (the measured cost). The
+  mechanism can generalize later.
+- **Committed non-playground agents.** They advertise only what their author declared; no problem
+  today.
+- **Dynamic real-name re-advertisement (M2).** Advertising a loaded op under its real name with a
+  schema-validated signature is a productionization option, evaluated after the POC — not built
+  here.
+
+## Product language
+
+- **Platform op** — an existing Agenta endpoint exposed to the agent as a tool, defined in the
+  code catalog `op_catalog.py` (e.g. `commit_revision`, `query_spans`).
+- **Advertised spec** — the `{name, description, inputSchema, …}` projection the model sees;
+  distinct from the **private resolved spec** the runner executes from.
+- **Discovery meta-toolset** — the small fixed set (`agenta_ops`, `agenta_op`) that stands in for
+  the op schemas: list, describe-on-demand, invoke.
+- **Disclosure** — moving an op's full schema out of the prompt (paid every turn) into a tool
+  result (paid once, only when fetched).
+
+## Success criteria
+
+1. A no-op turn's platform-op prompt cost drops from ~15K to a low-hundreds constant, and stays
+   flat as ops are added to the catalog.
+2. Capability parity: the build-an-agent lab loop (discover → wire → commit → test → schedule)
+   passes with the disclosed toolset.
+3. No safety regression: self-targeting `$ctx` bindings and per-op approval/permission behave
+   exactly as today, verified per mutating op.
+4. Reliability does not regress (target: fewer "wander" failures) on the lab matrix.
+5. Cost of laziness is bounded: ≤1 extra round-trip per distinct op used; a schema fetched at
+   most once per op per conversation.

docs/design/agent-workflows/projects/progressive-tool-disclosure/design.md

@@ -0,0 +1,101 @@
+# Design — the discovery meta-toolset
+
+## The core move
+
+Stop advertising N full op schemas every turn. Advertise a small fixed meta-toolset; keep the op
+specs resolved but **private** in runner memory; move each op's full schema from the *prompt*
+(paid every turn) to a *tool result* (paid once, only for ops actually used).
+
+Two model-facing tools (names illustrative):
+
+- **`agenta_ops(query?)`** — returns the op catalog as a compact list: `{op, one_line,
+  read_only}` per op, no input schemas. Optional `query` filters. A few hundred tokens, flat,
+  regardless of catalog size. This is the "what can I do on the platform" index.
+- **`agenta_op(op, args?)`** — the generic invoker. With no `args` (or `mode:"describe"`) it
+  returns **one** op's full `inputSchema` as a tool result; with `args` it executes that op.
+
+The 13 op specs stay in `plan.toolSpecs` and in `toolSpecsByName`. Only the *advertisement* is
+replaced. This is the pattern Claude Code uses on its own ~200 tools: names are listed, and a
+`ToolSearch` step loads a schema only when needed.
+
+## Why this is safe by construction
+
+Execution and permission read the **private** spec (research seams 2–4). The invoker feeds the
+target op's private `call` descriptor into the *unchanged* `direct.ts` path, so:
+
+- **Self-targeting is preserved.** `assembleBody` still fills `call.context` (`$ctx.*`) last, so
+  `commit_revision` still binds `$ctx.workflow.variant.id` and the model cannot retarget another
+  variant. The invoker never sees or forwards those fields — they were stripped at resolve time
+  and are re-applied below the invoker.
+- **SSRF guard is preserved.** `directCallUrl` host-locks to the run's Agenta origin.
+- **Approval is preserved — if the invoker gates per-op.** This is the one piece that is designed
+  in, not free (see below).
+
+## Execution path for `agenta_op(op, args)`
+
+The runner special-cases the invoker in its dispatch:
+
+1. **Resolve the target.** Look up `op` in `toolSpecsByName`. Unknown → tool error listing valid
+   ops (recoverable).
+2. **Describe mode** (no `args` / `mode:"describe"`) → return the target spec's
+   `resolved input schema` as the tool result. No side effects, no approval.
+3. **Execute mode** → build the permission **gate from the target op's spec** and run
+   `decide(gate, plan, stored)` (`permission-plan.ts:138`). `allow` runs; `deny` refuses;
+   `ask`/undecided pauses the turn and emits the normal `interaction_request(user_approval)` —
+   identical to calling the op directly today. Then run the target's `call` through
+   `assembleBody` → `directCallUrl` → `callDirect`, exactly as `executeRelayedTool` does now.
+4. **Return** the endpoint response verbatim.
+
+The invoker is one ordinary advertised tool on both delivery paths (Pi native, Claude MCP), so no
+harness-specific advertisement logic is needed.
+
+## Identifying the disclosure-eligible set
+
+A platform op is a `callback`-kind spec with a direct `call`; so is a `reference` (workflow)
+tool. There is no explicit marker today (research seam 5). Two options:
+
+- **Heuristic (zero wire change).** Collapse every direct-`call` callback spec into the
+  meta-toolset; leave builtins, `client`, `code`, `gateway` (callRef), and MCP advertised as-is.
+  The playground overlay only injects platform ops (+ the two client tools), so this covers the
+  measured cost. Risk: an author who added a `reference` tool would see it disclosed too. Fine for
+  a flagged POC.
+- **Marker (small wire add).** The platform resolver stamps a `source:"platform"` (or a
+  `disclosable` group tag) on the resolved spec; the runner collapses exactly that group. Precise,
+  and lets us disclose gateway/reference later on purpose. Costs a `protocol.ts` + `wire.py` +
+  golden change.
+
+Recommendation: heuristic for the POC to prove the numbers, marker before default-on.
+
+## The catalog summary (`agenta_ops`)
+
+Build `{op, one_line, read_only}` runner-side from the resolved specs already in memory — the
+`description` and `read_only`/`permission` fields ride along; only `inputSchema` is dropped. Zero
+new wire fields. (Alternative: thread a summary list from `op_catalog.py`; rejected for the POC as
+extra plumbing.)
+
+## Discoverability
+
+The always-loaded `build-an-agent` skill (~68 tokens) gains one line: "platform actions are listed
+by `agenta_ops`; fetch a schema with `agenta_op(op)` before calling it with args." Mirrors how the
+skill already routes the builder; negligible always-on weight.
+
+## Alternatives considered
+
+- **M2 — dynamic real-name advertisement.** Advertise names only; a `load_op` call registers the
+  real op spec into the harness registry mid-session (Pi extension re-register; Claude MCP
+  `tools/list_changed`) so the model calls the op by its real name with a schema-validated
+  signature and native per-op permission. Highest fidelity, but needs mid-session re-registration
+  on both harnesses and, under the cold-replay runtime, reconstruction of the "loaded" set each
+  turn. Deferred to productionization.
+- **Schema diet only.** Keep all ops advertised; replace the embedded ~5.5K agent-template delta
+  schema in `commit_revision`/`test_run` with an open object + a pointer to the skill's
+  `references/config-schema.md`. ~11K of ~15K is those two schemas, so this alone is a large,
+  near-zero-risk win. Complementary — it also shrinks each describe-mode fetch under the invoker.
+- **Mode-gating.** Drop the build kit in Chat mode, keep it in Build mode. Removes capability
+  rather than deferring it; can layer on top, not a substitute.
+
+## Cost of laziness
+
+One extra round-trip per *distinct* op used (describe → then execute), paid only for ops the run
+actually touches, and cacheable to once per op per conversation. A no-op turn pays nothing beyond
+the two meta-tools. Acceptable for a builder flow; measured in Slice 3.

docs/design/agent-workflows/projects/progressive-tool-disclosure/plan.md

@@ -0,0 +1,77 @@
+# Plan — sliced implementation
+
+## Scope now
+
+- Behind a flag/env (default OFF) so the disclosed path A/B's against today's always-advertise
+  path before it becomes default.
+- Playground platform ops only; no saved-agent change; no committed-agent behavior change.
+- The schema diet (Slice 1) is independently shippable and lands first.
+- Each slice leaves the tree working and testable. Order: baseline → diet → mechanism → measure.
+
+## Slice 0 — Baseline (pin the real cost)
+
+1. Write a measurement script (tiktoken `o200k_base`) that resolves the default build-kit overlay
+   and reports per-op advertised token cost + total, using the same `advertisedToolSpecs`
+   projection the runner ships (or a faithful mirror).
+2. Confirm which ops actually advertise live today — resolve with `AGENTA_AGENT_ENABLE_PLATFORM_HANDLERS`
+   off, and record whether `test_run` is in the advertised set.
+3. Add a runner unit test asserting the current behavior: every resolved platform op appears in
+   `advertisedToolSpecs(plan.toolSpecs)`. This is the invariant later slices intentionally flip.
+4. Check a `baseline.md` table into this folder (before-numbers, per op).
+
+**Exit:** `baseline.md` exists with per-op numbers; the "all platform ops advertised today" test
+passes on `main`.
+
+## Slice 1 — Schema diet (large, low-risk win, independent)
+
+1. In `op_catalog.py`, replace the embedded agent-template delta schema in `commit_revision`
+   (and `test_run`) with an open object (`{"type":"object"}`) plus a description pointing at the
+   skill's `references/config-schema.md`.
+2. Update the platform-op / wire contract tests that pin those schemas.
+3. Re-run the Slice 0 script; record the drop (expect ~11K).
+4. Lab check: an agent can still `commit_revision` with a valid config (the model has the shape
+   from the skill reference, and the server still validates).
+
+**Exit:** measured total drops by ~11K; contract tests green; a lab run commits a revision
+successfully.
+
+## Slice 2 — Disclosure mechanism (the POC), flagged
+
+1. Add the flag/env (default off) that turns disclosure on for a run.
+2. Introduce a disclosure transform applied to the advertised set at the two call sites
+   (`pi-assets.ts:353`, `environment.ts:721`): when on, replace the disclosure-eligible specs
+   (see design "Identifying the disclosure-eligible set" — heuristic for the POC) with the two
+   meta-tools `agenta_ops` + `agenta_op`; keep client tools and everything else advertised.
+   `plan.toolSpecs` / `toolSpecsByName` stay complete.
+3. Implement the invoker dispatch: `agenta_ops` returns `{op, one_line, read_only}` built
+   runner-side from the resolved specs; `agenta_op` describe-mode returns one op's input schema;
+   `agenta_op` execute-mode builds the gate from the TARGET op's spec, runs `decide()`, then
+   `assembleBody`/`directCallUrl`/`callDirect` (reuse the `executeRelayedTool` core).
+4. Unit tests: (a) with the flag on, only the meta-tools + client tools are advertised; (b) the
+   full private specs remain in `toolSpecsByName`; (c) per mutating op (`commit_revision`,
+   `create_schedule`, `remove_*`), `agenta_op` execute-mode produces the SAME approval verdict as
+   a direct call — no approval regression; (d) `$ctx` binding still fills server-side (the model
+   cannot retarget); (e) describe-mode has no side effect and no approval.
+5. Add the one-line nudge to the `build-an-agent` skill.
+
+**Exit:** flag on → a lab run completes discover → wire → commit → schedule using only the
+meta-tools; every mutating-op approval test passes; `tsc` + `pnpm test` green in `services/runner`.
+
+## Slice 3 — Measure, decide default, scope M2
+
+1. Re-run the Slice 0 script with the flag on; record the no-op turn cost (target: low hundreds).
+2. Run the build-an-agent lab / release gate with the flag on vs off; compare pass rate and
+   "wander" failures.
+3. Write a `results.md` comparison (tokens + reliability, before/after).
+4. Decide: flip default on? And is M2 (dynamic real-name advertisement) worth it for the ops that
+   most benefit from schema-validated calls?
+
+**Exit:** `results.md` checked in with the before/after; a go/no-go recommendation on
+default-on and on M2.
+
+## Not in this plan
+
+- Marker-based eligibility (Slice 2 uses the heuristic); adding a `source:"platform"` wire marker
+  is a follow-up if we default-on for gateway/reference too.
+- Disclosing gateway/code/client/MCP tools.
+- M2 dynamic advertisement implementation.