Agent evidence workflow: principles docs, e2e dev CLI, typed emulator client (#977)

* Document the service emulators: AGENTS.md note + emulate skill

When tests or demos need an upstream API, OAuth provider, or webhook
source, agents should reach for the @executor-js/emulate emulators
instead of hand-writing stubs. The new skill captures the control-plane
reference (/_emulate/openapi, /credentials, /ledger, /seed), the
hosted-instance addressing, the connect-handoff ledger-assertion
pattern, and the hard-won gotchas (Secure cookies need HTTPS
off-localhost; per-boot id counters need a fresh app DB).

.gitignore: carve .claude/skills/ out of the .claude/ ignore so skills
are shareable repo state while worktrees/settings stay local.

* Restructure agent docs: principles in AGENTS.md, mechanics in RUNNING.md

Root AGENTS.md becomes the stable contract layer and gains the missing
piece both recent agent sessions fumbled: a definition of done for
user-visible work. Done is something the user can open — (1) an e2e
scenario with direct links to the runs that prove the change, (2) the
session's dev server left up and reachable over the tailnet so the user
can take over, (3) an honest list of what to try by hand. The same
machinery runs in reverse: seed an environment into a state (live bug
repro, staged data) and hand across the link.

Mechanics move to a new RUNNING.md that opens by declaring itself
possibly stale — trust it as a starting point, verify against the code,
update on drift. It carries setup, the boot recipes (pointing at the
e2e globalsetup files as source of truth), run/viewer/sharing
operations, the leave-an-instance-up recipe with the Secure-cookie/
HTTPS constraint, and the hard-won environment gotchas.

e2e/AGENTS.md points at both. Docs deliberately avoid pinning URL
formats and flags that churn — agents discover those.

* Add the e2e dev CLI: the scenario primitives, interactive

cd e2e && bun run cli — boot a target and keep it up (up [--share]),
mint identities, make typed API calls, drive MCP sessions, read the
emulator request ledger, and tear down (down). The point: develop
against a live instance with the exact machinery the tests use, then
crystallize the journey into a scenario — and the booted instance IS
the handoff demo (AGENTS.md evidence contract) and the seeding vehicle
(drive a bug repro or staged state, hand across the URL).

The boot recipes move out of the vitest globalsetups into shared
setup/<target>.boot.ts modules — one boot path for suites (ephemeral)
and the CLI (persistent, detached runner owning cloud's in-process
WorkOS/Autumn emulators). up --share binds the tailnet: plain http for
selfhost; for cloud it fronts both the app and the WorkOS emulator
with tailscale-serve HTTPS (Secure cookies) and threads the public
origins through WORKOS_API_URL / emulator baseUrl / Vite allowed
hosts. Instances are tracked in e2e/.dev/<target>.json so a deliberate
long-lived instance is distinguishable from a leak; logs land next to
the state file (boot.ts gains logFile + pids for that).

Verified: both targets up/down cleanly through the CLI, identity/api/
mcp/ledger all answer, a scenario attaches to a CLI-booted instance
via E2E_SELFHOST_URL, both refactored globalsetups boot standalone
(api-tools green on cloud + selfhost), full suite run — the two cloud
failures (sources-api envelope, auth-methods mixed-source) reproduce
identically on the clean baseline, so they predate this change.

* Migrate to @executor-js/emulate 0.7.0's typed control-plane client

0.7.0 ships EmulatorClient + connectEmulator: one typed handle over the
/_emulate control plane (credentials.mint, ledger.list/clear, seed,
reset, manifest, ...) wherever the emulator runs — locally spawned or
hosted. Every hand-rolled fetch + response cast goes away:

- connect-handoff: mints via credentials.mint and asserts against
  LedgerEntry[] (match on request.body) instead of substring-searching
  the raw ledger text.
- targets/cloud setAccessTokenTtl: client.seed() instead of a raw POST.
- CLI ledger command: typed entries, JSON output.
- emulate skill: the client table replaces the raw-endpoint table as
  the primary interface; raw /_emulate stays documented for curl use.

Also folds the CLI onto src/ports.ts claimPorts (from the worktree-env
PR this branch now stacks on) — the CLI's ad-hoc probe-and-walk is
gone; a held block lock marks the instance and releases on down.

Verified: connect-handoff green on selfhost + cloud (the ledger
assertion exercises the hosted resend emulator's typed client), the
token-expiry mcp-execute scenario green on cloud (exercises seed).

Author: RhysSullivan

.claude/skills/emulate/SKILL.md

@@ -0,0 +1,101 @@
+---
+name: emulate
+description: Use the @executor-js/emulate service emulators (GitHub, Google, Stripe, Resend, WorkOS, …) to test integrations for real — full OpenAPI specs, working OAuth flows, mintable credentials, and a request ledger for assertions. Use when a test or demo needs a real-shaped upstream API, an OAuth/OIDC provider, a spec to feed addSpec, or proof that a request actually landed.
+---
+
+# Emulate: production-fidelity service emulators
+
+`@executor-js/emulate` (our fork of Vercel Labs' emulate, developed in
+`vendor/emulate` but ALWAYS consumed as the published npm package — never
+import from `vendor/`) provides stateful, wire-level emulators for 16
+services: `github vercel google okta microsoft spotify slack apple aws
+resend stripe mongoatlas clerk x workos autumn`. These are not mocks: real
+SDKs and real product code run against them unmodified — the cloud e2e
+target points the actual WorkOS SDK (sealed sessions, JWKS, hosted AuthKit
+login) and Autumn billing at emulators and exercises the product's real
+auth code.
+
+## Two ways to get one
+
+**Local, programmatic** (what `e2e/setup/cloud.boot.ts` does):
+
+```ts
+import { createEmulator } from "@executor-js/emulate";
+const github = await createEmulator({ service: "github", port: 4501 });
+// github.url, await github.close() — plus the full typed client below
+```
+
+`baseUrl` sets the _advertised_ origin (redirects, form actions, spec
+`servers`) when a proxy fronts the emulator — the bind stays on `port`.
+
+**Attach to a running one** (another process, or hosted on Cloudflare with
+Durable Object state at `https://<service>.emulators.dev` /
+`https://<service>.<instance>.emulators.dev` — catalog at
+`GET https://emulators.dev/_emulate/services`):
+
+```ts
+import { connectEmulator } from "@executor-js/emulate";
+const resend = await connectEmulator({ baseUrl: "https://resend.emulators.dev" });
+// optional: { service: "resend" } verifies the manifest on connect
+```
+
+Create a private hosted instance with `POST /_emulate/instances` on the
+service host.
+
+## The typed control-plane client
+
+Both `createEmulator` and `connectEmulator` return the same `EmulatorClient`
+surface (since 0.7.0) — use it instead of hand-rolling `/_emulate` fetches:
+
+| Call                                                               | Use                                                                                                                            |
+| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------ |
+| `client.openapiUrl`                                                | The spec URL — feed it straight to Executor's addSpec to register the emulator as an integration                               |
+| `client.credentials.mint({type:"api-key"})`                        | `IssuedCredential` in the service's real shape: API keys, bearer tokens, OAuth/OIDC clients, client-credentials apps           |
+| `client.ledger.list()` / `.clear()`                                | `LedgerEntry[]`: matched operationId, sanitized headers/body, auth identity, response status, webhook deliveries               |
+| `client.seed({...})`                                               | Add state via the service's seed schema (e.g. WorkOS `{oauth:{default_access_token_ttl_seconds:60}}` to compress token expiry) |
+| `client.reset()`                                                   | Reset state + logs, replay seed — works remotely, unlike the old local-only reset                                              |
+| `client.manifest()` / `.quickstart()` / `.specs()` / `.coverage()` | What the service is, which operations are real vs partial                                                                      |
+| `client.state()` / `.logs()` / `.connections()`                    | Store snapshot, webhook deliveries, copyable SDK/env/curl snippets                                                             |
+
+The same routes exist as raw HTTP under `/_emulate/*` (start at
+`GET /_emulate/quickstart`, written for agents) for curl/browser use — but in
+TypeScript, reach for the client; the types are the point.
+
+## Recipes
+
+**Test an integration end-to-end for real** (the `connect-handoff` pattern):
+`client.credentials.mint(...)` → register `client.openapiUrl` with the
+product → invoke a tool through the product → find the call in
+`client.ledger.list()` (match on `entry.request.body` / `operationId`). The
+ledger is the proof — "the product made this exact upstream call with this
+auth" — which beats asserting on the product's own response.
+
+**Real OAuth/OIDC flows**: google/okta/microsoft/apple/clerk/workos mint
+OAuth clients and run real authorize/token endpoints. The WorkOS emulator
+additionally serves hosted AuthKit login pages (any email signs in — users
+are minted on the fly, no password), an OAuth authorization server for MCP
+clients, and Vault KV. Real SDK + `WORKOS_API_URL` override = the product's
+untouched auth code against it. Set `EMULATE_WORKOS_AUDIENCE=<client_id>`
+before `createEmulator` so minted MCP access tokens carry the right audience.
+
+**A live, human-pokeable cloud instance with zero .env**:
+`cd e2e && bun run cli up cloud --share` — WorkOS + Autumn emulators + the
+app's real dev stack (recipe in `e2e/setup/cloud.boot.ts`), fronted with
+tailscale HTTPS.
+
+## Gotchas
+
+- **Secure cookies need HTTPS off-localhost.** Browser-driven flows work on
+  `127.0.0.1`, but from another device (tailnet) the app's `secure: true`
+  auth cookies are dropped over http → "Invalid login state". Front BOTH the
+  app and the emulator with HTTPS (`tailscale serve`), and give the emulator
+  its public origin via `baseUrl` AND the app's `WORKOS_API_URL` — the
+  authorize URL the browser follows is derived from the latter.
+- **State is per-process and id counters restart.** The WorkOS emulator
+  mints org ids from a per-boot counter — a persisted app DB from a previous
+  boot collides with new ids. Wipe the app's data dir when you restart the
+  emulator (the e2e globalsetup and cloud-demo both do).
+- **Don't hand-write fake upstreams.** If a scenario needs an upstream API,
+  OAuth provider, or webhook source, reach for an emulator before writing a
+  bespoke stub server — you get specs, auth, and the ledger for free, and
+  the e2e AGENTS.md "never modify product code or stubs" rule stays intact.

.gitignore

@@ -70,7 +70,9 @@ apps/host-selfhost/.e2e-data*/
 test-results/
 playwright-report/
 .last-run.json
-.claude/
+# .claude is local agent state (worktrees, settings), EXCEPT checked-in skills
+.claude/*
+!.claude/skills/
 .nitro/
 .output/
 .tanstack/

AGENTS.md

@@ -1,29 +1,11 @@
 # AGENTS.md
 
-## Fresh Checkout / Worktree Setup
-
-Run `bun run bootstrap` first in any fresh checkout or worktree. It is
-idempotent: runs `bun install` (whose prepare hook builds the internal
-packages dev servers fail without) and installs Playwright chromium.
-Skipping it is why fresh worktrees die with "Failed to resolve entry for
-package '@executor-js/vite-plugin'". The `vendor/` submodules are NOT
-needed — nothing imports from `vendor/` at runtime; those forks are consumed
-from npm (see `vendor/README.md`). `bun run bootstrap --forks` inits them
-only when you're deliberately developing a fork.
-
-## Environment Gotchas (learned the hard way)
-
-- The shell is fish, and the working directory resets between Bash calls. Use
-  absolute paths rooted at THIS worktree (check `pwd`), never
-  `/Users/rhys/src/executor` from memory, and don't rely on a prior `cd`.
-- Don't write probe scripts to `/tmp` — they can't resolve workspace packages
-  (`effect`, `playwright`, ...). Put scratch scripts under the repo root
-  (`scratch/` is gitignored) so bun resolves the workspace.
-- `bun.lock` conflicts on rebase/merge: take either side, then re-run
-  `bun install` to regenerate it — never hand-merge the lockfile.
-- e2e dev-server ports are derived per checkout (`cd e2e && bun run ports`).
-  If a boot reports a squatted port, an old dev server leaked — kill it by
-  PID from the error message; don't move your own ports to dodge it.
+This file is principles — the contracts that stay true while implementations
+churn. For how to actually run, boot, share, or navigate things today
+(fresh-worktree setup, dev servers, ports, environment gotchas), see
+[RUNNING.md](RUNNING.md) (which may lag reality; it says so itself). For
+writing e2e scenarios, see [e2e/AGENTS.md](e2e/AGENTS.md). Run
+`bun run bootstrap` first in any fresh checkout or worktree.
 
 ## Task Completion Requirements
 
@@ -36,6 +18,49 @@ only when you're deliberately developing a fork.
 - For broad or merge-ready changes, the full gates are `bun run format:check`,
   `bun run lint`, `bun run typecheck`, and `bun run test`.
 
+## Handing Back Work: Evidence, Not Assertions
+
+"Done" is something the user can open, not a claim. When work changes what a
+user sees or touches, the handoff has three parts, delivered unprompted:
+
+1. **Watch it** — an e2e scenario covers the change, and the handoff links
+   directly to the specific run(s) that prove it, with one line each on what
+   to look at. Never hand back a bare wall of green results: the user's
+   question is "show me the new thing working," not "is everything healthy?"
+2. **Touch it** — leave the session's dev server running and reachable over
+   the user's tailnet, with credentials, so they can take over and poke at
+   it. The instance you already booted for e2e IS this — leave it up rather
+   than standing up something separate.
+3. **What to try** — name the paths worth exercising by hand, especially
+   ones no scenario pins yet. Honesty about coverage gaps is part of the
+   handoff. A human driving a real browser from another device reaches
+   states the test harness structurally cannot; invite that.
+
+The same machinery runs in reverse: you can seed an environment INTO a
+state — reproduce a bug live, stage data for the user to take over, set up
+a walkthrough — and hand across the link. "Here's the broken state, live"
+beats a paragraph describing it.
+
+If no scenario covers the change yet, that is the cue to write one. When a
+change is user-visible, embed the run's recording in the PR description —
+reviewers should see the change, not just read about it.
+
+Don't memorize the mechanics (ports, viewer, sharing commands) — discover
+them from RUNNING.md and the code; they change.
+
+## Service Emulators
+
+When a test or demo needs an upstream API, OAuth/OIDC provider, or webhook
+source, use the `@executor-js/emulate` emulators (GitHub, Google, Stripe,
+Resend, WorkOS, and a dozen more) instead of writing a stub. They are
+wire-level and stateful — real SDKs run against them unmodified — and each
+serves a full OpenAPI spec ready for addSpec, mints real-shaped credentials,
+runs working OAuth flows, and records every call in a request ledger you can
+assert against. Hosted instances exist at `https://<service>.emulators.dev`
+with zero setup. See the `emulate` skill
+(`.claude/skills/emulate/SKILL.md`) for the control-plane reference and
+recipes.
+
 ## Attribution
 
 Do not add any AI assistant, Claude, Anthropic, or Co-Authored-By
@@ -44,21 +69,6 @@ attribution/trailers to commits, commit messages, PRs, or generated files.
 Pull request titles and descriptions are going to a public GitHub repo, so
 avoid using specific names or internal info unless explicitly stated to.
 
-## Show Changes in PR Descriptions
-
-When a change is user-visible and an e2e scenario covers it, embed the run's
-recording in the PR description — reviewers should see the change, not just
-read about it.
-
-```
-bun e2e/scripts/pr-media.ts e2e/runs/<target>/<scenario-slug>
-```
-
-converts the run's recording (browser `session.mp4` or `terminal.cast`) to a
-gif, uploads it to the `e2e-media` branch, and prints PR-ready markdown to
-paste into the body. Run screenshots (`*.png`) can be passed directly too. If
-no scenario covers the change yet, that is usually the cue to write one.
-
 ## Collaboration Notes
 
 The user uses speech to text occasionally, so if sentences are weird or words
@@ -102,4 +112,6 @@ before using it.
 
 ## Other
 
-Please make note of mistakes you make in MISTAKES.md. If you find you wish you had more context or tools, write that down in DESIRES.md. If you learn anything about your env write that down in LEARNINGS.md.
+Please make note of mistakes you make in MISTAKES.md. If you find you wish you
+had more context or tools, write that down in DESIRES.md. If you learn anything
+about your env write that down in LEARNINGS.md.

RUNNING.md

@@ -0,0 +1,117 @@
+# RUNNING.md — how things run today
+
+> **This document may be out of date.** It describes how things run today,
+> not how they must run. Trust it as a starting point; if you hit weirdness,
+> the implementation has probably moved and this file is why. Verify against
+> the code, then update this file when you notice drift. The principles in
+> [AGENTS.md](AGENTS.md) are the stable contract; everything below is
+> implementation detail that churns.
+
+## Fresh checkout / worktree setup
+
+`bun run bootstrap` from the repo root — idempotent: `bun install` (whose
+prepare hook builds `@executor-js/vite-plugin` and `packages/react`, the
+artifacts dev servers fail without) plus Playwright chromium. A fresh
+worktree that skips it dies with "Failed to resolve entry for package
+'@executor-js/vite-plugin'".
+
+The `vendor/` submodules (emulate, mcporter) are NOT required — nothing
+imports from `vendor/` at runtime; those packages come from npm
+(`@executor-js/emulate`, `@executor-js/mcporter`). `bun run bootstrap
+--forks` inits them only when deliberately developing a fork (see
+`vendor/README.md`).
+
+## Dev servers
+
+- Everything except desktop/cloud: `bun run dev` (turbo, from root)
+- One app: `bun run dev` from its `apps/<name>` directory
+- Self-host boots standalone with just env vars — see
+  `e2e/setup/selfhost.globalsetup.ts` for the canonical recipe (data dir,
+  bootstrap admin email/password, base URL, `EXECUTOR_ALLOW_LOCAL_NETWORK`)
+- Cloud needs WorkOS + Autumn; for a no-.env boot, point it at emulators —
+  see `e2e/setup/cloud.globalsetup.ts` for the canonical recipe (the real
+  SDKs against emulated services, PGlite dev DB)
+
+The e2e globalsetup files are the source of truth for "how do I boot a
+working instance of X" — read them before inventing a boot path.
+
+## E2E: running, viewing, sharing
+
+`e2e/AGENTS.md` covers writing scenarios. Operationally:
+
+- `cd e2e && bun run test` boots dev servers and runs everything;
+  `--project cloud|selfhost` narrows. `E2E_CLOUD_URL`/`E2E_SELFHOST_URL`
+  attach to an already-running server instead of booting.
+- Runs land in `e2e/runs/<target>/<scenario-slug>/` — `result.json`, step
+  screenshots, `session.mp4` + `trace.zip` for browser scenarios, and the
+  scenario source as `test.ts`.
+- `cd e2e && bun run serve` builds the viewer and serves the scenario ×
+  target matrix over HTTP, bound to all interfaces (reachable over the
+  tailnet). Individual runs are at `#/<target>/<slug>` hash routes — when
+  handing results to the user, link those directly, not the bare matrix.
+- `bun e2e/scripts/pr-media.ts e2e/runs/<target>/<slug>` converts a run's
+  recording to a gif, uploads it to the `e2e-media` branch, and prints
+  PR-ready markdown.
+
+E2E dev-server ports are derived and CLAIMED per checkout (`cd e2e && bun
+run ports` prints this checkout's block; see `e2e/src/ports.ts`) — each
+checkout hashes its repo root to a preferred block, atomically locks it,
+and walks to the next free block if squatted, so concurrent worktrees never
+collide or attach to each other's servers. `E2E_*_PORT` env vars pin ports
+explicitly. If a boot reports a squatted port, an old dev server leaked —
+`bun run reap` (repo root) lists and kills orphaned stacks.
+
+## The dev CLI: live instances, interactively
+
+`cd e2e && bun run cli` — the same primitives scenarios use, as commands.
+Boot a target, mint identities, make typed API calls, drive MCP, read the
+emulator ledger — develop interactively, then crystallize the journey into
+a scenario.
+
+```sh
+bun run cli up selfhost --share   # boot, reachable over the tailnet, stays up
+bun run cli up cloud --share      # emulated WorkOS+Autumn, tailscale-HTTPS fronted
+bun run cli status                # what's running, URLs, creds
+bun run cli identity selfhost     # fresh identity (headers / cookies / creds)
+bun run cli api selfhost tools.list
+bun run cli mcp selfhost call execute '{"code":"return 1+1;"}'
+bun run cli ledger cloud workos   # what hit the emulator
+bun run cli down selfhost         # tear down (also removes tailscale serves)
+```
+
+Instances persist until `down` — `up --share` IS the "touch it" handoff
+artifact, and the seeding direction too: boot, drive the product into a
+state (API/MCP/UI), hand across the URL. State files in `e2e/.dev/` mark
+deliberate long-lived instances (vs leaks); attach scenarios to a running
+instance with `E2E_<TARGET>_URL`.
+
+Why cloud `--share` is more involved (encoded in the CLI, kept here for
+when you hit it manually): the cloud app sets `secure: true` auth cookies,
+so login breaks over plain http from any non-localhost origin ("Invalid
+login state"). Both the app AND the WorkOS emulator get fronted with
+`tailscale serve` HTTPS, the emulator advertises its public URL on both
+sides (its `baseUrl` and the app's `WORKOS_API_URL` — the browser-facing
+authorize URL derives from the latter), and Vite must allow the public
+hostname (`__VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS`).
+
+## Environment gotchas (learned the hard way)
+
+- The shell is fish, and the working directory resets between Bash calls.
+  Use absolute paths rooted at THIS worktree; don't rely on a prior `cd`.
+- Don't write probe scripts to `/tmp` — they can't resolve workspace
+  packages (`effect`, `playwright`, …). Put scratch scripts under the repo
+  root (`scratch/` is gitignored) so bun resolves the workspace.
+- A fresh worktree's Vite dep-optimizer cache can serve PRE-REBASE code
+  (symptom: behavior matching old code only in dev servers, while unit
+  tests pass). Kill the server, clear `node_modules/.vite` /
+  `.tanstack`-adjacent caches, reboot.
+- The real Tailscale CLI on this machine is
+  `/opt/homebrew/opt/tailscale/bin/tailscale`; `/usr/local/bin/tailscale`
+  is a broken shim pointing at a deleted app. The tailnet IP is on the
+  `utun` interface (100.x.y.z) if the CLI fails.
+- `bun.lock` conflicts on rebase: take either side, re-run `bun install`,
+  never hand-merge.
+- Long-lived demo servers you left up for the user look like leaks to
+  cleanup tooling — `e2e/.dev/<target>.json` marks deliberate instances;
+  check it before reaping, and `bun run cli down <target>` is the clean
+  teardown.

bun.lock

@@ -0,0 +1 @@
+.dev/