Skip to content

Agent evidence workflow: principles docs, e2e dev CLI, typed emulator client#977

Merged
RhysSullivan merged 4 commits into
mainfrom
claude/blissful-pike-8710a7
Jun 12, 2026
Merged

Agent evidence workflow: principles docs, e2e dev CLI, typed emulator client#977
RhysSullivan merged 4 commits into
mainfrom
claude/blissful-pike-8710a7

Conversation

@RhysSullivan

@RhysSullivan RhysSullivan commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #973 (per-checkout port claiming + bootstrap).

What this adds

Docs restructured around principles vs mechanics. Root AGENTS.md becomes the stable contract layer and gains the definition of done for user-visible work: hand back evidence, not assertions — (1) an e2e scenario with direct links to the runs that prove the change, (2) the session's dev server left running and reachable, (3) what to try by hand. The same machinery runs in reverse: seed an environment into a state (live bug repro, staged data) and hand across the link. Operational mechanics move to a new RUNNING.md that opens by declaring itself possibly stale — trust, verify, update on drift.

An e2e dev CLI (cd e2e && bun run cli): the same primitives scenarios use, interactive. Boot a target and keep it up (up [--share]), mint identities, typed API calls, MCP sessions, emulator ledger reads, teardown. The boot recipes move out of the vitest globalsetups into shared setup/<target>.boot.ts modules — one boot path for suites (ephemeral) and the CLI (persistent). up --share binds the tailnet; for cloud it fronts both the app and the WorkOS emulator with tailscale-serve HTTPS (the app's auth cookies are Secure) and threads the public origins through. Instances are tracked in e2e/.dev/ so a deliberate long-lived instance is distinguishable from a leak. Ports come from the stacked PR's atomic block claiming.

@executor-js/emulate 0.7.0: the new typed control-plane client (EmulatorClient / connectEmulator) replaces every hand-rolled /_emulate fetch — connect-handoff mints via credentials.mint and asserts against typed LedgerEntry[] instead of substring-searching response text, the cloud target seeds token TTLs via client.seed(), and the CLI's ledger command prints typed entries.

Verified

  • Both targets boot/tear down cleanly through the CLI; identity/api/mcp/ledger all answer; --share reachable over the tailnet; scenarios attach to a CLI-booted instance via E2E_<TARGET>_URL.
  • Both refactored globalsetups boot standalone (api-tools green on cloud + selfhost).
  • connect-handoff green on selfhost + cloud (typed ledger assertion against the hosted resend emulator); the token-expiry mcp-execute scenario green on cloud (exercises seed).
  • Full-suite run: the only failures (sources-api envelope, mixed-auth-methods) reproduce identically on the clean baseline — pre-existing, not from this branch.

Stack

  1. Turnkey fresh-worktree dev env: atomic per-checkout e2e ports, bootstrap, leak reaper #973
  2. Agent evidence workflow: principles docs, e2e dev CLI, typed emulator client #977 👈 current

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 12, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
executor-marketing 8713bfb Commit Preview URL

Branch Preview URL
Jun 12 2026, 04:45 PM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 12, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
executor-cloud 8713bfb Jun 12 2026, 04:46 PM

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Cloudflare preview

Torn down — the PR is closed.

@pkg-pr-new

pkg-pr-new Bot commented Jun 12, 2026

Copy link
Copy Markdown

Open in StackBlitz

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@977

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@977

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@977

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@977

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@977

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@977

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@977

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@977

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@977

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@977

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@977

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@977

executor

npm i https://pkg.pr.new/executor@977

commit: 2e24da7

@RhysSullivan RhysSullivan changed the base branch from claude/gracious-satoshi-42e8e9 to main June 12, 2026 16:41
@RhysSullivan RhysSullivan force-pushed the claude/blissful-pike-8710a7 branch from 2e24da7 to 30e694c Compare June 12, 2026 16:42
When tests or demos need an upstream API, OAuth provider, or webhook
source, agents should reach for the @executor-js/emulate emulators
instead of hand-writing stubs. The new skill captures the control-plane
reference (/_emulate/openapi, /credentials, /ledger, /seed), the
hosted-instance addressing, the connect-handoff ledger-assertion
pattern, and the hard-won gotchas (Secure cookies need HTTPS
off-localhost; per-boot id counters need a fresh app DB).

.gitignore: carve .claude/skills/ out of the .claude/ ignore so skills
are shareable repo state while worktrees/settings stay local.
Root AGENTS.md becomes the stable contract layer and gains the missing
piece both recent agent sessions fumbled: a definition of done for
user-visible work. Done is something the user can open — (1) an e2e
scenario with direct links to the runs that prove the change, (2) the
session's dev server left up and reachable over the tailnet so the user
can take over, (3) an honest list of what to try by hand. The same
machinery runs in reverse: seed an environment into a state (live bug
repro, staged data) and hand across the link.

Mechanics move to a new RUNNING.md that opens by declaring itself
possibly stale — trust it as a starting point, verify against the code,
update on drift. It carries setup, the boot recipes (pointing at the
e2e globalsetup files as source of truth), run/viewer/sharing
operations, the leave-an-instance-up recipe with the Secure-cookie/
HTTPS constraint, and the hard-won environment gotchas.

e2e/AGENTS.md points at both. Docs deliberately avoid pinning URL
formats and flags that churn — agents discover those.
cd e2e && bun run cli — boot a target and keep it up (up [--share]),
mint identities, make typed API calls, drive MCP sessions, read the
emulator request ledger, and tear down (down). The point: develop
against a live instance with the exact machinery the tests use, then
crystallize the journey into a scenario — and the booted instance IS
the handoff demo (AGENTS.md evidence contract) and the seeding vehicle
(drive a bug repro or staged state, hand across the URL).

The boot recipes move out of the vitest globalsetups into shared
setup/<target>.boot.ts modules — one boot path for suites (ephemeral)
and the CLI (persistent, detached runner owning cloud's in-process
WorkOS/Autumn emulators). up --share binds the tailnet: plain http for
selfhost; for cloud it fronts both the app and the WorkOS emulator
with tailscale-serve HTTPS (Secure cookies) and threads the public
origins through WORKOS_API_URL / emulator baseUrl / Vite allowed
hosts. Instances are tracked in e2e/.dev/<target>.json so a deliberate
long-lived instance is distinguishable from a leak; logs land next to
the state file (boot.ts gains logFile + pids for that).

Verified: both targets up/down cleanly through the CLI, identity/api/
mcp/ledger all answer, a scenario attaches to a CLI-booted instance
via E2E_SELFHOST_URL, both refactored globalsetups boot standalone
(api-tools green on cloud + selfhost), full suite run — the two cloud
failures (sources-api envelope, auth-methods mixed-source) reproduce
identically on the clean baseline, so they predate this change.
0.7.0 ships EmulatorClient + connectEmulator: one typed handle over the
/_emulate control plane (credentials.mint, ledger.list/clear, seed,
reset, manifest, ...) wherever the emulator runs — locally spawned or
hosted. Every hand-rolled fetch + response cast goes away:

- connect-handoff: mints via credentials.mint and asserts against
  LedgerEntry[] (match on request.body) instead of substring-searching
  the raw ledger text.
- targets/cloud setAccessTokenTtl: client.seed() instead of a raw POST.
- CLI ledger command: typed entries, JSON output.
- emulate skill: the client table replaces the raw-endpoint table as
  the primary interface; raw /_emulate stays documented for curl use.

Also folds the CLI onto src/ports.ts claimPorts (from the worktree-env
PR this branch now stacks on) — the CLI's ad-hoc probe-and-walk is
gone; a held block lock marks the instance and releases on down.

Verified: connect-handoff green on selfhost + cloud (the ledger
assertion exercises the hosted resend emulator's typed client), the
token-expiry mcp-execute scenario green on cloud (exercises seed).
@RhysSullivan RhysSullivan force-pushed the claude/blissful-pike-8710a7 branch from 30e694c to 8713bfb Compare June 12, 2026 16:43
@RhysSullivan RhysSullivan merged commit ecd4322 into main Jun 12, 2026
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant