feat(wizard-ci): full e2e via control plane (--e2e) + replay (--replay)#2012
Draft
gewenyu99 wants to merge 1 commit into
Draft
feat(wizard-ci): full e2e via control plane (--e2e) + replay (--replay)#2012gewenyu99 wants to merge 1 commit into
gewenyu99 wants to merge 1 commit into
Conversation
Adds two modes to the existing wizard-ci, as an alternative to classic --ci (LoggingUI: agent-only, stdout-grep). --e2e drives the WHOLE interactive flow headlessly through the wizard-ci-tools control plane and asserts on structured state; --replay plays a recorded run back in the terminal. Core files: - services/wizard-ci/e2e.ts — runE2e(): /tmp app-copy isolation, env hygiene (strips host CLAUDE*/ANTHROPIC* so the spawned agent auths with the phx key instead of deferring to the host), scoped --project-id, the happy-path policy (skip mcp+slack, delete skills, continue past health issues), spawns the wizard repo's headless harness, then asserts the structured result (runPhase=completed, posthog dep/.env, reached keep-skills, skillsComplete). replayRecording(): shells to the wizard repo's terminal replayer. - services/wizard-ci/index.ts — wires --e2e (positional app, --project-id, --keep-skills) and --replay (--step/--delay) into the CLI + --help. Engine lives in the wizard repo (store + driver must run in-process); point WIZARD_PATH at it. See PostHog/wizard PR for src/lib/ci-driver + harness. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
--e2eand--replayto the existingwizard-ci, as a complement to classic--ci(which runs the agent underLoggingUIand exits — no interactive screens, only ANSI stdout to grep).--e2edrives the whole flow headlessly through thewizard-ci-toolscontrol plane and asserts on structured state.pnpm wizard-ci basic-integration/javascript-node/express-todo --e2e pnpm wizard-ci --replay /tmp/wizard-e2e-express-todo.recording.json # Enter ▸ stepCore files
services/wizard-ci/e2e.ts— the orchestration + assertion layer.runE2e(): copies the app to/tmp(never the real fixture); strips hostCLAUDE*/ANTHROPIC*env so the spawned agent auths with the phx key instead of deferring to the host (theapiKeySource: none→ 401 trap when run from inside a Claude Code session); passes the scoped--project-id; applies the happy-path policy (skip MCP + Slack, delete skills, continue past health-check issues); spawns the wizard repo's headless harness; then asserts the structured result —runPhase=completed, posthog dep /.envwritten, flow reachedkeep-skills,skillsComplete.replayRecording(): shells to the wizard repo's terminal replayer.services/wizard-ci/index.ts— wires--e2e(positional app,--project-id,--keep-skills) and--replay(--step/--delay <ms>) into the CLI +--help.Why
--cican't answer interactive prompts, skips the post-agent screens, and only offers stdout. The control plane drives every decision through the same store setters the real UI uses and asserts on structured state — same agent, full coverage, robust assertions. Validated end-to-end onexpress-todo(Node) and15-app-router-todo(Next.js): both real integrations,✓ E2E PASS.Depends on
The engine (store + driver run in-process with the wizard): PostHog/wizard companion PR adding
src/lib/ci-driver/+ the headless harness. SetWIZARD_PATHto the wizard repo.🤖 Generated with Claude Code