feat(cli): overhaul cli#617
Merged
Merged
Conversation
Core of the CLI overhaul: adds the ProgramCliSurface type, snapshots context-mill's cli-manifest.json into a build-time TS module (schema-validated via ajv in prebuild — a fetched-but-invalid manifest fails the build), adds the native + skill command factories, the `wizard audit` family with an interactive picker (recommended leaf pre-highlighted), and the `wizard skill` catalog (internal role hidden by default). Old single-purpose command entry files are replaced by manifest-derived commands. Manifest contract uses context-mill's role/command/skill/internal + recommended vocabulary (renamed from the earlier surface/public/catalog + default). Generated-By: PostHog Code Task-Id: 4debb1ba-a98a-494f-ba44-29df9fd1d87b
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
sarahxsanders
changed the title
…cute - flatSkillCommand: graceful fallback when a manifest entry is missing, instead of throwing at import (which took down the whole CLI); both flat commands now resolve by stable skillId - command.ts: declare positionals so `skill [name]` parses under strictOptions; correct the `recommended` doc comment - skill: `wizard skill` lists, `wizard skill <name>` runs, `wizard skill search <query>` searches; reject unknown skill names - agentSkillConfig: add a run recipe so `wizard skill <name>` and the narrow audit leaves actually run (were silently skipped via skipAgent) - family-picker: document the argv-bypass on the picker path - tests: regression guard for the agentSkillConfig run recipe Generated-By: PostHog Code Task-Id: b42696d8-ac3c-4cd2-ba12-747a846b8c23
…sive audit Move the `recommended` flag from `audit all` to `audit-events` so a bare `wizard audit` pre-highlights the events audit. `audit all` stays a selectable leaf — dropping it (and the auditConfig/screen cleanup that requires) is deferred to the multi-select PR. Mirrors the matching context-mill `cli:` change; the build currently sources this bootstrap snapshot since context-mill hasn't released the manifest yet. Generated-By: PostHog Code Task-Id: b42696d8-ac3c-4cd2-ba12-747a846b8c23
…ill id AuditRunScreen picked the slide deck by checking `session.skillId === 'events-audit'`, but context-mill renamed that skill to `audit-events` (the bootstrap manifest and the programs-cli dispatch test both use `audit-events`). The stale check silently fell through to the comprehensive AUDIT_AREA_SLIDES, so `wizard audit events` showed the wrong deck. Match the current id. Generated-By: PostHog Code Task-Id: 95407a79-1f13-4f1d-a37b-43fe4a62b857
Replaces the build-time CLI manifest snapshot with a runtime resolver
that reads cliEntries from skill-menu.json each invocation. Adding a
skill-backed subcommand is now a context-mill release; no wizard release
needed.
How it fits together:
- dispatch-family.ts owns the runtime resolution. dispatchFamily(family,
argv) tries the native handler registry first (today: just
audit/web-analytics), then fetches skill-menu.json and matches
parentCommand + command -> skillId. The comprehensive `audit all`
keeps its specialized auditConfig; everything else runs through the
generic agent-skill program with the resolved skillId injected.
- family-command-factory.ts wraps dispatchFamily as a yargs Command
(name: `<family> [skill]`). interactiveDefault opens the picker with
native + live entries combined; the recommended leaf is
pre-highlighted.
- audit.ts collapses to one factory call. migrate.ts / revenue.ts swap
flatSkillCommand for nativeCommandFactory now that the config has
the right command name and skillId. skill.ts fetches the catalog
at handler time instead of reading a baked snapshot.
Deletions:
- audit-3000 retired (skill, screens, registry entry, agent-interface
model branch, playground demo, screen-registry wiring)
- skill-command-factory.ts + flat-skill-command.ts: the runtime resolver
replaces both, including the empty-manifest fallback flat-skill-command
existed to work around
- cli-manifest.bootstrap.json, cli-manifest.schema.bootstrap.json,
scripts/generate-cli-manifest.cjs, src/lib/programs/cli-manifest.generated.ts
- ajv dev-dep; the generate-cli-manifest.cjs step in prebuild; three
.gitignore lines covering the deleted bootstrap apparatus
Test surface: programs-cli mocks fetchSkillMenu and asserts dispatchFamily
routing (skill-backed, native, comprehensive). family-picker keeps its
auditCommand shape assertion but flipped to "no static children by design".
program-registry catches the revenue-analytics command-name rename.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolves seven conflicts from main, which shipped #618 (`wizard skill` command) and #630 (source-maps → upload-source-maps rename) while this branch reworked the audit family and retired audit-3000. Resolutions: - skill.ts: hybrid. Keeps main's `wizard skill <skill-name>` contract (positional + runSkillMode) and adds `wizard skill list` as a child command that fetches the live skill-menu.json so users have an in-CLI way to discover skill ids. Drops the search subcommand from our pre-merge version — easy to add later if needed. - bin.ts, basic-integration/index.ts: take main's side. The `--skill` flag is gone for good (it's a real command now). - upload-sourcemaps.ts + error-tracking-upload-source-maps/index.ts: take main's renamed file and the matching `command: 'upload-source-maps'` config update. Drops our orphan `src/commands/source-maps.ts`. - audit-3000 modified files: kept deleted. We retired the program. - programs-cli.test.ts: combined imports from both sides. All tests from both sides pass after the merge. - wizard.test.ts: chased the rename through to its sourceMapsCommand import. Verification: pnpm typecheck green; pnpm jest 837/837 passing across 54 suites.
CI's `pnpm install --frozen-lockfile` rejected the previous push because the lockfile still pinned ajv@^8.20.0 as a direct dep even though package.json had already removed it. The remaining ajv entries are now transitive (via warlock and zod-to-json-schema), which is fine. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the command-overhaul mapping table to the README and a new AGENTS.md for agents working in the repo. Covers the command-vs-program distinction and where the command surface is defined. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Syncs the branch with main (was 19 commits behind). Brings in the `wizard slack` command (#654) and other post-2.21.0 work. Conflict resolution (src/commands/command.ts): combined main's `entryCommand`/`setEntryCommand` side-effect with the branch's `interactiveDefault` fallback in toCommandModule's handler, and kept both the branch's `PositionalOptions` import and main's `setEntryCommand` import. The 3-way merge correctly preserved the overhaul's command removals (integrate, events-audit, audit-3000 stay deleted) while adding main's new `slackCommand` to the rebuilt bin.ts .use() chain. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 16, 2026
Consolidates the repo guidance into AGENTS.md (the cross-tool standard so all agents read it) and reduces CLAUDE.md to an `@AGENTS.md` import. Avoids drift between two overlapping instruction files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Context-mill renamed the cli: block's `recommended` field to `default`. Update CliEntry and the family dispatcher to read `entry.default`, and align the doc comments. The wizard's Command.default field name is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bare `wizard audit` errored ("requires a subcommand") because the family
handler always ran dispatchFamily, which rejects an empty subcommand — the
interactiveDefault picker was never reached. Route the no-subcommand case to
the picker in an interactive terminal; keep the error in non-TTY/CI so we don't
hang on an Ink picker that can't render.
The picker shows only the default leaf (`audit events`) for now via the new
`pickerChildrenToShow` helper; the other audit subcommands stay runnable
directly (`wizard audit <name>`). Adds a test locking in that behavior.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
List every audit subcommand in README/AGENTS.md, add a "run a single skill" section, and explain that `wizard audit <subcommand>` chooses an audit area — it does not take a skill name (subcommands are skills promoted to commands; `wizard skill <name>` runs un-promoted ones). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…picker When a family surfaces a single option (today `audit` → `events`), run it directly so the user lands on its intro screen instead of a one-item picker. The picker still opens once a family surfaces more than one option, and non-TTY still errors "requires a subcommand". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Match the README to the actual behavior: bare `wizard audit` runs the events audit directly today, rather than opening an interactive picker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gewenyu99
approved these changes
Jun 17, 2026
gewenyu99
left a comment
gewenyu99
left a comment
Collaborator
There was a problem hiding this comment.
Some stuff to look at
| user:read,project:read,llm_gateway:read,dashboard:read,dashboard:write,insight:read,insight:write,query:read,notebook:read,notebook:write,health_issue:read,wizard_session:read,wizard_session:write,feature_flag:read,experiment:read,experiment_saved_metric:read,survey:read,session_recording:read,error_tracking:read,web_analytics:read,llm_analytics:read,cohort:read,person:read,annotation:read,annotation:write,activity_log:read,property_definition:read,event_definition:read,action:read,warehouse_table:read,warehouse_view:read,alert:read,subscription:read,feature_flag:write,integration:read,organization:read | ||
| ``` | ||
|
|
||
| # Command changes (CLI overhaul) |
Collaborator
There was a problem hiding this comment.
changes? or should we position this as just the way it works now?
|
|
||
| Wizard.use(basicIntegrationCommand) | ||
| .use(mcpCommand) | ||
| .use(integrateCommand) |
Collaborator
There was a problem hiding this comment.
What did this do btw I don't recall
| }, | ||
| // ── Internal modes ───────────────────────────────────────────────── | ||
| // Hidden from `--help`. See CONTRIBUTING.md for what each one does. | ||
| 'local-mcp': { |
Collaborator
There was a problem hiding this comment.
Make sure these are disabled in prod.
| */ | ||
| export interface ProgramCliSurface { | ||
| /** Where the program appears in the wizard CLI surface. */ | ||
| role: 'command' | 'skill' | 'internal'; |
Collaborator
There was a problem hiding this comment.
Ohhh btw, (not for this PR) can we standardize to variants such that
revenue skill maps to general command
revenue -> audit variant maps to npx wizard revenue audit?
No idea if that works but saying it in case it sparks
| steps: AGENT_SKILL_STEPS, | ||
| getContentBlocks: agentSkillContentBlocks, | ||
| allowedTools: ['Agent'], | ||
| run: (session) => { |
Collaborator
There was a problem hiding this comment.
This returns a promise. Are you sure you want run to return a promise?
It looks like you can just return
| * without touching the registry. Adding a native is a wizard PR. | ||
| */ | ||
|
|
||
| /** Wizard-native subcommands keyed by family. */ |
Collaborator
There was a problem hiding this comment.
This does feel like a bit of a hack vs. just registering them statically. 🤔 I'm not. entirely sure what these native handlers are
| * content blocks, screens). | ||
| */ | ||
| function configForCliEntry(entry: CliEntry): ProgramConfig { | ||
| if (entry.skillId === 'audit') return auditConfig; |
Collaborator
There was a problem hiding this comment.
🤔 Again if Audit has special screens tied to it, should it be statically defined?
| // case to the picker before this runs, so don't suggest opening it here. | ||
| process.stderr.write( | ||
| `\n\x1b[1;91m✖ \`wizard ${family}\` requires a subcommand.\x1b[0m\n` + | ||
| ` Pass one (e.g. \`wizard ${family} <subcommand>\`), or run it in an interactive terminal to pick from a menu.\n\n`, |
Collaborator
There was a problem hiding this comment.
Do we/should we list the valid ones?
Collaborator
There was a problem hiding this comment.
This file should probably have some test coverage just for developer/agent sanity moving forward to know what things were supposed to do. Lots of algo type stuff deserves tests to just preserve behavior for the next editor
gewenyu99
reviewed
Jun 17, 2026
| const children = buildFamilyPickerChildren(family, menu?.cliEntries ?? []); | ||
| const toShow = pickerChildrenToShow(children); | ||
| if (toShow.length === 1 && toShow[0]?.handler) { | ||
| await Promise.resolve(toShow[0].handler(argv)); |
Collaborator
There was a problem hiding this comment.
Wait this is weird too, you're resolving and awaiting on the same line
gewenyu99
reviewed
Jun 17, 2026
| // path — they only run when the leaf is invoked directly | ||
| // (`wizard audit events`). Harmless while leaves declare neither, but if a | ||
| // leaf ever grows a `check` or a defaulted option, this path will skip it. | ||
| await Promise.resolve(chosen.handler?.(argv)); |
An unrecognized command (e.g. `wizard asdf`) fell through to the default `$0` integration flow, because the CLI only ran `.strictOptions()` (rejects unknown flags, not unknown commands). Add `.strictCommands()` so unknown commands error. Also slim the failure output: the `.fail()` handler dumped the entire usage screen under every error; now it prints the error plus a one-line pointer to `wizard --help`. Updates provision-cli.test.ts: the test's no-op process.exit let the handler run after a validation `.fail()` once `showHelp()` was gone, so the exit mock now throws during the synchronous parse (matching how the real process.exit halts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gewenyu99
reviewed
Jun 17, 2026
…izard into posthog-code/cli-overhaul-core
yargs handlers are synchronous, so async command bodies were kicked off as fire-and-forget `void` promises — a rejection became an unhandled promise rejection (no message, wrong exit code). Add a shared `runCommandHandler` that awaits the work and turns any error into a clean `✖` message + exit 1, and use it for the family command handler (covering dispatchFamily and the picker) and `wizard skill list`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…into posthog-code/cli-overhaul-core
edwinyjlim
approved these changes
Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CLI OVERHAULLLL
context-mill #178 publishes a
cli:block per skill and emits theresolved entries inside
skill-menu.json. this PR rewires the wizardto consume that at runtime instead of baking a snapshot at build time.
three things happening here:
audit-3000is retiredhow dispatch works now
every family (
wizard audit,wizard migrate,wizard revenue-analytics)is a yargs command with a
[skill]positional and a handler. the handleris
dispatchFamily, which:audit web-analytics). runs immediately if matched — no network.skill-menu.json, finds the entry whereparentCommand + commandmatch, and runs the resolved skill.wizard auditwith no positional opens the existing family picker,which fetches at picker-open time and shows native + live entries
together. the
recommendedleaf from context-mill is pre-highlighted.adding a new audit subcommand (e.g.
wizard audit web-vitals) is now acontext-mill release with a new skill +
cli:block. no wizardrelease needed. adding a new family (
wizard investigate) is still awizard PR. that's the explicit split.
audit-3000
dropped on the wizard side too (context-mill already dropped it). this
removes:
src/lib/programs/audit-3000/src/ui/tui/screens/audit-3000/program-registry,screen-registry,screen-sequencesagent-interface.tsaudit-3000 · area slidesdeck in the playgroundtesting
pnpm typecheckgreenpnpm jest(no e2e): 817/817 tests passing across 53 suitesprograms-cli.test.tsrewritten to mockfetchSkillMenuand assertdispatchFamilyrouting (skill-backed entry, native handler, thecomprehensive
audit allspecial case)family-picker.test.tsflipped to match the newreality (
auditCommand.childrenis undefined by design now)program-registry.test.tsupdated for therevenue-analyticscommand-name rename (was
revenue)local end-to-end check (pointing at a local context-mill build via
--local-mcp/LOCAL_SKILLS_BASE_URL=http://localhost:8765):wizard audit eventsresolvesaudit-eventsover the wirewizard audit web-analyticsruns the native doctor (no network)wizard auditopens the picker with native + live entrieswizard audit unknown-thingprints the available list andexits non-zero
wizard auditwith the registry unreachable prints a clearerror and exits non-zero