Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/agent-cli-bench-discoverability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
"clerk": minor
---

Improve agent-CLI discoverability and add headless authentication.

- `clerk --help` now renders a top-level `Examples:` block and an `Environment:` section listing the `CLERK_*` env vars the CLI reads (`CLERK_SECRET_KEY`, `CLERK_MODE`, `CLERK_CONFIG_DIR`, `CLERK_UPDATE_CHANNEL`, `CLERK_NO_UPDATE_CHECK`).
- `clerk auth login` accepts `--token <key>` for headless authentication with a Clerk PLAPI access token. Pass `-` to read the token from stdin. The token is validated against `/oauth/userinfo`, stored without a refresh token, and surfaces a clear `AUTH_REQUIRED` error when it expires.
- `--json` option descriptions on `clerk apps list|create`, `clerk users list|create`, and `clerk doctor` now document the field shape so consumers know what to expect.
14 changes: 14 additions & 0 deletions .changeset/agent-cli-bench-parseability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
"clerk": minor
---

Improve agent-CLI parseability, discoverability, and recoverability per agentcli-bench rubric.

- **Global flags**: `--quiet` silences non-essential output (mirrors existing `--verbose`); `--no-color` disables ANSI sequences (complements `NO_COLOR` env). Both appear in `clerk --help`.
- **Exit codes** now align with BSD sysexits so agents can branch on the code alone: `EX_USAGE=64` (bad flag/subcommand/missing arg), `EX_NOPERM=77` (auth), `EX_TEMPFAIL=75` and `EX_UNAVAILABLE=69` (transient/upstream), `EX_DATAERR=65`, `EX_SOFTWARE=70`. Commander's `unknownOption` / `unknownCommand` / `missingArgument` errors now exit `64` instead of `1`.
- **Structured JSON errors** now include `retryable: boolean`, `nextStep: string`, and `docsUrl?: string`. 5xx and network failures (ECONNREFUSED/RESET/ETIMEDOUT/EAI_AGAIN/'fetch failed') are flagged retryable so agents can implement a single retry loop. The bad-flag JSON envelope points at `clerk --help`.
- **`clerk schema`**: new top-level subcommand that emits the full command tree (`{cli, version, schemaVersion, command}`) as JSON. Agents can walk every subcommand, argument, and option (with choices and defaults) without parsing `--help` text.
- **`clerk whoami --json`**: returns `{authenticated, user, linked, app, appName}`. Unauthenticated state is a value (`authenticated:false`), not a thrown error.
- **`clerk users list --json`** now includes `nextCursor` (offset-encoded) and a `pagination` envelope alongside the existing `data` and `hasMore` fields.
- **`clerk apps create --if-not-exists`**: idempotent flag that looks up an existing app by name and returns it (with `reused:true` in JSON) instead of creating a duplicate.
- **Top-level `--help`** gains a `Next:` block (`auth login`, `init`, `doctor`) and a `Documentation:` block linking to https://clerk.com/docs/cli and https://github.com/clerk/cli.
35 changes: 35 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ Options:
--mode <mode> Force interaction mode (human or agent). Defaults to
auto-detect based on TTY.
--verbose Show detailed output (enables debug messages)
--quiet Suppress non-essential output (info, warnings, spinners)
--no-color Disable ANSI color output (also respects the NO_COLOR env
var)
-h, --help Display help for command

Commands:
Expand All @@ -47,8 +50,40 @@ Commands:
disable Disable Clerk features on the linked instance
api [options] [endpoint] [filter] Make authenticated requests to the Clerk API
doctor [options] Check your project's Clerk integration health
schema [options] Print the full CLI command tree as JSON (for agents and tooling)
completion [shell] Generate shell autocompletion script
update [options] Update the Clerk CLI to the latest version
deploy Deploy a Clerk application to production
help [command] Display help for command

Examples:
$ clerk init Initialize Clerk in this project
$ clerk auth login Authenticate via browser OAuth
$ clerk apps list --json List applications as JSON (agent-pipeable)
$ clerk users list --json | jq '.data' Pipe user list to jq
$ clerk --mode agent api /users Force agent mode for non-interactive use

Environment:
CLERK_SECRET_KEY Backend API secret key for the linked instance
(sk_test_… / sk_live_…)
CLERK_MODE Force interaction mode: human or agent (default: TTY
auto-detect)
CLERK_CONFIG_DIR Override the directory for stored credentials and
config
CLERK_UPDATE_CHANNEL Release channel for `clerk update` (e.g. latest,
canary)
CLERK_NO_UPDATE_CHECK Set to any value to disable the post-command update
notification

Next:
$ clerk auth login Authenticate (or set CLERK_SECRET_KEY for headless use)
$ clerk init Set up Clerk in this project
$ clerk doctor Check that everything is wired up

Documentation:
https://clerk.com/docs/cli
https://github.com/clerk/cli

Give AI agents better Clerk context: install the Clerk skills
$ clerk skill install
```
66 changes: 66 additions & 0 deletions packages/cli-core/src/cli-program.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,72 @@ test("users create documents -d and --file for raw BAPI request bodies", () => {
expect(help).toContain("--file");
});

describe("agent-CLI discoverability surface", () => {
test("top-level --help renders Examples: with at least one agent-pipeable command", () => {
const help = createProgram().helpInformation();
expect(help).toContain("Examples:");
expect(help).toMatch(/clerk apps list --json/);
});

test("top-level --help renders Environment: with CLERK_* env vars actually read in cli-core", () => {
const help = createProgram().helpInformation();
expect(help).toContain("Environment:");
expect(help).toContain("CLERK_SECRET_KEY");
expect(help).toContain("CLERK_MODE");
});

test("data-returning subcommands document --json field shape", () => {
const program = createProgram();
const usersList = program.commands
.find((c) => c.name() === "users")!
.commands.find((c) => c.name() === "list")!;
const appsList = program.commands
.find((c) => c.name() === "apps")!
.commands.find((c) => c.name() === "list")!;

expect(usersList.helpInformation()).toMatch(/--json[^\n]*\bdata\b[^\n]*\bhasMore\b/);
expect(appsList.helpInformation()).toMatch(/--json[^\n]*\bapplication_id\b/);
});

test("auth login --help documents the headless path via --token and CLERK_SECRET_KEY", () => {
const program = createProgram();
const auth = program.commands.find((c) => c.name() === "auth")!;
const login = auth.commands.find((c) => c.name() === "login")!;
const help = login.helpInformation();

const optionNames = login.options.map((o) => o.long);
expect(optionNames).toContain("--token");
expect(help).toContain("CLERK_SECRET_KEY");
expect(help).toMatch(/headless/i);
});

// Names listed in `Environment:` must match what the CLI reads via
// process.env.CLERK_* — otherwise the help text drifts and lies.
const documentedEnvVars = [
...createProgram()
.helpInformation()
.matchAll(/\bCLERK_[A-Z0-9_]+\b/g),
].map((m) => m[0]);
const knownReadByCli = new Set([
"CLERK_SECRET_KEY",
"CLERK_MODE",
"CLERK_CONFIG_DIR",
"CLERK_UPDATE_CHANNEL",
"CLERK_NO_UPDATE_CHECK",
]);

test("setEnvVars documents at least one CLERK_* name", () => {
expect(documentedEnvVars.length).toBeGreaterThan(0);
});

test.each([...new Set(documentedEnvVars)])(
"documented env var %s is actually read by the CLI",
(name) => {
expect(knownReadByCli).toContain(name);
},
);
});

describe("formatApiBody", () => {
// --- Single error with meta ---

Expand Down
Loading