Skip to content

feat(cli): introduce structured CliError with typed error codes and Sentry routing#14749

Merged
FedeZara merged 8 commits intomainfrom
FedeZara/refactor/new-cli-error-system
Apr 14, 2026
Merged

feat(cli): introduce structured CliError with typed error codes and Sentry routing#14749
FedeZara merged 8 commits intomainfrom
FedeZara/refactor/new-cli-error-system

Conversation

@FedeZara
Copy link
Copy Markdown
Contributor

@FedeZara FedeZara commented Apr 8, 2026

Description

Introduces a structured CliError class in @fern-api/task-context with typed error codes and automatic Sentry routing. This replaces the ad-hoc new Error(...) pattern throughout the CLI with a system that categorizes errors into user-facing codes (e.g. CONFIG_ERROR, AUTH_ERROR) vs internal errors that should be reported to Sentry. It also simplifies error handling by removing redundant error classes and unifying the ones we already have.

Recommended review approach: commit-by-commit. Each commit is self-contained and builds on the previous one.

Prerequisite PRs

Follow-up PRs

Design Decisions

Two ways to trigger and track errors

There are two paths through which errors are captured and reported:

  1. Throwing CliError directly. Code anywhere in the CLI can throw new CliError({ message, code }). The top-level catch handler in each CLI entry point (CLI v1's runCli catch in cli.ts, CLI v2's withContext catch in withContext.ts) intercepts it, resolves the error code, and routes it to Sentry and/or PostHog.

  2. Calling failAndThrow / failWithoutThrowing on the task context. These methods log the error, resolve the code (explicit override > CliError.code > fallback), and report to Sentry if the code is Sentry-reportable. failAndThrow then throws a TaskAbortSignal to unwind the stack; failWithoutThrowing marks the task as failed and returns. This is the preferred path when the caller needs control over what happens after the error — for example, to continue processing other tasks or to clean up resources before aborting. It also adapts naturally to the existing error-handling patterns already used throughout the codebase.

Both paths converge on the same code-resolution logic (resolveErrorCode) and Sentry-routing rules (shouldReportToSentry), so tracking is consistent regardless of which path is used.

Default behavior: unclassified errors are internal

Any error that doesn't carry a CliError.Code (i.e. a plain new Error(...)) is treated as INTERNAL_ERROR at the top-level catch boundaries — and therefore reported to Sentry. The assumption is that if nobody explicitly categorized an error as user-facing, it's likely an internal bug.

Temporary exception: the follow-up PR #14750 temporarily downgrades this so unclassified errors skip Sentry during the migration period, to avoid noise from the ~34 packages that haven't been migrated yet. Once all migrations land, #14752 re-enables it.

Error code taxonomy

Errors are classified into 12 typed codes:

Code Sentry? Description
INTERNAL_ERROR Yes Unexpected bugs — should be investigated
RESOLUTION_ERROR Yes Type/reference resolution failures (likely IR bugs)
IR_CONVERSION_ERROR Yes IR generation failures
CONTAINER_ERROR Yes Docker container failures
VERSION_ERROR Yes Version parsing/compatibility issues
PARSE_ERROR No Malformed user input (YAML, OpenAPI, etc.)
ENVIRONMENT_ERROR No Missing env vars, wrong Node version, etc.
REFERENCE_ERROR No Dangling references in user config
VALIDATION_ERROR No Schema/rule validation failures
NETWORK_ERROR No HTTP failures, timeouts
AUTH_ERROR No Authentication/authorization issues
CONFIG_ERROR No Invalid generators.yml, fern.config.json, etc.

Only the first 5 codes are Sentry-reportable — they indicate bugs in Fern itself. The rest are user-actionable and would just create noise in Sentry.

Simplifying error handling by removing redundant classes

  • Unified CliError across CLI v1 and v2. CLI v2 had its own CliError class in packages/cli/cli-v2/src/errors/CliError.ts. This PR deletes it and makes both CLIs share the single CliError from @fern-api/task-context, ensuring consistent error codes and Sentry routing regardless of which CLI entry point is used.

  • Existing error classes now extend CliError. ValidationError, SourcedValidationError, and KeyringUnavailableError now extend CliError with their appropriate codes (VALIDATION_ERROR and AUTH_ERROR). This means resolveErrorCode() handles them automatically — no special-casing needed in every catch block.

  • Removed LoggableFernCliError. This was a wrapper that carried a log message alongside an error. With CliError now carrying a message field (it extends Error), the wrapper is redundant. All former LoggableFernCliError usages are replaced with CliError.

reportError — single error reporting path (CLI v2)

CLI v2's withContext.ts previously had separate shouldReportToSentry and extractErrorCode functions. These are consolidated into a single reportError(context, error, options?) function that:

  1. Skips TaskAbortSignal (already logged)
  2. Resolves the error code via resolveErrorCode() (explicit override > CliError.code > fallback to INTERNAL_ERROR)
  3. Reports to Sentry if the code is in SENTRY_REPORTABLE_CODES
  4. Always reports to PostHog with the error code as a property

Sentry tags

Error codes are now passed as Sentry tags (errorCode) on captured exceptions, making it possible to filter and alert on specific error categories in the Sentry dashboard.

Commits (review in order)

  1. add shared CliError class — introduces CliError in @fern-api/task-context with the code taxonomy, shouldReportToSentry, resolveErrorCode, and static factory methods
  2. unify CLI v2 CliError — deletes CLI v2's local CliError, switches to shared one, introduces reportError in withContext.ts
  3. clean up imports — mechanical import reordering for consistency
  4. make error classes extend CliErrorValidationError, SourcedValidationError, KeyringUnavailableError now extend CliError
  5. pass error code as Sentry tag — adds errorCode tag to captureException calls
  6. remove LoggableFernCliError — replaces all usages with CliError, deletes the class

Testing

  • Updated test helpers and mocks for new CliError / MockTaskContext signatures
  • Existing CLI v1 and v2 tests pass

Open with Devin

@FedeZara FedeZara self-assigned this Apr 8, 2026
Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 8, 2026

🌱 Seed Test Selector

Select languages to run seed tests for:

  • Python
  • TypeScript
  • Java
  • Go
  • Ruby
  • C#
  • PHP
  • Swift
  • Rust
  • OpenAPI
  • Postman

How to use: Click the ⋯ menu above → "Edit" → check the boxes you want → click "Update comment". Tests will run automatically and snapshots will be committed to this PR.

@FedeZara FedeZara changed the title refactor(cli): introduce structured CliError with typed error codes and Sentry routing feat(cli): introduce structured CliError with typed error codes and Sentry routing Apr 8, 2026
@FedeZara FedeZara force-pushed the FedeZara/refactor/telemetry-client-async-factory branch from a75a4d9 to 0be946b Compare April 9, 2026 12:23
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch 2 times, most recently from fbec724 to b016b76 Compare April 9, 2026 12:24
@FedeZara FedeZara force-pushed the FedeZara/refactor/telemetry-client-async-factory branch from 0be946b to aa3f135 Compare April 9, 2026 13:55
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch 2 times, most recently from c82a0cd to dcb9b87 Compare April 9, 2026 14:54
@FedeZara FedeZara force-pushed the FedeZara/refactor/telemetry-client-async-factory branch 2 times, most recently from 5542ab4 to 1008990 Compare April 9, 2026 15:50
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch 2 times, most recently from 81ea91e to 1ddf93c Compare April 9, 2026 15:50
errorCode: extractErrorCode(error)
});
reportError(context, error);
await context.telemetry.flush();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 👍

Comment on lines +138 to +146
this.instrumentPostHogEvent({
command: process.argv.join(" "),
properties: {
failed: true,
source: "cli",
error,
errorCode: code
}
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This supersedes the handling that was previously in packages/cli/cli/src/cli.ts, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly, I moved it here to have a single handling place. The behavior should be the same as before, but I'll check with some testing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make it simple, it's now resolveErrorCode responsible of categorizing the error and understand what we should do with it.

Comment on lines +103 to +111
this.instrumentPostHogEventImpl({
command: process.argv.join(" "),
properties: {
failed: true,
source: "task",
error,
errorCode: code
}
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to capture task events separately? We have the same command name, so I don't know how valuable this would be. It looks like we didn't have this before, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sending PostHog events also here because the error that will surface to the CLI main catch clause will be a generic TaskFailureSignal that is not being tracked anymore in PostHog. The behavior will change a little compared to the current, but we'll have better error categorization thanks to this change (before we would capture generic TaskFailureSignal in most of the cases)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if you agree on the change or you want this to be consistent with the past

@FedeZara FedeZara force-pushed the FedeZara/refactor/telemetry-client-async-factory branch from 1008990 to a300fbd Compare April 9, 2026 22:38
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch 2 times, most recently from 2dd5ea8 to cc732f3 Compare April 10, 2026 12:18
@@ -0,0 +1,111 @@
export const CliErrorCode = {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one thing we often do is create a namespace with the same name as the class, then exported types from it.
So you could do

import { CliError } from "...";

const code = CliError.Code.ConfigError;

Example from Claude on how to do this.

export class CliError extends Error {
    public readonly code: CliError.Code;

    public constructor(message: string, code: CliError.Code) {
        super(message);
        this.name = "CliError";
        this.code = code;
    }
}

export namespace CliError {
    export type Code = (typeof Code)[keyof typeof Code];
    export const Code = {
        ConfigError: "ConfigError",
        NetworkError: "NetworkError",
        AuthError: "AuthError",
    } as const;
}

@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from cc732f3 to 9db8d9a Compare April 10, 2026 14:44
Comment thread packages/seed/src/TaskContextImpl.ts Outdated
import { addPrefixToString } from "@fern-api/core-utils";
import { createLogger, LogLevel } from "@fern-api/logger";
import {
type CliErrorCode,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type CliErrorCode does not exist in the exported API. The correct type is CliError.Code.

This will cause a TypeScript compilation error because @fern-api/task-context exports CliError with a nested Code type, not a separate CliErrorCode type.

Fix:

import { type CliError } from "@fern-api/task-context";

Then update the parameter types on lines 82 and 88 from:

_options?: { code?: CliErrorCode }

to:

_options?: { code?: CliError.Code }
Suggested change
type CliErrorCode,
type CliError,

Spotted by Graphite

Fix in Graphite


Is this helpful? React 👍 or 👎 to let us know.

@FedeZara FedeZara force-pushed the FedeZara/refactor/telemetry-client-async-factory branch from a300fbd to 991545e Compare April 10, 2026 16:07
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from 963bccf to 2014468 Compare April 10, 2026 16:09
@FedeZara FedeZara force-pushed the FedeZara/refactor/telemetry-client-async-factory branch from 991545e to 1f3d977 Compare April 10, 2026 16:11
…entry routing

Introduce CliError with 12-code union (INTERNAL_ERROR, PARSE_ERROR,
CONFIG_ERROR, etc.) in @fern-api/task-context. Add captureException to
TaskContext interface. Wire Sentry reporting into failWithoutThrowing
at point of failure. Update top-level handlers in both CLI v1 and v2
to classify untyped errors as INTERNAL_ERROR.

Made-with: Cursor
…xt CliError

Delete the local cli-v2 CliError class and switch all 31 importing
files to use the shared CliError from @fern-api/task-context. Remap
v2 codes (AUTH_REQUIRED→AUTH_ERROR, EXIT→TaskAbortSignal, etc.),
replace ~35 CliError.exit() calls with TaskAbortSignal, and classify
~62 untyped new CliError sites with appropriate error codes.

Made-with: Cursor
…ingUnavailableError extend CliError

Each subclass now carries a default error code (VALIDATION_ERROR / AUTH_ERROR),
eliminating the need for the separate extractErrorCode and local
shouldReportToSentry functions in withContext.ts. reportError now relies
entirely on resolveErrorCode and the shared shouldReportToSentry from
@fern-api/task-context.

Made-with: Cursor
Adds the error code as an `error.code` tag to Sentry exceptions,
matching the v1 SentryClient behavior. The code is resolved via
resolveErrorCode in both reportError and TaskContextAdapter.

Made-with: Cursor
Replace all LoggableFernCliError throw sites with CliError.internalError()
and remove the dedicated catch branches since CliError is already handled
upstream. Deletes the now-unused LoggableFernCliError class.

Made-with: Cursor
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from e3ded50 to fdf7661 Compare April 14, 2026 18:22
@github-actions
Copy link
Copy Markdown
Contributor

SDK Generation Benchmark Results

Comparing PR branch against latest nightly baseline on main (2026-04-14T04:55:49Z).

Full benchmark table (click to expand)
Generator Spec main (generator) main (E2E) PR (generator) Delta
csharp-sdk square 92s 127s 89s -3s (-3.3%)
go-sdk square 90s 137s 103s +13s (+14.4%)
java-sdk square 173s 187s 163s -10s (-5.8%)
php-sdk square 84s 122s 88s +4s (+4.8%)
python-sdk square 128s 163s 126s -2s (-1.6%)
ruby-sdk-v2 square 113s 156s 117s +4s (+3.5%)
rust-sdk square 96s 92s 94s -2s (-2.1%)
swift-sdk square 86s 129s 84s -2s (-2.3%)
ts-sdk square 101s 136s 102s +1s (+1.0%)

main (generator): generator-only time via --skip-scripts (includes Docker image build, container startup, IR parsing, and code generation — this is the same Docker-based flow customers use via fern generate). main (E2E): full customer-observable time including build/test scripts (nightly baseline, informational). Delta is computed against generator-only baseline.
⚠️ = generation exited with a non-zero exit code (timing may not reflect a successful run).
Baseline from nightly runs on main (latest: 2026-04-14T04:55:49Z). Trigger benchmark-baseline to refresh.
Last updated: 2026-04-14 18:33 UTC

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 17 additional findings in Devin Review.

Open in Devin Review

Comment on lines +126 to +131
context.telemetry.sendLifecycleEvent({
status: "error",
command: context.info.command,
durationMs: Date.now() - context.createdAt,
errorCode: code
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Double lifecycle events in cli-v2 when TaskContextAdapter.failWithoutThrowing is called without a subsequent throw

In cli-v2, TaskContextAdapter.failWithoutThrowing at line 73 now calls reportError, which sends a PostHog lifecycle event with status: "error". If the command handler doesn't throw after this call (e.g., a legacy function calls failWithoutThrowing and then returns normally), execution continues to the success path in withContext.ts:33-36, which sends a second lifecycle event with status: "success". This results in contradictory duplicate lifecycle events for a single command invocation — one error and one success — polluting analytics data.

Prompt for agents
The issue is that reportError (called from TaskContextAdapter.failWithoutThrowing at line 73 of TaskContextAdapter.ts) sends a lifecycle event unconditionally, and then the withContext.ts success/error path sends another one. The lifecycle event should only be sent once per command invocation.

Possible approaches:
1. Track whether a lifecycle event has already been sent on the Context object (e.g. a boolean flag), and skip the duplicate in reportError or withContext.ts.
2. Only send the lifecycle event from withContext.ts (top-level), not from reportError. The reportError function would only handle Sentry reporting. This requires splitting the current reportError into Sentry-only and lifecycle+Sentry variants.
3. Have the reportError called from failWithoutThrowing only send Sentry (not lifecycle events), and leave lifecycle event sending exclusively to the withContext.ts catch/success blocks.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@FedeZara FedeZara merged commit 429be56 into main Apr 14, 2026
269 checks passed
Copy link
Copy Markdown
Contributor Author

Merge activity

@FedeZara FedeZara deleted the FedeZara/refactor/new-cli-error-system branch April 14, 2026 18:41
FedeZara added a commit that referenced this pull request Apr 14, 2026
…ors (#14750)

## Description

Part of the CLI error classification effort — see #14749 for full context.

Temporarily suppresses Sentry reporting for errors that haven't been classified yet. This is a safety net so that merging #14749 doesn't flood Sentry with noise while the ~35 follow-up package migration PRs are landing.

## Changes Made

- Added `UNCLASSIFIED` to `CliError.Code`
- Default error code fallback changed from `INTERNAL_ERROR` to `UNCLASSIFIED` in:
  - `cli.ts` (CLI v1 top-level catch)
  - `CliContext.ts` (CLI v1 task runner)
  - `withContext.ts` → `reportError` (CLI v2)
- `UNCLASSIFIED` is **not** in `SENTRY_REPORTABLE_CODES`, so these errors are silently tracked in PostHog but not sent to Sentry

## Rollback

Once all package migration PRs have landed, #14752 will remove `UNCLASSIFIED` and revert the fallback to `INTERNAL_ERROR`.

## Testing

- [x] Existing tests pass (only changes default string literals)
FedeZara added a commit that referenced this pull request Apr 14, 2026
## Description

Migrates `@fern-api/cli` (the main CLI package) to use explicit `CliError` error codes on every `failAndThrow` / `failWithoutThrowing` call site.

This is one of ~35 package migration PRs that follow the error classification system introduced in #14749.

## Changes Made

Assigned typed error codes across **42 files** in `packages/cli/cli/src/`, covering every `failAndThrow` and `failWithoutThrowing` invocation (excluding generic `catch` blocks where `resolveErrorCode` auto-extracts codes from `CliError` instances).

### Error codes used

| Code | Usage |
|------|-------|
| `CONFIG_ERROR` | Missing config, invalid flags, bad file paths, missing `generators.yml`, unknown group/generator names, missing `docs.yml` |
| `AUTH_ERROR` | Authentication failures (`fern login` required, token creation) |
| `NETWORK_ERROR` | HTTP/API call failures, translation service errors, generation task polling failures |
| `VALIDATION_ERROR` | Schema validation failures, broken-link checks, workspace validation |
| `PARSE_ERROR` | Malformed YAML/JSON, invalid IR files, export schema issues |
| `VERSION_ERROR` | Invalid semver, version already registered, generator version incompatibility |
| `ENVIRONMENT_ERROR` | Missing environment prerequisites, `self-update` failures, dependency installation |
| `INTERNAL_ERROR` | Unexpected states that indicate bugs, failed reruns, missing IR fields |

### Files touched (grouped by area)

- **Top-level CLI**: `cli.ts`, `cliV2.ts`, `rerunFernCliAtVersion.ts`, `resolveGroupGithubConfig.ts`
- **Generate commands**: `generateAPIWorkspace.ts`, `generateAPIWorkspaces.ts`, `generateDocsWorkspace.ts`
- **Docs commands**: `docsDiff.ts`, `generateLibraryDocs.ts`, `listDocsPreview.ts`, `deleteDocsPreview.ts`
- **SDK commands**: `sdkDiffCommand.ts`, `sdkPreview.ts`
- **Diff**: `diff.ts`
- **Validation**: `validateWorkspaces.ts`, `validateDocsBrokenLinks.ts`, `validateDocsWorkspaceAndLogIssues.ts`
- **Upgrade/Downgrade**: `upgrade.ts`, `upgradeGenerator.ts`, `downgrade.ts`
- **Self-update**: `selfUpdate.ts`
- **Export**: `servicesConverter.ts`, `typeConverter.ts`, `security.ts`
- **Other**: `token.ts`, `testOutput.ts`, `mockServer.ts`, `mergeOpenAPIWithOverrides.ts`, `compareOpenAPISpecs.ts`, `writeOverridesForWorkspaces.ts`, `getGeneratorList.ts`, `getOrganization.ts`, `installDependencies.ts`, `registerWorkspacesV1.ts`

## Testing

- [x] Existing tests pass
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants