feat(cli): introduce structured CliError with typed error codes and Sentry routing#14749
feat(cli): introduce structured CliError with typed error codes and Sentry routing#14749
Conversation
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
🌱 Seed Test SelectorSelect languages to run seed tests for:
How to use: Click the ⋯ menu above → "Edit" → check the boxes you want → click "Update comment". Tests will run automatically and snapshots will be committed to this PR. |
a75a4d9 to
0be946b
Compare
fbec724 to
b016b76
Compare
0be946b to
aa3f135
Compare
c82a0cd to
dcb9b87
Compare
5542ab4 to
1008990
Compare
81ea91e to
1ddf93c
Compare
| errorCode: extractErrorCode(error) | ||
| }); | ||
| reportError(context, error); | ||
| await context.telemetry.flush(); |
| this.instrumentPostHogEvent({ | ||
| command: process.argv.join(" "), | ||
| properties: { | ||
| failed: true, | ||
| source: "cli", | ||
| error, | ||
| errorCode: code | ||
| } | ||
| }); |
There was a problem hiding this comment.
This supersedes the handling that was previously in packages/cli/cli/src/cli.ts, right?
There was a problem hiding this comment.
Exactly, I moved it here to have a single handling place. The behavior should be the same as before, but I'll check with some testing.
There was a problem hiding this comment.
To make it simple, it's now resolveErrorCode responsible of categorizing the error and understand what we should do with it.
| this.instrumentPostHogEventImpl({ | ||
| command: process.argv.join(" "), | ||
| properties: { | ||
| failed: true, | ||
| source: "task", | ||
| error, | ||
| errorCode: code | ||
| } | ||
| }); |
There was a problem hiding this comment.
Do we need to capture task events separately? We have the same command name, so I don't know how valuable this would be. It looks like we didn't have this before, right?
There was a problem hiding this comment.
I'm sending PostHog events also here because the error that will surface to the CLI main catch clause will be a generic TaskFailureSignal that is not being tracked anymore in PostHog. The behavior will change a little compared to the current, but we'll have better error categorization thanks to this change (before we would capture generic TaskFailureSignal in most of the cases)
There was a problem hiding this comment.
Let me know if you agree on the change or you want this to be consistent with the past
1008990 to
a300fbd
Compare
2dd5ea8 to
cc732f3
Compare
| @@ -0,0 +1,111 @@ | |||
| export const CliErrorCode = { | |||
There was a problem hiding this comment.
one thing we often do is create a namespace with the same name as the class, then exported types from it.
So you could do
import { CliError } from "...";
const code = CliError.Code.ConfigError;Example from Claude on how to do this.
export class CliError extends Error {
public readonly code: CliError.Code;
public constructor(message: string, code: CliError.Code) {
super(message);
this.name = "CliError";
this.code = code;
}
}
export namespace CliError {
export type Code = (typeof Code)[keyof typeof Code];
export const Code = {
ConfigError: "ConfigError",
NetworkError: "NetworkError",
AuthError: "AuthError",
} as const;
}cc732f3 to
9db8d9a
Compare
| import { addPrefixToString } from "@fern-api/core-utils"; | ||
| import { createLogger, LogLevel } from "@fern-api/logger"; | ||
| import { | ||
| type CliErrorCode, |
There was a problem hiding this comment.
Type CliErrorCode does not exist in the exported API. The correct type is CliError.Code.
This will cause a TypeScript compilation error because @fern-api/task-context exports CliError with a nested Code type, not a separate CliErrorCode type.
Fix:
import { type CliError } from "@fern-api/task-context";Then update the parameter types on lines 82 and 88 from:
_options?: { code?: CliErrorCode }to:
_options?: { code?: CliError.Code }| type CliErrorCode, | |
| type CliError, |
Spotted by Graphite
Is this helpful? React 👍 or 👎 to let us know.
a300fbd to
991545e
Compare
963bccf to
2014468
Compare
991545e to
1f3d977
Compare
…entry routing Introduce CliError with 12-code union (INTERNAL_ERROR, PARSE_ERROR, CONFIG_ERROR, etc.) in @fern-api/task-context. Add captureException to TaskContext interface. Wire Sentry reporting into failWithoutThrowing at point of failure. Update top-level handlers in both CLI v1 and v2 to classify untyped errors as INTERNAL_ERROR. Made-with: Cursor
…xt CliError Delete the local cli-v2 CliError class and switch all 31 importing files to use the shared CliError from @fern-api/task-context. Remap v2 codes (AUTH_REQUIRED→AUTH_ERROR, EXIT→TaskAbortSignal, etc.), replace ~35 CliError.exit() calls with TaskAbortSignal, and classify ~62 untyped new CliError sites with appropriate error codes. Made-with: Cursor
Made-with: Cursor
…ingUnavailableError extend CliError Each subclass now carries a default error code (VALIDATION_ERROR / AUTH_ERROR), eliminating the need for the separate extractErrorCode and local shouldReportToSentry functions in withContext.ts. reportError now relies entirely on resolveErrorCode and the shared shouldReportToSentry from @fern-api/task-context. Made-with: Cursor
Adds the error code as an `error.code` tag to Sentry exceptions, matching the v1 SentryClient behavior. The code is resolved via resolveErrorCode in both reportError and TaskContextAdapter. Made-with: Cursor
Replace all LoggableFernCliError throw sites with CliError.internalError() and remove the dedicated catch branches since CliError is already handled upstream. Deletes the now-unused LoggableFernCliError class. Made-with: Cursor
e3ded50 to
fdf7661
Compare
SDK Generation Benchmark ResultsComparing PR branch against latest nightly baseline on Full benchmark table (click to expand)
main (generator): generator-only time via --skip-scripts (includes Docker image build, container startup, IR parsing, and code generation — this is the same Docker-based flow customers use via |
| context.telemetry.sendLifecycleEvent({ | ||
| status: "error", | ||
| command: context.info.command, | ||
| durationMs: Date.now() - context.createdAt, | ||
| errorCode: code | ||
| }); |
There was a problem hiding this comment.
🟡 Double lifecycle events in cli-v2 when TaskContextAdapter.failWithoutThrowing is called without a subsequent throw
In cli-v2, TaskContextAdapter.failWithoutThrowing at line 73 now calls reportError, which sends a PostHog lifecycle event with status: "error". If the command handler doesn't throw after this call (e.g., a legacy function calls failWithoutThrowing and then returns normally), execution continues to the success path in withContext.ts:33-36, which sends a second lifecycle event with status: "success". This results in contradictory duplicate lifecycle events for a single command invocation — one error and one success — polluting analytics data.
Prompt for agents
The issue is that reportError (called from TaskContextAdapter.failWithoutThrowing at line 73 of TaskContextAdapter.ts) sends a lifecycle event unconditionally, and then the withContext.ts success/error path sends another one. The lifecycle event should only be sent once per command invocation.
Possible approaches:
1. Track whether a lifecycle event has already been sent on the Context object (e.g. a boolean flag), and skip the duplicate in reportError or withContext.ts.
2. Only send the lifecycle event from withContext.ts (top-level), not from reportError. The reportError function would only handle Sentry reporting. This requires splitting the current reportError into Sentry-only and lifecycle+Sentry variants.
3. Have the reportError called from failWithoutThrowing only send Sentry (not lifecycle events), and leave lifecycle event sending exclusively to the withContext.ts catch/success blocks.
Was this helpful? React with 👍 or 👎 to provide feedback.
…ors (#14750) ## Description Part of the CLI error classification effort — see #14749 for full context. Temporarily suppresses Sentry reporting for errors that haven't been classified yet. This is a safety net so that merging #14749 doesn't flood Sentry with noise while the ~35 follow-up package migration PRs are landing. ## Changes Made - Added `UNCLASSIFIED` to `CliError.Code` - Default error code fallback changed from `INTERNAL_ERROR` to `UNCLASSIFIED` in: - `cli.ts` (CLI v1 top-level catch) - `CliContext.ts` (CLI v1 task runner) - `withContext.ts` → `reportError` (CLI v2) - `UNCLASSIFIED` is **not** in `SENTRY_REPORTABLE_CODES`, so these errors are silently tracked in PostHog but not sent to Sentry ## Rollback Once all package migration PRs have landed, #14752 will remove `UNCLASSIFIED` and revert the fallback to `INTERNAL_ERROR`. ## Testing - [x] Existing tests pass (only changes default string literals)
## Description Migrates `@fern-api/cli` (the main CLI package) to use explicit `CliError` error codes on every `failAndThrow` / `failWithoutThrowing` call site. This is one of ~35 package migration PRs that follow the error classification system introduced in #14749. ## Changes Made Assigned typed error codes across **42 files** in `packages/cli/cli/src/`, covering every `failAndThrow` and `failWithoutThrowing` invocation (excluding generic `catch` blocks where `resolveErrorCode` auto-extracts codes from `CliError` instances). ### Error codes used | Code | Usage | |------|-------| | `CONFIG_ERROR` | Missing config, invalid flags, bad file paths, missing `generators.yml`, unknown group/generator names, missing `docs.yml` | | `AUTH_ERROR` | Authentication failures (`fern login` required, token creation) | | `NETWORK_ERROR` | HTTP/API call failures, translation service errors, generation task polling failures | | `VALIDATION_ERROR` | Schema validation failures, broken-link checks, workspace validation | | `PARSE_ERROR` | Malformed YAML/JSON, invalid IR files, export schema issues | | `VERSION_ERROR` | Invalid semver, version already registered, generator version incompatibility | | `ENVIRONMENT_ERROR` | Missing environment prerequisites, `self-update` failures, dependency installation | | `INTERNAL_ERROR` | Unexpected states that indicate bugs, failed reruns, missing IR fields | ### Files touched (grouped by area) - **Top-level CLI**: `cli.ts`, `cliV2.ts`, `rerunFernCliAtVersion.ts`, `resolveGroupGithubConfig.ts` - **Generate commands**: `generateAPIWorkspace.ts`, `generateAPIWorkspaces.ts`, `generateDocsWorkspace.ts` - **Docs commands**: `docsDiff.ts`, `generateLibraryDocs.ts`, `listDocsPreview.ts`, `deleteDocsPreview.ts` - **SDK commands**: `sdkDiffCommand.ts`, `sdkPreview.ts` - **Diff**: `diff.ts` - **Validation**: `validateWorkspaces.ts`, `validateDocsBrokenLinks.ts`, `validateDocsWorkspaceAndLogIssues.ts` - **Upgrade/Downgrade**: `upgrade.ts`, `upgradeGenerator.ts`, `downgrade.ts` - **Self-update**: `selfUpdate.ts` - **Export**: `servicesConverter.ts`, `typeConverter.ts`, `security.ts` - **Other**: `token.ts`, `testOutput.ts`, `mockServer.ts`, `mergeOpenAPIWithOverrides.ts`, `compareOpenAPISpecs.ts`, `writeOverridesForWorkspaces.ts`, `getGeneratorList.ts`, `getOrganization.ts`, `installDependencies.ts`, `registerWorkspacesV1.ts` ## Testing - [x] Existing tests pass
Description
Introduces a structured
CliErrorclass in@fern-api/task-contextwith typed error codes and automatic Sentry routing. This replaces the ad-hocnew Error(...)pattern throughout the CLI with a system that categorizes errors into user-facing codes (e.g.CONFIG_ERROR,AUTH_ERROR) vs internal errors that should be reported to Sentry. It also simplifies error handling by removing redundant error classes and unifying the ones we already have.Prerequisite PRs
FernCliErrortoTaskAbortSignal(clears the naming space)failWithoutThrowingcan report errors)Follow-up PRs
@fern-api/clipackage (example of a package migration PR)new Error(...)→new CliError(...)and adding error codes tofailAndThrowandfailWithoutThrowingcalls)Design Decisions
Two ways to trigger and track errors
There are two paths through which errors are captured and reported:
Throwing
CliErrordirectly. Code anywhere in the CLI canthrow new CliError({ message, code }). The top-level catch handler in each CLI entry point (CLI v1'srunClicatch incli.ts, CLI v2'swithContextcatch inwithContext.ts) intercepts it, resolves the error code, and routes it to Sentry and/or PostHog.Calling
failAndThrow/failWithoutThrowingon the task context. These methods log the error, resolve the code (explicit override >CliError.code> fallback), and report to Sentry if the code is Sentry-reportable.failAndThrowthen throws aTaskAbortSignalto unwind the stack;failWithoutThrowingmarks the task as failed and returns. This is the preferred path when the caller needs control over what happens after the error — for example, to continue processing other tasks or to clean up resources before aborting. It also adapts naturally to the existing error-handling patterns already used throughout the codebase.Both paths converge on the same code-resolution logic (
resolveErrorCode) and Sentry-routing rules (shouldReportToSentry), so tracking is consistent regardless of which path is used.Default behavior: unclassified errors are internal
Any error that doesn't carry a
CliError.Code(i.e. a plainnew Error(...)) is treated asINTERNAL_ERRORat the top-level catch boundaries — and therefore reported to Sentry. The assumption is that if nobody explicitly categorized an error as user-facing, it's likely an internal bug.Temporary exception: the follow-up PR #14750 temporarily downgrades this so unclassified errors skip Sentry during the migration period, to avoid noise from the ~34 packages that haven't been migrated yet. Once all migrations land, #14752 re-enables it.
Error code taxonomy
Errors are classified into 12 typed codes:
INTERNAL_ERRORRESOLUTION_ERRORIR_CONVERSION_ERRORCONTAINER_ERRORVERSION_ERRORPARSE_ERRORENVIRONMENT_ERRORREFERENCE_ERRORVALIDATION_ERRORNETWORK_ERRORAUTH_ERRORCONFIG_ERROROnly the first 5 codes are Sentry-reportable — they indicate bugs in Fern itself. The rest are user-actionable and would just create noise in Sentry.
Simplifying error handling by removing redundant classes
Unified
CliErroracross CLI v1 and v2. CLI v2 had its ownCliErrorclass inpackages/cli/cli-v2/src/errors/CliError.ts. This PR deletes it and makes both CLIs share the singleCliErrorfrom@fern-api/task-context, ensuring consistent error codes and Sentry routing regardless of which CLI entry point is used.Existing error classes now extend
CliError.ValidationError,SourcedValidationError, andKeyringUnavailableErrornow extendCliErrorwith their appropriate codes (VALIDATION_ERRORandAUTH_ERROR). This meansresolveErrorCode()handles them automatically — no special-casing needed in every catch block.Removed
LoggableFernCliError. This was a wrapper that carried a log message alongside an error. WithCliErrornow carrying amessagefield (it extendsError), the wrapper is redundant. All formerLoggableFernCliErrorusages are replaced withCliError.reportError— single error reporting path (CLI v2)CLI v2's
withContext.tspreviously had separateshouldReportToSentryandextractErrorCodefunctions. These are consolidated into a singlereportError(context, error, options?)function that:TaskAbortSignal(already logged)resolveErrorCode()(explicit override >CliError.code> fallback toINTERNAL_ERROR)SENTRY_REPORTABLE_CODESSentry tags
Error codes are now passed as Sentry tags (
errorCode) on captured exceptions, making it possible to filter and alert on specific error categories in the Sentry dashboard.Commits (review in order)
add shared CliError class— introducesCliErrorin@fern-api/task-contextwith the code taxonomy,shouldReportToSentry,resolveErrorCode, and static factory methodsunify CLI v2 CliError— deletes CLI v2's localCliError, switches to shared one, introducesreportErrorinwithContext.tsclean up imports— mechanical import reordering for consistencymake error classes extend CliError—ValidationError,SourcedValidationError,KeyringUnavailableErrornow extendCliErrorpass error code as Sentry tag— addserrorCodetag tocaptureExceptioncallsremove LoggableFernCliError— replaces all usages withCliError, deletes the classTesting
CliError/MockTaskContextsignatures