Skip to content

chore(cli): temporarily disable Sentry reporting for unclassified errors#14750

Merged
FedeZara merged 3 commits intomainfrom
FedeZara/refactor/cli-error-disable-uncategorized-sentry
Apr 14, 2026
Merged

chore(cli): temporarily disable Sentry reporting for unclassified errors#14750
FedeZara merged 3 commits intomainfrom
FedeZara/refactor/cli-error-disable-uncategorized-sentry

Conversation

@FedeZara
Copy link
Copy Markdown
Contributor

@FedeZara FedeZara commented Apr 8, 2026

Description

Part of the CLI error classification effort — see #14749 for full context.

Temporarily suppresses Sentry reporting for errors that haven't been classified yet. This is a safety net so that merging #14749 doesn't flood Sentry with noise while the ~35 follow-up package migration PRs are landing.

Changes Made

  • Added UNCLASSIFIED to CliError.Code
  • Default error code fallback changed from INTERNAL_ERROR to UNCLASSIFIED in:
    • cli.ts (CLI v1 top-level catch)
    • CliContext.ts (CLI v1 task runner)
    • withContext.tsreportError (CLI v2)
  • UNCLASSIFIED is not in SENTRY_REPORTABLE_CODES, so these errors are silently tracked in PostHog but not sent to Sentry

Rollback

Once all package migration PRs have landed, #14752 will remove UNCLASSIFIED and revert the fallback to INTERNAL_ERROR.

Testing

  • Existing tests pass (only changes default string literals)

@FedeZara FedeZara self-assigned this Apr 8, 2026
Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 8, 2026

🌱 Seed Test Selector

Select languages to run seed tests for:

  • Python
  • TypeScript
  • Java
  • Go
  • Ruby
  • C#
  • PHP
  • Swift
  • Rust
  • OpenAPI
  • Postman

How to use: Click the ⋯ menu above → "Edit" → check the boxes you want → click "Update comment". Tests will run automatically and snapshots will be committed to this PR.

@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch 3 times, most recently from b016b76 to c82a0cd Compare April 9, 2026 13:56
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from 43e3daf to fd89650 Compare April 9, 2026 14:00
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch 3 times, most recently from 81ea91e to 1ddf93c Compare April 9, 2026 15:50
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from fd89650 to 6e6d684 Compare April 9, 2026 22:38
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from 1ddf93c to 2dd5ea8 Compare April 9, 2026 22:38
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch 2 times, most recently from ab973cc to fd585dd Compare April 10, 2026 12:18
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from 2dd5ea8 to cc732f3 Compare April 10, 2026 12:18
return "ENVIRONMENT_ERROR";
}
return "INTERNAL_ERROR";
return "UNCLASSIFIED";
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use your code map

@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from cc732f3 to 9db8d9a Compare April 10, 2026 14:44
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from fd585dd to 630e15e Compare April 10, 2026 14:44
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch 2 times, most recently from 2014468 to c0609d1 Compare April 10, 2026 16:11
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from 630e15e to 68c2b60 Compare April 10, 2026 16:19
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from c0609d1 to 2ecadc9 Compare April 10, 2026 16:20
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from 68c2b60 to 0549359 Compare April 10, 2026 16:20
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from 2ecadc9 to ee51994 Compare April 10, 2026 17:09
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from 0549359 to 813f087 Compare April 10, 2026 17:09
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from 813f087 to 21e744b Compare April 10, 2026 17:43
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from ee51994 to dbf93cf Compare April 10, 2026 18:33
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from 21e744b to b9b9c99 Compare April 10, 2026 18:33
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from dbf93cf to 8b3451a Compare April 10, 2026 18:42
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from b9b9c99 to a67cf84 Compare April 10, 2026 18:42
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from 8b3451a to 86d838d Compare April 10, 2026 18:58
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch 2 times, most recently from 0b71fc4 to dc33e43 Compare April 10, 2026 19:42
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch 2 times, most recently from 82915fb to ff37cd7 Compare April 10, 2026 21:55
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from dc33e43 to 7a7c757 Compare April 10, 2026 21:55
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from ff37cd7 to 0acdd65 Compare April 10, 2026 22:15
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from 7a7c757 to f473624 Compare April 10, 2026 22:16
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch 2 times, most recently from f67e32f to e3ded50 Compare April 10, 2026 22:24
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from f473624 to 41d0da0 Compare April 10, 2026 22:24
@FedeZara FedeZara force-pushed the FedeZara/refactor/new-cli-error-system branch from e3ded50 to fdf7661 Compare April 14, 2026 18:22
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from 41d0da0 to f45d891 Compare April 14, 2026 18:22
@FedeZara FedeZara changed the base branch from FedeZara/refactor/new-cli-error-system to graphite-base/14750 April 14, 2026 18:41
FedeZara added a commit that referenced this pull request Apr 14, 2026
…entry routing (#14749)

## Description

Introduces a structured `CliError` class in `@fern-api/task-context` with typed error codes and automatic Sentry routing. This replaces the ad-hoc `new Error(...)` pattern throughout the CLI with a system that categorizes errors into user-facing codes (e.g. `CONFIG_ERROR`, `AUTH_ERROR`) vs internal errors that should be reported to Sentry. It also simplifies error handling by removing redundant error classes and unifying the ones we already have.

> **Recommended review approach:** commit-by-commit. Each commit is self-contained and builds on the previous one.

### Prerequisite PRs

- #14746 — rename `FernCliError` to `TaskAbortSignal` (clears the naming space)
- #14747 — make CLI v1 telemetry synchronous (so `failWithoutThrowing` can report errors)
- #14748 — make CLI v2 telemetry synchronous (same, for CLI v2)

### Follow-up PRs

- #14750 — temporarily disable Sentry for unclassified errors (safety net while migrating)
- #14753 — migrate `@fern-api/cli` package (example of a package migration PR)
- ~34 more package migration PRs (one per package, converting `new Error(...)` → `new CliError(...)` and adding error codes to `failAndThrow` and `failWithoutThrowing` calls)
- #14752 — re-enable Sentry for unclassified errors (merge last, after all migrations)

## Design Decisions

### Two ways to trigger and track errors

There are two paths through which errors are captured and reported:

1. **Throwing `CliError` directly.** Code anywhere in the CLI can `throw new CliError({ message, code })`. The top-level catch handler in each CLI entry point (CLI v1's `runCli` catch in `cli.ts`, CLI v2's `withContext` catch in `withContext.ts`) intercepts it, resolves the error code, and routes it to Sentry and/or PostHog.

2. **Calling `failAndThrow` / `failWithoutThrowing` on the task context.** These methods log the error, resolve the code (explicit override > `CliError.code` > fallback), and report to Sentry if the code is Sentry-reportable. `failAndThrow` then throws a `TaskAbortSignal` to unwind the stack; `failWithoutThrowing` marks the task as failed and returns. This is the preferred path when the caller needs control over what happens after the error — for example, to continue processing other tasks or to clean up resources before aborting. It also adapts naturally to the existing error-handling patterns already used throughout the codebase.

Both paths converge on the same code-resolution logic (`resolveErrorCode`) and Sentry-routing rules (`shouldReportToSentry`), so tracking is consistent regardless of which path is used.

### Default behavior: unclassified errors are internal

Any error that doesn't carry a `CliError.Code` (i.e. a plain `new Error(...)`) is treated as `INTERNAL_ERROR` at the top-level catch boundaries — and therefore reported to Sentry. The assumption is that if nobody explicitly categorized an error as user-facing, it's likely an internal bug.

**Temporary exception:** the follow-up PR #14750 temporarily downgrades this so unclassified errors skip Sentry during the migration period, to avoid noise from the ~34 packages that haven't been migrated yet. Once all migrations land, #14752 re-enables it.

### Error code taxonomy

Errors are classified into 12 typed codes:

| Code | Sentry? | Description |
|------|---------|-------------|
| `INTERNAL_ERROR` | Yes | Unexpected bugs — should be investigated |
| `RESOLUTION_ERROR` | Yes | Type/reference resolution failures (likely IR bugs) |
| `IR_CONVERSION_ERROR` | Yes | IR generation failures |
| `CONTAINER_ERROR` | Yes | Docker container failures |
| `VERSION_ERROR` | Yes | Version parsing/compatibility issues |
| `PARSE_ERROR` | No | Malformed user input (YAML, OpenAPI, etc.) |
| `ENVIRONMENT_ERROR` | No | Missing env vars, wrong Node version, etc. |
| `REFERENCE_ERROR` | No | Dangling references in user config |
| `VALIDATION_ERROR` | No | Schema/rule validation failures |
| `NETWORK_ERROR` | No | HTTP failures, timeouts |
| `AUTH_ERROR` | No | Authentication/authorization issues |
| `CONFIG_ERROR` | No | Invalid generators.yml, fern.config.json, etc. |

Only the first 5 codes are Sentry-reportable — they indicate bugs in Fern itself. The rest are user-actionable and would just create noise in Sentry.

### Simplifying error handling by removing redundant classes

- **Unified `CliError` across CLI v1 and v2.** CLI v2 had its own `CliError` class in `packages/cli/cli-v2/src/errors/CliError.ts`. This PR deletes it and makes both CLIs share the single `CliError` from `@fern-api/task-context`, ensuring consistent error codes and Sentry routing regardless of which CLI entry point is used.

- **Existing error classes now extend `CliError`.** `ValidationError`, `SourcedValidationError`, and `KeyringUnavailableError` now extend `CliError` with their appropriate codes (`VALIDATION_ERROR` and `AUTH_ERROR`). This means `resolveErrorCode()` handles them automatically — no special-casing needed in every catch block.

- **Removed `LoggableFernCliError`.** This was a wrapper that carried a log message alongside an error. With `CliError` now carrying a `message` field (it extends `Error`), the wrapper is redundant. All former `LoggableFernCliError` usages are replaced with `CliError`.

### `reportError` — single error reporting path (CLI v2)

CLI v2's `withContext.ts` previously had separate `shouldReportToSentry` and `extractErrorCode` functions. These are consolidated into a single `reportError(context, error, options?)` function that:
1. Skips `TaskAbortSignal` (already logged)
2. Resolves the error code via `resolveErrorCode()` (explicit override > `CliError.code` > fallback to `INTERNAL_ERROR`)
3. Reports to Sentry if the code is in `SENTRY_REPORTABLE_CODES`
4. Always reports to PostHog with the error code as a property

### Sentry tags

Error codes are now passed as Sentry tags (`errorCode`) on captured exceptions, making it possible to filter and alert on specific error categories in the Sentry dashboard.

## Commits (review in order)

1. **`add shared CliError class`** — introduces `CliError` in `@fern-api/task-context` with the code taxonomy, `shouldReportToSentry`, `resolveErrorCode`, and static factory methods
2. **`unify CLI v2 CliError`** — deletes CLI v2's local `CliError`, switches to shared one, introduces `reportError` in `withContext.ts`
3. **`clean up imports`** — mechanical import reordering for consistency
4. **`make error classes extend CliError`** — `ValidationError`, `SourcedValidationError`, `KeyringUnavailableError` now extend `CliError`
5. **`pass error code as Sentry tag`** — adds `errorCode` tag to `captureException` calls
6. **`remove LoggableFernCliError`** — replaces all usages with `CliError`, deletes the class

## Testing

- [x] Updated test helpers and mocks for new `CliError` / `MockTaskContext` signatures
- [x] Existing CLI v1 and v2 tests pass

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/fern-api/fern/pull/14749" target="_blank">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
    <img src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1" alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
…xt CliError

Delete the local cli-v2 CliError class and switch all 31 importing
files to use the shared CliError from @fern-api/task-context. Remap
v2 codes (AUTH_REQUIRED→AUTH_ERROR, EXIT→TaskAbortSignal, etc.),
replace ~35 CliError.exit() calls with TaskAbortSignal, and classify
~62 untyped new CliError sites with appropriate error codes.

Made-with: Cursor
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from f45d891 to b49bf53 Compare April 14, 2026 18:42
@FedeZara FedeZara force-pushed the graphite-base/14750 branch from fdf7661 to 429be56 Compare April 14, 2026 18:42
@graphite-app graphite-app Bot changed the base branch from graphite-base/14750 to main April 14, 2026 18:43
Add UNCLASSIFIED code to CliError.Code and use it for catch-all error handlers
in cli.ts, CliContext.ts, and withContext.ts. Generic Error instances that are
not explicitly classified will no longer be reported to Sentry, reducing noise
while we migrate utility packages away from CliError.

Made-with: Cursor
@FedeZara FedeZara force-pushed the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch from b49bf53 to a00c1d7 Compare April 14, 2026 18:43
@FedeZara FedeZara merged commit 063113b into main Apr 14, 2026
57 checks passed
Copy link
Copy Markdown
Contributor Author

Merge activity

@FedeZara FedeZara deleted the FedeZara/refactor/cli-error-disable-uncategorized-sentry branch April 14, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants