Skip to content

chore(cli): make instrumentPostHogEvent synchronous#14747

Merged
FedeZara merged 1 commit intomainfrom
FedeZara/refactor/sync-instrument-posthog-event
Apr 10, 2026
Merged

chore(cli): make instrumentPostHogEvent synchronous#14747
FedeZara merged 1 commit intomainfrom
FedeZara/refactor/sync-instrument-posthog-event

Conversation

@FedeZara
Copy link
Copy Markdown
Contributor

@FedeZara FedeZara commented Apr 8, 2026

Description

Part of the CLI error classification effort — see #14749 for full context.

Makes instrumentPostHogEvent synchronous in CLI v1 by removing its async qualifier and dropping unnecessary await calls at all call sites. This is the CLI v1 counterpart to #14748 (CLI v2).

Why

The new error system (#14749) needs to call telemetry from failWithoutThrowing — a synchronous method on the TaskContext interface. If instrumentPostHogEvent remains async, every failWithoutThrowing call site would need to become async too, rippling changes through the entire TaskContext interface. Making telemetry synchronous avoids that: PostHog's capture() is fire-and-forget anyway (it buffers locally and flushes in the background), so the async was always unnecessary.

Changes Made

  • Changed instrumentPostHogEvent from async to synchronous in TaskContext interface, CliContext, TaskContextImpl, TaskContextAdapter, and test helpers
  • Removed await from all call sites (21 files)

Testing

  • Existing tests pass (behavioral no-op)

@FedeZara FedeZara self-assigned this Apr 8, 2026
Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 8, 2026

🌱 Seed Test Selector

Select languages to run seed tests for:

  • Python
  • TypeScript
  • Java
  • Go
  • Ruby
  • C#
  • PHP
  • Swift
  • Rust
  • OpenAPI

How to use: Click the ⋯ menu above → "Edit" → check the boxes you want → click "Update comment". Tests will run automatically and snapshots will be committed to this PR.

@FedeZara FedeZara changed the title refactor(cli): make instrumentPostHogEvent synchronous chore(cli): make instrumentPostHogEvent synchronous Apr 8, 2026
@FedeZara FedeZara force-pushed the FedeZara/refactor/rename-fern-cli-error-to-task-abort-signal branch from 2f457fe to b7324ed Compare April 9, 2026 12:22
@FedeZara FedeZara force-pushed the FedeZara/refactor/sync-instrument-posthog-event branch from 47cb12c to 5cda8a4 Compare April 9, 2026 12:23
@FedeZara FedeZara force-pushed the FedeZara/refactor/rename-fern-cli-error-to-task-abort-signal branch from b7324ed to b06c68c Compare April 9, 2026 13:55
@FedeZara FedeZara force-pushed the FedeZara/refactor/sync-instrument-posthog-event branch 2 times, most recently from 389f42d to 96bcc31 Compare April 9, 2026 14:54
@FedeZara FedeZara force-pushed the FedeZara/refactor/rename-fern-cli-error-to-task-abort-signal branch 2 times, most recently from 5842013 to 06590df Compare April 9, 2026 15:50
@FedeZara FedeZara force-pushed the FedeZara/refactor/sync-instrument-posthog-event branch 2 times, most recently from 0f6924a to fd138e2 Compare April 9, 2026 22:38
@FedeZara FedeZara force-pushed the FedeZara/refactor/rename-fern-cli-error-to-task-abort-signal branch from 06590df to 8c1a873 Compare April 9, 2026 22:38
@FedeZara FedeZara force-pushed the FedeZara/refactor/rename-fern-cli-error-to-task-abort-signal branch from 8c1a873 to 452b99d Compare April 10, 2026 16:06
@FedeZara FedeZara force-pushed the FedeZara/refactor/sync-instrument-posthog-event branch from fd138e2 to 42bf390 Compare April 10, 2026 16:07
@FedeZara FedeZara force-pushed the FedeZara/refactor/rename-fern-cli-error-to-task-abort-signal branch from 452b99d to 48cd7ac Compare April 10, 2026 16:10
@FedeZara FedeZara force-pushed the FedeZara/refactor/sync-instrument-posthog-event branch from 42bf390 to d105df5 Compare April 10, 2026 16:11
@FedeZara FedeZara force-pushed the FedeZara/refactor/rename-fern-cli-error-to-task-abort-signal branch from 48cd7ac to ddfbe75 Compare April 10, 2026 16:19
@FedeZara FedeZara force-pushed the FedeZara/refactor/sync-instrument-posthog-event branch 2 times, most recently from 6d3f221 to e7b45a4 Compare April 10, 2026 17:09
@FedeZara FedeZara force-pushed the FedeZara/refactor/rename-fern-cli-error-to-task-abort-signal branch from ddfbe75 to 5e8f14e Compare April 10, 2026 17:09
@FedeZara FedeZara force-pushed the FedeZara/refactor/sync-instrument-posthog-event branch from e7b45a4 to d20821f Compare April 10, 2026 18:33
@FedeZara FedeZara force-pushed the FedeZara/refactor/rename-fern-cli-error-to-task-abort-signal branch from 5e8f14e to 32b33b9 Compare April 10, 2026 18:33
@FedeZara FedeZara changed the base branch from FedeZara/refactor/rename-fern-cli-error-to-task-abort-signal to graphite-base/14747 April 10, 2026 19:53
@FedeZara FedeZara force-pushed the FedeZara/refactor/sync-instrument-posthog-event branch from d20821f to 1cb5ba5 Compare April 10, 2026 19:54
@FedeZara FedeZara force-pushed the graphite-base/14747 branch from 32b33b9 to 952cbd2 Compare April 10, 2026 19:54
@graphite-app graphite-app bot changed the base branch from graphite-base/14747 to main April 10, 2026 19:55
@FedeZara FedeZara force-pushed the FedeZara/refactor/sync-instrument-posthog-event branch from 1cb5ba5 to 6791d54 Compare April 10, 2026 19:55
@github-actions
Copy link
Copy Markdown
Contributor

SDK Generation Benchmark Results

Comparing PR branch against latest nightly baseline on main (2026-04-10T04:56:48Z).

Full benchmark table (click to expand)
Generator Spec main (generator) main (E2E) PR (generator) Delta
csharp-sdk square 100s 135s 91s -9s (-9.0%)
go-sdk square 104s 137s 104s +0s (+0.0%)
java-sdk square 157s 191s 149s -8s (-5.1%)
php-sdk square 86s 122s 81s -5s (-5.8%)
python-sdk square 124s 167s 129s +5s (+4.0%)
ruby-sdk-v2 square 120s 152s 117s -3s (-2.5%)
rust-sdk square 92s 94s 92s +0s (+0.0%)
swift-sdk square 105s 476s 81s -24s (-22.9%)
ts-sdk square 102s 129s 98s -4s (-3.9%)

main (generator): generator-only time via --skip-scripts (includes Docker image build, container startup, IR parsing, and code generation — this is the same Docker-based flow customers use via fern generate). main (E2E): full customer-observable time including build/test scripts (nightly baseline, informational). Delta is computed against generator-only baseline.
⚠️ = generation exited with a non-zero exit code (timing may not reflect a successful run).
Baseline from nightly runs on main (latest: 2026-04-10T04:56:48Z). Trigger benchmark-baseline to refresh.

@FedeZara FedeZara force-pushed the FedeZara/refactor/sync-instrument-posthog-event branch from 6791d54 to cbb6b50 Compare April 10, 2026 21:55
@github-actions
Copy link
Copy Markdown
Contributor

SDK Generation Benchmark Results

Comparing PR branch against latest nightly baseline on main (2026-04-10T04:56:48Z).

Full benchmark table (click to expand)
Generator Spec main (generator) main (E2E) PR (generator) Delta
csharp-sdk square 100s 135s 88s -12s (-12.0%)
go-sdk square 104s 137s 113s +9s (+8.7%)
java-sdk square 157s 191s 161s +4s (+2.5%)
php-sdk square 86s 122s 91s +5s (+5.8%)
python-sdk square 124s 167s 124s +0s (+0.0%)
ruby-sdk-v2 square 120s 152s 119s -1s (-0.8%)
rust-sdk square 92s 94s 91s -1s (-1.1%)
swift-sdk square 105s 476s 85s -20s (-19.0%)
ts-sdk square 102s 129s 105s +3s (+2.9%)

main (generator): generator-only time via --skip-scripts (includes Docker image build, container startup, IR parsing, and code generation — this is the same Docker-based flow customers use via fern generate). main (E2E): full customer-observable time including build/test scripts (nightly baseline, informational). Delta is computed against generator-only baseline.
⚠️ = generation exited with a non-zero exit code (timing may not reflect a successful run).
Baseline from nightly runs on main (latest: 2026-04-10T04:56:48Z). Trigger benchmark-baseline to refresh.

@FedeZara FedeZara merged commit 977810b into main Apr 10, 2026
264 checks passed
Copy link
Copy Markdown
Contributor Author

Merge activity

@FedeZara FedeZara deleted the FedeZara/refactor/sync-instrument-posthog-event branch April 10, 2026 22:13
FedeZara added a commit that referenced this pull request Apr 10, 2026
…ory (#14748)

## Description

Part of the CLI error classification effort — see #14749 for full context.

Makes telemetry synchronous in **CLI v2** by eagerly resolving the PostHog `distinctId` at startup via an async factory, so that `Context.telemetry` becomes a synchronous `TelemetryClient`. This is the CLI v2 counterpart to #14747 (CLI v1).

## Why

Same motivation as #14747: the new error system (#14749) introduces a `reportError` function called from `failWithoutThrowing` and top-level catch blocks. This function needs synchronous access to telemetry to capture Sentry exceptions with PostHog correlation. Previously `Context.telemetry` was a `Promise<TelemetryClient>`, which forced all error handlers to be async. Moving the async resolution to startup keeps the hot path simple and avoids changing the `TaskContext` interface.

## Changes Made

- Added `TelemetryClient.create()` async factory that resolves `distinctId` eagerly
- `Context.telemetry` is now a synchronous `TelemetryClient` (no more `Promise<TelemetryClient>`)
- Updated all test helpers to use the new factory (`TelemetryClient.create()` or direct construction)
- Adjusted CLI v2 `withContext` to `await` telemetry creation at startup

## Testing

- [x] Updated TelemetryClient tests for new factory pattern
- [x] Updated test context helpers
FedeZara added a commit that referenced this pull request Apr 14, 2026
…entry routing (#14749)

## Description

Introduces a structured `CliError` class in `@fern-api/task-context` with typed error codes and automatic Sentry routing. This replaces the ad-hoc `new Error(...)` pattern throughout the CLI with a system that categorizes errors into user-facing codes (e.g. `CONFIG_ERROR`, `AUTH_ERROR`) vs internal errors that should be reported to Sentry. It also simplifies error handling by removing redundant error classes and unifying the ones we already have.

> **Recommended review approach:** commit-by-commit. Each commit is self-contained and builds on the previous one.

### Prerequisite PRs

- #14746 — rename `FernCliError` to `TaskAbortSignal` (clears the naming space)
- #14747 — make CLI v1 telemetry synchronous (so `failWithoutThrowing` can report errors)
- #14748 — make CLI v2 telemetry synchronous (same, for CLI v2)

### Follow-up PRs

- #14750 — temporarily disable Sentry for unclassified errors (safety net while migrating)
- #14753 — migrate `@fern-api/cli` package (example of a package migration PR)
- ~34 more package migration PRs (one per package, converting `new Error(...)` → `new CliError(...)` and adding error codes to `failAndThrow` and `failWithoutThrowing` calls)
- #14752 — re-enable Sentry for unclassified errors (merge last, after all migrations)

## Design Decisions

### Two ways to trigger and track errors

There are two paths through which errors are captured and reported:

1. **Throwing `CliError` directly.** Code anywhere in the CLI can `throw new CliError({ message, code })`. The top-level catch handler in each CLI entry point (CLI v1's `runCli` catch in `cli.ts`, CLI v2's `withContext` catch in `withContext.ts`) intercepts it, resolves the error code, and routes it to Sentry and/or PostHog.

2. **Calling `failAndThrow` / `failWithoutThrowing` on the task context.** These methods log the error, resolve the code (explicit override > `CliError.code` > fallback), and report to Sentry if the code is Sentry-reportable. `failAndThrow` then throws a `TaskAbortSignal` to unwind the stack; `failWithoutThrowing` marks the task as failed and returns. This is the preferred path when the caller needs control over what happens after the error — for example, to continue processing other tasks or to clean up resources before aborting. It also adapts naturally to the existing error-handling patterns already used throughout the codebase.

Both paths converge on the same code-resolution logic (`resolveErrorCode`) and Sentry-routing rules (`shouldReportToSentry`), so tracking is consistent regardless of which path is used.

### Default behavior: unclassified errors are internal

Any error that doesn't carry a `CliError.Code` (i.e. a plain `new Error(...)`) is treated as `INTERNAL_ERROR` at the top-level catch boundaries — and therefore reported to Sentry. The assumption is that if nobody explicitly categorized an error as user-facing, it's likely an internal bug.

**Temporary exception:** the follow-up PR #14750 temporarily downgrades this so unclassified errors skip Sentry during the migration period, to avoid noise from the ~34 packages that haven't been migrated yet. Once all migrations land, #14752 re-enables it.

### Error code taxonomy

Errors are classified into 12 typed codes:

| Code | Sentry? | Description |
|------|---------|-------------|
| `INTERNAL_ERROR` | Yes | Unexpected bugs — should be investigated |
| `RESOLUTION_ERROR` | Yes | Type/reference resolution failures (likely IR bugs) |
| `IR_CONVERSION_ERROR` | Yes | IR generation failures |
| `CONTAINER_ERROR` | Yes | Docker container failures |
| `VERSION_ERROR` | Yes | Version parsing/compatibility issues |
| `PARSE_ERROR` | No | Malformed user input (YAML, OpenAPI, etc.) |
| `ENVIRONMENT_ERROR` | No | Missing env vars, wrong Node version, etc. |
| `REFERENCE_ERROR` | No | Dangling references in user config |
| `VALIDATION_ERROR` | No | Schema/rule validation failures |
| `NETWORK_ERROR` | No | HTTP failures, timeouts |
| `AUTH_ERROR` | No | Authentication/authorization issues |
| `CONFIG_ERROR` | No | Invalid generators.yml, fern.config.json, etc. |

Only the first 5 codes are Sentry-reportable — they indicate bugs in Fern itself. The rest are user-actionable and would just create noise in Sentry.

### Simplifying error handling by removing redundant classes

- **Unified `CliError` across CLI v1 and v2.** CLI v2 had its own `CliError` class in `packages/cli/cli-v2/src/errors/CliError.ts`. This PR deletes it and makes both CLIs share the single `CliError` from `@fern-api/task-context`, ensuring consistent error codes and Sentry routing regardless of which CLI entry point is used.

- **Existing error classes now extend `CliError`.** `ValidationError`, `SourcedValidationError`, and `KeyringUnavailableError` now extend `CliError` with their appropriate codes (`VALIDATION_ERROR` and `AUTH_ERROR`). This means `resolveErrorCode()` handles them automatically — no special-casing needed in every catch block.

- **Removed `LoggableFernCliError`.** This was a wrapper that carried a log message alongside an error. With `CliError` now carrying a `message` field (it extends `Error`), the wrapper is redundant. All former `LoggableFernCliError` usages are replaced with `CliError`.

### `reportError` — single error reporting path (CLI v2)

CLI v2's `withContext.ts` previously had separate `shouldReportToSentry` and `extractErrorCode` functions. These are consolidated into a single `reportError(context, error, options?)` function that:
1. Skips `TaskAbortSignal` (already logged)
2. Resolves the error code via `resolveErrorCode()` (explicit override > `CliError.code` > fallback to `INTERNAL_ERROR`)
3. Reports to Sentry if the code is in `SENTRY_REPORTABLE_CODES`
4. Always reports to PostHog with the error code as a property

### Sentry tags

Error codes are now passed as Sentry tags (`errorCode`) on captured exceptions, making it possible to filter and alert on specific error categories in the Sentry dashboard.

## Commits (review in order)

1. **`add shared CliError class`** — introduces `CliError` in `@fern-api/task-context` with the code taxonomy, `shouldReportToSentry`, `resolveErrorCode`, and static factory methods
2. **`unify CLI v2 CliError`** — deletes CLI v2's local `CliError`, switches to shared one, introduces `reportError` in `withContext.ts`
3. **`clean up imports`** — mechanical import reordering for consistency
4. **`make error classes extend CliError`** — `ValidationError`, `SourcedValidationError`, `KeyringUnavailableError` now extend `CliError`
5. **`pass error code as Sentry tag`** — adds `errorCode` tag to `captureException` calls
6. **`remove LoggableFernCliError`** — replaces all usages with `CliError`, deletes the class

## Testing

- [x] Updated test helpers and mocks for new `CliError` / `MockTaskContext` signatures
- [x] Existing CLI v1 and v2 tests pass

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/fern-api/fern/pull/14749" target="_blank">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
    <img src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1" alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants