Skip to content

Commit f997e3c

Browse files
authored
fix(code-reviews): billing error classification (#1332)
## Summary Billing errors from code reviews were being misclassified in two ways: v1 billing errors arrived as `interrupted` and got normalized to `cancelled` (hiding them from billing analytics), and v2 billing errors had the real error message discarded by the wrapper, storing the generic string `session.error` instead. This PR adds a `terminal_reason` column to track failure categories, fixes both classification paths, posts billing notices on PRs/MRs, and backfills ~13k historical rows. Three layers of fixes: - **Wrapper** (`cloud-agent-next`): passes the real billing error text from `properties.error` instead of discarding it and forwarding just the event type string. - **normalizePayload** (status callback handler): detects billing patterns in error messages when no explicit `terminalReason` is set, reclassifying `cancelled` billing errors as `failed` with `terminalReason: 'billing'`. - **Orchestrator**: catches 402 billing errors from `prepareSession`/`initiateSession`/`sendMessageV2`, tags them with `terminalReason: 'billing'`, and prevents the sendMessage fallback from retrying billing failures. Also adds billing PR/MR comment notices, admin dashboard improvements for billing vs. non-billing failure separation, and three backfill migrations for historical data. ## Verification - [x] `pnpm jest 'code-review-status.*route\.test'` — 17 tests pass (4 new: interrupted billing reclassification, failed billing inference, non-billing interrupted preserved, explicit terminalReason preserved) - [x] `pnpm typecheck` — passes across all workspace packages including cloud-agent-next wrapper - [x] Pre-push hooks passed (format, lint, typecheck) ## Visual Changes N/A ## Reviewer Notes - The v2 backfill migration (0057) uses exact `error_message = 'session.error'` matching rather than ILIKE patterns to avoid false positives from branch names containing billing keywords. - Migration 0056 already backfilled v1 `failed` billing rows (23,711 rows). Migration 0057 covers the remaining gaps: v1 `cancelled` billing rows (~12,461) and v2 `session.error` rows (742). - Deployment order matters: deploy the normalizePayload fix (Next.js) first so it's ready to handle callbacks, then apply the backfill migration, then deploy the wrapper fix (cloud-agent-next Worker). - The `!terminalReason` guard in normalizePayload ensures the billing inference only fires when no explicit reason was provided — orchestrator-path billing errors (which already send `terminalReason: 'billing'`) are not double-handled.
2 parents 117f947 + 9a77218 commit f997e3c

10 files changed

Lines changed: 14965 additions & 6 deletions

File tree

cloud-agent-next/wrapper/src/connection.ts

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -273,6 +273,28 @@ export function createConnectionManager(
273273
return false;
274274
}
275275

276+
function getTerminalErrorText(eventType: string, properties: Record<string, unknown>): string {
277+
const error = properties.error;
278+
if (typeof error === 'string') {
279+
return error;
280+
}
281+
282+
if (isRecord(error)) {
283+
if (typeof error.message === 'string') {
284+
return error.message;
285+
}
286+
287+
const data = error.data;
288+
if (isRecord(data) && typeof data.message === 'string') {
289+
return data.message;
290+
}
291+
292+
return JSON.stringify(error);
293+
}
294+
295+
return `Insufficient credits: ${eventType}`;
296+
}
297+
276298
/**
277299
* Start the SDK event subscription. Runs in the background.
278300
* Replaces the old SSE consumer with a typed event stream from the SDK.
@@ -367,7 +389,7 @@ export function createConnectionManager(
367389

368390
// Terminal error detection
369391
if (isTerminalError(eventType, properties)) {
370-
callbacks.onTerminalError(eventType);
392+
callbacks.onTerminalError(getTerminalErrorText(eventType, properties));
371393
return;
372394
}
373395

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
-- v1: cancelled billing reviews should be failed with terminal_reason='billing'
2+
-- These are from the cloud-agent callback path where payment_required_prompt
3+
-- was reported as 'interrupted' → normalized to 'cancelled'.
4+
UPDATE cloud_agent_code_reviews
5+
SET status = 'failed', terminal_reason = 'billing'
6+
WHERE status = 'cancelled'
7+
AND terminal_reason IS NULL
8+
AND (
9+
COALESCE(error_message, '') ILIKE '%Insufficient credits%'
10+
OR COALESCE(error_message, '') ILIKE '%paid model%'
11+
OR COALESCE(error_message, '') ILIKE '%add credits%'
12+
OR COALESCE(error_message, '') ILIKE '%Credits Required%'
13+
);
14+
15+
-- v2: wrapper terminal error path stored generic event type as error_message.
16+
-- The only path that produces error_message='session.error' is isTerminalError()
17+
-- in the wrapper, which exclusively triggers for billing/payment events.
18+
UPDATE cloud_agent_code_reviews
19+
SET terminal_reason = 'billing'
20+
WHERE status = 'failed'
21+
AND terminal_reason IS NULL
22+
AND agent_version = 'v2'
23+
AND error_message = 'session.error';

0 commit comments

Comments
 (0)