Skip to content

Commit b9b1693

Browse files
ndycodendycode
authored
fix(quota): detect unsupported Codex model from detail shape (#501) (#502)
* fix(quota): detect unsupported Codex model from detail error shape The Codex quota endpoint now returns model-not-supported errors as a flat {"detail": "..."} body instead of the nested {"error": {"message": "..."}} envelope. getUnsupportedCodexModelInfo only inspected errorBody.error, so detection failed: the model-fallback loop in fetchCodexQuotaSnapshot stopped continuing and the raw JSON leaked into "live check failed: {...}" output. - fetch-helpers: fall back to the top-level detail string when error is absent or not a record, reusing the existing unsupported-model patterns - quota-probe: track whether every probe model failed solely because the account lacks Codex entitlement; when so, throw a typed CodexUnavailableError - errors: add CodexUnavailableError plus isCodexUnavailableError guard - codex-manager: surface "(Codex not available for this account)" instead of leaking the raw probe error in the check command, still counted as a warning Adds regression tests for the detail shape, the all-unsupported vs mixed-failure paths, and the new error type and guard. Fixes #501 * fix(quota): surface Codex-unavailable note across all live-probe surfaces The first pass only fixed the check command. best, forecast, report, fix (repair), and the runtime deep-check reused fetchCodexQuotaSnapshot through their own catch blocks that rendered the raw error message, so they still leaked 'model is not supported when using Codex with a ChatGPT account' (and the Best Account / Auto-Fix screenshots in issue #501). - quota-probe: add CODEX_UNAVAILABLE_PROBE_NOTE and describeCodexProbeFailure(), centralizing the 'Codex not available for this account' wording and the CodexUnavailableError check so every surface renders it identically - best/forecast/report/forecast-report: route live-probe catch blocks through describeCodexProbeFailure (persist-patch catches left untouched) - repair-commands: emit 'refresh succeeded (Codex not available for this account)' instead of 'live probe failed: ...' for the unavailable case - runtime/account-check: same note for the deep-check probe path - codex-manager: reuse the shared constant in the two check-command branches Tests: describeCodexProbeFailure unit cases; partial quota-probe mocks in codex-manager-cli and repair-commands now spread importOriginal so the new exports resolve. Refs #501 * fix(quota): treat codex-unavailable as warning + add probe-path tests Addresses CodeRabbit/Greptile review on PR #502. Behavior fixes: - codex-manager: styleAccountDetailText now renders the codex-unavailable health detail in the warning tone (matches 'unavailable'/'not available') instead of the muted-success tone, so the dashboard no longer hides it - runtime/account-check: a CodexUnavailableError quota probe is counted as a warning (state.warnings) with the friendly note and no 'ERROR' prefix, instead of incrementing state.errors and misclassifying a working account; the results summary surfaces the warning bucket. Adds state.warnings field. Regression tests for every live-probe surface: - best/forecast/report: CodexUnavailableError yields the friendly note in probeErrors (and a non-unavailable failure stays normalized, no token leak) - repair-commands: refresh-then-probe-unavailable maps to warning-soft-failure carrying the note, account left enabled - runtime/account-check: warning (not error) accounting + note, no raw leak - codex-manager-cli: best --live shows the friendly note, never raw JSON - quota-probe: an instruction-fetch failure on a later model still rejects with the instruction error rather than masking it as CodexUnavailableError Refs #501
1 parent d0addd9 commit b9b1693

20 files changed

Lines changed: 707 additions & 41 deletions

‎lib/codex-manager.ts‎

Lines changed: 24 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,9 @@ import {
164164
type CodexQuotaSnapshot,
165165
fetchCodexQuotaSnapshot,
166166
formatQuotaSnapshotLine,
167+
CODEX_UNAVAILABLE_PROBE_NOTE,
167168
} from "./quota-probe.js";
169+
import { isCodexUnavailableError } from "./errors.js";
168170
import { queuedRefresh } from "./refresh-queue.js";
169171
import {
170172
type AccountMetadataV3,
@@ -420,7 +422,7 @@ function styleAccountDetailText(
420422
? "success"
421423
: fallbackTone;
422424
const suffixTone: PromptTone =
423-
/re-login|stale|warning|retry|fallback/i.test(suffix)
425+
/re-login|stale|warning|retry|fallback|unavailable|not available/i.test(suffix)
424426
? "warning"
425427
: /failed|error/i.test(suffix)
426428
? "danger"
@@ -434,7 +436,7 @@ function styleAccountDetailText(
434436
}
435437

436438
if (/rate-limited/i.test(compact)) return stylePromptText(compact, "danger");
437-
if (/re-login|stale|warning|fallback/i.test(compact))
439+
if (/re-login|stale|warning|fallback|unavailable|not available/i.test(compact))
438440
return stylePromptText(compact, "warning");
439441
if (/failed|error/i.test(compact)) return stylePromptText(compact, "danger");
440442
if (/ok|working|succeeded|valid/i.test(compact))
@@ -2248,12 +2250,17 @@ async function runHealthCheck(options: HealthCheckOptions = {}): Promise<void> {
22482250
}
22492251
healthDetail = formatQuotaSnapshotForDashboard(snapshot, display);
22502252
} catch (error) {
2251-
const message = normalizeFailureDetail(
2252-
error instanceof Error ? error.message : String(error),
2253-
undefined,
2254-
);
22552253
warnings += 1;
2256-
healthDetail = `signed in and working (live check failed: ${message})`;
2254+
if (isCodexUnavailableError(error)) {
2255+
healthDetail =
2256+
`signed in and working (${CODEX_UNAVAILABLE_PROBE_NOTE})`;
2257+
} else {
2258+
const message = normalizeFailureDetail(
2259+
error instanceof Error ? error.message : String(error),
2260+
undefined,
2261+
);
2262+
healthDetail = `signed in and working (live check failed: ${message})`;
2263+
}
22572264
}
22582265
}
22592266
}
@@ -2344,12 +2351,17 @@ async function runHealthCheck(options: HealthCheckOptions = {}): Promise<void> {
23442351
}
23452352
healthyMessage = formatQuotaSnapshotForDashboard(snapshot, display);
23462353
} catch (error) {
2347-
const message = normalizeFailureDetail(
2348-
error instanceof Error ? error.message : String(error),
2349-
undefined,
2350-
);
23512354
warnings += 1;
2352-
healthyMessage = `working now (live check failed: ${message})`;
2355+
if (isCodexUnavailableError(error)) {
2356+
healthyMessage =
2357+
`working now (${CODEX_UNAVAILABLE_PROBE_NOTE})`;
2358+
} else {
2359+
const message = normalizeFailureDetail(
2360+
error instanceof Error ? error.message : String(error),
2361+
undefined,
2362+
);
2363+
healthyMessage = `working now (live check failed: ${message})`;
2364+
}
23532365
}
23542366
}
23552367
}

‎lib/codex-manager/commands/best.ts‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import type { ForecastAccountResult } from "../../forecast.js";
2-
import type { CodexQuotaSnapshot } from "../../quota-probe.js";
2+
import { type CodexQuotaSnapshot, describeCodexProbeFailure } from "../../quota-probe.js";
33
import { resolveNormalizedModel } from "../../request/helpers/model-map.js";
44
import type { AccountStorageV3 } from "../../storage.js";
55
import type { TokenFailure, TokenResult } from "../../types.js";
@@ -208,9 +208,8 @@ export async function runBestCommand(
208208
});
209209
liveQuotaByIndex.set(i, liveQuota);
210210
} catch (error) {
211-
const message = deps.normalizeFailureDetail(
212-
error instanceof Error ? error.message : String(error),
213-
undefined,
211+
const message = describeCodexProbeFailure(error, (raw) =>
212+
deps.normalizeFailureDetail(raw, undefined),
214213
);
215214
probeErrors.push(`${deps.formatAccountLabel(account, i)}: ${message}`);
216215
}

‎lib/codex-manager/commands/forecast.ts‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ import {
1313
type RefreshedAccountPatch,
1414
} from "../forecast-report-shared.js";
1515
import type { QuotaCacheData } from "../../quota-cache.js";
16-
import type { CodexQuotaSnapshot } from "../../quota-probe.js";
16+
import { type CodexQuotaSnapshot, describeCodexProbeFailure } from "../../quota-probe.js";
1717
import { resolveNormalizedModel } from "../../request/helpers/model-map.js";
1818
import { type AccountMetadataV3, type AccountStorageV3 } from "../../storage.js";
1919
import type { TokenFailure, TokenResult } from "../../types.js";
@@ -347,9 +347,8 @@ export async function runForecastCommand(
347347
}
348348
}
349349
} catch (error) {
350-
const message = deps.normalizeFailureDetail(
351-
error instanceof Error ? error.message : String(error),
352-
undefined,
350+
const message = describeCodexProbeFailure(error, (raw) =>
351+
deps.normalizeFailureDetail(raw, undefined),
353352
);
354353
probeErrors.push(`${deps.formatAccountLabel(account, i)}: ${message}`);
355354
}

‎lib/codex-manager/commands/report.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ import {
2121
import {
2222
type CodexQuotaSnapshot,
2323
formatQuotaSnapshotLine,
24+
describeCodexProbeFailure,
2425
} from "../../quota-probe.js";
2526
import { type ModelFamily } from "../../prompts/codex.js";
2627
import {
@@ -447,9 +448,8 @@ export async function runReportCommand(
447448
});
448449
liveQuotaByIndex.set(i, liveQuota);
449450
} catch (error) {
450-
const message = deps.normalizeFailureDetail(
451-
error instanceof Error ? error.message : String(error),
452-
undefined,
451+
const message = describeCodexProbeFailure(error, (raw) =>
452+
deps.normalizeFailureDetail(raw, undefined),
453453
);
454454
probeErrors.push(`${formatAccountLabel(account, i)}: ${message}`);
455455
}

‎lib/codex-manager/forecast-report-commands.ts‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,11 @@ import {
1313
type ForecastAccountResult,
1414
} from "../forecast.js";
1515
import { loadQuotaCache, saveQuotaCache, type QuotaCacheData } from "../quota-cache.js";
16-
import { fetchCodexQuotaSnapshot, formatQuotaSnapshotLine } from "../quota-probe.js";
16+
import {
17+
fetchCodexQuotaSnapshot,
18+
formatQuotaSnapshotLine,
19+
describeCodexProbeFailure,
20+
} from "../quota-probe.js";
1721
import { queuedRefresh } from "../refresh-queue.js";
1822
import {
1923
getStoragePath,
@@ -349,9 +353,8 @@ export async function runForecast(
349353
}
350354
}
351355
} catch (error) {
352-
const message = deps.normalizeFailureDetail(
353-
error instanceof Error ? error.message : String(error),
354-
undefined,
356+
const message = describeCodexProbeFailure(error, (raw) =>
357+
deps.normalizeFailureDetail(raw, undefined),
355358
);
356359
probeErrors.push(`${formatAccountLabel(account, i)}: ${message}`);
357360
}
@@ -523,9 +526,8 @@ export async function runReport(
523526
});
524527
liveQuotaByIndex.set(i, liveQuota);
525528
} catch (error) {
526-
const message = deps.normalizeFailureDetail(
527-
error instanceof Error ? error.message : String(error),
528-
undefined,
529+
const message = describeCodexProbeFailure(error, (raw) =>
530+
deps.normalizeFailureDetail(raw, undefined),
529531
);
530532
probeErrors.push(`${formatAccountLabel(account, i)}: ${message}`);
531533
}

‎lib/codex-manager/repair-commands.ts‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,11 @@ import {
1313
sanitizeEmail,
1414
} from "../accounts.js";
1515
import { loadQuotaCache, saveQuotaCache, type QuotaCacheData } from "../quota-cache.js";
16-
import { fetchCodexQuotaSnapshot } from "../quota-probe.js";
16+
import {
17+
fetchCodexQuotaSnapshot,
18+
CODEX_UNAVAILABLE_PROBE_NOTE,
19+
} from "../quota-probe.js";
20+
import { isCodexUnavailableError } from "../errors.js";
1721
import { queuedRefresh } from "../refresh-queue.js";
1822
import {
1923
findMatchingAccountIndex,
@@ -1386,6 +1390,15 @@ export async function runFix(
13861390
});
13871391
continue;
13881392
} catch (error) {
1393+
if (isCodexUnavailableError(error)) {
1394+
reports.push({
1395+
index: i,
1396+
label,
1397+
outcome: "warning-soft-failure",
1398+
message: `refresh succeeded (${CODEX_UNAVAILABLE_PROBE_NOTE})`,
1399+
});
1400+
continue;
1401+
}
13891402
const message = deps.normalizeFailureDetail(
13901403
error instanceof Error ? error.message : String(error),
13911404
undefined,

‎lib/errors.ts‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ export const ErrorCode = {
1313
VALIDATION_ERROR: "CODEX_VALIDATION_ERROR",
1414
RATE_LIMIT: "CODEX_RATE_LIMIT",
1515
TIMEOUT: "CODEX_TIMEOUT",
16+
CODEX_UNAVAILABLE: "CODEX_UNAVAILABLE",
1617
} as const;
1718

1819
export type ErrorCodeType = (typeof ErrorCode)[keyof typeof ErrorCode];
@@ -185,3 +186,29 @@ export class StorageError extends CodexError {
185186
this.hint = hint;
186187
}
187188
}
189+
190+
/**
191+
* Raised when every probe model is rejected because the account/workspace has no
192+
* Codex entitlement (e.g. the API answers `model is not supported when using
193+
* Codex with a ChatGPT account` for all candidates). The account is otherwise
194+
* signed in and working; callers should surface this as a warning, not a failure.
195+
*/
196+
export class CodexUnavailableError extends CodexError {
197+
override readonly name = "CodexUnavailableError";
198+
199+
constructor(message: string, options?: CodexErrorOptions) {
200+
super(message, { ...options, code: options?.code ?? ErrorCode.CODEX_UNAVAILABLE });
201+
}
202+
}
203+
204+
/**
205+
* Type guard for `CodexUnavailableError` that survives cross-realm/duplicate-module
206+
* boundaries by also matching on the structural `code` marker.
207+
*/
208+
export function isCodexUnavailableError(error: unknown): error is CodexUnavailableError {
209+
if (error instanceof CodexUnavailableError) return true;
210+
return (
211+
error instanceof Error &&
212+
(error as { code?: unknown }).code === ErrorCode.CODEX_UNAVAILABLE
213+
);
214+
}

‎lib/quota-probe.ts‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,43 @@
11
import { CODEX_BASE_URL } from "./constants.js";
22
import { createCodexHeaders, getUnsupportedCodexModelInfo } from "./request/fetch-helpers.js";
3+
import { CodexUnavailableError, isCodexUnavailableError } from "./errors.js";
34
import { getCodexInstructions } from "./prompts/codex.js";
45
import { mutateRuntimeObservabilitySnapshot } from "./runtime/runtime-observability.js";
56
import type { RequestBody } from "./types.js";
67
import { isRecord } from "./utils.js";
78

9+
/**
10+
* Human-readable note shown when a live probe fails solely because the account
11+
* lacks Codex entitlement (see {@link CodexUnavailableError}). Centralized so
12+
* every check/forecast/repair surface renders the same wording instead of
13+
* leaking the raw upstream "model is not supported..." message (issue #501).
14+
*/
15+
export const CODEX_UNAVAILABLE_PROBE_NOTE = "Codex not available for this account";
16+
17+
/**
18+
* Turn a live-probe failure into a display string. When the failure is a
19+
* {@link CodexUnavailableError} (every probe model rejected for lack of Codex
20+
* entitlement), returns the friendly {@link CODEX_UNAVAILABLE_PROBE_NOTE};
21+
* otherwise falls back to the normalized error message.
22+
*
23+
* Pure and side-effect free; safe for concurrent use and performs no I/O.
24+
*
25+
* @param error - The thrown probe error.
26+
* @param normalize - Optional normalizer applied to non-unavailable messages
27+
* (e.g. `normalizeFailureDetail`); defaults to the raw error message.
28+
* @returns A display-ready failure description.
29+
*/
30+
export function describeCodexProbeFailure(
31+
error: unknown,
32+
normalize?: (message: string) => string,
33+
): string {
34+
if (isCodexUnavailableError(error)) {
35+
return CODEX_UNAVAILABLE_PROBE_NOTE;
36+
}
37+
const raw = error instanceof Error ? error.message : String(error);
38+
return normalize ? normalize(raw) : raw;
39+
}
40+
841
export interface CodexQuotaWindow {
942
usedPercent?: number;
1043
windowMinutes?: number;
@@ -330,8 +363,12 @@ export async function fetchCodexQuotaSnapshot(
330363
const models = normalizeProbeModels(options.model, options.fallbackModels);
331364
const timeoutMs = Math.max(1_000, Math.min(options.timeoutMs ?? 15_000, 60_000));
332365
let lastError: Error | null = null;
366+
let attemptedAnyModel = false;
367+
let sawUnsupportedModel = false;
368+
let sawOtherFailure = false;
333369

334370
for (const model of models) {
371+
attemptedAnyModel = true;
335372
try {
336373
const instructions = await getCodexInstructions(model);
337374
const probeBody: RequestBody = {
@@ -395,12 +432,14 @@ export async function fetchCodexQuotaSnapshot(
395432

396433
const unsupportedInfo = getUnsupportedCodexModelInfo(errorBody);
397434
if (unsupportedInfo.isUnsupported) {
435+
sawUnsupportedModel = true;
398436
lastError = new Error(
399437
unsupportedInfo.message ?? `Model '${model}' unsupported for this account`,
400438
);
401439
continue;
402440
}
403441

442+
sawOtherFailure = true;
404443
throw new Error(extractErrorMessage(bodyText, response.status));
405444
}
406445

@@ -409,11 +448,20 @@ export async function fetchCodexQuotaSnapshot(
409448
} catch {
410449
// Best effort cancellation.
411450
}
451+
sawOtherFailure = true;
412452
lastError = new Error("Codex response did not include quota headers");
413453
} catch (error) {
454+
sawOtherFailure = true;
414455
lastError = error instanceof Error ? error : new Error(String(error));
415456
}
416457
}
417458

459+
if (attemptedAnyModel && sawUnsupportedModel && !sawOtherFailure) {
460+
throw new CodexUnavailableError(
461+
lastError?.message ?? "Codex is not available for this account",
462+
{ cause: lastError ?? undefined },
463+
);
464+
}
465+
418466
throw lastError ?? new Error("Failed to fetch quotas");
419467
}

‎lib/request/fetch-helpers.ts‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -207,7 +207,25 @@ export function getUnsupportedCodexModelInfo(
207207

208208
const maybeError = errorBody.error;
209209
if (!isRecord(maybeError)) {
210-
return { isUnsupported: false };
210+
// Some upstreams (e.g. the Codex quota endpoint) return the flat
211+
// `{ "detail": "...model is not supported..." }` shape instead of the
212+
// nested `{ "error": { "message": "..." } }` envelope. Fall back to the
213+
// top-level `detail` string so model-fallback detection still works.
214+
const detail = typeof errorBody.detail === "string" ? errorBody.detail : undefined;
215+
if (!detail) {
216+
return { isUnsupported: false };
217+
}
218+
const isUnsupportedDetail =
219+
CHATGPT_CODEX_UNSUPPORTED_MODEL_PATTERN.test(detail) ||
220+
MODEL_ACCESS_DENIED_PATTERN.test(detail);
221+
if (!isUnsupportedDetail) {
222+
return { isUnsupported: false };
223+
}
224+
return {
225+
isUnsupported: true,
226+
message: detail,
227+
unsupportedModel: extractUnsupportedCodexModelFromText(detail) ?? undefined,
228+
};
211229
}
212230

213231
const code = typeof maybeError.code === "string" ? maybeError.code : undefined;

‎lib/runtime/account-check-types.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ export type AccountCheckWorkingState = {
55
flaggedChanged: boolean;
66
ok: number;
77
errors: number;
8+
warnings: number;
89
disabled: number;
910
removeFromActive: Set<string>;
1011
flaggedStorage: { version: 1; accounts: FlaggedAccountMetadataV1[] };
@@ -19,6 +20,7 @@ export function createAccountCheckWorkingState(flaggedStorage: {
1920
flaggedChanged: false,
2021
ok: 0,
2122
errors: 0,
23+
warnings: 0,
2224
disabled: 0,
2325
removeFromActive: new Set<string>(),
2426
flaggedStorage,

0 commit comments

Comments
 (0)