Skip to content

Commit 4569197

Browse files
authored
fix: detect token invalidation in 401 responses and stop cascade rotation (#496)
* fix: detect token invalidation in 401 responses and stop cascade rotation Upstream responses containing explicit token-invalidation messages ("invalidated oauth token", "authentication token has been invalidated", etc.) are now detected separately from generic 401 errors. When detected: - A long cooldown (default 5 min, CODEX_AUTH_TOKEN_INVALIDATION_COOLDOWN_MS) is applied to the affected account. - The 401 is returned directly to the client instead of rotating to the next account, preventing the cascade where each successive account token is invalidated in turn by OpenAI's anti-abuse detection. - Session affinity for that session key is cleared. Generic 401 responses (expired tokens, wrong credentials) continue to rotate as before. Fixes #495 * test: address CR review — cooldown duration assertions, edge cases, phrase provenance - Add comment above TOKEN_INVALIDATION_PHRASES explaining observed source providers (OpenAI/Microsoft), how to update the list, and reference to issue #495 for context. - Invalidation test: assert coolingDownUntil is ~5min (not 30s generic), confirming the long cooldown path is reached. - Generic 401 test: assert coolingDownUntil is ~30s (not 5min), confirming the short fallback path is unchanged. - Add edge case: empty 401 body does not trigger invalidation detection, rotation proceeds normally. - Add edge case: HTML 401 body containing invalidation phrase is detected and stops cascade rotation. * feat: add minRotationIntervalMs to throttle cross-request account switching Adds a global per-proxy rotation throttle that biases account selection toward the last successfully-served account within a configurable time window (default 60 seconds, env: CODEX_AUTH_MIN_ROTATION_INTERVAL_MS). When the last served account is within the window and still available, it receives a large score boost (1000) in the hybrid selection algorithm, overriding the freshness weight that previously caused the proxy to eagerly rotate to idle accounts on every request. This reduces how often different OAuth tokens are presented from the same IP in quick succession -- the primary fingerprint that triggers OpenAI's anti-abuse detection and causes cascade token invalidation (issue #495). The boost is applied only to available accounts, so rate-limited or cooling-down accounts are still skipped and rotation proceeds naturally. Setting minRotationIntervalMs to 0 disables the throttle entirely. * fix: detect token invalidation in refresh failures and stop cascade When the OAuth token refresh endpoint itself returns an explicit invalidation message (e.g. Microsoft/Outlook SSO tokens being revoked server-side on first use through the proxy), apply the long cooldown and return 401 directly to the client without rotating to the next account. This covers the second cascade vector reported in issue #495: Account 4 (Outlook) was getting invalidated immediately on the first request because ensureFreshAccessToken was calling queuedRefresh on a token with a missing or short expiry, and the refresh itself triggered Microsoft's session revocation. Rotating to the next account after that would then present another fresh token, cascading the invalidation. The fix adds tokenInvalidationCooldownMs to ensureFreshAccessToken, checks isTokenInvalidationError against the refresh failure message, and when matched: applies the long cooldown, clears session affinity, and returns the auth error to the client instead of continuing the loop. * fix: sliding sticky window, monotonic cooldown, and session affinity tests - Sliding sticky window: update lastGlobalSwitchAt on every successful serve, not only when the account index changes. Previously a request at t=61s would rotate even if the same account served at t=55s (only 6s ago) because the anchor stayed at t=0. Now the window slides so the 60s interval is measured from the last actual serve. - Monotonic auth-failure cooldown: replace direct markAccountCoolingDown calls in auth-failure paths with applyMonotonicAuthCooldown helper that only extends the cooldown if the proposed deadline exceeds the existing one. Two concurrent requests can no longer race so that a generic 401 truncates a longer invalidation cooldown already written by a parallel request. - Regression test: three requests across the window boundary using vi.useFakeTimers to verify the sliding behavior. - Session affinity test: refresh-invalidation test sends session_id and verifies subsequent request with same session routes to healthy account. * docs: document token-invalidation anti-abuse mitigations and Microsoft SSO warning - troubleshooting.md: add rows for progressive OAuth token invalidation and Microsoft/Outlook SSO immediate invalidation, with recovery steps and CODEX_AUTH_TOKEN_INVALIDATION_COOLDOWN_MS guidance. - configuration.md: add CODEX_AUTH_MIN_ROTATION_INTERVAL_MS and CODEX_AUTH_TOKEN_INVALIDATION_COOLDOWN_MS to the env var table. Expand the Runtime Rotation Proxy section with an "Anti-abuse protection" note explaining the two mitigations added in issue #495 (token-invalidation detection + rotation-rate throttle), and a Microsoft/Outlook SSO note for accounts that get invalidated on first proxy request.
1 parent 33e4b22 commit 4569197

9 files changed

Lines changed: 394 additions & 10 deletions

docs/configuration.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,8 @@ These are safe for most operators and frequently used in day-to-day workflows.
7373
| `CODEX_TUI_GLYPHS=ascii|unicode|auto` | Glyph mode selection |
7474
| `CODEX_AUTH_FETCH_TIMEOUT_MS=<ms>` | HTTP request timeout override |
7575
| `CODEX_AUTH_STREAM_STALL_TIMEOUT_MS=<ms>` | Stream stall timeout override |
76+
| `CODEX_AUTH_MIN_ROTATION_INTERVAL_MS=<ms>` | Minimum time between global account switches (default `60000`). The proxy biases selection toward the last-served account within this window to reduce the rate at which different OAuth tokens appear from the same IP. Set to `0` to disable. |
77+
| `CODEX_AUTH_TOKEN_INVALIDATION_COOLDOWN_MS=<ms>` | Cooldown applied to an account when the upstream or token-refresh endpoint explicitly revokes its OAuth token (default `300000`, 5 minutes). Raise this if accounts continue to be re-invalidated after re-login. |
7678

7779
---
7880

@@ -110,6 +112,13 @@ Keep these enabled for most environments:
110112

111113
The proxy preserves request bodies and streaming responses, replaces outbound auth headers with the selected managed account, and rotates to another account before response bytes are streamed when it sees rate limits, server errors, network failures, or refresh failures. It removes hop-by-hop headers, private account metadata headers, and stale decoded `content-encoding` from client responses. If every account is unavailable, the proxy returns a structured pool-exhaustion error that points to `codex-multi-auth rotation status`.
112114

115+
**Anti-abuse protection.** Rapidly switching OAuth tokens from the same IP can trigger OpenAI's anti-abuse detection and cause accounts to be invalidated in sequence. The proxy includes two mitigations:
116+
117+
- **Token-invalidation detection**: when the upstream or the token-refresh endpoint returns an explicit OAuth revocation message, the proxy returns the error directly to the client instead of rotating to the next account. The affected account receives a 5-minute cooldown (`tokenInvalidationCooldownMs`, default `300000`) instead of the generic 30-second auth-failure cooldown. Configure via `CODEX_AUTH_TOKEN_INVALIDATION_COOLDOWN_MS`.
118+
- **Rotation-rate throttle**: the proxy biases account selection toward the last-served account for a configurable window (default 60 seconds, `minRotationIntervalMs`). Accounts that are rate-limited or cooling down are still rotated around. Configure via `CODEX_AUTH_MIN_ROTATION_INTERVAL_MS` or set to `0` to disable.
119+
120+
Microsoft/Outlook SSO accounts may be more sensitive to proxy-mediated token use. If an Outlook-linked account is invalidated on every first request through the proxy but works normally on ChatGPT web, the root cause is likely IP or device binding on the Microsoft side. Raising `CODEX_AUTH_TOKEN_INVALIDATION_COOLDOWN_MS` and re-logging in the affected account typically resolves the cascade. If the problem persists, consider excluding the Microsoft account from the rotation pool via `codex-multi-auth switch`.
121+
113122
For `codex app` launches that go through the wrapper, the wrapper automatically starts a small internal helper so rotation can keep working if the desktop app launcher detaches. The helper stores only local runtime status, uses the same per-session proxy client key as the CLI path, and exits after an idle timeout.
114123

115124
`codex-multi-auth rotation enable` also binds the packaged desktop app to a persistent localhost router. This backs up the real Codex `config.toml`, writes the `codex-multi-auth-runtime-proxy` provider into the real Codex home, starts the router immediately, and installs a user login startup entry: a Startup `.cmd` on Windows or a LaunchAgent on macOS. The persistent provider is marked as not requiring OpenAI auth and uses a local app-bind client token, so the desktop runtime does not display the selected multi-auth account while codex-multi-auth status and quota views still read the router's last-account telemetry. `codex-multi-auth rotation disable` and `codex-multi-auth rotation unbind-app` stop that router, remove the startup entry, and restore the backed-up Codex config. The official app files are not patched.

docs/troubleshooting.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,8 @@ The package does not publish a global `codex` binary. `codex-multi-auth ...` is
7676
| `codex-multi-auth rotation status` says disabled | Stored setting or env override is off | Run `codex-multi-auth rotation enable`, remove `CODEX_MULTI_AUTH_RUNTIME_ROTATION_PROXY=0`, or set `CODEX_MULTI_AUTH_RUNTIME_ROTATION_PROXY=1` for one process |
7777
| Forwarded Codex session does not show the local provider | Command is help/non-requesting, rotation is disabled, or the official CLI was not launched through the wrapper | Check `where codex-multi-auth-codex`, then run `codex-multi-auth rotation status` |
7878
| Pool exhausted error from the proxy | Every managed account is unavailable for that model/family | Run `codex-multi-auth rotation status`, then `codex-multi-auth forecast --live` |
79+
| Accounts progressively lose OAuth tokens while the proxy is active | Rapid account rotation triggers OpenAI's anti-abuse detection, which invalidates tokens in sequence | The proxy detects explicit token-invalidation responses and stops rotating; re-login any invalidated accounts and ensure `minRotationIntervalMs` is at least `60000` (default) |
80+
| Microsoft/Outlook SSO account gets invalidated on every first request through the proxy | Microsoft OAuth tokens may be invalidated when the proxy presents them from a different IP or device context than where they were issued | The proxy now detects invalidation at both the upstream request and the token-refresh stage; if the problem persists, set `CODEX_AUTH_TOKEN_INVALIDATION_COOLDOWN_MS=600000` (10 min) and re-login, or keep the Microsoft account disabled from the rotation pool via `codex-multi-auth rotation status` |
7981
| Packaged app still uses normal Codex routing | App bind was not installed or was removed | Run `codex-multi-auth rotation bind-app`, then reopen the app |
8082
| Codex Desktop history disappears after app bind | Current Codex Desktop builds can filter local threads by the active provider, and app bind switches the real config to `codex-multi-auth-runtime-proxy` | The data is normally still under `~/.codex`; run `codex-multi-auth rotation unbind-app` or `codex-multi-auth rotation disable` to restore the original provider/config before browsing old history |
8183
| Model speed controls are not visible with rotation | Speed/reasoning controls remain owned by Codex config or CLI flags; the app bind only routes Responses traffic | Set `model_reasoning_effort` in `~/.codex/config.toml` or pass `-c model_reasoning_effort=<level>` for wrapper-launched CLI sessions |

lib/config.ts

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -198,6 +198,8 @@ export const DEFAULT_PLUGIN_CONFIG: PluginConfig = {
198198
proactiveRefreshBufferMs: 5 * 60_000,
199199
networkErrorCooldownMs: 6_000,
200200
serverErrorCooldownMs: 4_000,
201+
tokenInvalidationCooldownMs: 5 * 60_000,
202+
minRotationIntervalMs: 60_000,
201203
storageBackupEnabled: true,
202204
preemptiveQuotaEnabled: true,
203205
preemptiveQuotaRemainingPercent5h: 5,
@@ -1402,6 +1404,48 @@ export function getServerErrorCooldownMs(pluginConfig: PluginConfig): number {
14021404
);
14031405
}
14041406

1407+
/**
1408+
* Get the cooldown duration in milliseconds to apply when an OAuth token has been
1409+
* explicitly invalidated by the upstream (distinct from a generic 401).
1410+
*
1411+
* A longer default (5 minutes) prevents the cascade where rapid account rotation
1412+
* causes each successive account's token to be invalidated in turn by OpenAI's
1413+
* anti-abuse detection.
1414+
*
1415+
* @param pluginConfig - Plugin configuration used to resolve the setting
1416+
* @returns The cooldown in milliseconds (minimum 0, default 300000)
1417+
*/
1418+
export function getTokenInvalidationCooldownMs(pluginConfig: PluginConfig): number {
1419+
return resolveNumberSetting(
1420+
"CODEX_AUTH_TOKEN_INVALIDATION_COOLDOWN_MS",
1421+
pluginConfig.tokenInvalidationCooldownMs,
1422+
5 * 60_000,
1423+
{ min: 0 },
1424+
);
1425+
}
1426+
1427+
/**
1428+
* Get the minimum time in milliseconds that must elapse between global account
1429+
* switches across requests. When the last served account is still within this
1430+
* window and is available, it receives a large selection-score boost so the
1431+
* proxy stays on it rather than rotating to a fresher idle account.
1432+
*
1433+
* Setting this to 0 disables the throttle. Default is 60 seconds, which
1434+
* reduces the rate at which different OAuth tokens are presented from the same
1435+
* IP and helps avoid OpenAI's anti-abuse detection (see issue #495).
1436+
*
1437+
* @param pluginConfig - Plugin configuration used to resolve the setting
1438+
* @returns The minimum rotation interval in milliseconds (minimum 0, default 60000)
1439+
*/
1440+
export function getMinRotationIntervalMs(pluginConfig: PluginConfig): number {
1441+
return resolveNumberSetting(
1442+
"CODEX_AUTH_MIN_ROTATION_INTERVAL_MS",
1443+
pluginConfig.minRotationIntervalMs,
1444+
60_000,
1445+
{ min: 0 },
1446+
);
1447+
}
1448+
14051449
/**
14061450
* Determines whether periodic storage backups are enabled.
14071451
*
@@ -1822,6 +1866,16 @@ const CONFIG_EXPLAIN_ENTRIES: ConfigExplainMeta[] = [
18221866
envNames: ["CODEX_AUTH_SERVER_ERROR_COOLDOWN_MS"],
18231867
getValue: getServerErrorCooldownMs,
18241868
},
1869+
{
1870+
key: "tokenInvalidationCooldownMs",
1871+
envNames: ["CODEX_AUTH_TOKEN_INVALIDATION_COOLDOWN_MS"],
1872+
getValue: getTokenInvalidationCooldownMs,
1873+
},
1874+
{
1875+
key: "minRotationIntervalMs",
1876+
envNames: ["CODEX_AUTH_MIN_ROTATION_INTERVAL_MS"],
1877+
getValue: getMinRotationIntervalMs,
1878+
},
18251879
{
18261880
key: "storageBackupEnabled",
18271881
envNames: ["CODEX_AUTH_STORAGE_BACKUP_ENABLED"],

lib/runtime-rotation-proxy.ts

Lines changed: 120 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ import {
1717
getSessionAffinityMaxEntries,
1818
getSessionAffinityTtlMs,
1919
getStreamStallTimeoutMs,
20+
getMinRotationIntervalMs,
21+
getTokenInvalidationCooldownMs,
2022
getTokenRefreshSkewMs,
2123
loadPluginConfig,
2224
} from "./config.js";
@@ -116,7 +118,25 @@ interface RuntimeRotationAccountIdentity {
116118
const DEFAULT_HOST = "127.0.0.1";
117119
const DEFAULT_QUOTA_REMAINING_THRESHOLD = 10;
118120
const DEFAULT_AUTH_FAILURE_COOLDOWN_MS = 30_000;
121+
119122
const DEFAULT_MAX_RUNTIME_ACCOUNT_ATTEMPTS = 4;
123+
124+
// Phrases observed in upstream 401 response bodies when OpenAI/Microsoft has
125+
// explicitly revoked an OAuth token (as opposed to a generic expired-token 401
126+
// that can be retried after a refresh). Matching is case-insensitive substring.
127+
// If anti-abuse detection triggers different wording in production, add the new
128+
// phrase here and record the source provider and date. See issue #495.
129+
const TOKEN_INVALIDATION_PHRASES = [
130+
"invalidated oauth token",
131+
"authentication token has been invalidated",
132+
"oauth token has been invalidated",
133+
"token has been invalidated",
134+
] as const;
135+
136+
function isTokenInvalidationError(bodyText: string): boolean {
137+
const lower = bodyText.toLowerCase();
138+
return TOKEN_INVALIDATION_PHRASES.some((phrase) => lower.includes(phrase));
139+
}
120140
const MAX_REQUEST_BODY_BYTES = 64 * 1024 * 1024;
121141
const MAX_THREAD_GOAL_FALLBACKS = 512;
122142
const HOP_BY_HOP_HEADERS = new Set([
@@ -636,6 +656,21 @@ function buildUpstreamUrl(
636656
return upstream.toString();
637657
}
638658

659+
// Monotonic auth-failure cooldown: only extend, never shorten. Two concurrent
660+
// requests on the same account can race so that an invalidation path sets a
661+
// long cooldown (5 min) and a subsequent generic 401 truncates it (30 s).
662+
// Reading the live coolingDownUntil before writing prevents that race.
663+
function applyMonotonicAuthCooldown(
664+
accountManager: AccountManager,
665+
account: ManagedAccount,
666+
cooldownMs: number,
667+
): void {
668+
const existing = accountManager.getAccountByIndex(account.index)?.coolingDownUntil ?? 0;
669+
if (Date.now() + cooldownMs > existing) {
670+
accountManager.markAccountCoolingDown(account, cooldownMs, "auth-failure");
671+
}
672+
}
673+
639674
function hasUsableAccessToken(
640675
account: ManagedAccount,
641676
now: number,
@@ -699,8 +734,13 @@ async function ensureFreshAccessToken(params: {
699734
model: string | null;
700735
now: number;
701736
tokenRefreshSkewMs: number;
702-
}): Promise<{ ok: true; accessToken: string; account: ManagedAccount } | { ok: false; retryable: boolean }> {
703-
const { accountManager, account, family, model, now, tokenRefreshSkewMs } = params;
737+
tokenInvalidationCooldownMs: number;
738+
}): Promise<
739+
| { ok: true; accessToken: string; account: ManagedAccount }
740+
| { ok: false; retryable: boolean; invalidated?: boolean }
741+
> {
742+
const { accountManager, account, family, model, now, tokenRefreshSkewMs, tokenInvalidationCooldownMs } =
743+
params;
704744
if (hasUsableAccessToken(account, now, tokenRefreshSkewMs)) {
705745
return { ok: true, accessToken: account.access ?? "", account };
706746
}
@@ -709,13 +749,18 @@ async function ensureFreshAccessToken(params: {
709749
if (refreshResult.type === "failed") {
710750
accountManager.recordFailure(account, family, model);
711751
accountManager.incrementAuthFailures(account);
712-
accountManager.markAccountCoolingDown(
752+
// If the refresh endpoint itself returns an explicit invalidation message
753+
// (e.g. Microsoft/Outlook SSO revokes the refresh token server-side), apply
754+
// the long cooldown and signal to the caller to stop rotating rather than
755+
// presenting other accounts' tokens from the same IP.
756+
const invalidated = isTokenInvalidationError(refreshResult.message ?? "");
757+
applyMonotonicAuthCooldown(
758+
accountManager,
713759
account,
714-
DEFAULT_AUTH_FAILURE_COOLDOWN_MS,
715-
"auth-failure",
760+
invalidated ? tokenInvalidationCooldownMs : DEFAULT_AUTH_FAILURE_COOLDOWN_MS,
716761
);
717762
accountManager.saveToDiskDebounced();
718-
return { ok: false, retryable: isTokenRefreshRetryable(refreshResult) };
763+
return { ok: false, retryable: isTokenRefreshRetryable(refreshResult), invalidated };
719764
}
720765

721766
const auth: OAuthAuthDetails = {
@@ -869,6 +914,7 @@ export function chooseAccount(params: {
869914
policy: RuntimePolicyDecision | null;
870915
pinnedIndex: number | null;
871916
skipReasons?: Map<number, string>;
917+
stickyBoostByAccount?: Record<number, number>;
872918
}): ManagedAccount | null {
873919
const {
874920
accountManager,
@@ -881,6 +927,7 @@ export function chooseAccount(params: {
881927
policy,
882928
pinnedIndex,
883929
skipReasons,
930+
stickyBoostByAccount,
884931
} = params;
885932

886933
// Manual pin (from `codex-multi-auth switch <n>`) overrides every other
@@ -940,7 +987,10 @@ export function chooseAccount(params: {
940987
}
941988

942989
const selected = accountManager.getCurrentOrNextForFamilyHybrid(family, model, {
943-
scoreBoostByAccount: policy?.scoreBoostByAccount,
990+
scoreBoostByAccount: {
991+
...(policy?.scoreBoostByAccount ?? {}),
992+
...(stickyBoostByAccount ?? {}),
993+
},
944994
});
945995
if (
946996
selected &&
@@ -1179,6 +1229,10 @@ export async function startRuntimeRotationProxy(
11791229
const tokenRefreshSkewMs = getTokenRefreshSkewMs(pluginConfig);
11801230
const networkErrorCooldownMs = getNetworkErrorCooldownMs(pluginConfig);
11811231
const serverErrorCooldownMs = getServerErrorCooldownMs(pluginConfig);
1232+
const tokenInvalidationCooldownMs = getTokenInvalidationCooldownMs(pluginConfig);
1233+
const minRotationIntervalMs = getMinRotationIntervalMs(pluginConfig);
1234+
let lastGlobalAccountIndex: number | null = null;
1235+
let lastGlobalSwitchAt = 0;
11821236
const fetchTimeoutMs = options.fetchTimeoutMs ?? getFetchTimeoutMs(pluginConfig);
11831237
const streamStallTimeoutMs =
11841238
options.streamStallTimeoutMs ?? getStreamStallTimeoutMs(pluginConfig);
@@ -1388,6 +1442,12 @@ export async function startRuntimeRotationProxy(
13881442
attemptedIndexes.size < accountCount &&
13891443
transientAttempts < transientAttemptLimit
13901444
) {
1445+
const rotationStickyBoost: Record<number, number> =
1446+
minRotationIntervalMs > 0 &&
1447+
lastGlobalAccountIndex !== null &&
1448+
now() - lastGlobalSwitchAt < minRotationIntervalMs
1449+
? { [lastGlobalAccountIndex]: 1000 }
1450+
: {};
13911451
const selected = chooseAccount({
13921452
accountManager,
13931453
sessionAffinityStore,
@@ -1399,6 +1459,7 @@ export async function startRuntimeRotationProxy(
13991459
policy: policyDecision,
14001460
pinnedIndex,
14011461
skipReasons: accountSkipReasons,
1462+
stickyBoostByAccount: rotationStickyBoost,
14021463
});
14031464
if (!selected) {
14041465
if (
@@ -1445,10 +1506,32 @@ export async function startRuntimeRotationProxy(
14451506
model: context.model,
14461507
now: now(),
14471508
tokenRefreshSkewMs,
1509+
tokenInvalidationCooldownMs,
14481510
});
14491511
if (!refreshed.ok) {
14501512
accountManager.refundToken(selected, context.family, context.model);
14511513
exhaustionReason = "auth-failure";
1514+
if (refreshed.invalidated) {
1515+
// Refresh endpoint explicitly revoked the token. Stop cascade:
1516+
// return auth error to client instead of rotating to the next account.
1517+
sessionAffinityStore?.forgetSession(context.sessionKey);
1518+
res.writeHead(HTTP_STATUS.UNAUTHORIZED, { "content-type": "application/json" });
1519+
res.end(
1520+
JSON.stringify({
1521+
error: {
1522+
message: "OAuth token has been invalidated. Please re-login.",
1523+
code: "token_invalidated",
1524+
},
1525+
}),
1526+
);
1527+
await usageRecorder.record({
1528+
outcome: "failure",
1529+
statusCode: HTTP_STATUS.UNAUTHORIZED,
1530+
errorCode: "token_invalidated",
1531+
account: selected,
1532+
});
1533+
return;
1534+
}
14521535
if (!refreshed.retryable) continue;
14531536
transientAttempts += 1;
14541537
transientExhaustionReason = "auth-failure";
@@ -1643,13 +1726,36 @@ export async function startRuntimeRotationProxy(
16431726
}
16441727

16451728
if (upstream.status === HTTP_STATUS.UNAUTHORIZED) {
1646-
await readErrorBody(upstream);
1729+
const bodyText = await readErrorBody(upstream);
16471730
accountManager.refundToken(refreshed.account, context.family, context.model);
16481731
accountManager.recordFailure(refreshed.account, context.family, context.model);
1649-
accountManager.markAccountCoolingDown(
1732+
if (isTokenInvalidationError(bodyText)) {
1733+
// The upstream explicitly revoked this OAuth token. Applying a long
1734+
// cooldown prevents cascade invalidation: rapidly presenting each
1735+
// account's token from the same IP triggers OpenAI's anti-abuse
1736+
// detection and invalidates them in sequence. Return the 401 directly
1737+
// rather than rotating so the client can prompt for re-login.
1738+
applyMonotonicAuthCooldown(
1739+
accountManager,
1740+
refreshed.account,
1741+
tokenInvalidationCooldownMs,
1742+
);
1743+
sessionAffinityStore?.forgetSession(context.sessionKey);
1744+
accountManager.saveToDiskDebounced();
1745+
res.writeHead(upstream.status, responseHeadersForClient(upstream.headers));
1746+
res.end(bodyText);
1747+
await usageRecorder.record({
1748+
outcome: "failure",
1749+
statusCode: upstream.status,
1750+
errorCode: "token_invalidated",
1751+
account: refreshed.account,
1752+
});
1753+
return;
1754+
}
1755+
applyMonotonicAuthCooldown(
1756+
accountManager,
16501757
refreshed.account,
16511758
DEFAULT_AUTH_FAILURE_COOLDOWN_MS,
1652-
"auth-failure",
16531759
);
16541760
accountManager.saveToDiskDebounced();
16551761
exhaustionReason = "auth-failure";
@@ -1727,6 +1833,10 @@ export async function startRuntimeRotationProxy(
17271833
refreshed.account.index,
17281834
now(),
17291835
);
1836+
if (refreshed.account.index !== lastGlobalAccountIndex) {
1837+
lastGlobalAccountIndex = refreshed.account.index;
1838+
}
1839+
lastGlobalSwitchAt = now();
17301840
}
17311841
await persistRuntimeActiveAccount(
17321842
accountManager,

lib/schemas.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,8 @@ export const PluginConfigSchema = z.object({
7070
proactiveRefreshBufferMs: z.number().min(30_000).optional(),
7171
networkErrorCooldownMs: z.number().min(0).optional(),
7272
serverErrorCooldownMs: z.number().min(0).optional(),
73+
tokenInvalidationCooldownMs: z.number().min(0).optional(),
74+
minRotationIntervalMs: z.number().min(0).optional(),
7375
storageBackupEnabled: z.boolean().optional(),
7476
preemptiveQuotaEnabled: z.boolean().optional(),
7577
preemptiveQuotaRemainingPercent5h: z.number().min(0).max(100).optional(),

test/codex-manager-cli.test.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1175,6 +1175,8 @@ describe("codex manager cli commands", () => {
11751175
proactiveRefreshBufferMs: 300000,
11761176
networkErrorCooldownMs: 6000,
11771177
serverErrorCooldownMs: 4000,
1178+
tokenInvalidationCooldownMs: 300000,
1179+
minRotationIntervalMs: 60000,
11781180
storageBackupEnabled: true,
11791181
preemptiveQuotaEnabled: true,
11801182
preemptiveQuotaRemainingPercent5h: 5,

0 commit comments

Comments
 (0)