feat(proxy): opt-in same-target 429 wait-and-retry before key failover (#487) - #865
Open
harryzhou2000 wants to merge 14 commits into
Open
feat(proxy): opt-in same-target 429 wait-and-retry before key failover (#487)#865harryzhou2000 wants to merge 14 commits into
harryzhou2000 wants to merge 14 commits into
Conversation
lidge-jun#487) Codex never retries HTTP 429 (openai/codex#30471 keeps retry_429=false and the misleading 'exceeded retry limit' error), and single-key pools have no failover, so provider-level retryOn429 waits (Retry-After or fixed interval) and replays the identical pre-stream request on the same key before any key rotation. - types.ts: RateLimitRetryPolicy + OcxProviderConfig.retryOn429 - config.ts: lenient zod validation (strip unknown; typo degrades, never rejects config) - key-failover.ts: rateLimitRetryPolicyFor + rateLimitRetryDelayMs (Retry-After capped at maxIntervalMs) - responses/core.ts: recovery-loop replay before multi-key failover; abort-aware sleep; covers Responses, chat completions, and routed Claude messages - usage/log.ts: AttemptRecoveryKind 'rate-limit-429' - docs-site configuration reference + structure/04 transport note + devlog plan unit - tests: policy unit tests + e2e (single-key replay, passthrough without knob, exhaustion, retry-before-failover ordering) Verified: typecheck, privacy scan, 43/43 retry-related tests; full suite failures are pre-existing on pristine dev in this environment (WS/auth/connection-refused) plus missing-gui-deps artifacts that pass once installed.
📝 WalkthroughWalkthroughAdds configurable, API-key-only HTTP 429 retries. Requests replay on the same key before failover, use bounded ChangesRate-limit retry policy
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant ResponsesCore
participant UpstreamProvider
participant KeyFailover
Client->>ResponsesCore: Send request
ResponsesCore->>UpstreamProvider: Send request with current API key
UpstreamProvider-->>ResponsesCore: Return HTTP 429
ResponsesCore->>ResponsesCore: Wait using retry policy
ResponsesCore->>UpstreamProvider: Replay identical request on same key
UpstreamProvider-->>ResponsesCore: Return success or HTTP 429
ResponsesCore->>KeyFailover: Fail over after retry attempts
KeyFailover-->>ResponsesCore: Provide next key
ResponsesCore-->>Client: Return final response
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
- usage/log: whitelist the rate-limit-429 recovery kind so persisted usage rows and post-restart /api/logs keep the reason - key-failover: gate retryOn429 to key-auth providers (no same-token OAuth replays, no silent no-op on forward passthrough); accept Retry-After 0 as immediate - config: cap maxIntervalMs at the effective 10-minute cooldown ceiling - core: hoist the retry budget outside the recovery loop so 413/401 replays cannot re-arm it; cancel the unread 429 body before aborting on client cancel - gui: add rate-limit-429 (and pre-existing anthropic-oauth-429) to the Logs union - tests: OAuth/forward gating, HTTP-date Retry-After, Retry-After 0, persisted recovery-kind, and a deterministic direct-handler abort test (real-socket disconnect propagation is async in Bun, so the e2e version was timing-flaky) - docs: key-auth-only note, maxIntervalMs cap, abort-propagation nuance, locale rows
… xhigh) - core: the Responses passthrough wire (openai-responses key-auth gateways, e.g. the built-in DeepSeek preset) now replays 429 on the same key pre-relay, before the forward-pool logic; Anthropic terminal-guard continuations replay before key/account failover - images/loop + web-search/loop: same-target replay before on429 key rotation via a new retryOn429Policy dep (abort-aware, heartbeat seams preserved) - key-failover: the fixed fallback is now capped at maxIntervalMs (a single wait never exceeds the cap); authMode local runtimes are gated out alongside oauth/forward, so the knob matches the documented API-key scope exactly - tests: key-auth openai-responses passthrough e2e, terminal-guard continuation replay, image/web-search same-key replay (rotation stays zero), fallback cap, local-mode gating - docs: coverage and cap wording in devlog 010, structure/04, and all five configuration locale rows
harryzhou2000
marked this pull request as ready for review
August 1, 2026 18:54
harryzhou2000
requested review from
Ingwannu,
Wibias and
lidge-jun
as code owners
August 1, 2026 18:54
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@devlog/_plan/260802_429_same_target_retry/010_design.md`:
- Around line 57-67: Update the latency and concurrency sections of the retry
design to distinguish retry-wait time from total request latency, noting that
connection, response, and configured timeout durations also contribute. Revise
the request-volume bound to account for multi-key failover, including the
possibility of exhausting the retry budget on one key before attempting another,
and document the combined bound alongside the existing failover behavior.
- Around line 7-9: Update same-key retry handling around the
continuation-request rebuild in core response processing so every retry replays
a cached, body-safe representation of the identical upstream request, including
serialized body and authentication headers, rather than rebuilding it each time.
Apply this to passthrough Responses and Anthropic terminal continuations; only
rebuild after the target or adapter changes, defining deterministic equivalence
where adapters must rebuild. Extend the rate-limit retry and terminal-guard
tests to assert body and authentication/header equality, not just send counts.
- Around line 32-34: Update rate-limit retry authentication handling to fail
closed: normalize omitted key-provider auth modes to "key", preserve "local" in
providerConfigSeed (including the mapping in derive), and make
rateLimitRetryPolicyFor allow retries only when authMode === "key". Add
regression coverage for omitted, local, and unknown authentication modes while
preserving existing key-provider behavior.
In `@docs-site/src/content/docs/reference/configuration.md`:
- Line 353: Update the retryOn429 documentation across the English, Japanese,
Korean, Russian, and Chinese provider and adapter reference pages. State that it
applies only to authMode: "key" and excludes oauth, forward, and local
providers; document same-key replay, raw openai-responses passthrough,
translated openai-chat/Anthropic requests, and that custom runTurn transports
are excluded. Keep the existing defaults and behavior description consistent
across all pages.
In `@src/images/loop.ts`:
- Around line 467-490: The 429 retry waits must not consume the cumulative
header deadline in either loop. In src/images/loop.ts lines 467-490, update the
flow around the headerDeadline and rateLimitRetryPolicy retry loop so each
deliberate wait is excluded, while preserving the configured retry outcome and
preventing the 504 header-timeout path from firing due to that wait; apply the
identical change in src/web-search/loop.ts lines 361-384 around its
headerDeadline and retry loop.
In `@src/providers/key-failover.ts`:
- Around line 76-96: Ensure local providers cannot enter the key-failover retry
path when authMode is undefined: preserve authMode: "local" through the provider
derivation/routing flow or pass the registry auth kind into
rateLimitRetryPolicyFor, while retaining undefined as the default for custom
API-key providers. Add a regression test covering a local provider configured
with retryOn429: {}.
In `@src/server/responses/core.ts`:
- Around line 1631-1671: Update the retry closure in the passthrough 429 loop to
pass `"rate-limit-429"` to `noteAttemptSend` whenever `fetchWithTransientRetry`
provides no recovery kind, while preserving any recovery kind it does provide.
Keep the existing `rateLimitPolicy` retry flow unchanged.
- Around line 2629-2665: Hoist the rateLimitPolicy and rateLimitRetries
declarations out of the terminal continuation while loop, placing them beside
imageTierBias so the retry budget persists across key rotation, account
rotation, and image-tier 413 continues. Resolve rateLimitPolicy once from the
initial route.provider and preserve the existing retry loop behavior while
preventing the budget from resetting per iteration.
In `@tests/rate-limit-retry.test.ts`:
- Around line 99-159: The retry tests lack coverage proving the retry budget is
scoped per request rather than reset during failover. Add focused regression
coverage near the existing retry-loop tests using a two-key provider pool,
retryOn429 with attempts set to 1, and upstream responses that always return
429; assert the total upstream send count remains within the request-wide bound,
covering both the main recovery loop and terminal-continuation path.
In `@tests/usage-log.test.ts`:
- Around line 39-62: Replace the as never cast in the
persists-the-rate-limit-429-recovery-kind-on-attempts test with a
PersistedUsageEntry-typed object. Keep the recoveryKinds literal type-checked
against the recovery-kind union, and cast only individual fields that genuinely
require it while preserving the existing round-trip assertion.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: a0cd7539-4937-4e9d-bf2a-e78d059e114a
📒 Files selected for processing (22)
devlog/_plan/260802_429_same_target_retry/000_research.mddevlog/_plan/260802_429_same_target_retry/010_design.mddocs-site/src/content/docs/ja/reference/configuration.mddocs-site/src/content/docs/ko/reference/configuration.mddocs-site/src/content/docs/reference/configuration.mddocs-site/src/content/docs/ru/reference/configuration.mddocs-site/src/content/docs/zh-cn/reference/configuration.mdgui/src/pages/Logs.tsxsrc/config.tssrc/images/loop.tssrc/providers/key-failover.tssrc/server/responses/core.tssrc/types.tssrc/usage/log.tssrc/web-search/loop.tsstructure/04_transports-and-sidecars.mdtests/images/loop.test.tstests/rate-limit-retry.test.tstests/server-rate-limit-retry-e2e.test.tstests/terminal-guard-server.test.tstests/usage-log.test.tstests/web-search.test.ts
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a06a416024
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… deadlines, recovery labels) - key-failover: fail closed - only authMode key (or the documented omitted default) may replay; unknown modes rejected; an already-expired HTTP-date Retry-After retries immediately (like Retry-After: 0) - derive: providerConfigSeed preserves the registry auth kind (incl. local) so the gate survives the seed round-trip and routing - core: release the unread 429 body BEFORE every backoff (main loop, passthrough, continuations); passthrough and continuation replay sends record the rate-limit-429 recovery kind; continuation budget hoisted outside the failover loop so rotation/413 cannot re-arm it; continuation waits sleep on the upstream signal so an SSE body-cancel aborts them too - images/web-search loops: release the 429 body before the wait, restart the response-header deadline after each deliberate wait (backoffs never consume the connect budget or surface as 504), and record rate-limit-429 on replay sends - config: load-time degradation for invalid optional retryOn429 fields (warn + drop the field) instead of tripping the whole schema and hiding all providers behind a default config; the management write boundary still rejects - tests: fail-closed auth modes (incl. unknown), expired HTTP-date, per-request budget across 2-key pools (e2e + terminal continuation), byte-identical replay bodies/auth (passthrough + continuation), recovery telemetry in both loops, config load degradation, registry auth-kind preservation, typed usage-log entry - docs: retry-wait vs total-latency and attempts+poolKeys volume bounds, identical-replay equivalence, deadline/backoff behavior in devlog 010 and structure/04; retryOn429 boundary notes in the provider guide and adapter reference (English + ja/ko/ru/zh-cn)
Author
|
All 17 inline review comments are addressed in 58c36c9 (plus docstrings in 0feede1 for the coverage check). Highlights: fail-closed auth (registry now preserves authMode "local"), expired Retry-After dates retry immediately, unread 429 bodies are released before every backoff, rate-limit-429 is recorded on every retry surface, bridge loops restart their header deadline after each wait, the continuation budget is per-request, invalid optional retryOn429 fields degrade at load instead of discarding the config, and replay identity is asserted byte-for-byte in tests. Replies were posted on each thread; the 7 open threads are resolved. The PR remains a draft. |
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
src/server/responses/core.ts (1)
2647-2685: 🩺 Stability & Availability | 🔴 Critical | 🏗️ Heavy liftEmit upstream heartbeats during 429 backoff
The bridge checks upstream activity every 2,000 ms. It increments
stallTickswhen no adapter event arrives and aborts atceil(stallTimeoutSec * 1000 / 2000)ticks. Wire heartbeats do not reset this counter.The retry waits can be silent for up to 600,000 ms. This exceeds the default 300-second core budget and the default 330-second web-search budget.
Apply the fix to these live generators:
src/server/responses/core.ts:2647-2685src/web-search/loop.ts:363-393src/images/loop.ts:469-498Split each
sleepWithAbortcall into chunks shorter than 2,000 ms, such as 1,000 ms. Yield{ type: "heartbeat" }after every chunk, including the final chunk. Add regression coverage for a backoff longer thanstallTimeoutSec. The pre-stream retry loops insrc/server/responses/core.tsdo not require this change.Update the web-search timeout documentation and tests to state that retry backoff remains live through these upstream heartbeats.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/server/responses/core.ts` around lines 2647 - 2685, Update the live retry-backoff loops in src/server/responses/core.ts:2647-2685, src/web-search/loop.ts:363-393, and src/images/loop.ts:469-498 to split sleepWithAbort waits into sub-2,000 ms chunks, yielding a heartbeat after every chunk including the final one, while preserving abort handling; leave pre-stream retry loops unchanged. Add regression coverage for backoffs exceeding stallTimeoutSec, and update web-search timeout documentation and tests to describe continued liveness through upstream heartbeats.src/images/loop.ts (1)
469-476: 🚀 Performance & Scalability | 🟠 Major | ⚡ Quick winKeep the retry budget scoped consistently.
src/images/loop.tsinitializesrateLimitRetriesinsideprepareIterationEvents, while onerunWithImageBridgerequest can execute multiple image-loop iterations. This can re-arm the same-key budget and exceed the documented per-request send bound.
src/images/loop.ts#L469-L476: move the policy and counter to request scope, or explicitly define the budget per iteration.structure/04_transports-and-sidecars.md#L235-L235: update theattempts + poolKeysbound if per-iteration scope is intentional.devlog/_plan/260802_429_same_target_retry/010_design.md#L95-L110: add a multi-iteration image-bridge regression test.#!/usr/bin/env bash set -euo pipefail rg -n -C 8 'prepareIterationEvents|rateLimitRetries|rateLimitRetryPolicy|HARD_CAP|retryOn429Policy' \ src/images/loop.ts tests/images/loop.test.tsAs per path instructions, add a focused regression in the flat Bun tests for this
src/**behavior change.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/images/loop.ts` around lines 469 - 476, Keep the 429 retry policy and counter request-scoped in prepareIterationEvents/runWithImageBridge so multiple image-loop iterations cannot reset the same-key retry budget; preserve the documented per-request send bound. Update structure/04_transports-and-sidecars.md:235 to reflect the chosen request-scoped bound, and extend devlog/_plan/260802_429_same_target_retry/010_design.md:95-110 with a multi-iteration image-bridge regression scenario. Add a focused regression to the flat Bun tests covering this behavior.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@devlog/_plan/260802_429_same_target_retry/010_design.md`:
- Around line 88-89: Align the design and associated tests with the current
implementation: update the expired HTTP-date and numeric Retry-After: 0 cases to
expect the configured fallback interval when the parser returns undefined,
unless the parser and rateLimitRetryDelayMs are intentionally changed to
propagate a zero delay. Keep the documented behavior, parser expectations, and
tests consistent.
In `@docs-site/src/content/docs/ja/reference/adapters.md`:
- Around line 41-43: Update the retry descriptions in
docs-site/src/content/docs/ja/reference/adapters.md:41-43,
docs-site/src/content/docs/ko/reference/adapters.md:48-50,
docs-site/src/content/docs/ru/reference/adapters.md:51-53, and
docs-site/src/content/docs/zh-cn/reference/adapters.md:46-48 to state that
same-key 429 replay occurs before other handling or failover, while preserving
the existing translated behavior and custom runTurn exception.
In `@docs-site/src/content/docs/zh-cn/guides/providers.md`:
- Around line 37-40: Update the Chinese provider guide passage describing
retryOn429 to explicitly state that it is opt-in and disabled by default unless
configured, while preserving the existing API-key-only restriction and OAuth,
forward, and local exclusions. Keep the wording consistent with the
corresponding English provider guide.
In `@src/config.ts`:
- Around line 1135-1149: Update the retryOn429 sanitization around the fields
definition and loop to iterate over all keys in policy, warning and ignoring any
key not in the recognized field set. Preserve the existing validators and
warning behavior for known fields with invalid values, and continue rebuilding
p.retryOn429 from only accepted entries.
In `@src/images/loop.ts`:
- Around line 479-486: Await the unread 429 response body cancellation in the
retry flow around sleepWithAbort, while preserving handling for already-closed
bodies and cancellation failures. Update
devlog/_plan/260802_429_same_target_retry/010_design.md lines 53-55 and
structure/04_transports-and-sidecars.md lines 236-237 only as needed to keep
their “release before backoff” claims accurate after the implementation change.
- Around line 482-498: The retry flow around sleepWithAbort in
src/images/loop.ts lines 482-498 must clear headerDeadline before sleeping,
check signal.aborted immediately afterward and throw the existing 499 LoopError
before telemetry or replay, then create the replacement deadline before
onRateLimitRetrySend and fetchOnce. Update
structure/04_transports-and-sidecars.md lines 238-242 to preserve and document
the 499-before-replay guarantee; no other behavior needs changing.
---
Outside diff comments:
In `@src/images/loop.ts`:
- Around line 469-476: Keep the 429 retry policy and counter request-scoped in
prepareIterationEvents/runWithImageBridge so multiple image-loop iterations
cannot reset the same-key retry budget; preserve the documented per-request send
bound. Update structure/04_transports-and-sidecars.md:235 to reflect the chosen
request-scoped bound, and extend
devlog/_plan/260802_429_same_target_retry/010_design.md:95-110 with a
multi-iteration image-bridge regression scenario. Add a focused regression to
the flat Bun tests covering this behavior.
In `@src/server/responses/core.ts`:
- Around line 2647-2685: Update the live retry-backoff loops in
src/server/responses/core.ts:2647-2685, src/web-search/loop.ts:363-393, and
src/images/loop.ts:469-498 to split sleepWithAbort waits into sub-2,000 ms
chunks, yielding a heartbeat after every chunk including the final one, while
preserving abort handling; leave pre-stream retry loops unchanged. Add
regression coverage for backoffs exceeding stallTimeoutSec, and update
web-search timeout documentation and tests to describe continued liveness
through upstream heartbeats.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9881d8f6-9808-4382-a953-a3a067cfc38b
📒 Files selected for processing (26)
devlog/_plan/260802_429_same_target_retry/010_design.mddocs-site/src/content/docs/guides/providers.mddocs-site/src/content/docs/ja/guides/providers.mddocs-site/src/content/docs/ja/reference/adapters.mddocs-site/src/content/docs/ko/guides/providers.mddocs-site/src/content/docs/ko/reference/adapters.mddocs-site/src/content/docs/reference/adapters.mddocs-site/src/content/docs/ru/guides/providers.mddocs-site/src/content/docs/ru/reference/adapters.mddocs-site/src/content/docs/zh-cn/guides/providers.mddocs-site/src/content/docs/zh-cn/reference/adapters.mdsrc/config.tssrc/images/loop.tssrc/providers/derive.tssrc/providers/key-failover.tssrc/server/responses/core.tssrc/web-search/loop.tsstructure/04_transports-and-sidecars.mdtests/config-user-edits.test.tstests/images/loop.test.tstests/provider-registry-parity.test.tstests/rate-limit-retry.test.tstests/server-rate-limit-retry-e2e.test.tstests/terminal-guard-server.test.tstests/usage-log.test.tstests/web-search.test.ts
…ne race, config warnings, locale docs) - images/web-search loops: AWAIT the unread 429 body cancellation before the backoff; clear the old header deadline BEFORE the sleep; re-check client cancellation after the wait so 499 wins over stale-deadline edges; start the fresh deadline before telemetry and replay - config: sanitizeRetryOn429ForLoad warns about misnamed keys (e.g. attempt) instead of silently dropping them - docs: locale adapter pages state same-key replay runs before other handling/failover; provider guide (English + ja/ko/ru/zh-cn) states retryOn429 is opt-in (absent = off); devlog and structure/04 note the awaited cancellation and the 499-before-replay guarantee - test: config-user-edits covers the misnamed-key drop
harryzhou2000
marked this pull request as draft
August 1, 2026 19:50
harryzhou2000
marked this pull request as ready for review
August 1, 2026 19:57
harryzhou2000
marked this pull request as draft
August 1, 2026 19:59
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
devlog/_plan/260802_429_same_target_retry/010_design.md (1)
47-53: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winDocument the
runTurnexception for bridge retries.
src/images/loop.tsenters theadapter.runTurnbranch at Lines 355-433 and returns before the HTTP 429 retry loop at Lines 477-513. A key-auth customrunTurnadapter therefore does not receive this HTTP retry policy. State that bridge retries apply to HTTP adapters only and exclude customrunTurntransports.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@devlog/_plan/260802_429_same_target_retry/010_design.md` around lines 47 - 53, Update the retry-policy documentation near the bridge retry references to state that image/video bridge retries apply only to HTTP adapters. Explicitly exclude custom adapters using the adapter.runTurn path in src/images/loop.ts, which returns before the HTTP 429 retry loop; do not imply that runTurn transports receive the same wait-and-replay behavior.src/server/responses/core.ts (1)
2668-2707: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winAwait terminal-continuation body cancellation before the backoff.
At Line 2679,
response.body.cancel()is detached withvoid.sleepWithAbort()then starts at Line 2681 while cancellation can still be pending. Under repeated 429 responses, unread bodies can remain active through the retry wait and replay.Await cancellation before sleeping.
Proposed fix
- try { void response.body?.cancel().catch(() => {}); } catch { /* already closed */ } + try { await response.body?.cancel().catch(() => {}); } catch { /* already closed */ }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/server/responses/core.ts` around lines 2668 - 2707, In the terminal-continuation retry loop, update the response body cleanup before sleepWithAbort to await response.body cancellation rather than detaching the promise. Preserve the existing safe handling for absent or already-closed bodies, and ensure the backoff starts only after cancellation has settled.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@devlog/_plan/260802_429_same_target_retry/010_design.md`:
- Around line 47-53: Update the retry-policy documentation near the bridge retry
references to state that image/video bridge retries apply only to HTTP adapters.
Explicitly exclude custom adapters using the adapter.runTurn path in
src/images/loop.ts, which returns before the HTTP 429 retry loop; do not imply
that runTurn transports receive the same wait-and-replay behavior.
In `@src/server/responses/core.ts`:
- Around line 2668-2707: In the terminal-continuation retry loop, update the
response body cleanup before sleepWithAbort to await response.body cancellation
rather than detaching the promise. Preserve the existing safe handling for
absent or already-closed bodies, and ensure the backoff starts only after
cancellation has settled.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 72915daa-eace-482c-83cd-3c1d098cc169
📒 Files selected for processing (24)
devlog/_plan/260802_429_same_target_retry/010_design.mddocs-site/src/content/docs/guides/providers.mddocs-site/src/content/docs/ja/guides/providers.mddocs-site/src/content/docs/ja/reference/adapters.mddocs-site/src/content/docs/ko/guides/providers.mddocs-site/src/content/docs/ko/reference/adapters.mddocs-site/src/content/docs/ru/guides/providers.mddocs-site/src/content/docs/ru/reference/adapters.mddocs-site/src/content/docs/zh-cn/guides/providers.mddocs-site/src/content/docs/zh-cn/reference/adapters.mdsrc/config.tssrc/images/loop.tssrc/providers/derive.tssrc/providers/key-failover.tssrc/server/responses/core.tssrc/types.tssrc/web-search/loop.tsstructure/04_transports-and-sidecars.mdtests/config-user-edits.test.tstests/images/loop.test.tstests/server-rate-limit-retry-e2e.test.tstests/terminal-guard-server.test.tstests/usage-log.test.tstests/web-search.test.ts
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1af83c39cb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… send telemetry, invalid master switch, secret-safe warnings)
harryzhou2000
marked this pull request as ready for review
August 1, 2026 20:17
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d4e1978821
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ncel, post-sleep abort re-checks, pinned xAI req-id)
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
src/server/responses/core.ts (1)
2617-2666: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueReduce the duplicated recovery-label expression in
fetchContinuation.Lines 2638 and 2647 both derive the same label from
replay. The second one additionally merges the transient-retry kind. The logic is correct, but the label rule now lives in two places, so a future change to the label (for example a distinct kind for continuation replays) can easily update only one branch.♻️ Optional consolidation
if (continuationEstimate !== undefined) logCtx.usageLogInputTokens = continuationEstimate; + const replayKind: AttemptRecoveryKind | undefined = replay ? "rate-limit-429" : undefined; try { if (activeAdapter.fetchResponse) { - noteAttemptSend(logCtx.activeAttempt, continuationEstimate, replay ? "rate-limit-429" : undefined); + noteAttemptSend(logCtx.activeAttempt, continuationEstimate, replayKind); return await activeAdapter.fetchResponse(continuationRequest, { @@ return await fetchWithResetRetry( recovery => { - noteAttemptSend(logCtx.activeAttempt, continuationEstimate, recovery ?? (replay ? "rate-limit-429" : undefined)); + noteAttemptSend(logCtx.activeAttempt, continuationEstimate, recovery ?? replayKind);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/server/responses/core.ts` around lines 2617 - 2666, Consolidate the replay-derived recovery label in fetchContinuation by computing it once before the fetch branches, then reuse that value in both noteAttemptSend calls while preserving the fetchWithResetRetry recovery override behavior.src/config.ts (1)
1144-1158: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueInclude the accepted range in the warning for out-of-range numbers.
The validators at Lines 1146-1148 reject both wrong types and out-of-range numbers, but the warning at Line 1157 reports only
typeof value. A user who writes"attempts": 999seesproviders.x.retryOn429.attempts (number) is invalid, which gives no hint about the accepted bound. Numbers and booleans cannot carry secret material, so the range can be stated safely while the value itself stays unlogged.♻️ Proposed diagnostic improvement
- const fields: Array<[string, (value: unknown) => boolean]> = [ - ["enabled", value => typeof value === "boolean"], - ["attempts", value => typeof value === "number" && Number.isInteger(value) && value >= 1 && value <= 20], - ["intervalMs", value => typeof value === "number" && Number.isInteger(value) && value >= 100 && value <= 600_000], - ["maxIntervalMs", value => typeof value === "number" && Number.isInteger(value) && value >= 100 && value <= 600_000], - ["respectRetryAfter", value => typeof value === "boolean"], + const fields: Array<[string, (value: unknown) => boolean, string]> = [ + ["enabled", value => typeof value === "boolean", "expected boolean"], + ["attempts", value => typeof value === "number" && Number.isInteger(value) && value >= 1 && value <= 20, "expected integer 1-20"], + ["intervalMs", value => typeof value === "number" && Number.isInteger(value) && value >= 100 && value <= 600_000, "expected integer 100-600000"], + ["maxIntervalMs", value => typeof value === "number" && Number.isInteger(value) && value >= 100 && value <= 600_000, "expected integer 100-600000"], + ["respectRetryAfter", value => typeof value === "boolean", "expected boolean"], ]; const cleaned: Record<string, unknown> = {}; - for (const [key, isValid] of fields) { + for (const [key, isValid, expectation] of fields) { const value = policyRecord[key]; if (value === undefined) continue; if (isValid(value)) cleaned[key] = value; // Log only the received type, never the value (provider config can hold secrets). - else console.warn(`⚠️ config.json providers.${name}.retryOn429.${key} (${typeof value}) is invalid — ignoring the field`); + else console.warn(`⚠️ config.json providers.${name}.retryOn429.${key} (${typeof value}) is invalid — ${expectation}; ignoring the field`); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/config.ts` around lines 1144 - 1158, Update the validation warning in the retry policy field loop around the `fields` validators to include the accepted ranges for numeric keys such as `attempts`, `intervalMs`, and `maxIntervalMs`, while retaining type-only reporting and never logging the received value. Keep boolean warnings unchanged and preserve the existing validation and field-cleaning behavior.src/images/loop.ts (1)
483-515: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winBound pre-header 429 retries before creating SSE
For non-
runTurnimage requests and all web-search requests,prepareIterationDrainedconsumes the retry heartbeat beforebridgeToResponsesSSEis created. The client receives no response headers while same-target 429 retries wait. The default policy can wait 3 minutes; valid configuration can wait up to 200 minutes (20 × 600_000ms). Bound the aggregate eager-phase wait to the request header budget, or move these retries into the liveproduce()phase. Apply the fix to both loops.runTurnimage requests already skip the eager drain.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/images/loop.ts` around lines 483 - 515, Bound same-target 429 retry waiting during the eager preparation phase so it cannot exceed the request header-timeout budget before SSE creation, or defer the retries into the live produce phase. Apply the corresponding fix to the retry loop in src/images/loop.ts lines 483-515 and src/web-search/loop.ts lines 380-412; preserve the existing runTurn image behavior, which already skips eager draining.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/providers/xai-transport.ts`:
- Around line 135-138: Rename the second parameter of withGeneratedRequestId to
reflect that it receives the already-resolved request ID, whether configured or
generated. Update all references within the helper while preserving the single
fallback in the request setup where configuredRequestId ?? randomUUID() is
computed, ensuring retries continue using the same ID.
In `@tests/config-user-edits.test.ts`:
- Around line 183-185: In the test assertion for the retryOn429 diagnostic,
replace the loose "string" substring check with an assertion anchored to the
retryOn429 field and its parenthesized received type. Keep the secret-redaction
assertion unchanged and ensure the expectation specifically verifies the
type-only warning from loadConfig().
---
Outside diff comments:
In `@src/config.ts`:
- Around line 1144-1158: Update the validation warning in the retry policy field
loop around the `fields` validators to include the accepted ranges for numeric
keys such as `attempts`, `intervalMs`, and `maxIntervalMs`, while retaining
type-only reporting and never logging the received value. Keep boolean warnings
unchanged and preserve the existing validation and field-cleaning behavior.
In `@src/images/loop.ts`:
- Around line 483-515: Bound same-target 429 retry waiting during the eager
preparation phase so it cannot exceed the request header-timeout budget before
SSE creation, or defer the retries into the live produce phase. Apply the
corresponding fix to the retry loop in src/images/loop.ts lines 483-515 and
src/web-search/loop.ts lines 380-412; preserve the existing runTurn image
behavior, which already skips eager draining.
In `@src/server/responses/core.ts`:
- Around line 2617-2666: Consolidate the replay-derived recovery label in
fetchContinuation by computing it once before the fetch branches, then reuse
that value in both noteAttemptSend calls while preserving the
fetchWithResetRetry recovery override behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 3f71039b-4261-460f-853e-cf5dc91ff470
📒 Files selected for processing (11)
src/config.tssrc/images/loop.tssrc/providers/xai-transport.tssrc/server/responses/core.tssrc/web-search/loop.tstests/config-user-edits.test.tstests/images/loop.test.tstests/rate-limit-retry.test.tstests/terminal-guard-server.test.tstests/web-search.test.tstests/xai-transport.test.ts
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/config.ts (1)
1384-1384: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winSanitize
configDiagnosticsFromRawbefore schema validation.loadConfigsanitizes before both parses, butsrc/config.ts:1540-1546does not. An invalidretryOn429then returns the default fallback, andsrc/cli/config-command.ts:101-105can save that fallback over the user's providers and keys. CallsanitizeRetryOn429ForLoad(parsed)afterJSON.parseand add a regression test.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/config.ts` at line 1384, Update the config loading path around configDiagnosticsFromRaw so sanitizeRetryOn429ForLoad(parsed) runs immediately after JSON.parse and before schema validation, matching loadConfig’s sanitization order. Preserve the existing validation and fallback behavior otherwise, and add a regression test confirming invalid retryOn429 does not cause the fallback to overwrite user providers or keys when saving via the config command.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/config.ts`:
- Around line 1163-1166: Update the warning in the retryOn429 unknown-field
handling to apply a log-safe encoding to the result of redactSecretString(key)
before interpolation. Preserve the existing secret redaction and readable
ordinary field names while escaping control and ANSI characters in the logged
field name.
---
Outside diff comments:
In `@src/config.ts`:
- Line 1384: Update the config loading path around configDiagnosticsFromRaw so
sanitizeRetryOn429ForLoad(parsed) runs immediately after JSON.parse and before
schema validation, matching loadConfig’s sanitization order. Preserve the
existing validation and fallback behavior otherwise, and add a regression test
confirming invalid retryOn429 does not cause the fallback to overwrite user
providers or keys when saving via the config command.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 8b7029ea-0a8e-4f20-88be-326c30edde53
📒 Files selected for processing (2)
src/config.tstests/config-user-edits.test.ts
Author
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/config.ts`:
- Around line 1163-1167: Update sanitizeRetryOn429ForLoad() to create one safe
provider-name value using JSON.stringify(redactSecretString(name)), then use
that value in every warning emitted by the function instead of interpolating
name directly. Add coverage verifying secret-shaped provider names are redacted
and control-character names are safely JSON-escaped.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 350a5f0d-56af-496c-a07b-844d73515bd6
📒 Files selected for processing (3)
src/config.tssrc/providers/xai-transport.tstests/config-user-edits.test.ts
Owner
|
Maintainer architecture review on head Verified FIXED: (a) backoff no longer consumes the header deadline in images/web-search loops; (c) retry budget is request-wide across key/account rotation and image-tier continues; (d) auth gating rejects oauth/forward/local and registry preserves Still blocking:
Items 1-2 are the architectural core; everything else is close. If you are short on cycles, say so — a maintainer will take over the remaining items on top of your branch. Thank you for the substantial work here; the resolved set shows the design is sound. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an opt-in, provider-level
retryOn429policy: on HTTP 429 the proxy waits (upstreamRetry-Afteror a fixed interval, capped) and replays the identical pre-stream request on the same key before any multi-key failover.Why
retry_429=falsein codex-rs endpoint configs, surfaced as the misleading "exceeded retry limit" error (HTTP 429 is surfaced as "exceeded retry limit" without retrying; expose retry_429 or clarify error openai/codex#30471, still open; the suggested fix only rewords the message).Retry-Afterso clients that honor it (Claude Code) recover; Codex does not act on it.Behavior
providers.<name>.retryOn429 = { enabled?, attempts?, intervalMs?, maxIntervalMs?, respectRetryAfter? }(defaults: enabled=true, attempts=3, intervalMs=5000, maxIntervalMs=60000, respectRetryAfter=true). Default off when absent → zero behavior change.authMode: "key", or the documented omitted default for custom API-key providers). Fail closed: OAuth/forward credentials are never replayed on the same token; local runtimes (Ollama etc.) have no remote key to preserve; unknown/custom auth modes are rejected rather than guessed at.providerConfigSeednow preserves the registry auth kind (including"local") so the gate survives the seed round-trip.Retry-After.attempts)./v1/responses,/v1/chat/completions, and routed/v1/messages(all enterhandleResponses), plus the other key-auth surfaces that bypass that loop: the Responses passthrough wire (openai-responseskey-auth gateways, e.g. the built-in DeepSeek preset), the image/video bridge and web-search sidecar loops (before theiron429key rotation), and Anthropic terminal-guard continuations (before key/account failover).attemptsbudget.Retry-Afteror the fixed fallback — is capped atmaxIntervalMs; the schema capsmaxIntervalMsat 600000 (the effective per-wait cooldown ceiling). An already-expired HTTP-dateRetry-Afterretries immediately (likeRetry-After: 0).rate-limit-429recovery kind on replay sends; the image/video and web-search bridge loops restart their response-header deadline after each deliberate wait, so backoffs never consume the connect budget or surface as a 504.retryOn429fields with a warning instead of tripping the whole schema (which would hide every provider/key behind a default config); the management write boundary still rejects invalid policies.Files
src/types.ts—RateLimitRetryPolicy+OcxProviderConfig.retryOn429src/config.ts— zod validation (outer provider schema stays passthrough; a typo insideretryOn429degrades, never rejects the whole config),maxIntervalMs≤ 600000src/providers/key-failover.ts— policy normalization gated to key-auth + delay computation (Retry-After seconds/HTTP-date/0, capped)src/providers/derive.ts—providerConfigSeedpreserves the registry auth kind (incl."local")src/server/responses/core.ts— recovery-loop replay before the multi-key failoverwhile; passthrough-wire replay before the forward-pool logic; terminal-guard continuation replay before key/account failover; abort path cancels the unread 429 body firstsrc/images/loop.ts,src/web-search/loop.ts— same-target replay before theiron429key rotation (newretryOn429Policydep)src/usage/log.ts—AttemptRecoveryKindmemberrate-limit-429and its persisted-usage whitelist entrygui/src/pages/Logs.tsx— recovery-kind union includesrate-limit-429(and pre-existinganthropic-oauth-429)docs-siteconfiguration reference (all five locales),structure/04transport note,devlog/_plan/260802_429_same_target_retry/Test plan
bun test tests/rate-limit-retry.test.ts tests/server-rate-limit-retry-e2e.test.ts tests/usage-log.test.ts tests/key-failover.test.ts tests/retry-after-429.test.ts— 79 pass (policy/delay units incl. fail-closed auth gating, expired HTTP-date,Retry-After: 0, fallback cap; persistedrate-limit-429recovery kind; deterministic direct-handler abort test; e2e: replay-to-success with byte-identical bodies, opt-in passthrough, exhaustion, retry-before-failover ordering, key-authopenai-responsespassthrough replay with identical body/auth, per-request budget across a 2-key pool)tests/terminal-guard-server.test.ts); image bridge + web-search loop same-key replay before rotation withrate-limit-429telemetry (tests/images/loop.test.ts,tests/web-search.test.ts); config load degradation (tests/config-user-edits.test.ts); registry auth-kind preservation (tests/provider-registry-parity.test.ts)bun run typecheckbun run privacy:scancd gui && bun run lintdevin this environment (WS/auth/connection-refused) plus GUI tests that pass oncegui/deps are installedCommits
667ad08f— feature implementation2efb887b— audit round-1 fixes (usage-log whitelist, key-auth gating, budget scope, abort body cancel, schema cap, GUI union, docs)a06a4160— audit round-3 xhigh fixes: coverage extended to passthrough wire / image+web-search bridges / terminal continuations, fixed-fallback cap, local-mode gating, docs + tests58c36c9e— review-bot round: fail-closed auth (registry preserves local), expired Retry-After → immediate, release 429 bodies before backoff,rate-limit-429recovery on every surface, bridge header-deadline restart, continuation budget hoisted + upstream-signal sleep, config load degradation, identical-replay + budget regression tests, provider/adapter docs (5 locales)568e565c— review-bot round 2: awaited body-cancel before backoff, stale-deadline cleared pre-sleep + 499 re-check post-sleep, misnamed retryOn429 keys warn at load, opt-in + ordering wording in provider/adapter guides (5 locales)3c19337e— docstring coverage pass on the diff (JSDoc for seeded provider config, responses core helpers, image/web-search iteration prep, terminal-guard continuation, and test helpers)1af83c39— attach JSDoc directly to the remaining diff-touched declarations (terminal-guard continuation, image/web-search fetchOnce, cooldown check, provider-config interface, persisted-usage helper)d4e19788— review round 3: retry budget hoisted per request across bridge iterations, single dispatch-time attempt telemetry (recovery kind passed through), invalidenabledmaster switch discards the whole policy, secret-safe config warnings (path + type only)58bf694f— local audit round (gpt-5.6-terra + DeepSeek-V4-Flash): one request-wide 429 budget shared between the main recovery loop and the terminal-guard continuation; awaited 429-body cancellation before every backoff (all surfaces); post-sleep client-abort re-check before dispatching every replay; xAIx-grok-req-idpinned per logical request so same-key replays are byte-identical; new regression tests (shared continuation budget, continuation abort-during-wait, bridge budget not re-armed after rotation, far-future HTTP-date cap, stable xAI request id)4fee87b5— review round 4: unrecognizedretryOn429field NAMES are redacted before logging (secret-shaped property names become[REDACTED], ordinary typos stay readable)5945711b— review round 4 cleanup: rename the xAI request-id param topinnedRequestId(fallback only at transport resolution); anchor the secret-warning test assertion to the exact field+type diagnostice65106f0— review round 4 follow-up: JSON-escape the redacted field name in load warnings so control-character property names (newline/ANSI) cannot forge log lines89535fb7— review round 5: redact and JSON-escape the PROVIDER name in allretryOn429load warnings (sanitizer runs pre-validation, so the name is untrusted; secret-shaped names log as[REDACTED], control characters escaped)Draft — opened for review, not for merge.