Skip to content

Add jittered retry backoff controls#1667

Open
RitwijParmar wants to merge 1 commit into
Portkey-AI:mainfrom
RitwijParmar:codex/portkey-jittered-retry-backoff
Open

Add jittered retry backoff controls#1667
RitwijParmar wants to merge 1 commit into
Portkey-AI:mainfrom
RitwijParmar:codex/portkey-jittered-retry-backoff

Conversation

@RitwijParmar

Copy link
Copy Markdown

Summary

  • default gateway retries to randomized exponential backoff so transient 5xx/429 bursts do not retry on a fixed synchronized schedule
  • add retry config knobs for min_timeout, max_timeout, factor, and randomize, wired through validation, request context, and retryRequest
  • document the bounded backoff config in the automatic retries cookbook

Why

Issue #1404 reports retry-induced p95/p99 tail amplification during upstream turbulence. The previous retryRequest path passed randomize: false to async-retry, which can make many retrying requests fire again on the same deterministic cadence. This gives operators a small, backwards-compatible way to jitter and bound retry delays while preserving the existing attempts/status-code behavior.

Validation

  • npx jest tests/unit/src/handlers/retryHandler.test.ts --runInBand
  • npx prettier --check cookbook/getting-started/automatic-retries-on-failures.md src/handlers/retryHandler.ts src/handlers/handlerUtils.ts src/handlers/services/requestContext.ts src/middlewares/requestValidator/schema/config.ts src/types/requestBody.ts tests/unit/src/handlers/retryHandler.test.ts
  • git diff --check
  • npm run build (passes; existing repo warnings about external deps/circular deps and pre-existing provider type warnings still appear)

Closes #1404

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FYI: SEV-lite — retry-induced tail amplification during upstream 5xx burst

1 participant