Skip to content

feat: make retry-policy knobs configurable via connect() (both backends)#433

Merged
msrathore-db merged 1 commit into
mainfrom
feat/configurable-retry-policy-options
Jun 11, 2026
Merged

feat: make retry-policy knobs configurable via connect() (both backends)#433
msrathore-db merged 1 commit into
mainfrom
feat/configurable-retry-policy-options

Conversation

@msrathore-db

Copy link
Copy Markdown
Contributor

Problem

The retry-policy knobs — retryMaxAttempts, retriesTimeout, retryDelayMin, retryDelayMax — lived only in the internal ClientConfig (defaults 5 / 15min / 1s / 60s). The Thrift HttpRetryPolicy reads them, and #420 forwarded them to the kernel on the SEA path via buildKernelRetryOptions(getConfig()). But:

  • they were not declared on ConnectionOptions, and
  • connect() never copied them from user options into ClientConfig (it copies metricView / precision / telemetry / userAgentEntry / customHeaders, but not these).

So a caller passing retryMaxAttempts: 2 had no effect on either backend — the hardcoded defaults always applied. (#420 made SEA consume the same ClientConfig knobs as Thrift — parity of consumption — but did not add a way to set them.)

Confirmed empirically before the fix (persistent-429 fault on SEA):

retryMaxAttempts=2 → 5 attempts   (default, override ignored)
retryMaxAttempts=0 → 5 attempts   (override ignored)

Change

  • Declare the four knobs on ConnectionOptions (with JSDoc).
  • Ingest them into ClientConfig in connect() (per-key narrowed copy, matching the existing metricView / precision / telemetry handling).

Because both backends already read ClientConfig — Thrift via HttpRetryPolicy.getConfig(), SEA via buildKernelRetryOptions(getConfig()) — this single change makes the knobs honored on Thrift and SEA simultaneously, at parity. No kernel change required.

Semantics

retryMaxAttempts=N caps at N total attempts (initial + retries combined); 0/1 both mean a single attempt, no retry. This is consistent across backends — Thrift's HttpRetryPolicy increments attempt then stops at attempt >= retryMaxAttempts, and the kernel converts the total to its after-initial retry count.

Verification

After the fix, end-to-end on SEA (persistent-429 fault, countSeaCalls('ExecuteStatement')):

retryMaxAttempts=2 → 2 attempts
retryMaxAttempts=0 → 1 attempt

(previously both produced the default 5). Plus unit tests asserting connect() ingests the options into ClientConfig and keeps defaults when omitted (2 passing). Source files lint clean.

This pull request and its description were written by Isaac.

`retryMaxAttempts` / `retriesTimeout` / `retryDelayMin` / `retryDelayMax` lived
only in the internal `ClientConfig` (with defaults 5 / 15min / 1s / 60s) and were
consumed by the Thrift `HttpRetryPolicy` and forwarded to the kernel on the SEA
path (#420) — but `connect()` never ingested them from user options, and they
were not declared on `ConnectionOptions`. So a caller setting `retryMaxAttempts`
had no effect on either backend; the hardcoded defaults always applied.

Declare the four knobs on `ConnectionOptions` and copy them into `ClientConfig`
in `connect()` (per-key narrowed copy, matching the metricView / precision /
telemetry knobs). Because both backends already read `ClientConfig` — Thrift via
`HttpRetryPolicy.getConfig()`, SEA via `buildKernelRetryOptions(getConfig())` —
this one change makes the knobs honored on Thrift and SEA simultaneously, at
parity.

Semantics are total-attempts on both backends: `retryMaxAttempts=N` caps at N
total attempts (initial + retries); `0`/`1` both mean a single attempt. Verified
end-to-end on SEA against a persistent-429 fault: `retryMaxAttempts=2` → 2
attempts, `=0` → 1 attempt (previously both produced the default 5). Thrift's
`HttpRetryPolicy` increments-then-checks `attempt >= retryMaxAttempts`, giving
the same total-attempt count.

Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
@msrathore-db msrathore-db added this pull request to the merge queue Jun 11, 2026
Merged via the queue into main with commit 6110300 Jun 11, 2026
25 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants