Skip to content

sync: add --concurrency and --retry-timeout flags#5201

Closed
luketchang wants to merge 1 commit into
databricks:mainfrom
replit:feat/sync-concurrency-flag
Closed

sync: add --concurrency and --retry-timeout flags#5201
luketchang wants to merge 1 commit into
databricks:mainfrom
replit:feat/sync-concurrency-flag

Conversation

@luketchang

Copy link
Copy Markdown

The sync engine hardcoded its in-flight request limit at 20 and relied on the SDK's retry layer, which only retries 429 and 504 (databricks-sdk-go httpclient/errors.go DefaultErrorRetriable). 502 and 503 surface as fatal errors that fail the entire sync.

  • Add SyncOptions.Concurrency and SyncOptions.RetryTimeout; default to MaxRequestsInFlight and DefaultRetryTimeout respectively when zero so existing in-process callers (bundle deploy/files) need no changes.
  • Wrap each filer call (PUT, delete, mkdir, rmdir) in retries.Poll, retrying on 502/503/504 within the deadline and logging each retry at warn level. Uses the same retries.Poll helper used elsewhere in the CLI for state polling.
  • Surface --concurrency (default 5) and --retry-timeout (default 30s) on 'databricks sync' and 'databricks bundle sync'.

429 stays out of the new retry layer because the SDK already handles it with rate-limit-aware backoff. 500 stays out because it is not always transient for Databricks APIs.

Slack thread:
https://replit.slack.com/archives/C0A2Z9042FR/p1778082184841149

Changes

Why

Tests

The sync engine hardcoded its in-flight request limit at 20 and relied
on the SDK's retry layer, which only retries 429 and 504
(databricks-sdk-go httpclient/errors.go DefaultErrorRetriable). 502 and
503 surface as fatal errors that fail the entire sync.

- Add SyncOptions.Concurrency and SyncOptions.RetryTimeout; default to
  MaxRequestsInFlight and DefaultRetryTimeout respectively when zero so
  existing in-process callers (bundle deploy/files) need no changes.
- Wrap each filer call (PUT, delete, mkdir, rmdir) in retries.Poll,
  retrying on 502/503/504 within the deadline and logging each retry
  at warn level. Uses the same retries.Poll helper used elsewhere in
  the CLI for state polling.
- Surface --concurrency (default 5) and --retry-timeout (default 30s)
  on 'databricks sync' and 'databricks bundle sync'.

429 stays out of the new retry layer because the SDK already handles it
with rate-limit-aware backoff. 500 stays out because it is not always
transient for Databricks APIs.

Slack thread:
https://replit.slack.com/archives/C0A2Z9042FR/p1778082184841149
@luketchang luketchang closed this May 7, 2026
@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 5201
  • Commit SHA: 7566146e6647a5f480ea298e0d45ad4407d75c3b

Checks will be approved automatically on success.

@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/cmd/bundle/ - needs approval

Files: cmd/bundle/sync.go
Suggested: @pietern
Also eligible: @denik, @anton-107, @shreyas-goenka, @andrewnester, @janniklasrose, @lennartkats-db

/cmd/sync/ - needs approval

Files: cmd/sync/sync.go
Suggested: @simonfaltum
Also eligible: @renaudhartert-db, @hectorcast-db, @parthban-db, @tanmay-db, @Divyansh-db, @tejaskochar-db, @mihaimitrea-db, @chrisst, @rauchy

/libs/sync/ - needs approval

4 files changed
Suggested: @simonfaltum
Also eligible: @renaudhartert-db, @hectorcast-db, @parthban-db, @tanmay-db, @Divyansh-db, @tejaskochar-db, @mihaimitrea-db, @chrisst, @rauchy

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

@luketchang luketchang deleted the feat/sync-concurrency-flag branch May 7, 2026 02:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant