Skip to content

Increase burst for rate limiter#1280

Merged
denik merged 6 commits into
mainfrom
denik/increase-burst
Sep 17, 2025
Merged

Increase burst for rate limiter#1280
denik merged 6 commits into
mainfrom
denik/increase-burst

Conversation

@denik

@denik denik commented Sep 10, 2025

Copy link
Copy Markdown
Contributor

What changes are proposed in this pull request?

Increase burst in rate limiter to the value of RateLimitPerSecond.

The current value of 1 means that even CLI commands that do 2 requests can have a sleep between them. This will increase speed of short-lived commands that do a few requests only.

The burst=1 also means that effective RPS is lower than target 15 RPS because if there is a mix of requests that are longer than 1/15s (default rate limit) and shorter than 1/15s the tokens are wasted.

The burst=1 comes from the initial commit 027c878#diff-99ba62d54caf56e573f16c036acfae28d76ddaace9a914740a10f4e1d50b1bd8R420

which comes from this terraform provider commit databricks/terraform-provider-databricks@6669a94#diff-e24a91e4fac6650a021a72e8527d08f88b5775ee989f109f5c6ead0c3737c7a6R201

How is this tested?

Existing tests.

@denik denik force-pushed the denik/increase-burst branch from a89cdb6 to 386ec91 Compare September 17, 2025 07:54
@denik denik temporarily deployed to test-trigger-is September 17, 2025 07:54 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is September 17, 2025 07:55 — with GitHub Actions Inactive
Comment thread httpclient/api_client.go Outdated
Comment on lines +133 to +136
burst := int(rateLimit)
if burst < 1 {
burst = 1
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] This makes it clear that fractional limits need to be considered.

Alternatively, add a comment to clarify the intent in the rounding logic.

Suggested change
burst := int(rateLimit)
if burst < 1 {
burst = 1
}
burst := int(math.Ceil(rateLimit))

Comment thread httpclient/api_client.go Outdated
return &ApiClient{
config: cfg,
rateLimiter: rate.NewLimiter(rateLimit, 1),
rateLimiter: rate.NewLimiter(rateLimit, burst),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, the orDefault call above is duplicated for cfg.RateLimitPerSecond.

It means the rate limit in this field is always > 0, and always an integer.

You can use cfg.RateLimitPerSecond directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks. 3b7960d

I removed the duplicated default as well, to make sure the first one applies to both.

@github-actions

Copy link
Copy Markdown

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-go

Inputs:

  • PR number: 1280
  • Commit SHA: 3b7960deb95b912208bd5cacb9ec48c714652a98

Checks will be approved automatically on success.

@denik denik added this pull request to the merge queue Sep 17, 2025
Merged via the queue into main with commit a323662 Sep 17, 2025
15 checks passed
@denik denik deleted the denik/increase-burst branch September 17, 2025 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants