Increase burst for rate limiter#1280
Conversation
a89cdb6 to
386ec91
Compare
| burst := int(rateLimit) | ||
| if burst < 1 { | ||
| burst = 1 | ||
| } |
There was a problem hiding this comment.
[nit] This makes it clear that fractional limits need to be considered.
Alternatively, add a comment to clarify the intent in the rounding logic.
| burst := int(rateLimit) | |
| if burst < 1 { | |
| burst = 1 | |
| } | |
| burst := int(math.Ceil(rateLimit)) |
| return &ApiClient{ | ||
| config: cfg, | ||
| rateLimiter: rate.NewLimiter(rateLimit, 1), | ||
| rateLimiter: rate.NewLimiter(rateLimit, burst), |
There was a problem hiding this comment.
FWIW, the orDefault call above is duplicated for cfg.RateLimitPerSecond.
It means the rate limit in this field is always > 0, and always an integer.
You can use cfg.RateLimitPerSecond directly.
There was a problem hiding this comment.
Makes sense, thanks. 3b7960d
I removed the duplicated default as well, to make sure the first one applies to both.
|
If integration tests don't run automatically, an authorized user can run them manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
What changes are proposed in this pull request?
Increase burst in rate limiter to the value of RateLimitPerSecond.
The current value of 1 means that even CLI commands that do 2 requests can have a sleep between them. This will increase speed of short-lived commands that do a few requests only.
The burst=1 also means that effective RPS is lower than target 15 RPS because if there is a mix of requests that are longer than 1/15s (default rate limit) and shorter than 1/15s the tokens are wasted.
The burst=1 comes from the initial commit 027c878#diff-99ba62d54caf56e573f16c036acfae28d76ddaace9a914740a10f4e1d50b1bd8R420
which comes from this terraform provider commit databricks/terraform-provider-databricks@6669a94#diff-e24a91e4fac6650a021a72e8527d08f88b5775ee989f109f5c6ead0c3737c7a6R201
How is this tested?
Existing tests.