feat: add token bucket rate limiter for API throttling

[bnusunny]

Adds a thread-safe token bucket rate limiter for throttling AWS API calls. Supports blocking and non-blocking acquire modes.

[bnusunny]

@cortex-assistant Please review this PR.

[bnusunny]

@cortex-assistant Please review this PR.

[bnusunny]

@cortex-assistant Please review this PR.

[bnusunny]

@cortex-assistant Please review this PR.

[cortex-assistant]

Review: feat: add token bucket rate limiter for API throttling

The core token bucket algorithm is well-implemented and the code is clean. However, there are a few issues that should be addressed before merging.

Issues

#	Severity	Description
1	🔴 Critical	rate=0 causes an unhandled ZeroDivisionError on line 41 (1.0 / self._rate). No input validation on rate or capacity.
2	🔴 Major	No unit tests. Every existing utility in samcli/lib/utils/ has a corresponding test file in tests/unit/lib/utils/. A concurrency primitive needs tests for race conditions, timeout behavior, edge cases, and burst behavior.
3	🟡 Major	The poll-sleep loop in acquire() is inefficient. Consider using threading.Condition with wait(timeout) — more idiomatic and avoids wasting CPU cycles. The project already uses threading primitives extensively (see lock_distributor.py).
4	🟡 Minor	Sleep duration doesn't account for remaining time until deadline — can overshoot timeout by up to 0.1s. Should be min(sleep_interval, deadline - time.monotonic()).
5	🟡 Minor	Missing type hints (_refill needs -> None), timeout should be Optional[float], and docstrings on __init__/_refill are missing — inconsistent with project conventions (see lock_distributor.py for reference).

Summary

Please add input validation (raise ValueError for rate <= 0 or capacity <= 0), add unit tests, and consider replacing the spin-loop with threading.Condition. The style nits are minor but worth aligning with the rest of the codebase.

[cortex-assistant]

Critical / Major: No input validation on rate or capacity. rate=0 will cause a ZeroDivisionError on line 41, and negative values produce nonsensical behavior.

Suggested fix:

def __init__(self, rate: float, capacity: int = 10):
    if rate <= 0:
        raise ValueError(f"rate must be positive, got {rate}")
    if capacity <= 0:
        raise ValueError(f"capacity must be positive, got {capacity}")
    ...

Also, consider adding a docstring with Parameters section to match the project convention (see lock_distributor.py).

[cortex-assistant]

Major: This poll-sleep loop is inefficient for a production concurrency primitive. Two issues:

1. 1.0 / self._rate will raise ZeroDivisionError if rate=0 (see input validation comment above).
2. The sleep doesn't account for remaining time until deadline — can overshoot the timeout by up to 0.1s. Should be:

   sleep_time = min(1.0 / self._rate, 0.1)
   if deadline:
       sleep_time = min(sleep_time, max(0, deadline - time.monotonic()))
   time.sleep(sleep_time)

Longer term, consider replacing with threading.Condition + wait(timeout) for a more efficient and idiomatic approach.

[cortex-assistant]

Minor: Type hint should be Optional[float] for clarity:

from typing import Optional

def acquire(self, timeout: Optional[float] = None) -> bool:

This is consistent with the project's use of Optional in other utils.

[cortex-assistant]

I've completed the review of PR #18. Here's a summary of what I found:

Verdict: Request Changes 🔴

The token bucket algorithm itself is solid, but there are a few issues to address:

1. Critical — ZeroDivisionError: No input validation on rate or capacity. Passing rate=0 crashes on 1.0 / self._rate.
2. Major — No unit tests: Every other utility in samcli/lib/utils/ has corresponding tests. A concurrency primitive especially needs them.
3. Major — Inefficient spin-loop: The acquire() method uses time.sleep polling instead of threading.Condition, and the sleep doesn't account for remaining deadline time.
4. Minor — Style inconsistencies: Missing type hints, Optional[float] not used, and missing docstrings compared to project conventions.

I left 3 inline comments on the specific lines and labeled the PR as size:S.