Use token-bucket timed rate limiting

[thomhurst]

Summary

• replace per-worker minimum dwell timing with a shared TokenBucketRateLimiter
• limit operation starts to a configured permit count per replenishment window
• add a three-argument builder overload that separates permits per window from maximum in-flight concurrency
• preserve the existing two-argument overload by mapping its parallelism value to both limits
• keep zero-duration windows unthrottled and retain cancellation/validation behavior

Impact

Long-running operations no longer delay future rate permits when callers configure a larger concurrency cap. Timed processors now model start rate independently from work duration, while existing call sites remain source-compatible.

Validation

• legacy timed rate-limit tests pass
• added timing-tolerant coverage for independent concurrency, replenishment burst size, and max-concurrency validation
• dotnet test: 1,617 tests passed across net8.0, net9.0, and net10.0

Closes #335

[greptile-apps]

Greptile Summary

This PR replaces per-worker delay throttling with shared token-bucket rate limiting. The main changes are:

• Separate permits per window from maximum in-flight concurrency.
• Preserve the existing two-argument builder overloads.
• Keep zero-duration windows unthrottled.
• Add validation and timing coverage for the new behavior.

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the updated code.
• Timed processor variants consistently use the shared rate limiter.
• Validation covers the separate concurrency limit.

Important Files Changed

Filename	Overview
EnumerableAsyncProcessor/WorkerPool.cs	Adds shared token-bucket permit acquisition while retaining the unthrottled zero-window path.
EnumerableAsyncProcessor/Builders/ActionAsyncProcessorBuilder.cs	Adds an overload with separate permit and concurrency limits while preserving the existing API.
EnumerableAsyncProcessor/Builders/ItemActionAsyncProcessorBuilder_2.cs	Adds independent rate and concurrency settings for item-to-result processing.
EnumerableAsyncProcessor/RunnableProcessors/TimedRateLimitedParallelAsyncProcessor.cs	Validates and forwards separate permit, window, and concurrency values.
EnumerableAsyncProcessor/RunnableProcessors/ResultProcessors/ResultTimedRateLimitedParallelAsyncProcessor_2.cs	Applies the separate rate and concurrency model to result processing.
EnumerableAsyncProcessor/EnumerableAsyncProcessor.csproj	Adds the rate-limiting package used by the worker pool.
EnumerableAsyncProcessor.UnitTests/WorkerPoolBehaviourTests.cs	Adds coverage for independent concurrency and token replenishment.
EnumerableAsyncProcessor.UnitTests/ValidationRegressionTests.cs	Adds validation coverage for a zero maximum-concurrency argument.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Builder overload] --> B[Timed processor]
    B --> C{Window is zero?}
    C -->|Yes| D[Worker pool without throttling]
    C -->|No| E[Shared token bucket]
    E --> F[Acquire start permit]
    D --> G[Run with concurrency limit]
    F --> G

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart LR
    A[Builder overload] --> B[Timed processor]
    B --> C{Window is zero?}
    C -->|Yes| D[Worker pool without throttling]
    C -->|No| E[Shared token bucket]
    E --> F[Acquire start permit]
    D --> G[Run with concurrency limit]
    F --> G

Loading

_{Reviews (3): Last reviewed commit: "feat: add token-bucket rate limiting" | Re-trigger Greptile}