Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,6 @@ dotnet run --project EnumerableAsyncProcessor.UnitTests -f net10.0 -- --treenode

TUnit test projects compile to executables; VSTest-style `dotnet test --filter` does not work. See the `tunit-testing` skill for full filter syntax.

**TUnit itself depends on this library.** `TUnit.Engine` is compiled against EnumerableAsyncProcessor v3, and the locally built assembly shadows the copy TUnit shipped with (same assembly identity), so removing or changing a public member that TUnit.Engine binds to crashes test discovery with `MissingMethodException` before any test runs. The exact signatures TUnit needs are pinned by `V3BinaryCompatibilityTests` and marked as binary-compat members in the source — do not remove them until TUnit ships a build compiled against v4.

CI (`.github/workflows/dotnet.yml`) runs the `EnumerableAsyncProcessor.Pipeline` project (a ModularPipelines app, `dotnet run -c Release` from that directory), which builds, tests, packs, and — on `main` — publishes to NuGet. Versioning comes from GitVersion (`GitVersion.yml` pins `next-version: 4.0.0`; keep that file present, its absence makes ModularPipelines generate a Mainline config that crashes on GitHub PR merge commits).

## Architecture
Expand Down
47 changes: 0 additions & 47 deletions EnumerableAsyncProcessor.UnitTests/V3BinaryCompatibilityTests.cs

This file was deleted.

10 changes: 0 additions & 10 deletions EnumerableAsyncProcessor/Builders/ActionAsyncProcessorBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,6 @@ public IAsyncProcessor ProcessInParallel(int? maxConcurrency = null, bool schedu
return new ParallelAsyncProcessor(_count, _taskSelector, _cancellationTokenSource, maxConcurrency, scheduleOnThreadPool).StartProcessing();
}

/// <summary>
/// Processes items in parallel with bounded concurrency. Binary-compatible with assemblies
/// compiled against v3 (equivalent to <c>ProcessInParallel(maxConcurrency: n)</c>).
/// </summary>
[System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)]
public IAsyncProcessor ProcessInParallel(int maxConcurrency)
{
return ProcessInParallel((int?)maxConcurrency);
}

public IAsyncProcessor ProcessOneAtATime()
{
return new OneAtATimeAsyncProcessor(_count, _taskSelector, _cancellationTokenSource).StartProcessing();
Expand Down
10 changes: 0 additions & 10 deletions EnumerableAsyncProcessor/Builders/ActionAsyncProcessorBuilder_1.cs
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,6 @@ public IAsyncProcessor<TOutput> ProcessInParallel(int? maxConcurrency = null, bo
return new ResultParallelAsyncProcessor<TOutput>(_count, _taskSelector, _cancellationTokenSource, maxConcurrency, scheduleOnThreadPool).StartProcessing();
}

/// <summary>
/// Processes items in parallel with bounded concurrency. Binary-compatible with assemblies
/// compiled against v3 (equivalent to <c>ProcessInParallel(maxConcurrency: n)</c>).
/// </summary>
[System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)]
public IAsyncProcessor<TOutput> ProcessInParallel(int maxConcurrency)
{
return ProcessInParallel((int?)maxConcurrency);
}

public IAsyncProcessor<TOutput> ProcessOneAtATime()
{
return new ResultOneAtATimeAsyncProcessor<TOutput>(_count, _taskSelector, _cancellationTokenSource).StartProcessing();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,16 +57,6 @@ public IAsyncProcessor ProcessInParallel(int? maxConcurrency = null, bool schedu
.StartProcessing();
}

/// <summary>
/// Processes items in parallel with bounded concurrency. Binary-compatible with assemblies
/// compiled against v3 (equivalent to <c>ProcessInParallel(maxConcurrency: n)</c>).
/// </summary>
[System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)]
public IAsyncProcessor ProcessInParallel(int maxConcurrency)
{
return ProcessInParallel((int?)maxConcurrency);
}

public IAsyncProcessor ProcessOneAtATime()
{
return new OneAtATimeAsyncProcessor<TInput>(_items, _taskSelector, _cancellationTokenSource)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -83,16 +83,6 @@ public IAsyncProcessor<TOutput> ProcessInParallel(int? maxConcurrency = null, bo
return new ResultParallelAsyncProcessor<TInput, TOutput>(_items, _taskSelector, _cancellationTokenSource, maxConcurrency, scheduleOnThreadPool).StartProcessing();
}

/// <summary>
/// Processes items in parallel with bounded concurrency. Binary-compatible with assemblies
/// compiled against v3 (equivalent to <c>ProcessInParallel(maxConcurrency: n)</c>).
/// </summary>
[System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)]
public IAsyncProcessor<TOutput> ProcessInParallel(int maxConcurrency)
{
return ProcessInParallel((int?)maxConcurrency);
}

/// <summary>
/// Process items one at a time sequentially.
/// </summary>
Expand Down
20 changes: 0 additions & 20 deletions EnumerableAsyncProcessor/Extensions/AsyncEnumerableExtensions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -150,26 +150,6 @@ public static async IAsyncEnumerable<TOutput> SelectManyAsync<T, TOutput>(
}
}

/// <summary>
/// Collects every item from the source into a list. This overload has no work to parallelize;
/// it is kept for binary compatibility with assemblies compiled against v3 (notably TUnit).
/// </summary>
[Obsolete("This overload only collects the stream and parallelizes nothing. Pass a selector, or enumerate the stream directly. It exists for binary compatibility with assemblies compiled against v3.")]
[System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)]
public static async Task<IEnumerable<T>> ProcessInParallel<T>(
this IAsyncEnumerable<T> items,
CancellationToken cancellationToken = default)
{
var results = new List<T>();

await foreach (var item in items.WithCancellation(cancellationToken).ConfigureAwait(false))
{
results.Add(item);
}

return results;
}

/// <summary>
/// Process items in parallel with transformation and return all results as IEnumerable when awaited.
/// </summary>
Expand Down
5 changes: 0 additions & 5 deletions EnumerableAsyncProcessor/PublicAPI.Shipped.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,12 @@ EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder.ActionAsyncProcessorBuilder(int count, System.Func<System.Threading.CancellationToken, System.Threading.Tasks.Task!>! taskSelector, System.Threading.CancellationToken cancellationToken) -> void
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder.ActionAsyncProcessorBuilder(int count, System.Func<System.Threading.Tasks.Task!>! taskSelector, System.Threading.CancellationToken cancellationToken) -> void
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder.ProcessInBatches(int batchSize) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder.ProcessInParallel(int maxConcurrency) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder.ProcessInParallel(int maxConcurrency, System.TimeSpan timeSpan) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder.ProcessInParallel(int permitsPerWindow, System.TimeSpan window, int maxConcurrency) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder.ProcessInParallel(int? maxConcurrency = null, bool scheduleOnThreadPool = false) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder.ProcessOneAtATime() -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder<TOutput>
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder<TOutput>.ProcessInBatches(int batchSize) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor<TOutput>!
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder<TOutput>.ProcessInParallel(int maxConcurrency) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor<TOutput>!
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder<TOutput>.ProcessInParallel(int maxConcurrency, System.TimeSpan timeSpan) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor<TOutput>!
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder<TOutput>.ProcessInParallel(int permitsPerWindow, System.TimeSpan window, int maxConcurrency) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor<TOutput>!
EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder<TOutput>.ProcessInParallel(int? maxConcurrency = null, bool scheduleOnThreadPool = false) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor<TOutput>!
Expand Down Expand Up @@ -48,7 +46,6 @@ EnumerableAsyncProcessor.Builders.ExecutionCountAsyncProcessorBuilder.SelectAsyn
EnumerableAsyncProcessor.Builders.ExecutionCountAsyncProcessorBuilder.SelectAsync<TOutput>(System.Func<System.Threading.Tasks.Task<TOutput>!>! taskSelector, System.Threading.CancellationToken cancellationToken) -> EnumerableAsyncProcessor.Builders.ActionAsyncProcessorBuilder<TOutput>!
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput, TOutput>
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput, TOutput>.ProcessInBatches(int batchSize) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor<TOutput>!
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput, TOutput>.ProcessInParallel(int maxConcurrency) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor<TOutput>!
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput, TOutput>.ProcessInParallel(int maxConcurrency, System.TimeSpan timeSpan) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor<TOutput>!
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput, TOutput>.ProcessInParallel(int permitsPerWindow, System.TimeSpan window, int maxConcurrency) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor<TOutput>!
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput, TOutput>.ProcessInParallel(int? maxConcurrency = null, bool scheduleOnThreadPool = false) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor<TOutput>!
Expand All @@ -57,7 +54,6 @@ EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput>
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput>.ItemActionAsyncProcessorBuilder(System.Collections.Generic.IEnumerable<TInput>! items, System.Func<TInput, System.Threading.CancellationToken, System.Threading.Tasks.Task!>! taskSelector, System.Threading.CancellationToken cancellationToken) -> void
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput>.ItemActionAsyncProcessorBuilder(System.Collections.Generic.IEnumerable<TInput>! items, System.Func<TInput, System.Threading.Tasks.Task!>! taskSelector, System.Threading.CancellationToken cancellationToken) -> void
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput>.ProcessInBatches(int batchSize) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput>.ProcessInParallel(int maxConcurrency) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput>.ProcessInParallel(int maxConcurrency, System.TimeSpan timeSpan) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput>.ProcessInParallel(int permitsPerWindow, System.TimeSpan window, int maxConcurrency) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
EnumerableAsyncProcessor.Builders.ItemActionAsyncProcessorBuilder<TInput>.ProcessInParallel(int? maxConcurrency = null, bool scheduleOnThreadPool = false) -> EnumerableAsyncProcessor.Interfaces.IAsyncProcessor!
Expand Down Expand Up @@ -155,7 +151,6 @@ static EnumerableAsyncProcessor.Extensions.AsyncEnumerableExtensions.ForEachAsyn
static EnumerableAsyncProcessor.Extensions.AsyncEnumerableExtensions.ProcessInParallel<T, TOutput>(this System.Collections.Generic.IAsyncEnumerable<T>! items, System.Func<T, System.Threading.Tasks.Task<TOutput>!>! taskSelector, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable<TOutput>!>!
static EnumerableAsyncProcessor.Extensions.AsyncEnumerableExtensions.ProcessInParallel<T, TOutput>(this System.Collections.Generic.IAsyncEnumerable<T>! items, System.Func<T, System.Threading.Tasks.Task<TOutput>!>! taskSelector, int maxConcurrency, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable<TOutput>!>!
static EnumerableAsyncProcessor.Extensions.AsyncEnumerableExtensions.ProcessInParallel<T, TOutput>(this System.Collections.Generic.IAsyncEnumerable<T>! items, System.Func<T, System.Threading.Tasks.Task<TOutput>!>! taskSelector, int? maxConcurrency, bool scheduleOnThreadPool, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable<TOutput>!>!
static EnumerableAsyncProcessor.Extensions.AsyncEnumerableExtensions.ProcessInParallel<T>(this System.Collections.Generic.IAsyncEnumerable<T>! items, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable<T>!>!
static EnumerableAsyncProcessor.Extensions.AsyncEnumerableExtensions.SelectAsync<T, TOutput>(this System.Collections.Generic.IAsyncEnumerable<T>! items, System.Func<T, System.Threading.CancellationToken, System.Threading.Tasks.Task<TOutput>!>! taskSelector, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> EnumerableAsyncProcessor.Builders.AsyncEnumerableActionAsyncProcessorBuilder<T, TOutput>!
static EnumerableAsyncProcessor.Extensions.AsyncEnumerableExtensions.SelectAsync<T, TOutput>(this System.Collections.Generic.IAsyncEnumerable<T>! items, System.Func<T, System.Threading.Tasks.Task<TOutput>!>! taskSelector, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> EnumerableAsyncProcessor.Builders.AsyncEnumerableActionAsyncProcessorBuilder<T, TOutput>!
static EnumerableAsyncProcessor.Extensions.AsyncEnumerableExtensions.SelectMany<T, TOutput>(this System.Collections.Generic.IAsyncEnumerable<T>! items, System.Func<T, System.Collections.Generic.IAsyncEnumerable<TOutput>!>! selector, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Collections.Generic.IAsyncEnumerable<TOutput>!
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Version 4 is a major release. Review these source and behaviour changes before u

- **.NET 8 is the minimum runtime.** The `net6.0` and `netstandard2.0` targets, the Polyfill dependency, and the `TaskCompletionSource` compatibility shim were removed. The package targets and tests `net8.0`, `net9.0`, and `net10.0`.
- **Parallel processing has one API.** Use `ProcessInParallel(maxConcurrency: 100)` for bounded concurrency and `ProcessInParallel()` for unbounded concurrency. The old non-timed `RateLimitedParallelAsyncProcessor*` types and duplicate overload family were removed. Rename named `levelOfParallelism` arguments to `maxConcurrency` on the `ProcessInParallel` builder family (`InParallelAsync` keeps its `levelOfParallelism` parameter name); replace positional `ProcessInParallel(true)` calls with `ProcessInParallel(scheduleOnThreadPool: true)`.
- **The parameterized no-selector `IAsyncEnumerable<T>.ProcessInParallel(maxConcurrency, ...)` overloads were removed.** They only buffered the stream into a list — their `maxConcurrency`/`scheduleOnThreadPool` parameters had no effect. Pass a selector (`items.ProcessInParallel(item => Task.FromResult(item), maxConcurrency)`) instead. The parameterless `ProcessInParallel(cancellationToken)` overload remains for binary compatibility with v3-compiled assemblies (notably TUnit) and is documented as a plain collect.
- **The no-selector `IAsyncEnumerable<T>.ProcessInParallel(...)` overloads were removed.** They only buffered the stream into a list — their `maxConcurrency`/`scheduleOnThreadPool` parameters had no effect. Pass a selector (`items.ProcessInParallel(item => Task.FromResult(item), maxConcurrency)`) instead, or enumerate the stream directly.
- **`IAsyncEnumerableProcessor` moved to `EnumerableAsyncProcessor.Interfaces`** (previously `EnumerableAsyncProcessor.Extensions`) and now implements `IAsyncDisposable`/`IDisposable`. Processors returned by the `IAsyncEnumerable<T>` builder path dispose their internal resources automatically when `ExecuteAsync` completes; they are single-use.
- **Processor and builder classes are sealed.** None of them were externally subclassable in practice (their pipelines hinge on internal members); v4 makes that explicit.
- **Timed processing is a real start-rate limit.** It now uses a shared token bucket instead of holding each worker slot for at least one window. `ProcessInParallel(permitsPerWindow, window, maxConcurrency)` controls start rate and in-flight concurrency independently. The existing two-argument overload remains and uses its first value for both limits.
Expand Down