Skip to content

Commit 1c74be7

Browse files
authored
refactor!: remove result-discarding InParallelAsync overloads and polish packaging (#373)
* refactor!: remove result-discarding InParallelAsync overloads and polish packaging The Func<TSource, Task<TResult>> and Func<TSource, TResult> overloads returned plain Task and silently dropped every result. Callers that want results already have SelectAsync(...).ProcessInParallel(...); result-returning delegates still bind to the surviving Func<TSource, Task> and Action<TSource> overloads with identical behaviour. Also: XML docs for ParallelExtensions and IAsyncProcessor, NuGet package icon, refreshed package description covering IAsyncEnumerable streaming and timed rate limiting. * docs: correct InParallelAsync migration and cancellation doc claims Async value-returning lambdas and explicitly typed result delegates do not bind to the surviving overloads, and item faults take precedence over cancellation on the returned task.
1 parent 136b15e commit 1c74be7

6 files changed

Lines changed: 81 additions & 56 deletions

File tree

EnumerableAsyncProcessor/EnumerableAsyncProcessor.csproj

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,9 @@
1717
<PropertyGroup>
1818
<PackageLicenseExpression>MIT</PackageLicenseExpression>
1919
<PackageReadmeFile>README.md</PackageReadmeFile>
20+
<PackageIcon>icon.png</PackageIcon>
2021
<Authors>Tom Longhurst</Authors>
21-
<Description>Various Enumerable Async Processors - Batch / Parallel / Rate Limited / One at a time</Description>
22+
<Description>Process async tasks over IEnumerable and IAsyncEnumerable sources with controlled concurrency: one at a time, batched, bounded parallel, timed rate limited (e.g. requests per second), or fully parallel. Fluent API with streaming results, exception fidelity, cancellation and disposal support.</Description>
2223
<RepositoryType>git</RepositoryType>
2324
<PackageProjectUrl>https://github.com/thomhurst/EnumerableAsyncProcessor</PackageProjectUrl>
2425
<RepositoryUrl>https://github.com/thomhurst/EnumerableAsyncProcessor</RepositoryUrl>
@@ -38,6 +39,7 @@
3839
</AssemblyAttribute>
3940

4041
<None Include="$(MSBuildThisFileDirectory)..\README.md" Pack="true" PackagePath="\" />
42+
<None Include="$(MSBuildThisFileDirectory)..\icon.png" Pack="true" PackagePath="\" />
4143

4244
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="10.0.301" PrivateAssets="All"/>
4345
<PackageReference Include="System.Threading.RateLimiting" Version="10.0.10" />

EnumerableAsyncProcessor/Extensions/ParallelExtensions.cs

Lines changed: 39 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -2,25 +2,23 @@
22

33
namespace EnumerableAsyncProcessor.Extensions;
44

5+
/// <summary>
6+
/// Fire-and-await parallel processing over an <see cref="IEnumerable{T}"/> without building a processor object.
7+
/// To collect results, use <c>SelectAsync(...).ProcessInParallel(...)</c> instead.
8+
/// </summary>
59
public static class ParallelExtensions
610
{
7-
public static Task InParallelAsync<TSource, TResult>(
8-
this IEnumerable<TSource> source,
9-
int levelOfParallelism,
10-
Func<TSource, Task<TResult>> taskSelector)
11-
{
12-
return InParallelAsync(source, levelOfParallelism, taskSelector, CancellationToken.None);
13-
}
14-
15-
public static Task InParallelAsync<TSource, TResult>(
16-
this IEnumerable<TSource> source,
17-
int levelOfParallelism,
18-
Func<TSource, Task<TResult>> taskSelector,
19-
CancellationToken cancellationToken)
20-
{
21-
return StartWorkers(source, levelOfParallelism, taskSelector, cancellationToken);
22-
}
23-
11+
/// <summary>
12+
/// Runs <paramref name="taskSelector"/> for every item on a fixed pool of workers.
13+
/// </summary>
14+
/// <typeparam name="TSource">The input item type.</typeparam>
15+
/// <param name="source">The items to process. The sequence is materialized once before workers start.</param>
16+
/// <param name="levelOfParallelism">Maximum concurrent operations. Zero or negative uses <see cref="Environment.ProcessorCount"/>.</param>
17+
/// <param name="taskSelector">The async operation to perform on each item. Any result carried by a returned <c>Task&lt;T&gt;</c> is not collected.</param>
18+
/// <returns>
19+
/// A task that completes when every item has been processed. When multiple items fail, awaiting it
20+
/// throws the first failure while <c>Task.Exception.InnerExceptions</c> carries every failure.
21+
/// </returns>
2422
public static Task InParallelAsync<TSource>(
2523
this IEnumerable<TSource> source,
2624
int levelOfParallelism,
@@ -29,6 +27,18 @@ public static Task InParallelAsync<TSource>(
2927
return InParallelAsync(source, levelOfParallelism, taskSelector, CancellationToken.None);
3028
}
3129

30+
/// <summary>
31+
/// Runs <paramref name="taskSelector"/> for every item on a fixed pool of workers.
32+
/// </summary>
33+
/// <typeparam name="TSource">The input item type.</typeparam>
34+
/// <param name="source">The items to process. The sequence is materialized once before workers start.</param>
35+
/// <param name="levelOfParallelism">Maximum concurrent operations. Zero or negative uses <see cref="Environment.ProcessorCount"/>.</param>
36+
/// <param name="taskSelector">The async operation to perform on each item. Any result carried by a returned <c>Task&lt;T&gt;</c> is not collected.</param>
37+
/// <param name="cancellationToken">Stops claiming new items when cancelled. The returned task then completes as canceled, unless items had already failed - their exceptions take precedence.</param>
38+
/// <returns>
39+
/// A task that completes when every item has been processed. When multiple items fail, awaiting it
40+
/// throws the first failure while <c>Task.Exception.InnerExceptions</c> carries every failure.
41+
/// </returns>
3242
public static Task InParallelAsync<TSource>(
3343
this IEnumerable<TSource> source,
3444
int levelOfParallelism,
@@ -38,24 +48,18 @@ public static Task InParallelAsync<TSource>(
3848
return StartWorkers(source, levelOfParallelism, taskSelector, cancellationToken);
3949
}
4050

41-
// Overloads for CPU-bound processing
42-
public static Task InParallelAsync<TSource, TResult>(
43-
this IEnumerable<TSource> source,
44-
int levelOfParallelism,
45-
Func<TSource, TResult> taskSelector,
46-
CancellationToken cancellationToken = default)
47-
{
48-
return StartWorkers(
49-
source,
50-
levelOfParallelism,
51-
item =>
52-
{
53-
_ = taskSelector(item);
54-
return Task.CompletedTask;
55-
},
56-
cancellationToken);
57-
}
58-
51+
/// <summary>
52+
/// Runs a synchronous, CPU-bound <paramref name="taskSelector"/> for every item on a fixed pool of thread-pool workers.
53+
/// </summary>
54+
/// <typeparam name="TSource">The input item type.</typeparam>
55+
/// <param name="source">The items to process. The sequence is materialized once before workers start.</param>
56+
/// <param name="levelOfParallelism">Maximum concurrent operations. Zero or negative uses <see cref="Environment.ProcessorCount"/>.</param>
57+
/// <param name="taskSelector">The synchronous operation to perform on each item.</param>
58+
/// <param name="cancellationToken">Stops claiming new items when cancelled. The returned task then completes as canceled, unless items had already failed - their exceptions take precedence.</param>
59+
/// <returns>
60+
/// A task that completes when every item has been processed. When multiple items fail, awaiting it
61+
/// throws the first failure while <c>Task.Exception.InnerExceptions</c> carries every failure.
62+
/// </returns>
5963
public static Task InParallelAsync<TSource>(
6064
this IEnumerable<TSource> source,
6165
int levelOfParallelism,
Lines changed: 37 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,45 @@
1-
using System.Runtime.CompilerServices;
1+
using System.Runtime.CompilerServices;
22

33
namespace EnumerableAsyncProcessor.Interfaces;
44

5+
/// <summary>
6+
/// Represents an async processor that performs operations without returning results.
7+
/// </summary>
8+
/// <remarks>
9+
/// This interface implements both IDisposable and IAsyncDisposable. Processors should be properly disposed
10+
/// to ensure cleanup of internal resources and cancellation of running tasks.
11+
/// Use 'await using var processor = ...' for automatic disposal, or call DisposeAsync() manually.
12+
/// </remarks>
513
public interface IAsyncProcessor : IAsyncDisposable, IDisposable
614
{
7-
/**
8-
* <summary>
9-
* A collection of all the asynchronous Tasks, which could be pending or complete.
10-
* </summary>
11-
*/
12-
IEnumerable<Task> GetEnumerableTasks();
15+
/// <summary>
16+
/// Gets a collection of all the asynchronous Tasks, which could be pending or complete.
17+
/// Tasks appear in source order.
18+
/// </summary>
19+
/// <returns>An enumerable of tasks that may be running, completed, or faulted.</returns>
20+
IEnumerable<Task> GetEnumerableTasks();
1321

14-
TaskAwaiter GetAwaiter();
22+
/// <summary>
23+
/// Gets a task awaiter for awaiting completion of all operations directly.
24+
/// </summary>
25+
/// <returns>A task awaiter that completes when all processing is done.</returns>
26+
TaskAwaiter GetAwaiter();
1527

16-
Task WaitAsync();
28+
/// <summary>
29+
/// Gets a task that completes when all operations have finished.
30+
/// </summary>
31+
/// <returns>
32+
/// A task that completes when all processing is done. When multiple items fail, awaiting it
33+
/// throws the first failure while <c>Task.Exception.InnerExceptions</c> carries every failure.
34+
/// </returns>
35+
Task WaitAsync();
1736

18-
/**
19-
* <summary>
20-
* Try to cancel all un-finished tasks
21-
* </summary>
22-
*/
23-
void CancelAll();
24-
}
37+
/// <summary>
38+
/// Attempts to cancel all unfinished tasks.
39+
/// </summary>
40+
/// <remarks>
41+
/// This method is called automatically during disposal but can be called manually
42+
/// if you need to cancel processing before disposal.
43+
/// </remarks>
44+
void CancelAll();
45+
}

EnumerableAsyncProcessor/PublicAPI.Shipped.txt

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -169,9 +169,6 @@ static EnumerableAsyncProcessor.Extensions.EnumerableExtensions.SelectManyAsync<
169169
static EnumerableAsyncProcessor.Extensions.EnumerableExtensions.SelectManyAsync<T, TOutput>(this System.Collections.Generic.IEnumerable<T>! items, System.Func<T, System.Threading.Tasks.Task<System.Collections.Generic.IEnumerable<TOutput>!>!>! selector, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Collections.Generic.IAsyncEnumerable<TOutput>!
170170
static EnumerableAsyncProcessor.Extensions.EnumerableExtensions.SelectManyAsync<T, TOutput>(this System.Collections.Generic.IEnumerable<T>! items, System.Func<T, System.Threading.Tasks.Task<TOutput[]!>!>! selector, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Collections.Generic.IAsyncEnumerable<TOutput>!
171171
static EnumerableAsyncProcessor.Extensions.EnumerableExtensions.ToAsyncProcessorBuilder<T>(this System.Collections.Generic.IEnumerable<T>! items) -> EnumerableAsyncProcessor.Builders.ItemAsyncProcessorBuilder<T>!
172-
static EnumerableAsyncProcessor.Extensions.ParallelExtensions.InParallelAsync<TSource, TResult>(this System.Collections.Generic.IEnumerable<TSource>! source, int levelOfParallelism, System.Func<TSource, System.Threading.Tasks.Task<TResult>!>! taskSelector) -> System.Threading.Tasks.Task!
173-
static EnumerableAsyncProcessor.Extensions.ParallelExtensions.InParallelAsync<TSource, TResult>(this System.Collections.Generic.IEnumerable<TSource>! source, int levelOfParallelism, System.Func<TSource, System.Threading.Tasks.Task<TResult>!>! taskSelector, System.Threading.CancellationToken cancellationToken) -> System.Threading.Tasks.Task!
174-
static EnumerableAsyncProcessor.Extensions.ParallelExtensions.InParallelAsync<TSource, TResult>(this System.Collections.Generic.IEnumerable<TSource>! source, int levelOfParallelism, System.Func<TSource, TResult>! taskSelector, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!
175172
static EnumerableAsyncProcessor.Extensions.ParallelExtensions.InParallelAsync<TSource>(this System.Collections.Generic.IEnumerable<TSource>! source, int levelOfParallelism, System.Action<TSource>! taskSelector, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) -> System.Threading.Tasks.Task!
176173
static EnumerableAsyncProcessor.Extensions.ParallelExtensions.InParallelAsync<TSource>(this System.Collections.Generic.IEnumerable<TSource>! source, int levelOfParallelism, System.Func<TSource, System.Threading.Tasks.Task!>! taskSelector) -> System.Threading.Tasks.Task!
177174
static EnumerableAsyncProcessor.Extensions.ParallelExtensions.InParallelAsync<TSource>(this System.Collections.Generic.IEnumerable<TSource>! source, int levelOfParallelism, System.Func<TSource, System.Threading.Tasks.Task!>! taskSelector, System.Threading.CancellationToken cancellationToken) -> System.Threading.Tasks.Task!

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,8 @@ Version 4 is a major release. Review these source and behaviour changes before u
3535
- **Arbitrary upper limits were removed.** Task counts and batch sizes may exceed 10,000, and time windows may exceed 24 hours. Validity checks remain: counts, batch sizes, concurrency, and permit counts must be positive; time windows cannot be negative.
3636
- **Validation is consistent and eager.** Invalid `maxConcurrency`, parallelism, batch, and timed-rate arguments now throw while the processor is built for both action and result variants.
3737
- **`TaskWrapper` types and processor plumbing are internal.** The `ActionTaskWrapper`/`ItemTaskWrapper` structs and the previously `protected` members of the abstract processor base classes (`TaskWrappers`, `EnumerableTaskCompletionSources`, `CancellationToken`, constructors) were implementation details that could not be meaningfully used outside the library, and are no longer public.
38-
- **Synchronous `InParallelAsync` delegates now run concurrently.** The `Func<TSource, TResult>` and `Action<TSource>` overloads use thread-pool workers instead of executing delegates sequentially on the caller. Do not depend on their previous thread affinity or execution order.
38+
- **Synchronous `InParallelAsync` delegates now run concurrently.** The `Action<TSource>` overload uses thread-pool workers instead of executing delegates sequentially on the caller. Do not depend on its previous thread affinity or execution order.
39+
- **The result-selector `InParallelAsync` overloads were removed.** The `Func<TSource, Task<TResult>>` and `Func<TSource, TResult>` overloads returned a plain `Task` and silently discarded every result. Use `SelectAsync(...).ProcessInParallel(...)` to collect results. Fire-and-await callers can keep non-async lambdas returning `Task<T>` (they bind to the `Func<TSource, Task>` overload), but async lambdas that return a value and explicitly typed result-returning delegates must be rewritten to discard the result (e.g. `async x => { _ = await GetAsync(x); }`).
3940
- **An already-cancelled token yields cancelled tasks, not `ArgumentException`.** Building a processor with a token that is already cancelled (or that cancels between building and the terminal call) now produces a processor whose per-item tasks are cancelled; `WaitAsync`/`GetResultsAsync` throw `OperationCanceledException`. Previously this threw `ArgumentException` from an internal constructor.
4041
- **`IAsyncEnumerable<T>` processors preserve every failure.** The `Task` returned by `ExecuteAsync()` now carries all failures via `Task.Exception.InnerExceptions` (awaiting still throws the first), matching the `IEnumerable<T>` processors. Previously only the first failure was observable and the rest were lost.
4142
- **`ExecuteAsync()` enforces single use.** Calling it a second time throws `InvalidOperationException`; calling it after disposal throws `ObjectDisposedException`. Previously a second call failed with a confusing `ObjectDisposedException` from internal plumbing.

icon.png

8.5 KB
Loading

0 commit comments

Comments
 (0)