PurgeAllInstancesAsync - Durable SQL Provider - Deletes only 1000 instances maximum

[CGSK]

We see a behavior, where, only a 1000 instances have been deleted though there are many eligible instances are available in the 'Instances' table.

Is this behavior a default one?
Is there a way to increase this limit of 1000?

Our intention is to run the PurgeAll once every day but with this restriction we left with millions of records in our sql db, indirectly hindering our running durable orchestrators performance.

[Function("PurgeDurableFunctionsHistory")]
public async Task RunAsync([TimerTrigger("%PurgeDurableFunctionsHistory:Schedule%")] TimerInfo myTimer, [DurableClient] DurableTaskClient client, CancellationToken cancellationToken)
{
PurgeResult result = null;
if (!int.TryParse(Environment.GetEnvironmentVariable("PurgeDurableFunctionsHistoryDays"), out int deletePurgeHistoryDays))
{
deletePurgeHistoryDays = 30;
}

var orchStatus = new List<OrchestrationRuntimeStatus>
{
	OrchestrationRuntimeStatus.Completed,
};
var purgeFilter = new PurgeInstancesFilter(
	DateTime.SpecifyKind(DateTime.MinValue, DateTimeKind.Utc),
	DateTime.SpecifyKind(DateTime.UtcNow.AddDays(deletePurgeHistoryDays * -1), DateTimeKind.Utc),
	orchStatus);

result = await client.PurgeAllInstancesAsync(purgeFilter, cancellationToken).ConfigureAwait(false);
}

NuGet used:

<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.DurableTask" Version="1.1.4" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.DurableTask.SqlServer" Version="1.3.0" />		
<PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.22.0" />

[lilyjma]

@CGSK - We're limiting the purge operation to just 1,000 instances in a single call for the MSSQL backend. The guidance here would be to call DurableTaskClient.PurgeAllInstancesAsync in a loop if you need to purge more. We'll update our docs to call this out.

Closing this issue for now. Please reopen if needed. Thanks!

[lilyjma]

Confirming that this is a bug. Reopening the issue until it's resolved.

[usemam]

@lilyjma @cgillum - tried to address this in #308. Thanks.