Azure Storage Provider

The Azure Storage provider uses Azure Storage queues, tables, and blobs to persist orchestration state. It's a self-managed option suitable for existing Azure Storage deployments.

When to Use Azure Storage

✅ Good for:

Existing Azure Storage infrastructure
Cost-sensitive workloads with low-to-moderate throughput
Internal Azure services in Ring-1 or lower

⚠️ Consider Durable Task Scheduler instead for:

New projects
Enterprise support requirements
High-throughput scenarios
Zero infrastructure management
Internal Azure services in Ring-2 or higher

Installation

dotnet add package Microsoft.Azure.DurableTask.AzureStorage

Configuration

Basic Setup

using DurableTask.AzureStorage;
using DurableTask.Core;
using Microsoft.Extensions.Logging;

using ILoggerFactory loggerFactory = LoggerFactory.Create(builder =>
{
    builder.AddConsole();
    builder.SetMinimumLevel(LogLevel.Information);
});

var settings = new AzureStorageOrchestrationServiceSettings
{
    StorageConnectionString = "DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;",
    TaskHubName = "MyTaskHub",
    LoggerFactory = loggerFactory
};

var service = new AzureStorageOrchestrationService(settings);
await service.CreateIfNotExistsAsync();

var worker = new TaskHubWorker(service, loggerFactory);
var client = new TaskHubClient(service, loggerFactory: loggerFactory);

Using Managed Identity

using Azure.Identity;
using DurableTask.AzureStorage;
using DurableTask.Core;
using Microsoft.Extensions.Logging;

using ILoggerFactory loggerFactory = LoggerFactory.Create(builder =>
{
    builder.AddConsole();
    builder.SetMinimumLevel(LogLevel.Information);
});

// Uses DefaultAzureCredential
var credential = new DefaultAzureCredential();

var settings = new AzureStorageOrchestrationServiceSettings
{
    TaskHubName = "MyTaskHub",
    StorageAccountClientProvider = new StorageAccountClientProvider("mystorageaccount", credential),
    LoggerFactory = loggerFactory
};

var service = new AzureStorageOrchestrationService(settings);

Tip

For complete runnable examples using managed identity, see the Managed Identity Samples.

Configuration Options

Core Settings

Setting	Description	Default
`TaskHubName`	Name of the task hub (alphanumeric, 3-45 chars)	Required
`StorageConnectionString`	Azure Storage connection string	Required*
`StorageAccountName`	Storage account name (for managed identity)	Required*

*Either StorageConnectionString or StorageAccountName with credentials is required.

Performance Settings

Setting	Description	Default
`PartitionCount`	Number of control queue partitions (1-16)	4
`ControlQueueBufferThreshold`	Max messages prefetched and buffered per partition	64
`MaxConcurrentTaskOrchestrationWorkItems`	Max concurrent orchestrations	100
`MaxConcurrentTaskActivityWorkItems`	Max concurrent activities	10

Partition Management

Setting	Description	Default
`UseTablePartitionManagement`	Use table-based partition management (recommended)	`true`
`UseLegacyPartitionManagement`	Use legacy blob-based partition management	`false`
`LeaseRenewInterval`	Interval for renewing partition leases	10 seconds
`LeaseInterval`	Lease duration before expiration	30 seconds
`LeaseAcquireInterval`	Interval for checking partition balance	10 seconds

Note

Table-based partition management (UseTablePartitionManagement = true) is the default and recommended option. It provides better reliability for partition distribution and uses a {taskhub}Partitions table instead of blob leases. It's also significantly less expensive in terms of Azure Storage operations.

Example Configuration

var settings = new AzureStorageOrchestrationServiceSettings
{
    StorageConnectionString = connectionString,
    TaskHubName = "MyTaskHub",
    
    // Performance tuning
    PartitionCount = 8,
    ControlQueueBufferThreshold = 128,
    MaxConcurrentTaskOrchestrationWorkItems = 200,
    MaxConcurrentTaskActivityWorkItems = 200,
    
    // Lease settings
    LeaseInterval = TimeSpan.FromSeconds(30),
    LeaseRenewInterval = TimeSpan.FromSeconds(10)
};

Architecture

Storage Resources

The Azure Storage provider creates these resources:

Resource Type	Name Pattern	Purpose
Control Queues	`{taskhub}-control-{0..N}`	Orchestration messages
Work Item Queue	`{taskhub}-workitems`	Activity messages
History Table	`{taskhub}History`	Orchestration history
Instances Table	`{taskhub}Instances`	Instance metadata
Partitions Table	`{taskhub}Partitions`	Partition leases (table manager)
Lease Blobs	`{taskhub}-leases/`	Partition leases (blob manager)

Partitioning

The Azure Storage provider uses partitions to distribute orchestration workloads across workers. Each partition corresponds to exactly one control queue.

How Partitioning Works

Orchestrations and entities are assigned to partitions by hashing the instance ID
Instance IDs are random GUIDs by default, ensuring even distribution across partitions
A single orchestration instance is always processed by one partition (and therefore one worker) at a time
The PartitionCount setting (1–16, default 4) determines how many control queues are created

Queue Architecture

The task hub uses two types of queues:

Queue Type	Count	Purpose	Processing
Control queues	`PartitionCount`	Orchestration lifecycle messages	Partitioned — each queue owned by one worker
Work item queue	1	Activity function messages	Shared — all workers compete for messages

┌──────────────────────────────────────────────────────────────────────┐
│                            Task Hub                                  │
│                                                                      │
│  CONTROL QUEUES (partitioned)           WORK ITEM QUEUE (shared)     │
│  ┌──────────────────────────────────┐   ┌─────────────────────────┐  │
│  │ control-00 │ control-01 │ ...   │   │      workitems          │  │
│  │ (Worker A) │ (Worker B) │       │   │                         │  │
│  │            │            │       │   │  All workers compete    │  │
│  │ • Start    │ • Start    │       │   │  for activity messages  │  │
│  │ • Timer    │ • Timer    │       │   │                         │  │
│  │ • Activity │ • Activity │       │   └─────────────────────────┘  │
│  │   complete │   complete │       │              ▲                 │
│  │ • External │ • External │       │              │                 │
│  │   event    │   event    │       │   Activities scheduled by      │
│  └────────────┴────────────┴───────┘   orchestrators go here        │
│         │            │                                              │
│         ▼            ▼                                              │
│  ┌──────────────────────────────────┐                               │
│  │      ORCHESTRATION INSTANCES     │                               │
│  │  Hash(InstanceID) → Partition    │                               │
│  └──────────────────────────────────┘                               │
└──────────────────────────────────────────────────────────────────────┘

Control Queue Messages

Control queues contain orchestration lifecycle messages:

ExecutionStarted — New orchestration started
TaskCompleted — Activity function completed
TimerFired — Durable timer expired
EventRaised — External event received
SubOrchestrationCompleted — Child orchestration completed

When messages are dequeued, up to 32 messages are fetched in a single poll. Messages for the same instance are batched together for efficient processing.

Work Item Queue

The work item queue is a simple, non-partitioned queue for activity function messages:

All workers compete to dequeue activity messages
Activities are stateless — any worker can execute any activity
Activities can scale out infinitely (limited only by worker count)

Partition Count Guidance

Workload	Recommended `PartitionCount`
Development/testing	1–2
Low-to-moderate throughput	4 (default)
High throughput	8–16

Important

Partition count cannot be changed after task hub creation. Set it high enough to accommodate future scale-out needs. The maximum number of workers that can process orchestrations concurrently equals the partition count. Note that higher partition counts increase Azure Storage costs due to more queue and table operations.

Lease Management

Workers compete for partition ownership using one of two partition managers:

Table Partition Manager (Default)

When UseTablePartitionManagement = true (default):

Partition leases are stored in the {taskhub}Partitions table
Uses Azure Table ETags for concurrency control
Provides better reliability due to transactional updates

Blob Partition Manager (Legacy)

When UseTablePartitionManagement = false:

Partition leases are stored as blobs in {taskhub}-leases/
Uses Azure Blob leases for concurrency control
Available in "safe" (UseLegacyPartitionManagement = false) and "legacy" (UseLegacyPartitionManagement = true) variants

Partition lifecycle

Workers acquire leases to claim partition ownership
Leases are renewed at LeaseRenewInterval (default 10s)
Leases expire after LeaseInterval (default 30s) if not renewed
Partitions are automatically balanced across workers

Message Processing

Prefetching: Messages are prefetched from control queues in batches
Batching: Messages for the same instance are grouped together
History fetch: Orchestration history is loaded from Table Storage
Processing: Orchestration code runs with the loaded history
Checkpoint: New history and messages are appended

Checkpoint Order

Checkpoints are written in this order to ensure that no data is lost if there is a failure:

New messages → Storage queues
New history → Table storage
Delete processed messages

Because the checkpoints aren't atomic, duplicates may occur. The replay model handles history duplicates gracefully. Message duplicates may result in activities being executed multiple times if an unexpected failure occurs.

Scaling

Horizontal Scaling

Multiple workers can connect to the same task hub:

// Worker 1, 2, 3... all connect to same task hub
var service = new AzureStorageOrchestrationService(settings);
var worker = new TaskHubWorker(service, loggerFactory);
await worker.StartAsync();

Partitions are automatically distributed across workers.

Partition Count

For high-throughput scenarios, increase partition count:

var settings = new AzureStorageOrchestrationServiceSettings
{
    PartitionCount = 16  // More partitions = more parallelism
};

Warning

Partition count cannot be changed after task hub creation.

Operations

Create Task Hub

await service.CreateIfNotExistsAsync();

Delete Task Hub

await service.DeleteAsync();

Purge History

await service.PurgeOrchestrationHistoryAsync(
    DateTime.UtcNow.AddDays(-30),  // Older than 30 days
    OrchestrationStateTimeRangeFilterType.OrchestrationCompletedTimeFilter);