Skip to content

Commit f02ed62

Browse files
authored
Add allow_parallel to control concurrent task runs (#10)
## Summary Add allow_parallel flag to control whether the same task can run concurrently across multiple instances. The setting follows a cascade resolution: task > task group > global config, defaulting to True (preserving backward compatibility). When set to False, the coordinator checks the Redis running key and skips tasks that are already executing.
1 parent ea63360 commit f02ed62

17 files changed

Lines changed: 628 additions & 456 deletions

File tree

CHANGELOG.md

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,41 @@
11
# Release Notes
22

3+
## Latest
4+
5+
## 1.1.0 - 2026-04-15
6+
7+
### Features
8+
9+
* 🚀 **Parallel execution control (`allow_parallel`)**:
10+
Add configurable `allow_parallel` setting to control whether the same task can run multiple times concurrently.
11+
When set to `False`, the coordinator skips scheduling if the task is already running on any worker (checked via
12+
Redis heartbeat key).
13+
Supports three-level cascade resolution: **task > task group > global config**, the most specific non-null
14+
value wins.
15+
Available on `Config` (global default), `TaskGroup` (group-level override), `@task_group.add_task()` decorator, `task_group.add_dynamic_task()`, and the `POST /tasks` REST API. Dynamic task definitions include the setting in Redis persistence for restart survival.
16+
17+
### Internals
18+
* ⬆️ Bump dependencies in uv.lock file, for dev purposes:
19+
- anyio from 4.12.1 to 4.13.0
20+
- charset-normalizer from 3.4.5 to 3.4.7
21+
- coverage from 7.13.4 to 7.13.5
22+
- fastapi from 0.135.1 to 0.135.3
23+
- mkdocs-get-deps from 0.2.0 to 0.2.2
24+
- mkdocs-material from 9.7.4 to 9.7.6
25+
- platformdirs from 4.9.4 to 4.9.6
26+
- pydantic from 2.12.5 to 2.13.0
27+
- pydantic-core from 2.41.5 to 2.46.0
28+
- pygments from 2.19.2 to 2.20.0
29+
- pymdown-extensions from 10.21 to 10.21.2
30+
- pytest from 9.0.2 to 9.0.3
31+
- redis from 7.3.0 to 7.4.0
32+
- requests from 2.32.5 to 2.33.1
33+
- ruff from 0.15.5 to 0.15.10
34+
- setuptools from 82.0.0 to 82.0.1
35+
- starlette from 0.52.1 to 1.0.0
36+
- termynal from 0.13.1 to 0.14.0
37+
- ty from 0.0.21 to 0.0.29
38+
339
## 1.0.0 - 2026-03-09
440

541
### Features
@@ -33,7 +69,7 @@
3369
* ⬆️ Pre-commit bump uv-pre-commit from 0.9.7 to 0.9.18.
3470
* ⬆️ Pre-commit bump ruff-pre-commit from 0.14.3 to 0.14.10.
3571
* ⬆️ Bump dependencies in uv.lock file, for dev purposes:
36-
* - annotated-doc added 0.0.4
72+
- annotated-doc added 0.0.4
3773
- anyio from 4.11.0 to 4.12.1
3874
- backrefs from 6.1 to 6.2
3975
- certifi from 2026.1.4 to 2026.2.25

docs/docs/features.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,7 @@ Built with production environments in mind, FastAPI Task Manager includes compre
100100
**Built-in safeguards:**
101101

102102
- Exponential backoff retry with configurable delays and per-task overrides
103+
- Per-task parallel execution control (`allow_parallel=False` to prevent overlapping runs)
103104
- Task heartbeat monitoring for crash detection
104105
- Automatic reconciliation of stale and failed tasks
105106
- Health check endpoints for monitoring

docs/docs/learn/api-reference.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,7 @@ Retrieve detailed information for all tasks, including running state and retry b
9696
"retry_backoff": null,
9797
"retry_backoff_max": null,
9898
"dynamic": false,
99+
"allow_parallel": null,
99100
"task_group_name": "My Example Task Group",
100101
"kwargs": null,
101102
"function_name": null,
@@ -246,7 +247,8 @@ Create a new dynamic task from a registered function.
246247
"high_priority": false,
247248
"tags": ["reports"],
248249
"retry_backoff": 5.0,
249-
"retry_backoff_max": 120.0
250+
"retry_backoff_max": 120.0,
251+
"allow_parallel": false
250252
}
251253
```
252254

@@ -262,6 +264,7 @@ Create a new dynamic task from a registered function.
262264
| `tags` | `list[string]` | No | Tags for filtering |
263265
| `retry_backoff` | `float` | No | Per-task initial backoff override |
264266
| `retry_backoff_max` | `float` | No | Per-task max backoff override |
267+
| `allow_parallel` | `bool \| null` | No | Allow concurrent executions (`null` = inherit from group/config) |
265268

266269
**Response model:** `DynamicTaskResponse`
267270

docs/docs/learn/dynamic-tasks.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,22 @@ curl -X POST http://localhost:8000/task-manager/tasks \
148148
}'
149149
```
150150

151+
### Disable Parallel Execution
152+
153+
Set `allow_parallel: false` to prevent a task from being scheduled while a previous execution is still running. This is checked via the Redis heartbeat key, so it works correctly across multiple workers.
154+
155+
```bash
156+
curl -X POST http://localhost:8000/task-manager/tasks \
157+
-H "Content-Type: application/json" \
158+
-d '{
159+
"task_group_name": "Reports",
160+
"function_name": "send_report",
161+
"cron_expression": "*/5 * * * *",
162+
"name": "long_running_report",
163+
"allow_parallel": false
164+
}'
165+
```
166+
151167
### Custom Task Names
152168

153169
If you don't provide a `name`, one is auto-generated from the function name plus a hash of the kwargs and cron expression. Providing explicit names makes tasks easier to manage and monitor.
@@ -189,6 +205,7 @@ The method accepts the same parameters available in the REST API:
189205
| `tags` | `list[str]` | No | Tags for filtering |
190206
| `retry_backoff` | `float` | No | Initial retry delay in seconds |
191207
| `retry_backoff_max` | `float` | No | Maximum retry delay in seconds |
208+
| `allow_parallel` | `bool \| None` | No | Allow concurrent executions (`None` = inherit from group/config) |
192209

193210
The method returns the created `Task` object and raises `RuntimeError` if the function is not registered or the task name is already taken.
194211

docs/docs/learn/getting_started/configurations.md

Lines changed: 67 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -37,24 +37,24 @@ Default value is `500`.
3737
## Redis Configuration
3838

3939
### Redis Host
40-
{* ./docs_src/tutorial/configurations_py310.py ln[10] *}
40+
{* ./docs_src/tutorial/configurations_py310.py ln[11] *}
4141

4242
The hostname or IP address of the Redis server. This is the only **required** configuration parameter with no default value.
4343

4444
### Redis Port
45-
{* ./docs_src/tutorial/configurations_py310.py ln[11] *}
45+
{* ./docs_src/tutorial/configurations_py310.py ln[12] *}
4646

4747
The port number of the Redis server.
4848
Default value is `6379`.
4949

5050
### Redis Password
51-
{* ./docs_src/tutorial/configurations_py310.py ln[12] *}
51+
{* ./docs_src/tutorial/configurations_py310.py ln[13] *}
5252

5353
The password for authenticating with the Redis server. Set to `None` if your Redis instance does not require authentication.
5454
Default value is `None`.
5555

5656
### Redis DB
57-
{* ./docs_src/tutorial/configurations_py310.py ln[13] *}
57+
{* ./docs_src/tutorial/configurations_py310.py ln[14] *}
5858

5959
The Redis database number to use. Redis supports multiple databases (0-15 by default).
6060
Default value is `0`.
@@ -66,13 +66,13 @@ Default value is `0`.
6666
These settings control the core task scheduling loop.
6767

6868
### Poll Interval
69-
{* ./docs_src/tutorial/configurations_py310.py ln[15] *}
69+
{* ./docs_src/tutorial/configurations_py310.py ln[16] *}
7070

7171
The interval (in seconds) between coordinator scheduling cycles. Lower values mean tasks are picked up faster but increase Redis load.
7272
Default value is `0.1`.
7373

7474
### Worker Service Name
75-
{* ./docs_src/tutorial/configurations_py310.py ln[16] *}
75+
{* ./docs_src/tutorial/configurations_py310.py ln[17] *}
7676

7777
The service name used for worker identification. This appears in health check responses and logs to help identify which service a worker belongs to.
7878
Default value is `"fastapi-task-manager"`.
@@ -84,7 +84,7 @@ Default value is `"fastapi-task-manager"`.
8484
These settings control the Redis Streams-based task distribution system.
8585

8686
### Stream Block Timeout
87-
{* ./docs_src/tutorial/configurations_py310.py ln[18] *}
87+
{* ./docs_src/tutorial/configurations_py310.py ln[19] *}
8888

8989
The block timeout (in milliseconds) for `XREADGROUP` when consumers wait for new messages. Higher values reduce Redis round-trips but increase shutdown latency.
9090
Default value is `1000`.
@@ -96,13 +96,13 @@ Default value is `1000`.
9696
FastAPI Task Manager uses distributed leader election via Redis to ensure that only one instance schedules tasks at a time, while all instances can execute them.
9797

9898
### Leader Heartbeat Interval
99-
{* ./docs_src/tutorial/configurations_py310.py ln[20] *}
99+
{* ./docs_src/tutorial/configurations_py310.py ln[21] *}
100100

101101
The interval (in seconds) between leader lock renewals. The leader periodically renews its lock to signal it is still alive.
102102
Default value is `3.0`.
103103

104104
### Leader Retry Interval
105-
{* ./docs_src/tutorial/configurations_py310.py ln[21] *}
105+
{* ./docs_src/tutorial/configurations_py310.py ln[22] *}
106106

107107
The interval (in seconds) between leadership acquisition attempts for follower instances. When a worker is not the leader, it tries to acquire leadership at this interval.
108108
Default value is `5.0`.
@@ -114,7 +114,7 @@ Default value is `5.0`.
114114
The reconciler detects and recovers stale or failed tasks. It runs only on the leader instance.
115115

116116
### Reconciliation Interval
117-
{* ./docs_src/tutorial/configurations_py310.py ln[23] *}
117+
{* ./docs_src/tutorial/configurations_py310.py ln[24] *}
118118

119119
The interval (in seconds) between reconciliation checks. The reconciler scans for overdue or stuck tasks at this interval.
120120
Default value is `30`.
@@ -126,7 +126,7 @@ Default value is `30`.
126126
When a task fails, FastAPI Task Manager applies exponential backoff to delay re-execution. This prevents rapid failure loops and gives external dependencies time to recover.
127127

128128
### Retry Backoff
129-
{* ./docs_src/tutorial/configurations_py310.py ln[25] *}
129+
{* ./docs_src/tutorial/configurations_py310.py ln[26] *}
130130

131131
The initial backoff delay (in seconds) after a task failure. The first retry will be delayed by this amount.
132132
Default value is `1.0`.
@@ -136,7 +136,7 @@ This setting can be overridden on individual tasks via the `retry_backoff` param
136136
////
137137

138138
### Retry Backoff Max
139-
{* ./docs_src/tutorial/configurations_py310.py ln[26] *}
139+
{* ./docs_src/tutorial/configurations_py310.py ln[27] *}
140140

141141
The maximum backoff delay (in seconds). The delay will never exceed this value, regardless of how many consecutive failures occur.
142142
Default value is `60.0`.
@@ -146,19 +146,72 @@ This setting can be overridden on individual tasks via the `retry_backoff_max` p
146146
////
147147

148148
### Retry Backoff Multiplier
149-
{* ./docs_src/tutorial/configurations_py310.py ln[27] *}
149+
{* ./docs_src/tutorial/configurations_py310.py ln[28] *}
150150

151151
The multiplier applied to the current delay after each consecutive failure. For example, with default settings: 1s, 2s, 4s, 8s, 16s, 32s, 60s (capped).
152152
Default value is `2.0`.
153153

154154
---
155155

156+
## Parallel Execution Control
157+
158+
The `allow_parallel` setting controls whether the same task can run multiple times concurrently. When set to `False`, the coordinator will **skip scheduling** the task if it is already running on any worker. This is useful for long-running tasks where overlapping executions would be problematic.
159+
160+
This setting can be configured at **three levels**, with a cascade resolution order: **task > task group > global config**.
161+
162+
- If a task sets `allow_parallel`, that value is used.
163+
- Otherwise, if the task group sets `allow_parallel`, that value is used.
164+
- Otherwise, the global `Config.allow_parallel` value is used (default: `True`).
165+
166+
### Global (Config)
167+
168+
{* ./docs_src/tutorial/configurations_py310.py ln[9] *}
169+
170+
Sets the default for all tasks across the entire application. Default value is `True`.
171+
172+
### Task Group
173+
174+
```python
175+
# All tasks in this group default to no parallel execution
176+
my_tasks = TaskGroup(name="My Tasks", allow_parallel=False)
177+
```
178+
179+
Sets the default for all tasks within the group. Set to `None` (the default) to inherit from the global config.
180+
181+
### Task
182+
183+
```python
184+
# This specific task cannot run in parallel, even if the group allows it
185+
@my_tasks.add_task(
186+
"*/5 * * * *",
187+
name="slow_sync",
188+
allow_parallel=False,
189+
)
190+
async def slow_sync():
191+
await asyncio.sleep(7)
192+
```
193+
194+
Overrides both the group and global setting for this specific task. Set to `None` (the default) to inherit from the task group.
195+
196+
//// tip | Cascade example
197+
With `Config(allow_parallel=True)` and `TaskGroup(allow_parallel=False)`:
198+
199+
- A task with `allow_parallel=True` **will** allow parallel runs (task wins)
200+
- A task with `allow_parallel=None` **will not** allow parallel runs (inherits from group)
201+
////
202+
203+
//// note | Multi-worker safe
204+
This check uses the Redis `running` heartbeat key, which is shared across all workers. Even if the task is running on a different instance, the coordinator will correctly skip scheduling it.
205+
////
206+
207+
---
208+
156209
## Running Heartbeat Configuration
157210

158211
While a task is executing, the worker periodically renews a heartbeat key in Redis. If a worker crashes, the key expires, signaling that the task is no longer being executed.
159212

160213
### Running Heartbeat Interval
161-
{* ./docs_src/tutorial/configurations_py310.py ln[29] *}
214+
{* ./docs_src/tutorial/configurations_py310.py ln[30] *}
162215

163216
The interval (in seconds) between heartbeat renewals while a task is executing. The worker renews the running key at this interval.
164217
Default value is `3.0`.

docs_src/tutorial/configurations_py310.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
concurrent_tasks=2,
77
statistics_redis_expiration=432_000,
88
statistics_history_runs=30,
9+
allow_parallel=True,
910
# --------- Redis config ---------
1011
redis_host="localhost",
1112
redis_port=6379,

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[project]
22
name = "fastapi-task-manager"
3-
version = "1.0.0.post2"
3+
version = "1.1.0"
44
description = "A task manager for FastAPI. Robust Scheduling, Distributed Safety"
55
readme = "README.md"
66
authors = [

src/fastapi_task_manager/config.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ class Config(BaseModel):
77
concurrent_tasks: int = 2
88
statistics_redis_expiration: int = 432_000 # 5 days
99
statistics_history_runs: int = 500
10+
# Global default for allowing parallel executions of the same task.
11+
# Can be overridden at task group or individual task level.
12+
allow_parallel: bool = True
1013
# --------- End of app config variables ---------
1114

1215
# --------- Redis config variables ---------

src/fastapi_task_manager/coordinator.py

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -152,7 +152,9 @@ async def _is_task_due(self, task_group: TaskGroup, task: Task) -> bool:
152152
153153
A task is due if:
154154
1. It's not disabled
155-
2. Its next_run time is in the past (or not set)
155+
2. It's not in backoff
156+
3. If allow_parallel is False, it's not already running
157+
4. Its next_run time is in the past (or not set)
156158
157159
Args:
158160
task_group: The TaskGroup containing the task.
@@ -179,6 +181,24 @@ async def _is_task_due(self, task_group: TaskGroup, task: Task) -> bool:
179181
)
180182
return False
181183

184+
# Resolve allow_parallel with cascade: task > task_group > config
185+
effective_allow_parallel = task.allow_parallel
186+
if effective_allow_parallel is None:
187+
effective_allow_parallel = task_group.allow_parallel
188+
if effective_allow_parallel is None:
189+
effective_allow_parallel = self._task_manager.config.allow_parallel
190+
191+
# When parallel execution is disabled, skip if the task is already running
192+
if not effective_allow_parallel:
193+
running_key = self._keys.running_task_key(task_group.name, task.name)
194+
if await self._redis.exists(running_key):
195+
logger.debug(
196+
"Task %s/%s is already running and allow_parallel=False, skipping",
197+
task_group.name,
198+
task.name,
199+
)
200+
return False
201+
182202
# Check next_run time
183203
next_run_b = await self._redis.get(keys.next_run)
184204
if next_run_b is None:

src/fastapi_task_manager/schema/health.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ class ConfigResponse(BaseModel):
2020
concurrent_tasks: int
2121
statistics_history_runs: int
2222
statistics_redis_expiration: int
23+
allow_parallel: bool
2324

2425
# Runner
2526
poll_interval: float

0 commit comments

Comments
 (0)