You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add allow_parallel to control concurrent task runs (#10)
## Summary
Add allow_parallel flag to control whether the same task can run
concurrently across multiple instances. The setting follows a cascade
resolution: task > task group > global config, defaulting to True
(preserving backward compatibility). When set to False, the coordinator
checks the Redis running key and skips tasks that are already executing.
Copy file name to clipboardExpand all lines: CHANGELOG.md
+37-1Lines changed: 37 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,41 @@
1
1
# Release Notes
2
2
3
+
## Latest
4
+
5
+
## 1.1.0 - 2026-04-15
6
+
7
+
### Features
8
+
9
+
* 🚀 **Parallel execution control (`allow_parallel`)**:
10
+
Add configurable `allow_parallel` setting to control whether the same task can run multiple times concurrently.
11
+
When set to `False`, the coordinator skips scheduling if the task is already running on any worker (checked via
12
+
Redis heartbeat key).
13
+
Supports three-level cascade resolution: **task > task group > global config**, the most specific non-null
14
+
value wins.
15
+
Available on `Config` (global default), `TaskGroup` (group-level override), `@task_group.add_task()` decorator, `task_group.add_dynamic_task()`, and the `POST /tasks` REST API. Dynamic task definitions include the setting in Redis persistence for restart survival.
16
+
17
+
### Internals
18
+
* ⬆️ Bump dependencies in uv.lock file, for dev purposes:
19
+
- anyio from 4.12.1 to 4.13.0
20
+
- charset-normalizer from 3.4.5 to 3.4.7
21
+
- coverage from 7.13.4 to 7.13.5
22
+
- fastapi from 0.135.1 to 0.135.3
23
+
- mkdocs-get-deps from 0.2.0 to 0.2.2
24
+
- mkdocs-material from 9.7.4 to 9.7.6
25
+
- platformdirs from 4.9.4 to 4.9.6
26
+
- pydantic from 2.12.5 to 2.13.0
27
+
- pydantic-core from 2.41.5 to 2.46.0
28
+
- pygments from 2.19.2 to 2.20.0
29
+
- pymdown-extensions from 10.21 to 10.21.2
30
+
- pytest from 9.0.2 to 9.0.3
31
+
- redis from 7.3.0 to 7.4.0
32
+
- requests from 2.32.5 to 2.33.1
33
+
- ruff from 0.15.5 to 0.15.10
34
+
- setuptools from 82.0.0 to 82.0.1
35
+
- starlette from 0.52.1 to 1.0.0
36
+
- termynal from 0.13.1 to 0.14.0
37
+
- ty from 0.0.21 to 0.0.29
38
+
3
39
## 1.0.0 - 2026-03-09
4
40
5
41
### Features
@@ -33,7 +69,7 @@
33
69
* ⬆️ Pre-commit bump uv-pre-commit from 0.9.7 to 0.9.18.
34
70
* ⬆️ Pre-commit bump ruff-pre-commit from 0.14.3 to 0.14.10.
35
71
* ⬆️ Bump dependencies in uv.lock file, for dev purposes:
Copy file name to clipboardExpand all lines: docs/docs/learn/dynamic-tasks.md
+17Lines changed: 17 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -148,6 +148,22 @@ curl -X POST http://localhost:8000/task-manager/tasks \
148
148
}'
149
149
```
150
150
151
+
### Disable Parallel Execution
152
+
153
+
Set `allow_parallel: false` to prevent a task from being scheduled while a previous execution is still running. This is checked via the Redis heartbeat key, so it works correctly across multiple workers.
154
+
155
+
```bash
156
+
curl -X POST http://localhost:8000/task-manager/tasks \
157
+
-H "Content-Type: application/json" \
158
+
-d '{
159
+
"task_group_name": "Reports",
160
+
"function_name": "send_report",
161
+
"cron_expression": "*/5 * * * *",
162
+
"name": "long_running_report",
163
+
"allow_parallel": false
164
+
}'
165
+
```
166
+
151
167
### Custom Task Names
152
168
153
169
If you don't provide a `name`, one is auto-generated from the function name plus a hash of the kwargs and cron expression. Providing explicit names makes tasks easier to manage and monitor.
@@ -189,6 +205,7 @@ The method accepts the same parameters available in the REST API:
189
205
|`tags`|`list[str]`| No | Tags for filtering |
190
206
|`retry_backoff`|`float`| No | Initial retry delay in seconds |
191
207
|`retry_backoff_max`|`float`| No | Maximum retry delay in seconds |
208
+
|`allow_parallel`|`bool \| None`| No | Allow concurrent executions (`None` = inherit from group/config) |
192
209
193
210
The method returns the created `Task` object and raises `RuntimeError` if the function is not registered or the task name is already taken.
The block timeout (in milliseconds) for `XREADGROUP` when consumers wait for new messages. Higher values reduce Redis round-trips but increase shutdown latency.
90
90
Default value is `1000`.
@@ -96,13 +96,13 @@ Default value is `1000`.
96
96
FastAPI Task Manager uses distributed leader election via Redis to ensure that only one instance schedules tasks at a time, while all instances can execute them.
The interval (in seconds) between leadership acquisition attempts for follower instances. When a worker is not the leader, it tries to acquire leadership at this interval.
108
108
Default value is `5.0`.
@@ -114,7 +114,7 @@ Default value is `5.0`.
114
114
The reconciler detects and recovers stale or failed tasks. It runs only on the leader instance.
The interval (in seconds) between reconciliation checks. The reconciler scans for overdue or stuck tasks at this interval.
120
120
Default value is `30`.
@@ -126,7 +126,7 @@ Default value is `30`.
126
126
When a task fails, FastAPI Task Manager applies exponential backoff to delay re-execution. This prevents rapid failure loops and gives external dependencies time to recover.
The multiplier applied to the current delay after each consecutive failure. For example, with default settings: 1s, 2s, 4s, 8s, 16s, 32s, 60s (capped).
152
152
Default value is `2.0`.
153
153
154
154
---
155
155
156
+
## Parallel Execution Control
157
+
158
+
The `allow_parallel` setting controls whether the same task can run multiple times concurrently. When set to `False`, the coordinator will **skip scheduling** the task if it is already running on any worker. This is useful for long-running tasks where overlapping executions would be problematic.
159
+
160
+
This setting can be configured at **three levels**, with a cascade resolution order: **task > task group > global config**.
161
+
162
+
- If a task sets `allow_parallel`, that value is used.
163
+
- Otherwise, if the task group sets `allow_parallel`, that value is used.
164
+
- Otherwise, the global `Config.allow_parallel` value is used (default: `True`).
Sets the default for all tasks within the group. Set to `None` (the default) to inherit from the global config.
180
+
181
+
### Task
182
+
183
+
```python
184
+
# This specific task cannot run in parallel, even if the group allows it
185
+
@my_tasks.add_task(
186
+
"*/5 * * * *",
187
+
name="slow_sync",
188
+
allow_parallel=False,
189
+
)
190
+
asyncdefslow_sync():
191
+
await asyncio.sleep(7)
192
+
```
193
+
194
+
Overrides both the group and global setting for this specific task. Set to `None` (the default) to inherit from the task group.
195
+
196
+
//// tip | Cascade example
197
+
With `Config(allow_parallel=True)` and `TaskGroup(allow_parallel=False)`:
198
+
199
+
- A task with `allow_parallel=True`**will** allow parallel runs (task wins)
200
+
- A task with `allow_parallel=None`**will not** allow parallel runs (inherits from group)
201
+
////
202
+
203
+
//// note | Multi-worker safe
204
+
This check uses the Redis `running` heartbeat key, which is shared across all workers. Even if the task is running on a different instance, the coordinator will correctly skip scheduling it.
205
+
////
206
+
207
+
---
208
+
156
209
## Running Heartbeat Configuration
157
210
158
211
While a task is executing, the worker periodically renews a heartbeat key in Redis. If a worker crashes, the key expires, signaling that the task is no longer being executed.
0 commit comments