Skip to content

Commit 83bf0a7

Browse files
committed
docs: add orchestration guide and update README/WORKPLAN
Add guides/orchestration.md with examples for call_workflow and start_workflow. Update README with orchestration in features list, API reference, and guides section. Update WORKPLAN to mark orchestration complete (~268 tests).
1 parent b877b74 commit 83bf0a7

6 files changed

Lines changed: 540 additions & 87 deletions

File tree

README.md

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ A durable, resumable workflow engine for Elixir. Similar to Temporal/Inngest.
1414
- **Compensations** - Saga pattern with automatic rollback
1515
- **Cron Scheduling** - Recurring workflows with cron expressions
1616
- **Reliability** - Automatic retries with exponential/linear/constant backoff
17+
- **Orchestration** - Parent/child workflow composition
1718
- **Persistence** - PostgreSQL-backed execution state
1819

1920
## Installation
@@ -439,6 +440,21 @@ hours(2) # 7_200_000 ms
439440
days(7) # 604_800_000 ms
440441
```
441442

443+
### Orchestration
444+
445+
```elixir
446+
use Durable.Orchestration
447+
448+
# Synchronous: call child and wait for result
449+
case call_workflow(MyApp.PaymentWorkflow, %{"amount" => 100}, timeout: hours(1)) do
450+
{:ok, result} -> {:ok, assign(data, :payment, result)}
451+
{:error, reason} -> {:error, reason}
452+
end
453+
454+
# Fire-and-forget: start child and continue
455+
{:ok, child_id} = start_workflow(MyApp.EmailWorkflow, %{"to" => email}, ref: :welcome)
456+
```
457+
442458
### API
443459

444460
```elixir
@@ -449,6 +465,7 @@ Durable.list_executions(workflow: Module, status: :running)
449465
Durable.cancel(id, "reason")
450466
Durable.send_event(id, "event", payload)
451467
Durable.provide_input(id, "input_name", data)
468+
Durable.list_children(parent_id)
452469
```
453470

454471
## Guides
@@ -457,10 +474,10 @@ Durable.provide_input(id, "input_name", data)
457474
- [Parallel](guides/parallel.md) - Concurrent execution
458475
- [Compensations](guides/compensations.md) - Saga pattern
459476
- [Waiting](guides/waiting.md) - Sleep, events, human input
477+
- [Orchestration](guides/orchestration.md) - Parent/child workflow composition
460478

461479
## Coming Soon
462480

463-
- Workflow orchestration (parent/child workflows)
464481
- Phoenix LiveView dashboard
465482

466483
## License
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,13 @@
11
# DurableWorkflow Implementation Plan
22

3+
> **⚠️ ARCHIVED (2026-01-23):** This document is no longer maintained.
4+
> See `WORKPLAN.md` for current status and `arch.md` for technical reference.
5+
>
6+
> Key changes since this document was written:
7+
> - ForEach primitive was **removed** (use `Enum.map` instead)
8+
> - Parallel execution uses new results model (`__results__`, `into:`, `returns:`)
9+
> - Loop primitive was **never implemented** (use step retries or `Enum` functions)
10+
311
## Executive Summary
412

513
This document outlines the complete implementation plan for **Durable**, a durable, resumable workflow engine for Elixir.

agents/WORKPLAN.md

Lines changed: 70 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
11
# Durable Workflow Engine - Work Plan
22

3-
**Last Updated:** 2026-01-03
3+
**Last Updated:** 2026-02-27
44
**Overall Progress:** ~75% Complete
5-
**Reference:** See `IMPLEMENTATION_PLAN.md` for detailed design and code examples
5+
**Reference:** See `arch.md` for technical architecture
66

77
---
88

99
## Quick Stats
1010

1111
| Metric | Value |
1212
|--------|-------|
13-
| Source Modules | 41 |
14-
| Passing Tests | 214 |
13+
| Source Modules | 40 |
14+
| Passing Tests | ~268 |
1515
| Documentation Guides | 6 |
1616
| Lines of Code | ~8,500 |
1717

@@ -24,7 +24,7 @@
2424
| 0 | Project Foundation | Complete | 100% |
2525
| 1 | Core MVP | Complete | 100% |
2626
| 2 | Observability | Partial | 40% |
27-
| 3 | Advanced Features | Mostly Complete | 85% |
27+
| 3 | Advanced Features | Mostly Complete | 90% |
2828
| 4 | Scalability | Not Started | 0% |
2929
| 5 | Developer Experience | Partial | 35% |
3030

@@ -86,7 +86,7 @@
8686

8787
---
8888

89-
## Phase 3: Advanced Features [85%]
89+
## Phase 3: Advanced Features [90%]
9090

9191
### 3.1-3.3 Wait Primitives [COMPLETE - 46 tests]
9292

@@ -117,26 +117,27 @@
117117

118118
### 3.5 Loops [SKIPPED]
119119

120-
Intentionally skipped - use step-level retries or `foreach` instead.
120+
Intentionally skipped - use step-level retries or Elixir's `Enum` functions instead.
121121

122122
### 3.6 Parallel Execution [COMPLETE - 13 tests]
123123

124124
| Feature | Status |
125125
|---------|--------|
126126
| `parallel do` macro | Complete |
127-
| Context merge strategies | Complete |
127+
| Results model (`__results__`) | Complete |
128+
| `into:` custom merge function | Complete |
129+
| `returns:` option | Complete |
128130
| Error strategies | Complete |
129131
| Resume durability | Complete |
130132

131-
### 3.7 ForEach [COMPLETE - 13 tests]
133+
See `guides/parallel.md` for comprehensive documentation.
132134

133-
| Feature | Status |
134-
|---------|--------|
135-
| `foreach` macro | Complete |
136-
| `current_item/0`, `current_index/0` | Complete |
137-
| Sequential/Concurrent modes | Complete |
138-
| `:collect_as` option | Complete |
139-
| `:on_error` handling | Complete |
135+
### 3.7 ForEach [REMOVED]
136+
137+
**Decision (2026-01-23):** The `foreach` primitive was removed. Users should use
138+
Elixir's built-in enumeration tools (`Enum.map`, `Task.async_stream`) for batch
139+
processing instead. This simplifies the DSL while providing the same functionality
140+
through idiomatic Elixir.
140141

141142
### 3.8 Switch/Case [NOT STARTED]
142143

@@ -165,12 +166,24 @@ Low priority - `branch` macro covers most cases.
165166
| Manual trigger | Complete |
166167
| Telemetry events | Complete |
167168

169+
### 3.11 Workflow Orchestration [COMPLETE - 12 tests]
170+
171+
| Feature | Status |
172+
|---------|--------|
173+
| `call_workflow/3` (synchronous) | Complete |
174+
| `start_workflow/3` (fire-and-forget) | Complete |
175+
| Idempotent resume | Complete |
176+
| Cascade cancellation | Complete |
177+
| Parent notification on child complete/fail | Complete |
178+
| Nested workflows (A → B → C) | Complete |
179+
| `Durable.list_children/2` API | Complete |
180+
181+
See `guides/orchestration.md` for comprehensive documentation.
182+
168183
### Remaining Phase 3 Work
169184

170185
| Feature | Priority | Complexity |
171186
|---------|----------|------------|
172-
| Workflow Orchestration (`call_workflow`) | High | Medium |
173-
| Parent-child tracking | High | Low |
174187
| Switch/Case macro | Low | Low |
175188
| Pipe-based API | Low | Medium |
176189

@@ -204,7 +217,7 @@ Note: Multi-node scheduling already works via `FOR UPDATE SKIP LOCKED`.
204217
| Module docs (@moduledoc) | Complete |
205218
| Function docs (@doc) | Complete |
206219
| Typespecs (@spec) | Complete |
207-
| 6 Documentation Guides | Complete |
220+
| 7 Documentation Guides | Complete |
208221

209222
### Remaining
210223

@@ -229,25 +242,24 @@ Note: Multi-node scheduling already works via `FOR UPDATE SKIP LOCKED`.
229242
### High Priority
230243

231244
1. Guide: Getting Started
232-
2. Workflow Orchestration (`call_workflow`)
233-
3. HexDocs Publishing
234-
4. `mix durable.status`
245+
2. HexDocs Publishing
246+
3. `mix durable.status`
235247

236248
### Medium Priority
237249

238-
5. Guide: Testing Workflows
239-
6. `Durable.TestCase`
240-
7. Graph Generation
241-
8. `mix durable.list`
242-
9. pg_notify Message Bus
250+
4. Guide: Testing Workflows
251+
5. `Durable.TestCase`
252+
6. Graph Generation
253+
7. `mix durable.list`
254+
8. pg_notify Message Bus
243255

244256
### Lower Priority
245257

246-
10. Switch/Case macro
247-
11. Redis Queue Adapter
248-
12. Phoenix Dashboard
249-
13. Example Project
250-
14. Pipe-based API
258+
9. Switch/Case macro
259+
10. Redis Queue Adapter
260+
11. Phoenix Dashboard
261+
12. Example Project
262+
13. Pipe-based API
251263

252264
---
253265

@@ -259,30 +271,50 @@ Note: Multi-node scheduling already works via `FOR UPDATE SKIP LOCKED`.
259271
| wait_test.exs | 46 | Wait primitives |
260272
| decision_test.exs | 13 | Decision steps |
261273
| parallel_test.exs | 13 | Parallel execution |
262-
| foreach_test.exs | 13 | ForEach iteration |
263274
| log_capture_test.exs | 13 | Log/IO capture |
264275
| integration_test.exs | 11 | End-to-end flows |
265276
| branch_test.exs | 10 | Branch macro |
266277
| durable_test.exs | 8 | Core API |
267278
| compensation_test.exs | 6 | Saga pattern |
268279
| Other | ~36 | Queue, handlers, etc. |
269-
| **Total** | **214** | |
280+
| orchestration_test.exs | 12 | Workflow orchestration |
281+
| **Total** | **~268** | |
270282

271283
---
272284

273285
## Known Limitations
274286

275-
1. Wait primitives not supported in parallel/foreach blocks
287+
1. Wait primitives not supported in parallel blocks
276288
2. No backward jumps in decision steps (forward-only by design)
277289
3. Context is single-level atomized (top-level keys only)
278290
4. No workflow versioning
291+
5. No foreach/loop DSL primitives (use Elixir's `Enum` functions)
279292

280293
---
281294

282295
## Next Steps
283296

284297
1. **Documentation** - Getting Started guide and HexDocs publishing
285-
2. **Workflow Orchestration** - Child workflow support (`call_workflow`)
286-
3. **Graph Visualization** - Understanding complex workflows
298+
2. **Graph Visualization** - Understanding complex workflows
299+
3. **Testing Helpers** - `Durable.TestCase` for easier workflow testing
300+
301+
The existing ~268 tests provide good confidence in implemented features. Suitable for internal use; additional documentation needed before public release.
302+
303+
---
287304

288-
The existing 214 tests provide good confidence in implemented features. Suitable for internal use; additional documentation needed before public release.
305+
## Changelog
306+
307+
### 2026-02-27
308+
- Added workflow orchestration: `call_workflow/3` (synchronous) and `start_workflow/3` (fire-and-forget)
309+
- Added `Durable.Orchestration` module with `use Durable.Orchestration` macro
310+
- Added cascade cancellation (cancelling parent cancels active children)
311+
- Added parent notification on child completion/failure
312+
- Added `Durable.list_children/2` API
313+
- Added `guides/orchestration.md` documentation
314+
- 12 new tests for orchestration (total: ~268)
315+
316+
### 2026-01-23
317+
- Removed `foreach` primitive (use `Enum.map` or `Task.async_stream` instead)
318+
- Updated parallel execution with new results model (`__results__`, `into:`, `returns:`)
319+
- Updated documentation in `guides/parallel.md`
320+
- Archived `IMPLEMENTATION_PLAN.md` (now `IMPLEMENTATION_PLAN_ARCHIVED.md`)

0 commit comments

Comments
 (0)