Skip to content

Commit 773510b

Browse files
deploy: 6c0bdbf
1 parent 0dc5819 commit 773510b

285 files changed

Lines changed: 1596 additions & 1072 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

2.0/llms-full.txt

Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11569,6 +11569,11 @@ execute the same workflows.
1156911569

1157011570
Use `DW_V2_GUARDRAILS_BOOT` in CI and deployment manifests. The older `WORKFLOW_V2_GUARDRAILS_BOOT` name is retained only so `env:audit` can point alpha-era operators at the rename; the workflow package no longer reads it as a runtime fallback.
1157111571

11572+
Use [Task Matching and Dispatch](/docs/2.0/polyglot/task-matching-dispatch)
11573+
when you configure `DW_V2_MATCHING_ROLE_QUEUE_WAKE` or a dedicated
11574+
`workflow:v2:repair-pass --loop` daemon. Those settings change where
11575+
ready-task discovery runs, not the worker-protocol contract itself.
11576+
1157211577
## Runtime Infrastructure Variables
1157311578

1157411579
Runtime infrastructure variables are framework and dependency controls. The
@@ -14725,6 +14730,8 @@ Before polling, each worker must register its namespace, task queue, runtime,
1472514730
supported type keys, and local capacity through `POST /api/worker/register`.
1472614731
The [Namespace, Auth, And Worker Registration](/docs/2.0/polyglot/namespace-auth-workers)
1472714732
reference freezes that registration payload and the role-scoped auth contract.
14733+
For the broader ready-task discovery and lease-assignment contract behind these
14734+
verbs, see [Task Matching and Dispatch](/docs/2.0/polyglot/task-matching-dispatch).
1472814735

1472914736
## Workflow Task Bridge
1473014737

@@ -15128,6 +15135,194 @@ $activityBridge = app(ActivityTaskBridge::class);
1512815135
activity-grade worker, bridge, and handler contracts that build on the worker
1512915136
protocol.
1513015137

15138+
<!-- Source: docs/polyglot/task-matching-dispatch.md -->
15139+
15140+
# Task Matching and Dispatch
15141+
15142+
Use this guide when you need to reason about how Durable Workflow finds ready
15143+
work, assigns it to compatible workers, and scales that assignment path beyond
15144+
"every node polls everything." This is the contract behind workflow-task polls,
15145+
activity-task polls, queue wake behavior, and dedicated matching-role
15146+
deployments.
15147+
15148+
The core idea is that Durable Workflow separates two concerns:
15149+
15150+
- durable history and task state stay in the database
15151+
- task matching decides which live worker gets the next eligible task
15152+
15153+
That separation matters even before you introduce a separate matching service.
15154+
It lets operators talk about ready-task discovery, queue ownership, lease
15155+
churn, and backpressure as one explicit role instead of as incidental side
15156+
effects of every worker process.
15157+
15158+
## What the matching role does
15159+
15160+
The matching role is responsible for:
15161+
15162+
- discovering ready workflow and activity tasks
15163+
- narrowing the eligible set by namespace, connection, queue, and
15164+
compatibility family
15165+
- surfacing tasks to one worker at a time
15166+
- converting a successful claim into a lease with expiry
15167+
- preserving the same task when a claim fails, a lease expires, or a worker
15168+
disappears
15169+
- making wake and backlog state visible to operators
15170+
15171+
Matching does not replace durable history, workflow replay, or task execution.
15172+
It decides who gets a chance to execute next.
15173+
15174+
## Deployment shapes
15175+
15176+
Durable Workflow supports three practical shapes today:
15177+
15178+
### In-worker matching
15179+
15180+
This is the default shape. Worker processes long-poll for ready work and use
15181+
claim-time fencing to ensure only one worker owns a task lease at a time.
15182+
15183+
Use this when:
15184+
15185+
- you run a single node or a small fleet
15186+
- queue pressure is moderate
15187+
- you do not need a dedicated matching daemon yet
15188+
15189+
### In-server HTTP matching
15190+
15191+
In standalone-server deployments, external workers reach the same matching
15192+
contract through the worker protocol:
15193+
15194+
- `POST /api/worker/workflow-tasks/poll`
15195+
- `POST /api/worker/activity-tasks/poll`
15196+
15197+
The server becomes the network entrypoint for the same ready-task discovery,
15198+
claim, heartbeat, and completion flow.
15199+
15200+
### Dedicated matching-role deployment
15201+
15202+
Larger fleets can concentrate broad ready-task discovery into a dedicated
15203+
matching-role process. The documented operator shape today is:
15204+
15205+
```bash
15206+
php artisan workflow:v2:repair-pass --loop
15207+
```
15208+
15209+
Run that daemon in a dedicated process and set
15210+
`DW_V2_MATCHING_ROLE_QUEUE_WAKE=0` on execution-only nodes so they stop doing
15211+
the broad queue-worker wake on every loop tick.
15212+
15213+
Use this shape when:
15214+
15215+
- execution nodes should focus on replay and activity execution
15216+
- broad independent polling is creating unnecessary contention
15217+
- operators want an explicit place to own ready-task discovery and sweep
15218+
cadence
15219+
15220+
The dedicated role changes where matching happens, not the worker-protocol
15221+
contract. Poll, claim, lease, and redelivery semantics stay the same.
15222+
15223+
## Ready-task discovery
15224+
15225+
The matching role looks for durable tasks that are actually ready to run now:
15226+
15227+
- the task kind matches the poller (`workflow` or `activity`)
15228+
- the task is in `ready` state
15229+
- the task's `available_at` has arrived
15230+
- namespace, queue, connection, and compatibility filters still match
15231+
- activity polls also require a matching advertised activity type
15232+
15233+
Long-poll wakeups are an acceleration path, not the correctness path. If the
15234+
wake signal is delayed or missing, the task still becomes visible through
15235+
durable polling. A wake failure should raise latency, not strand work.
15236+
15237+
## Claim, lease, and backpressure
15238+
15239+
Matching only offers an opportunity. Ownership starts when one worker
15240+
successfully claims the task and receives a lease.
15241+
15242+
That lease gives Durable Workflow its backpressure behavior:
15243+
15244+
- one worker owns the task until the lease is renewed, completed, released, or
15245+
expired
15246+
- a failed or stale worker does not permanently trap the task
15247+
- redelivery is normal and must be handled as part of at-least-once execution
15248+
- incompatible workers do not steal the lease; they get an explicit rejection
15249+
15250+
Because work is lease-based, queue pressure shows up as backlog, stale leases,
15251+
or repeated redelivery instead of as hidden in-memory loss.
15252+
15253+
## Queue partitioning
15254+
15255+
The main partitioning primitives are:
15256+
15257+
- **namespace** for tenant or environment boundaries
15258+
- **connection** and **queue** for work-routing boundaries
15259+
- **compatibility family** for mixed-version routing safety
15260+
- **activity type filters** for workers that only execute selected activities
15261+
15262+
Use separate queues when you want stronger isolation, independent worker pools,
15263+
or different downstream budgets. Use compatibility families when the same
15264+
queue must stay live across a rollout or rollback without letting in-flight
15265+
work drift to incompatible executors.
15266+
15267+
## Wake signals and dedicated sweeps
15268+
15269+
Wake signals shorten the time between "task became ready" and "a poller noticed
15270+
it." They are not the durable source of truth.
15271+
15272+
In the default shape, queue workers can emit wake signals as part of their
15273+
normal loop. In a dedicated matching-role deployment, execution-only nodes
15274+
disable that broad wake path and the matching daemon owns the sweep cadence
15275+
instead.
15276+
15277+
Treat these as tuning levers:
15278+
15279+
- `DW_V2_MATCHING_ROLE_QUEUE_WAKE` controls whether execution nodes perform the
15280+
in-worker wake
15281+
- `DW_WAKE_SIGNAL_TTL_SECONDS` controls how long wake markers remain visible
15282+
- long-poll timeout settings control how long pollers stay parked waiting for
15283+
work
15284+
15285+
## What operators should watch
15286+
15287+
Use these surfaces together:
15288+
15289+
- task-queue visibility for ready depth, active slots, and throttling
15290+
- worker fleet visibility for active vs stale pollers on each queue
15291+
- health and Waterline diagnostics for unhealthy tasks, stale leases, and
15292+
compatible-worker gaps
15293+
- rolling-upgrade checks when mixed versions are live and matching must block
15294+
unsafe claims
15295+
15296+
If the oldest ready-task age grows while compatible workers are available, the
15297+
matching path is not making forward progress quickly enough. If ready tasks are
15298+
preserved but no compatible worker can claim them, that is a compatibility
15299+
problem, not a matching-loss problem.
15300+
15301+
## When to introduce a dedicated matching role
15302+
15303+
Move from default in-worker matching to a dedicated matching-role shape when:
15304+
15305+
- broad polling is creating visible database or cache contention
15306+
- execution nodes should stop paying the overhead of ready-task sweeps
15307+
- queue fairness and dispatch ownership need to be reasoned about as one
15308+
explicit subsystem
15309+
- operators want a distinct process to scale, supervise, and debug for
15310+
ready-task discovery
15311+
15312+
Stay on the default shape when the current fleet is small and the main need is
15313+
clear rollout and compatibility behavior rather than a new topology.
15314+
15315+
## Related references
15316+
15317+
- [Worker Protocol](/docs/2.0/polyglot/worker-protocol) for poll, heartbeat,
15318+
complete, and fail verbs
15319+
- [Task Queue Admission](/docs/2.0/polyglot/task-queue-admission) for slot,
15320+
lease, and dispatch budgets on top of matching
15321+
- [Rolling Upgrades](/docs/2.0/rolling-upgrades) for rollout procedure when
15322+
matching and compatibility are both in play
15323+
- [Server Config Reference](/docs/2.0/polyglot/server-config-reference) for
15324+
`DW_V2_MATCHING_ROLE_QUEUE_WAKE`, poll timing, and wake-signal settings
15325+
1513115326
<!-- Source: docs/polyglot/task-queue-admission.md -->
1513215327

1513315328
# Task Queue Admission
@@ -15142,6 +15337,11 @@ Task queue admission keeps one queue, tenant, or downstream dependency from cons
1514215337

1514315338
Use admission controls when a queue is tied to a rate-limited dependency, tenants share the same server, or operators need to prove why a workflow is waiting.
1514415339

15340+
Admission sits on top of the matching contract. Read
15341+
[Task Matching and Dispatch](/docs/2.0/polyglot/task-matching-dispatch) for
15342+
how ready work is discovered and leased before these budgets decide whether the
15343+
next task is allowed through.
15344+
1514515345
## How The Budget Is Applied
1514615346

1514715347
Workflow and activity polling starts with the workers that are currently registered for a namespace and task queue. Each worker advertises `max_concurrent_workflow_tasks` and `max_concurrent_activity_tasks`; the server sums active, non-stale workers to calculate the queue's registered slot capacity.
@@ -15683,6 +15883,13 @@ has not run yet, and start the new one. The window between the two is
1568315883
bounded by how long it takes the new container to come up; schedule
1568415884
firing resumes from the persisted state on next tick.
1568515885

15886+
If the rollout also introduces a dedicated matching-role deployment, make that
15887+
topology change explicit instead of assuming every execution node will keep
15888+
doing broad ready-task sweeps. See
15889+
[Task Matching and Dispatch](/docs/2.0/polyglot/task-matching-dispatch) for
15890+
the documented `workflow:v2:repair-pass --loop` plus
15891+
`DW_V2_MATCHING_ROLE_QUEUE_WAKE=0` shape.
15892+
1568615893
## Readiness and cutover
1568715894

1568815895
Use the readiness contract — not just the liveness probe — to decide

0 commit comments

Comments
 (0)