Skip to content

Commit 53ffa30

Browse files
committed
spike(workflow): concurrent dynamic dispatch harness (DynamicNodeSupervisor + ctx.pipeline)
RFC google#92 reference harness. DynamicNodeSupervisor (gate-on-leaf, TaskGroup fan-out) + ctx.pipeline/ctx.parallel on the real ADK Workflow engine. 11 deterministic CI-safe tests (no LLM) + an env-gated live Gemini E2E. Proves barrier-free execution, failed-item isolation, control-exception cancellation (requires TaskGroup, not gather), nested no-deadlock with leaf gating (+ driver-gating deadlock contrast), and resume exactly-once for completed children. pyink/isort/mdformat clean.
1 parent 7d29bfa commit 53ffa30

4 files changed

Lines changed: 905 additions & 0 deletions

File tree

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# Dynamic Supervisor Spike — concurrent dynamic dispatch for ADK Workflows
2+
3+
Reproducible harness for an RFC proposing leaf-gated concurrent dynamic
4+
dispatch (`ctx.pipeline` / `ctx.parallel`) on the ADK Workflow engine.
5+
6+
**The harness exists to prove the design on the real engine, not to ship an
7+
API.** It pins exactly which properties hold: all five merge-gate properties
8+
hold with a wrapper supervisor on the unmodified engine. The v1 interrupt
9+
behavior is decided — cancel in-flight siblings and re-run them on resume;
10+
checkpoint-then-pause is a deferred v2 product decision.
11+
12+
## Environment
13+
14+
- Built/run against `google/adk-python` (branch rebased onto current `main`).
15+
- *Historical run evidence below was captured on ADK `2.0.0` at `origin/main` @ `4006fe40`; results re-verified on the rebased branch.*
16+
- Python: 3.11+ (uses `asyncio.TaskGroup` + `except*`)
17+
18+
## Files
19+
20+
| File | Purpose |
21+
| ---------------------------------- | ------------------------------------------------------------------------------------------------------ |
22+
| `supervisor.py` | Prototype `DynamicNodeSupervisor` (gate-on-leaf + `TaskGroup` fan-out) over the real `ctx.run_node()`. |
23+
| `test_dynamic_supervisor_spike.py` | Deterministic regression harness (no LLM). The trustworthy artifact. |
24+
| `test_live_gemini_e2e.py` | OPTIONAL live-model evidence; env-gated, skipped by default. |
25+
26+
## Run the deterministic harness (CI-safe, no network)
27+
28+
```bash
29+
pytest contributing/samples/workflows/dynamic_supervisor_spike/test_dynamic_supervisor_spike.py -q
30+
```
31+
32+
### Expected current result: **11 passed**
33+
34+
1. `test_concurrent_dispatch_correct_and_barrier_free` — concurrent `ctx.run_node`
35+
executes correctly (distinct results, no corruption), wall ≈ max-delay not serial sum.
36+
1. `test_pipeline_barrier_free` — item 0 enters stage 2 before item 1 finishes stage 1.
37+
1. `test_parallel_failed_item_isolation` — ordinary error → `None`, siblings unaffected.
38+
1. `test_control_exception_propagates_and_cancels_siblings``NodeInterruptedError`
39+
propagates **and cancels the running sibling**. Requires `asyncio.TaskGroup`:
40+
`asyncio.gather` propagates but does **not** cancel siblings, so the supervisor
41+
contract mandates TaskGroup-equivalent structured concurrency.
42+
1. `test_nested_combinator_no_deadlock_leaf_gating` — a pipeline stage calling
43+
`parallel` with `gate=2` completes; peak in-flight ≤ gate.
44+
1. `test_driver_gating_deadlocks_as_predicted` — CONTRAST: gating *drivers* instead
45+
of *leaves* deadlocks (timeout). Proves the leaf-gating decision empirically.
46+
1. `test_sequential_resume_is_exactly_once` — sequential dispatch resumes
47+
exactly-once (completed children fast-forward; interrupted node re-runs).
48+
1. `test_concurrent_resume_completed_children_fast_forward`**the merge gate.**
49+
Under *concurrent* dispatch, children that COMPLETE before the interrupt
50+
fast-forward on resume (exactly-once). No double-spend.
51+
1. `test_concurrent_inflight_children_cancelled_on_interrupt_rerun` — pins the
52+
**decided v1 semantic**: a sibling that interrupts while others are still IN
53+
FLIGHT cancels them; cancelled (never-completed) children correctly re-run on
54+
resume. (Checkpoint-then-pause is deferred to v2.)
55+
1. `test_child_cancellederror_does_not_cancel_siblings` — a branch-originated
56+
`asyncio.CancelledError` is asyncio task-cancellation: not propagated, siblings
57+
untouched, slot left `None`. Only `NodeInterruptedError` / non-cancellation
58+
`BaseException` cancel siblings.
59+
1. `test_gate_must_be_positive``gate=0`/negative raises `ValueError` at
60+
construction (would otherwise deadlock every dispatch).
61+
62+
## Resume exactly-once: there is no engine gap (a correction)
63+
64+
An earlier draft of this harness reported a resume "engine gap." That was a
65+
**test artifact and has been retracted.** The earlier test let the
66+
`RequestInput` child interrupt *before* its siblings finished, so the
67+
`TaskGroup` cancelled still-running siblings; those **cancelled (never
68+
completed)** children then re-ran on resume — which is *correct*, not a bug.
69+
70+
With the timing separated (test 8 vs test 9), the truth is:
71+
72+
- **Completed** concurrent children **fast-forward** on resume (exactly-once) —
73+
identical to sequential. No double-spend of completed LLM work.
74+
- **In-flight** children cancelled by an interrupting sibling **re-run** on
75+
resume — correctness-preserving (they never completed).
76+
77+
The `"Workflow ...: cancelling N leftover tasks"` log is **benign cleanup** — it
78+
appears even in the sequential exactly-once run, and completion is still
79+
checkpointed correctly. It is not corruption.
80+
81+
**Net: all five merge-gate properties hold with a wrapper supervisor + the real
82+
engine; no `_workflow.py` change is required for resume correctness.** The one
83+
behavior worth calling out in the RFC is a design trade-off, not a bug:
84+
interrupting one branch cancels in-flight siblings and discards their partial
85+
progress. If preserving that progress is desired, that is a separate design
86+
decision (e.g. checkpoint-then-pause instead of cancel).
87+
88+
## Optional: live model evidence (supporting only)
89+
90+
Skipped unless explicitly configured — never runs in CI by accident:
91+
92+
```bash
93+
export SPIKE_LIVE=1
94+
export GOOGLE_GENAI_USE_VERTEXAI=1
95+
export GOOGLE_CLOUD_PROJECT=<your-project>
96+
export GOOGLE_CLOUD_LOCATION=global # gemini-3.5-flash serves here
97+
export SPIKE_GEMINI_MODEL=gemini-3.5-flash # or any flash model you can access
98+
pytest contributing/samples/workflows/dynamic_supervisor_spike/test_live_gemini_e2e.py -q -s
99+
```
100+
101+
Asserts the concurrent pipeline wall-clock is well under the serial sum of
102+
per-call latencies. The deterministic engine tests — not this — are the
103+
artifact maintainers should trust.
Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# Copyright 2026 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Prototype DynamicNodeSupervisor for concurrent dynamic dispatch.
16+
17+
This is the RFC spike artifact (see README.md). It layers concurrent
18+
``ctx.run_node()`` orchestration on the real ADK Workflow engine via a
19+
framework-owned supervisor. Two design decisions are encoded here and
20+
verified by the tests:
21+
22+
1. The concurrency gate is acquired around each LEAF dispatch (a single
23+
``ctx.run_node`` call), NOT around drivers. Orchestration frames
24+
(drivers, fan-out, nested pipeline/parallel) hold no permit while
25+
awaiting children, so nesting a combinator inside a stage cannot
26+
deadlock. (Gating drivers DOES deadlock — see the contrast test.)
27+
28+
2. Fan-out uses ``asyncio.TaskGroup`` (structured concurrency), NOT
29+
``asyncio.gather``. ``gather`` propagates an exception but does not
30+
cancel siblings.
31+
32+
Failure / cancellation contract (verified by the tests):
33+
34+
* Ordinary ``Exception`` in a branch -> that branch becomes ``None``;
35+
siblings are unaffected.
36+
* ``NodeInterruptedError`` (and other non-cancellation ``BaseException``
37+
such as ``KeyboardInterrupt`` / ``SystemExit``) -> propagates, and
38+
``TaskGroup`` cancels the remaining branches.
39+
* External cancellation of the combinator -> propagates down to the
40+
in-flight branches (standard structured concurrency).
41+
* A branch raising ``asyncio.CancelledError`` itself is treated by asyncio
42+
as that task's own cancellation: ``TaskGroup`` does NOT propagate it and
43+
does NOT cancel siblings; the branch's slot is left ``None`` and the
44+
others run to completion. (This is asyncio semantics, not something the
45+
supervisor can override without bespoke handling — see the test.)
46+
"""
47+
48+
from __future__ import annotations
49+
50+
import asyncio
51+
import os
52+
from typing import Any
53+
from typing import Awaitable
54+
from typing import Callable
55+
from typing import Sequence
56+
57+
from google.adk.workflow._errors import NodeInterruptedError
58+
59+
# Control exceptions are NEVER converted to None. NodeInterruptedError is a
60+
# BaseException by design so it cannot be swallowed by ``except Exception``.
61+
_CONTROL_EXC = (
62+
NodeInterruptedError,
63+
asyncio.CancelledError,
64+
KeyboardInterrupt,
65+
SystemExit,
66+
)
67+
68+
69+
def default_gate() -> int:
70+
"""min(16, cpu-2): matches the Claude Code reference runtime cap."""
71+
return min(16, max(1, (os.cpu_count() or 3) - 2))
72+
73+
74+
class DynamicNodeSupervisor:
75+
"""Drives concurrent dynamic ``ctx.run_node()`` chains under one parent."""
76+
77+
def __init__(self, ctx, *, gate: int | None = None) -> None:
78+
self.ctx = ctx
79+
resolved_gate = gate if gate is not None else default_gate()
80+
if resolved_gate < 1:
81+
raise ValueError(f"gate must be >= 1, got {resolved_gate}")
82+
self.gate = asyncio.Semaphore(resolved_gate)
83+
self.peak_in_flight = 0
84+
self._in_flight = 0
85+
86+
async def dispatch(
87+
self, child, *, node_input: Any = None, run_id: str | None = None
88+
) -> Any:
89+
"""One leaf dispatch. The gate is held ONLY for the child execution."""
90+
async with self.gate:
91+
self._in_flight += 1
92+
self.peak_in_flight = max(self.peak_in_flight, self._in_flight)
93+
try:
94+
return await self.ctx.run_node(
95+
child, node_input=node_input, run_id=run_id
96+
)
97+
finally:
98+
self._in_flight -= 1
99+
100+
async def _guard_ordinary(self, factory: Callable[[], Awaitable[Any]]) -> Any:
101+
"""Ordinary Exception -> None (drop the branch). Control exceptions escape."""
102+
try:
103+
return await factory()
104+
except _CONTROL_EXC:
105+
raise
106+
except Exception: # noqa: BLE001 - includes DynamicNodeFailError
107+
return None
108+
109+
async def _supervise(
110+
self, factories: Sequence[Callable[[], Awaitable[Any]]]
111+
) -> list[Any]:
112+
"""Structured fan-out via TaskGroup. See the failure/cancellation contract
113+
in the module docstring: ordinary failure -> None; NodeInterruptedError
114+
(and other non-cancellation BaseException) propagates and cancels the rest;
115+
a branch's own CancelledError leaves its slot None without cancelling
116+
siblings. Results preserve input order.
117+
"""
118+
results: list[Any] = [None] * len(factories)
119+
120+
async def _run_one(i: int, f: Callable[[], Awaitable[Any]]) -> None:
121+
results[i] = await self._guard_ordinary(f)
122+
123+
try:
124+
async with asyncio.TaskGroup() as tg:
125+
for i, f in enumerate(factories):
126+
tg.create_task(_run_one(i, f))
127+
except* NodeInterruptedError:
128+
raise NodeInterruptedError()
129+
return results
130+
131+
async def parallel(
132+
self, thunks: Sequence[Callable[[], Awaitable[Any]]]
133+
) -> list[Any]:
134+
"""BARRIER fan-out. thunks: zero-arg callables returning awaitables."""
135+
return await self._supervise(thunks)
136+
137+
async def pipeline(
138+
self,
139+
items: Sequence[Any],
140+
*stages: Callable[[Any, Any, int], Awaitable[Any]],
141+
gate_drivers: bool = False,
142+
) -> list[Any]:
143+
"""Barrier-free per-item pipelining. Stage signature: (prev, item, index).
144+
145+
Each item flows through all stages independently; item A may be in stage
146+
k while item B is in stage 1. An ordinary Exception in a stage drops that
147+
item to None; control exceptions propagate.
148+
149+
``gate_drivers=True`` is the intentionally-BUGGY variant used by the
150+
contrast test to demonstrate the nested-combinator deadlock.
151+
"""
152+
153+
def make_driver(item: Any, i: int) -> Callable[[], Awaitable[Any]]:
154+
async def drive() -> Any:
155+
prev = item
156+
for stage in stages:
157+
prev = await stage(prev, item, i)
158+
return prev
159+
160+
if gate_drivers:
161+
162+
async def gated() -> Any:
163+
async with self.gate: # gating the DRIVER -> deadlock on nesting
164+
return await drive()
165+
166+
return gated
167+
return drive
168+
169+
return await self._supervise(
170+
[make_driver(it, i) for i, it in enumerate(items)]
171+
)

0 commit comments

Comments
 (0)