Skip to content

Commit da4f7aa

Browse files
committed
spike(workflow): RFC google#93 authored-workflow demand-gate artifact
A model emits a declarative, validated WorkflowSpec (typed data, not code) that the framework validates and executes on the real ADK engine via the google#92 supervisor. authoring.py (WorkflowSpec plain kind-tagged recursive union; CapabilityRegistry; WorkflowSpecValidator; SpecInterpreter for step/fan_out/ branch/loop_until), 10 deterministic tests, env-gated live planner sweep (multi-stage/branch/loop on gemini-3.5-flash, shape-specific assertions). Findings folded into the RFC: open dict[str,X] maps are a structured-output hazard (Branch.routes -> list[Route]); Gemini response_schema rejects Field(discriminator=...), so the vocabulary is a plain kind-tagged union; planning vs capability quality are separable. pyink/isort/mdformat clean.
1 parent 53ffa30 commit da4f7aa

4 files changed

Lines changed: 1106 additions & 0 deletions

File tree

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Authored Workflow Spike — demand gate for RFC #93
2+
3+
Reference spike for **agent-authored typed Workflows** (RFC #93): a model emits
4+
a declarative, validated `WorkflowSpec` (typed data, **not** code) that the
5+
framework validates and executes on the real ADK Workflow engine via the #92
6+
`DynamicNodeSupervisor`. This directory is the re-runnable demand-gate artifact
7+
behind the RFC's "can a model author good plans?" question.
8+
9+
## Environment
10+
11+
- ADK: `2.1.0`
12+
- Built against `google/adk-python` upstream `main`.
13+
- Python 3.11+ (recursive `kind`-tagged unions; `asyncio.TaskGroup` in #92).
14+
15+
## Files
16+
17+
| File | Purpose |
18+
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
19+
| `authoring.py` | `WorkflowSpec` (plain `kind`-tagged recursive tree), `CapabilityRegistry`, `WorkflowSpecValidator`, `SpecInterpreter` (step / fan_out / branch / loop_until). |
20+
| `test_authoring.py` | Deterministic, CI-safe tests (no LLM). The trustworthy artifact. |
21+
| `test_live_planner_sweep.py` | OPTIONAL env-gated live planner sweep across plan shapes. |
22+
23+
## Deterministic tests (CI-safe, no network)
24+
25+
```bash
26+
pytest contributing/samples/workflows/authored_workflow_spike/test_authoring.py -q
27+
```
28+
29+
Expected: **10 passed**`Binding` invariant, `max_iters>=1`, validator accepts a
30+
valid spec and rejects unknown capability / non-preceding binding / duplicate id,
31+
the open-map warning, and interpreter execution of fan_out→aggregate, branch
32+
(correct route), and loop_until (stops + correct output).
33+
34+
## Live planner sweep (optional evidence)
35+
36+
Skipped unless configured — no hardcoded project/model:
37+
38+
```bash
39+
export SPIKE_LIVE=1 GOOGLE_GENAI_USE_VERTEXAI=1
40+
export GOOGLE_CLOUD_PROJECT=<project> GOOGLE_CLOUD_LOCATION=global
41+
export SPIKE_GEMINI_MODEL=gemini-3.5-flash # 3.5 serves from `global`
42+
pytest contributing/samples/workflows/authored_workflow_spike/test_live_planner_sweep.py -q -s
43+
```
44+
45+
## Gate results (run on `gemini-3.5-flash`)
46+
47+
**Initial gate (codebase audit):** planner authored a valid, sensible, executable
48+
plan (`fan_out reviewer → triager`) matching a hand-wired baseline. **PASS.**
49+
50+
**Shape sweep (this directory):** the planner authored + validated + executed all
51+
three shapes:
52+
53+
| Shape | Authored steps | Result |
54+
| ----------- | ----------------------- | --------------------------------------- |
55+
| multi-stage | `fan_out → step → step` | report → formatted note |
56+
| branch | `step → branch` | took the matched route, produced a note |
57+
| loop_until | `loop_until` | iterated to a headline |
58+
59+
## Findings that fell out (and shaped the RFC)
60+
61+
1. **Open-ended `dict[str, X]` maps are a structured-output reliability hazard.**
62+
Surfaced **twice**: a capability's `counts: dict[str,int]` came back empty, and
63+
the spec's own `Branch.routes: dict[str, list]` came back empty. **Both fixed by
64+
using enumerated/list structures** — capability outputs use fixed severity
65+
fields; `Branch.routes` is now a `list[Route]`, not a map. The validator also
66+
warns on open-map capability outputs.
67+
1. **The strict `unmatched=fail` branch contract earns its keep** — when the planner
68+
bound a branch switch to a whole object instead of its field, execution failed
69+
loudly instead of silently mis-routing.
70+
1. **Gemini `response_schema` rejects Pydantic's `Field(discriminator=...)`.** The
71+
plan vocabulary is a PLAIN union of models that each carry a `kind` literal (a
72+
*structurally-tagged* union). The strict discriminated form emits a
73+
`discriminator` keyword that genai's `response_schema` refuses
74+
(`Schema: extra_forbidden`, verified on `gemini-3.5-flash`); the `kind` tags
75+
still make parsing and switching unambiguous.
76+
1. **Planning vs capability quality are separable** — authoring/structure was
77+
reliably good; the residual variance was per-capability output quality
78+
(prompts/schemas/retries), not planning.
79+
80+
This is a demand-gate artifact, not production code.

0 commit comments

Comments
 (0)