Skip to content

Commit e4bc6a4

Browse files
ling-senpeng13claude
ling-senpeng13
and
claude
committed
docs(examples): add example-catalog verification report; document agent server-version requirement
Report-only PR — no code changes. Tracks the verified status of every SDK example against conductor-oss 3.32.0-rc.8 (agents suite: 96 PASS / 46 SKIP / 2 ERROR / 2 HUNG with the companion fixes PR applied; 3 PASS / 103 ERROR on the CLI's `latest` 3.30.x server), the bugs found, and the remaining known failures. Fixes are proposed separately as a draft PR. Docs: the agent examples require conductor-oss >= 3.32.0-rc.8 (agent runtime on by default); documented in examples/agents/README.md and examples/README.md with `conductor server start --version 3.32.0-rc.8`. Also fixed the provider/model table (OpenAI row showed the Anthropic default string). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 7481aed commit e4bc6a4

3 files changed

Lines changed: 203 additions & 5 deletions

File tree

‎examples/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,12 @@ Durable agent authoring (`Agent`, `AgentRuntime`, tools, guardrails, handoffs, m
8989
strategies) — a separate, more extensive catalog of 270+ examples, requiring
9090
`pip install 'conductor-python[agents]'`. See [agents/README.md](agents/README.md).
9191

92+
> **Server version:** the agent examples require a Conductor server with the agent
93+
> runtime — conductor-oss **`3.32.0-rc.8` or newer**. The stable `latest` installed by
94+
> `conductor server start` (3.30.x) does not expose `/api/agent/*` and every agent
95+
> example fails with `AgentNotFoundError: HTTP 404`. Use
96+
> `conductor server start --version 3.32.0-rc.8`.
97+
9298
---
9399

94100
### Monitoring

‎examples/VERIFICATION_REPORT.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# Examples Verification Report
2+
3+
**Date:** 2026-07-18 · **Base:** `7481aed2`
4+
5+
Full verification of the SDK example catalog against a live Conductor server. This
6+
report tracks the status of each suite so it can be re-checked as the SDK and server
7+
evolve. **This PR is report + documentation only** — fixes for the bugs found are
8+
proposed separately as a draft PR, pending a decision on whether to take them.
9+
10+
## Environment
11+
12+
| Component | Value |
13+
|-----------|-------|
14+
| Server | conductor-oss **3.32.0-rc.8** boot JAR (Maven Central), `java -jar --server.port=8080` |
15+
| Python | 3.12.13 (uv venv), `pip install -e '.[agents]'` |
16+
| Platform | macOS (darwin arm64) — worker processes use the **spawn** start method |
17+
| LLM | `AGENTSPAN_LLM_MODEL=openai/gpt-4o-mini`, `OPENAI_API_KEY` + `ANTHROPIC_API_KEY` set |
18+
| Harness | `examples/agents/run_all_examples.py --jobs 8 --timeout 360` + curated core-example runner |
19+
20+
### ⚠️ Server version requirement (root cause of the most common failure)
21+
22+
The agent examples need the **agent runtime**, on by default from conductor-oss
23+
**3.32.0-rc.8** (the version pinned in `.github/workflows/agent-e2e.yml`). The `latest`
24+
JAR installed by `conductor server start` is the 3.30.x stable line, which has **no
25+
`/api/agent/*` endpoints** — every agent example fails with
26+
`AgentNotFoundError: HTTP 404 ... api/agent/start`. On 3.30.2 the suite scored
27+
3 PASS / 103 ERROR; on 3.32.0-rc.8 (same code) it scored 96 PASS / 2 ERROR.
28+
29+
Workaround (works today, `conductor-server-3.32.0-rc.8.jar` is already in the CLI's S3
30+
bucket): `conductor server start --version 3.32.0-rc.8`. Decision: do **not** repoint
31+
the CLI's floating `latest` at an rc — this resolves itself when 3.32.0 GA ships.
32+
33+
## Results — agents suite (`examples/agents/`, 146 files)
34+
35+
`run_all_examples.py` with the fixes from the companion fixes PR applied:
36+
37+
| Status | Count | Notes |
38+
|--------|------:|-------|
39+
| PASS | 96 | end-to-end against the live server, real LLM calls |
40+
| SKIP | 46 | interactive / daemon-by-design / external infra (MCP, Docker, Kafka, Slack, OCG, skills repo, messages API) |
41+
| ERROR | 2 | see "Remaining failures" |
42+
| HUNG | 2 | see "Remaining failures" |
43+
44+
Before fixes (same server): 91 PASS / 12 ERROR / 3 HUNG / 40 SKIP.
45+
46+
## Results — core suite (curated, 18 examples)
47+
48+
| Example | Status | Notes |
49+
|---------|--------|-------|
50+
| workers_e2e.py | ✅ fixed | workers run clean; e2e workflow COMPLETED server-side; blocks on `join_processes()` by design |
51+
| worker_example.py | ✅ fixed | runs clean as a worker daemon |
52+
| task_listener_example.py | ✅ fixed | all workers start, listeners fire, no spawn errors |
53+
| helloworld/helloworld.py | ✅ fixed | exits 0 |
54+
| worker_configuration_example.py | ✅ | |
55+
| task_workers.py | ✅ | |
56+
| dynamic_workflow.py | ✅ | |
57+
| workflow_ops.py | ✅ | |
58+
| workflow_status_listner.py | ✅ | |
59+
| kitchensink.py | ✅ | |
60+
| task_configure.py | ✅ | |
61+
| metadata_journey_oss.py | ✅ | |
62+
| schedule_journey.py | ✅ | |
63+
| event_listener_examples.py | ✅ | |
64+
| lease_extension_example.py | ✅ | ~62s |
65+
| agentic_workflows/llm_chat.py | ✅ fixed | previously a **false pass** — reported success while the workflow FAILED server-side |
66+
| task_context_example.py | ➖ daemon | runs until Ctrl+C by design; workers run clean |
67+
| agentic_workflows/function_calling_example.py | ➖ interactive | reads stdin; not runnable headless |
68+
69+
Static check: every `.py` under `examples/` compiles (`compileall`). Unit tests:
70+
`tests/unit/ai` — 1744 passed.
71+
72+
## Bugs found (fixes proposed in the companion draft PR)
73+
74+
Statuses marked "fixed" below and in the tables above reflect runs **with the
75+
companion PR's fixes applied**; without them, each item reproduces as described.
76+
77+
1. **`examples/helloworld/greetings_worker.py` — duplicate `def greet`.** The module
78+
defined `greet` twice (tasks `greet` and `greet_sync`); the second shadowed the
79+
first, so spawn-pickling the first failed with
80+
`PicklingError: not the same object as helloworld.greetings_worker.greet`. Broke
81+
`workers_e2e.py`, `helloworld.py`, and `task_listener_example.py` on macOS/Windows.
82+
Renamed the second function `greet_sync`.
83+
84+
2. **`src/conductor/ai/agents/tool.py` — SDK bug: worker-task tool lookup broken after
85+
re-registration.** `ToolRegistry.register_tool_workers` overwrites a
86+
`@worker_task` tool's `_decorated_functions` entry with a spawn-safe
87+
`ToolWorkerEntry` wrapper; the identity check in `_try_worker_task` then failed for
88+
the same tool later in the same run
89+
(`TypeError: Expected a @tool-decorated function ...`). Broke
90+
`14_existing_workers.py`. The lookup now walks `__wrapped__` chains and the
91+
wrapper's `fn_direct`/`fn_ref` carriers. Two regression tests added.
92+
93+
3. **`examples/worker_example.py` — hardcoded `/Users/viren/` metrics path.**
94+
`PermissionError` on any other machine. Now uses `tempfile.gettempdir()`.
95+
96+
4. **`examples/user_example/user_workers.py` — inconsistent package identity.**
97+
Imported `examples.user_example.models` while every consumer loads the module as
98+
`user_example.user_workers`; spawn children (which only inherit `PYTHONPATH`, not
99+
runtime `sys.path` edits) crashed. Import is now package-relative-consistent.
100+
101+
5. **`examples/agents/kitchen_sink.py` — `@agent` classifiers declared a required
102+
`prompt` arg.** `@agent`-decorated functions are invoked with zero args at compile
103+
time for dynamic instructions → `TypeError`. Signatures fixed.
104+
105+
6. **`examples/agents/16i/16j` — tools defined inside factory functions.**
106+
`<locals>` callables can't be re-imported by spawn workers →
107+
`SpawnSafetyError`. Tools hoisted to module level; both now pass.
108+
109+
7. **`examples/agents/94_openai_runner_tools.py` — `@function_tool` rebinding.** The
110+
decorator rebinds the module global to a `FunctionTool`, so the extracted original
111+
couldn't be pickled by reference. The example now keeps the plain function at module
112+
level and applies `function_tool()` at `Agent(...)` construction; passes end-to-end.
113+
114+
8. **`examples/agentic_workflows/llm_chat.py` — two bugs.** (a) Two LLM tasks sent
115+
**system-only** message lists; the server requires a non-empty user message and
116+
failed the workflow before the first task ran. (b) The example treated any terminal
117+
state as success (`is_completed()`), printing "Conversation complete." over a FAILED
118+
workflow and exiting 0. Both fixed; a full 3-turn conversation now completes.
119+
120+
9. **`examples/agents/run_all_examples.py` — skip-list updates.** Six examples that
121+
need infra this environment (and CI's pinned server) don't provide are now
122+
classified SKIP with reasons: `30`/`32` (need the `dg` skill cloned into
123+
`~/.claude/skills/dg`), `75`/`82`/`83`/`84` (need the workflow *messages* API,
124+
`GET /api/workflow/{id}/messages`, which is not in conductor-oss 3.32.0-rc.8).
125+
126+
10. **`examples/agents/README.md`** — added the server-version requirement + start
127+
commands; fixed the provider/model table (OpenAI row showed the Anthropic default
128+
string); renumbered setup steps. **`examples/README.md`** — added the same server
129+
note to the AI Agents section.
130+
131+
## Remaining failures (known, not fixed here)
132+
133+
| Example | Status | Diagnosis |
134+
|---------|--------|-----------|
135+
| `agents/kitchen_sink.py` | ERROR | Fixed classifier TypeError, now fails server-side: `HTTP 500 — The Task translation_swarm defined as a sub-workflow has no workflow definition available`. Sub-workflow (SWARM strategy) definitions aren't registered before `/agent/start` on conductor-oss rc.8. Needs SDK/server investigation. |
136+
| `agents/74_cli_error_output.py` | FLAKY | LLM-behavior assertion: the agent paraphrases stderr instead of quoting it verbatim; passes/fails depending on model output. Consider loosening the assertion or pinning a stricter prompt. |
137+
| `agents/86_coding_agent.py` | HUNG | Does not finish within 420s even running solo. Needs investigation (or reclassification if it is expected to be long-running). |
138+
| `agents/68_context_condensation.py` | SLOW | Passes solo in ~346s; exceeds the suite timeout under 8-way contention. Consider `--timeout 480` for suite runs or trimming the example. |
139+
140+
## SDK observations (follow-ups, out of scope for this PR)
141+
142+
- **`AgentNotFoundError` should name the server-version requirement.** When
143+
`/api/agent/start` 404s ("No static resource"), the error in
144+
`orkes_agent_client.py` should suggest conductor-oss ≥ 3.32.0-rc.8 and the
145+
`conductor server start --version` command. This would have made the most common
146+
failure self-explanatory.
147+
- **`run_all_examples.py` preflight**: check `GET /api/version` + probe an agent
148+
endpoint before launching 100+ subprocesses against a server that can't run them.
149+
- **openai-agents `@function_tool` at module level is spawn-unsafe by construction**
150+
(the global is rebound to a `FunctionTool`; the original function is only reachable
151+
through a closure, which `FunctionRef` can't express). Teaching the spawn-safety
152+
layer a closure hop — or documenting the `function_tool(fn)`-at-construction pattern
153+
— would let users keep the upstream OpenAI sample shape.
154+
155+
## How to reproduce
156+
157+
```bash
158+
# server
159+
curl -fL -o conductor-server.jar "https://repo1.maven.org/maven2/org/conductoross/conductor-server/3.32.0-rc.8/conductor-server-3.32.0-rc.8-boot.jar"
160+
OPENAI_API_KEY=... ANTHROPIC_API_KEY=... java -jar conductor-server.jar --server.port=8080
161+
162+
# SDK + suite
163+
uv venv --python 3.12 .venv && VIRTUAL_ENV=$PWD/.venv uv pip install -e '.[agents]'
164+
export CONDUCTOR_SERVER_URL=http://localhost:8080/api AGENTSPAN_LLM_MODEL=openai/gpt-4o-mini
165+
python examples/agents/run_all_examples.py --jobs 8 --timeout 360
166+
```

‎examples/agents/README.md‎

Lines changed: 31 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,33 @@ To install all framework dependencies at once:
120120
uv pip install langchain langchain-core langchain-openai langgraph openai-agents google-adk
121121
```
122122

123-
### 2. Configure your environment
123+
### 2. Start a Conductor server
124+
125+
The agent examples need a Conductor server with the **agent runtime**, which is on by
126+
default from **conductor-oss `3.32.0-rc.8`** onward (the same version pinned by this
127+
repo's agent-e2e CI). Older servers — including the `latest` stable line installed by
128+
`conductor server start` (3.30.x at the time of writing) — do not expose the
129+
`/api/agent/*` endpoints, and every example fails with
130+
`AgentNotFoundError: HTTP 404 ... api/agent/start`.
131+
132+
Start a known-good version with the [Conductor CLI](https://github.com/conductor-oss/conductor-cli):
133+
134+
```bash
135+
conductor server start --version 3.32.0-rc.8
136+
```
137+
138+
Or run the boot JAR from Maven Central directly:
139+
140+
```bash
141+
curl -fL -o conductor-server.jar \
142+
"https://repo1.maven.org/maven2/org/conductoross/conductor-server/3.32.0-rc.8/conductor-server-3.32.0-rc.8-boot.jar"
143+
java -jar conductor-server.jar --server.port=8080
144+
```
145+
146+
Export your LLM provider API keys (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`) in the
147+
shell that starts the server — the server auto-enables the matching providers.
148+
149+
### 3. Configure your environment
124150

125151
Export environment variables:
126152

@@ -131,14 +157,14 @@ export AGENTSPAN_SERVER_URL=http://localhost:8080/api
131157
# export AGENTSPAN_AUTH_SECRET=<secret>
132158
```
133159

134-
#### 2.1. Choose a model
160+
#### 3.1. Choose a model
135161

136162
The `AGENTSPAN_LLM_MODEL` variable uses the `provider/model-name` format. Examples:
137163

138164
| Provider | Model string | API key env var |
139165
|----------|-------------|-----------------|
140-
| OpenAI | `anthropic/claude-sonnet-4-6` (default) | `OPENAI_API_KEY` |
141-
| Anthropic | `anthropic/claude-sonnet-4-20250514` | `ANTHROPIC_API_KEY` |
166+
| OpenAI | `openai/gpt-4o-mini` | `OPENAI_API_KEY` |
167+
| Anthropic | `anthropic/claude-sonnet-4-6` (default) | `ANTHROPIC_API_KEY` |
142168
| Google Gemini | `google_gemini/gemini-2.0-flash` | `GOOGLE_GEMINI_API_KEY` |
143169
| AWS Bedrock | `aws_bedrock/...` | AWS credentials |
144170
| Azure OpenAI | `azure_openai/...` | Azure credentials |
@@ -147,7 +173,7 @@ All supported providers: `openai`, `anthropic`, `google_gemini`, `google_vertex_
147173
`azure_openai`, `aws_bedrock`, `cohere`, `mistral`, `groq`, `perplexity`,
148174
`hugging_face`, `deepseek`.
149175

150-
### 3. Run an example
176+
### 4. Run an example
151177

152178
```bash
153179
# Core SDK examples

0 commit comments

Comments
 (0)