Skip to content

Commit c6bc7b3

Browse files
ling-senpeng13claude
ling-senpeng13
and
claude
committed
fix(examples): verify example catalog against conductor-oss 3.32.0-rc.8; fix 9 broken examples + worker-tool registry lookup
Full verification run of the example catalog (agents suite: 96 PASS / 46 SKIP / 2 ERROR / 2 HUNG; core suite: 13 PASS / 4 daemon-by-design / 1 interactive) — see examples/VERIFICATION_REPORT.md for status tracking, environment, root causes, and remaining known failures. Example fixes (all verified end-to-end against a live server): - greetings_worker.py: duplicate `def greet` shadowed the registered worker and broke spawn pickling on macOS/Windows (workers_e2e, helloworld, task_listener_example) - worker_example.py: hardcoded /Users/viren metrics dir -> tempfile - user_workers.py: import via examples.* package identity crashed spawn children that load the module as user_example.* - kitchen_sink.py: @agent classifier functions must take zero args - 16i/16j: tools defined inside factory functions are not importable by spawn workers (SpawnSafetyError) — hoisted to module level - 94_openai_runner_tools.py: keep the plain function at module level and apply function_tool() at Agent construction so the module global is not rebound to an unpicklable FunctionTool - llm_chat.py: server requires a non-empty user message (two tasks sent system-only lists); also exit non-zero when the workflow ends FAILED instead of reporting success on any terminal state - run_all_examples.py: classify 6 env-dependent examples as SKIP with reasons (dg skill repo; workflow messages API absent in rc.8) SDK fix: - tool.py _try_worker_task: registry identity check failed once ToolRegistry.register_tool_workers re-registered the task name with a spawn-safe ToolWorkerEntry wrapper; lookup now walks __wrapped__ / fn_direct / fn_ref carriers (broke 14_existing_workers.py; 2 regression tests added, tests/unit/ai: 1744 passed) Docs: - examples/agents/README.md + examples/README.md: agent examples require conductor-oss >= 3.32.0-rc.8 (`conductor server start --version 3.32.0-rc.8`); the CLI's `latest` (3.30.x) has no /api/agent/* and fails with AgentNotFoundError HTTP 404. Also fixed the provider/model table (OpenAI row showed the Anthropic default string). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 7481aed commit c6bc7b3

14 files changed

Lines changed: 370 additions & 39 deletions

File tree

‎examples/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,12 @@ Durable agent authoring (`Agent`, `AgentRuntime`, tools, guardrails, handoffs, m
8989
strategies) — a separate, more extensive catalog of 270+ examples, requiring
9090
`pip install 'conductor-python[agents]'`. See [agents/README.md](agents/README.md).
9191

92+
> **Server version:** the agent examples require a Conductor server with the agent
93+
> runtime — conductor-oss **`3.32.0-rc.8` or newer**. The stable `latest` installed by
94+
> `conductor server start` (3.30.x) does not expose `/api/agent/*` and every agent
95+
> example fails with `AgentNotFoundError: HTTP 404`. Use
96+
> `conductor server start --version 3.32.0-rc.8`.
97+
9298
---
9399

94100
### Monitoring

‎examples/VERIFICATION_REPORT.md‎

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
# Examples Verification Report
2+
3+
**Date:** 2026-07-18 · **Branch:** `examples/verification-and-fixes` · **Base:** `7481aed2`
4+
5+
Full verification of the SDK example catalog against a live Conductor server, with
6+
fixes for every reproducible example bug found. This report tracks the status of each
7+
suite so it can be re-checked as the SDK and server evolve.
8+
9+
## Environment
10+
11+
| Component | Value |
12+
|-----------|-------|
13+
| Server | conductor-oss **3.32.0-rc.8** boot JAR (Maven Central), `java -jar --server.port=8080` |
14+
| Python | 3.12.13 (uv venv), `pip install -e '.[agents]'` |
15+
| Platform | macOS (darwin arm64) — worker processes use the **spawn** start method |
16+
| LLM | `AGENTSPAN_LLM_MODEL=openai/gpt-4o-mini`, `OPENAI_API_KEY` + `ANTHROPIC_API_KEY` set |
17+
| Harness | `examples/agents/run_all_examples.py --jobs 8 --timeout 360` + curated core-example runner |
18+
19+
### ⚠️ Server version requirement (root cause of the most common failure)
20+
21+
The agent examples need the **agent runtime**, on by default from conductor-oss
22+
**3.32.0-rc.8** (the version pinned in `.github/workflows/agent-e2e.yml`). The `latest`
23+
JAR installed by `conductor server start` is the 3.30.x stable line, which has **no
24+
`/api/agent/*` endpoints** — every agent example fails with
25+
`AgentNotFoundError: HTTP 404 ... api/agent/start`. On 3.30.2 the suite scored
26+
3 PASS / 103 ERROR; on 3.32.0-rc.8 (same code) it scored 96 PASS / 2 ERROR.
27+
28+
Workaround (works today, `conductor-server-3.32.0-rc.8.jar` is already in the CLI's S3
29+
bucket): `conductor server start --version 3.32.0-rc.8`. Decision: do **not** repoint
30+
the CLI's floating `latest` at an rc — this resolves itself when 3.32.0 GA ships.
31+
32+
## Results — agents suite (`examples/agents/`, 146 files)
33+
34+
`run_all_examples.py` after the fixes in this PR:
35+
36+
| Status | Count | Notes |
37+
|--------|------:|-------|
38+
| PASS | 96 | end-to-end against the live server, real LLM calls |
39+
| SKIP | 46 | interactive / daemon-by-design / external infra (MCP, Docker, Kafka, Slack, OCG, skills repo, messages API) |
40+
| ERROR | 2 | see "Remaining failures" |
41+
| HUNG | 2 | see "Remaining failures" |
42+
43+
Before fixes (same server): 91 PASS / 12 ERROR / 3 HUNG / 40 SKIP.
44+
45+
## Results — core suite (curated, 18 examples)
46+
47+
| Example | Status | Notes |
48+
|---------|--------|-------|
49+
| workers_e2e.py | ✅ fixed | workers run clean; e2e workflow COMPLETED server-side; blocks on `join_processes()` by design |
50+
| worker_example.py | ✅ fixed | runs clean as a worker daemon |
51+
| task_listener_example.py | ✅ fixed | all workers start, listeners fire, no spawn errors |
52+
| helloworld/helloworld.py | ✅ fixed | exits 0 |
53+
| worker_configuration_example.py | ✅ | |
54+
| task_workers.py | ✅ | |
55+
| dynamic_workflow.py | ✅ | |
56+
| workflow_ops.py | ✅ | |
57+
| workflow_status_listner.py | ✅ | |
58+
| kitchensink.py | ✅ | |
59+
| task_configure.py | ✅ | |
60+
| metadata_journey_oss.py | ✅ | |
61+
| schedule_journey.py | ✅ | |
62+
| event_listener_examples.py | ✅ | |
63+
| lease_extension_example.py | ✅ | ~62s |
64+
| agentic_workflows/llm_chat.py | ✅ fixed | previously a **false pass** — reported success while the workflow FAILED server-side |
65+
| task_context_example.py | ➖ daemon | runs until Ctrl+C by design; workers run clean |
66+
| agentic_workflows/function_calling_example.py | ➖ interactive | reads stdin; not runnable headless |
67+
68+
Static check: every `.py` under `examples/` compiles (`compileall`). Unit tests:
69+
`tests/unit/ai` — 1744 passed.
70+
71+
## Bugs found and fixed in this PR
72+
73+
1. **`examples/helloworld/greetings_worker.py` — duplicate `def greet`.** The module
74+
defined `greet` twice (tasks `greet` and `greet_sync`); the second shadowed the
75+
first, so spawn-pickling the first failed with
76+
`PicklingError: not the same object as helloworld.greetings_worker.greet`. Broke
77+
`workers_e2e.py`, `helloworld.py`, and `task_listener_example.py` on macOS/Windows.
78+
Renamed the second function `greet_sync`.
79+
80+
2. **`src/conductor/ai/agents/tool.py` — SDK bug: worker-task tool lookup broken after
81+
re-registration.** `ToolRegistry.register_tool_workers` overwrites a
82+
`@worker_task` tool's `_decorated_functions` entry with a spawn-safe
83+
`ToolWorkerEntry` wrapper; the identity check in `_try_worker_task` then failed for
84+
the same tool later in the same run
85+
(`TypeError: Expected a @tool-decorated function ...`). Broke
86+
`14_existing_workers.py`. The lookup now walks `__wrapped__` chains and the
87+
wrapper's `fn_direct`/`fn_ref` carriers. Two regression tests added.
88+
89+
3. **`examples/worker_example.py` — hardcoded `/Users/viren/` metrics path.**
90+
`PermissionError` on any other machine. Now uses `tempfile.gettempdir()`.
91+
92+
4. **`examples/user_example/user_workers.py` — inconsistent package identity.**
93+
Imported `examples.user_example.models` while every consumer loads the module as
94+
`user_example.user_workers`; spawn children (which only inherit `PYTHONPATH`, not
95+
runtime `sys.path` edits) crashed. Import is now package-relative-consistent.
96+
97+
5. **`examples/agents/kitchen_sink.py` — `@agent` classifiers declared a required
98+
`prompt` arg.** `@agent`-decorated functions are invoked with zero args at compile
99+
time for dynamic instructions → `TypeError`. Signatures fixed.
100+
101+
6. **`examples/agents/16i/16j` — tools defined inside factory functions.**
102+
`<locals>` callables can't be re-imported by spawn workers →
103+
`SpawnSafetyError`. Tools hoisted to module level; both now pass.
104+
105+
7. **`examples/agents/94_openai_runner_tools.py` — `@function_tool` rebinding.** The
106+
decorator rebinds the module global to a `FunctionTool`, so the extracted original
107+
couldn't be pickled by reference. The example now keeps the plain function at module
108+
level and applies `function_tool()` at `Agent(...)` construction; passes end-to-end.
109+
110+
8. **`examples/agentic_workflows/llm_chat.py` — two bugs.** (a) Two LLM tasks sent
111+
**system-only** message lists; the server requires a non-empty user message and
112+
failed the workflow before the first task ran. (b) The example treated any terminal
113+
state as success (`is_completed()`), printing "Conversation complete." over a FAILED
114+
workflow and exiting 0. Both fixed; a full 3-turn conversation now completes.
115+
116+
9. **`examples/agents/run_all_examples.py` — skip-list updates.** Six examples that
117+
need infra this environment (and CI's pinned server) don't provide are now
118+
classified SKIP with reasons: `30`/`32` (need the `dg` skill cloned into
119+
`~/.claude/skills/dg`), `75`/`82`/`83`/`84` (need the workflow *messages* API,
120+
`GET /api/workflow/{id}/messages`, which is not in conductor-oss 3.32.0-rc.8).
121+
122+
10. **`examples/agents/README.md`** — added the server-version requirement + start
123+
commands; fixed the provider/model table (OpenAI row showed the Anthropic default
124+
string); renumbered setup steps. **`examples/README.md`** — added the same server
125+
note to the AI Agents section.
126+
127+
## Remaining failures (known, not fixed here)
128+
129+
| Example | Status | Diagnosis |
130+
|---------|--------|-----------|
131+
| `agents/kitchen_sink.py` | ERROR | Fixed classifier TypeError, now fails server-side: `HTTP 500 — The Task translation_swarm defined as a sub-workflow has no workflow definition available`. Sub-workflow (SWARM strategy) definitions aren't registered before `/agent/start` on conductor-oss rc.8. Needs SDK/server investigation. |
132+
| `agents/74_cli_error_output.py` | FLAKY | LLM-behavior assertion: the agent paraphrases stderr instead of quoting it verbatim; passes/fails depending on model output. Consider loosening the assertion or pinning a stricter prompt. |
133+
| `agents/86_coding_agent.py` | HUNG | Does not finish within 420s even running solo. Needs investigation (or reclassification if it is expected to be long-running). |
134+
| `agents/68_context_condensation.py` | SLOW | Passes solo in ~346s; exceeds the suite timeout under 8-way contention. Consider `--timeout 480` for suite runs or trimming the example. |
135+
136+
## SDK observations (follow-ups, out of scope for this PR)
137+
138+
- **`AgentNotFoundError` should name the server-version requirement.** When
139+
`/api/agent/start` 404s ("No static resource"), the error in
140+
`orkes_agent_client.py` should suggest conductor-oss ≥ 3.32.0-rc.8 and the
141+
`conductor server start --version` command. This would have made the most common
142+
failure self-explanatory.
143+
- **`run_all_examples.py` preflight**: check `GET /api/version` + probe an agent
144+
endpoint before launching 100+ subprocesses against a server that can't run them.
145+
- **openai-agents `@function_tool` at module level is spawn-unsafe by construction**
146+
(the global is rebound to a `FunctionTool`; the original function is only reachable
147+
through a closure, which `FunctionRef` can't express). Teaching the spawn-safety
148+
layer a closure hop — or documenting the `function_tool(fn)`-at-construction pattern
149+
— would let users keep the upstream OpenAI sample shape.
150+
151+
## How to reproduce
152+
153+
```bash
154+
# server
155+
curl -fL -o conductor-server.jar "https://repo1.maven.org/maven2/org/conductoross/conductor-server/3.32.0-rc.8/conductor-server-3.32.0-rc.8-boot.jar"
156+
OPENAI_API_KEY=... ANTHROPIC_API_KEY=... java -jar conductor-server.jar --server.port=8080
157+
158+
# SDK + suite
159+
uv venv --python 3.12 .venv && VIRTUAL_ENV=$PWD/.venv uv pip install -e '.[agents]'
160+
export CONDUCTOR_SERVER_URL=http://localhost:8080/api AGENTSPAN_LLM_MODEL=openai/gpt-4o-mini
161+
python examples/agents/run_all_examples.py --jobs 8 --timeout 360
162+
```

‎examples/agentic_workflows/llm_chat.py‎

Lines changed: 27 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -87,8 +87,14 @@ def create_chat_workflow(executor) -> ConductorWorkflow:
8787
messages=[
8888
ChatMessage(
8989
role="system",
90-
message="You are an expert in science. Think of a random scientific "
91-
"discovery and create a short, interesting question about it.",
90+
message="You are an expert in science.",
91+
),
92+
# The server requires a non-empty user message — a system-only
93+
# conversation fails validation before the first task runs.
94+
ChatMessage(
95+
role="user",
96+
message="Think of a random scientific discovery and create a "
97+
"short, interesting question about it.",
9298
),
9399
],
94100
temperature=0.7,
@@ -121,9 +127,16 @@ def create_chat_workflow(executor) -> ConductorWorkflow:
121127
ChatMessage(
122128
role="system",
123129
message=(
124-
"You are an expert in science. Given the context below, "
130+
"You are an expert in science. Given the context provided, "
125131
"generate a follow-up question to dive deeper into the topic. "
126-
"Do not repeat previous questions.\n\n"
132+
"Do not repeat previous questions."
133+
),
134+
),
135+
# The server requires a non-empty user message — a system-only
136+
# conversation fails validation before the task runs.
137+
ChatMessage(
138+
role="user",
139+
message=(
127140
"Context:\n${chat_complete_ref.output.result}\n\n"
128141
"Previous questions:\n"
129142
"${collect_history_ref.input.history}"
@@ -198,6 +211,16 @@ def main():
198211
printed_tasks.add(ref)
199212
time.sleep(2)
200213

214+
# is_completed() is true for any terminal state — verify the workflow
215+
# actually COMPLETED rather than FAILED/TERMINATED/TIMED_OUT.
216+
result = workflow_client.get_workflow(workflow_id=workflow_id, include_tasks=False)
217+
if result.status != "COMPLETED":
218+
print("=" * 70)
219+
print(f"Workflow ended {result.status}: {result.reason_for_incompletion}")
220+
print(f"Full execution: {api_config.ui_host}/execution/{workflow_id}")
221+
print("=" * 70)
222+
raise SystemExit(1)
223+
201224
print("=" * 70)
202225
print("Conversation complete.")
203226
print(f"Full execution: {api_config.ui_host}/execution/{workflow_id}")

‎examples/agents/16i_credentials_langchain.py‎

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -23,19 +23,22 @@
2323
from settings import settings
2424

2525

26+
# Tool callables must live at module level: worker processes are spawned on
27+
# macOS/Windows and re-import this module, so a tool defined inside a factory
28+
# function ("<locals>") cannot be resolved by qualified name (SpawnSafetyError).
29+
def check_github_token() -> str:
30+
"""Check if GitHub token is available in the environment."""
31+
token = os.environ.get("GITHUB_TOKEN", "")
32+
if token:
33+
return f"GitHub token available (starts with {token[:4]}...)"
34+
return "GitHub token is NOT available"
35+
36+
2637
def create_langchain_agent():
2738
"""Create a LangChain agent with a tool that uses GITHUB_TOKEN."""
2839
from langchain.agents import create_agent
2940
from langchain_core.tools import tool as lc_tool
3041

31-
@lc_tool
32-
def check_github_token() -> str:
33-
"""Check if GitHub token is available in the environment."""
34-
token = os.environ.get("GITHUB_TOKEN", "")
35-
if token:
36-
return f"GitHub token available (starts with {token[:4]}...)"
37-
return "GitHub token is NOT available"
38-
3942
model_str = settings.llm_model
4043
# create_agent accepts "provider:model" format (e.g. "openai:gpt-4o")
4144
if "/" in model_str:
@@ -44,7 +47,7 @@ def check_github_token() -> str:
4447

4548
agent = create_agent(
4649
model_str,
47-
tools=[check_github_token],
50+
tools=[lc_tool(check_github_token)],
4851
system_prompt="You are a helpful assistant. Use tools when asked.",
4952
)
5053
return agent

‎examples/agents/16j_credentials_openai_sdk.py‎

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -22,22 +22,27 @@
2222
from conductor.ai.agents import AgentRuntime
2323

2424

25+
# Tool callables must live at module level: worker processes are spawned on
26+
# macOS/Windows and re-import this module, so a tool defined inside a factory
27+
# function ("<locals>") cannot be resolved by qualified name (SpawnSafetyError).
28+
# Keep the plain function importable and apply @function_tool at agent
29+
# construction, so the module global is not rebound to a FunctionTool.
30+
def check_github_auth() -> str:
31+
"""Check if GitHub authentication is available."""
32+
token = os.environ.get("GITHUB_TOKEN", "")
33+
if token:
34+
return f"GitHub token is set (starts with {token[:4]}...)"
35+
return "GitHub token is NOT set"
36+
37+
2538
def create_openai_agent():
2639
"""Create an OpenAI Agent SDK agent with a credential-aware tool."""
2740
from agents import Agent, function_tool
2841

29-
@function_tool
30-
def check_github_auth() -> str:
31-
"""Check if GitHub authentication is available."""
32-
token = os.environ.get("GITHUB_TOKEN", "")
33-
if token:
34-
return f"GitHub token is set (starts with {token[:4]}...)"
35-
return "GitHub token is NOT set"
36-
3742
agent = Agent(
3843
name="github_checker",
3944
instructions="You check GitHub authentication status. Use the tool when asked.",
40-
tools=[check_github_auth],
45+
tools=[function_tool(check_github_auth)],
4146
)
4247
return agent
4348

‎examples/agents/94_openai_runner_tools.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,10 @@ class Weather(BaseModel):
4949
conditions: str = Field(description="The weather conditions")
5050

5151

52-
@function_tool
52+
# Keep the plain function importable at module level and apply
53+
# @function_tool at Agent construction: worker processes are spawned on
54+
# macOS/Windows and re-import this module by qualified name, which fails
55+
# when the decorator has rebound the module global to a FunctionTool.
5356
def get_weather(city: Annotated[str, "The city to get the weather for"]) -> Weather:
5457
"""Get the current weather information for a specified city."""
5558
print("[debug] get_weather called")
@@ -59,7 +62,7 @@ def get_weather(city: Annotated[str, "The city to get the weather for"]) -> Weat
5962
agent = Agent(
6063
name="weather_agent",
6164
instructions="You are a helpful agent.",
62-
tools=[get_weather],
65+
tools=[function_tool(get_weather)],
6366
)
6467

6568

‎examples/agents/README.md‎

Lines changed: 31 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,33 @@ To install all framework dependencies at once:
120120
uv pip install langchain langchain-core langchain-openai langgraph openai-agents google-adk
121121
```
122122

123-
### 2. Configure your environment
123+
### 2. Start a Conductor server
124+
125+
The agent examples need a Conductor server with the **agent runtime**, which is on by
126+
default from **conductor-oss `3.32.0-rc.8`** onward (the same version pinned by this
127+
repo's agent-e2e CI). Older servers — including the `latest` stable line installed by
128+
`conductor server start` (3.30.x at the time of writing) — do not expose the
129+
`/api/agent/*` endpoints, and every example fails with
130+
`AgentNotFoundError: HTTP 404 ... api/agent/start`.
131+
132+
Start a known-good version with the [Conductor CLI](https://github.com/conductor-oss/conductor-cli):
133+
134+
```bash
135+
conductor server start --version 3.32.0-rc.8
136+
```
137+
138+
Or run the boot JAR from Maven Central directly:
139+
140+
```bash
141+
curl -fL -o conductor-server.jar \
142+
"https://repo1.maven.org/maven2/org/conductoross/conductor-server/3.32.0-rc.8/conductor-server-3.32.0-rc.8-boot.jar"
143+
java -jar conductor-server.jar --server.port=8080
144+
```
145+
146+
Export your LLM provider API keys (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`) in the
147+
shell that starts the server — the server auto-enables the matching providers.
148+
149+
### 3. Configure your environment
124150

125151
Export environment variables:
126152

@@ -131,14 +157,14 @@ export AGENTSPAN_SERVER_URL=http://localhost:8080/api
131157
# export AGENTSPAN_AUTH_SECRET=<secret>
132158
```
133159

134-
#### 2.1. Choose a model
160+
#### 3.1. Choose a model
135161

136162
The `AGENTSPAN_LLM_MODEL` variable uses the `provider/model-name` format. Examples:
137163

138164
| Provider | Model string | API key env var |
139165
|----------|-------------|-----------------|
140-
| OpenAI | `anthropic/claude-sonnet-4-6` (default) | `OPENAI_API_KEY` |
141-
| Anthropic | `anthropic/claude-sonnet-4-20250514` | `ANTHROPIC_API_KEY` |
166+
| OpenAI | `openai/gpt-4o-mini` | `OPENAI_API_KEY` |
167+
| Anthropic | `anthropic/claude-sonnet-4-6` (default) | `ANTHROPIC_API_KEY` |
142168
| Google Gemini | `google_gemini/gemini-2.0-flash` | `GOOGLE_GEMINI_API_KEY` |
143169
| AWS Bedrock | `aws_bedrock/...` | AWS credentials |
144170
| Azure OpenAI | `azure_openai/...` | Azure credentials |
@@ -147,7 +173,7 @@ All supported providers: `openai`, `anthropic`, `google_gemini`, `google_vertex_
147173
`azure_openai`, `aws_bedrock`, `cohere`, `mistral`, `groq`, `perplexity`,
148174
`hugging_face`, `deepseek`.
149175

150-
### 3. Run an example
176+
### 4. Run an example
151177

152178
```bash
153179
# Core SDK examples

0 commit comments

Comments
 (0)