Skip to content

Commit d500843

Browse files
Adonis-a233claude
andcommitted
feat(examples): add reproducible eval+optimize closed-loop example
Implements the six-stage Evaluation + Optimization pipeline required by the issue: baseline evaluation, rule-based failure attribution with an in-report accuracy self-check, candidate search (scripted in fake mode, real GEPA via AgentOptimizer.optimize + TargetPrompt in live mode), candidate validation with per-case deltas, a validation-first five-check acceptance gate, and append-only audit persistence under timestamped runs/ directories. - fake mode is deterministic and needs no API key or network calls - live agent bridge retries with exponential backoff and per-call timeout, and accumulates token usage so evaluation spend is audited alongside optimizer spend in the cost gate - optimizer.json is validated at startup (metric weights, gate keys) - attribution, rubric, gate, diff, self-check, and config validation are covered by 33 IO-free unit tests under tests/ - generated reports are gitignored; frozen JSON/Markdown samples are committed under sample_output/ - README ships a design note covering attribution, gating, overfit protection, and the audit trail Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 8080800 commit d500843

15 files changed

Lines changed: 2776 additions & 0 deletions
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Everything below is regenerated on each run. A frozen sample report is
2+
# committed under sample_output/ so reviewers can see the expected shape
3+
# without run-to-run diff churn.
4+
__pycache__/
5+
_sdk_eval_metrics.json
6+
runs/
7+
optimization_report.json
8+
optimization_report.md
Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# Evaluation + Optimization Loop
2+
3+
## 1. Purpose
4+
5+
This example implements the issue requirement for a reproducible Evaluation + Optimization pipeline. It is not only an `AgentOptimizer` quickstart: it wraps optimization with baseline evaluation, failure attribution, validation regression, gate decisions, and audit artifacts.
6+
7+
The default `fake` mode runs without model credentials. The `live` mode uses a real `LlmAgent` bridge and invokes `AgentOptimizer.optimize` against a `TargetPrompt`.
8+
9+
## 2. Pipeline Stages
10+
11+
The pipeline runs six stages:
12+
13+
1. Baseline evaluation: score train and validation sets separately, including metric scores, pass/fail, reasons, and key trace fields.
14+
2. Failure attribution: cluster failures into `final_response_mismatch`, `tool_call_error`, `parameter_error`, `llm_rubric_not_met`, `knowledge_recall_insufficient`, and `format_error`.
15+
3. Optimization execution: fake mode applies a deterministic candidate; live mode calls `AgentOptimizer.optimize` with `TargetPrompt.add_path("system_prompt", ...)`.
16+
4. Candidate validation: rerun train and validation sets and compute per-case deltas such as `new_pass`, `new_fail`, `score_up`, and `score_down`.
17+
5. Acceptance gate: require validation gain, no new hard fail, no key-case regression, no train-up/validation-down overfit, and cost within budget.
18+
6. Audit persistence: write prompt snapshots, scores, deltas, gate reasons, cost, duration, and config snapshots into an append-only per-run directory.
19+
20+
## 3. Directory Layout
21+
22+
```text
23+
examples/optimization/eval_optimize_loop/
24+
├── agent/
25+
│ ├── __init__.py
26+
│ └── agent.py
27+
├── prompts/
28+
│ └── system.md
29+
├── sample_output/
30+
│ ├── optimization_report.sample.json
31+
│ └── optimization_report.sample.md
32+
├── tests/
33+
│ ├── __init__.py
34+
│ └── test_pipeline_units.py
35+
├── train.evalset.json
36+
├── val.evalset.json
37+
├── case_meta.json
38+
├── optimizer.json
39+
├── optimizer.sdk.json
40+
└── run.py
41+
```
42+
43+
## 4. Inputs
44+
45+
- `train.evalset.json`: training evaluation set.
46+
- `val.evalset.json`: validation evaluation set; it must be a different file from train.
47+
- `optimizer.json`: outer-loop configuration for mode, metric weights, fake candidate patch, and gate thresholds. It is validated at startup (`validate_config`): the three metric weights must sum to 1.0 and all gate keys must be present.
48+
- `prompts/system.md`: baseline prompt source registered as the optimization target.
49+
- `case_meta.json`: out-of-schema metadata for key cases, rubric kinds, and attribution hints. The `category` field declares the expected failure category per case; the report's attribution self-check measures rule-based attribution accuracy against it.
50+
- `optimizer.sdk.json`: live-only SDK optimizer config passed to `AgentOptimizer.optimize`. This is also where the GEPA `seed` and the in-run spend cap (`max_metric_calls`) live.
51+
52+
## 5. Outputs
53+
54+
All outputs are generated at runtime and gitignored; a frozen sample report is committed under `sample_output/` for reference.
55+
56+
- `runs/<timestamp>_<run_id>/`: append-only per-run audit directory containing `baseline_prompt.md`, `candidate_prompt.md`, `optimization_report.json`, `optimization_report.md`, and (live only) raw SDK artifacts under `agent_optimizer/` (`RoundRecord`-backed round files, `result.json`, `summary.txt`, `best_prompts/`).
57+
- `runs/latest/`: convenience mirror of the newest run directory.
58+
- `optimization_report.json` / `optimization_report.md`: convenience copies of the newest report at the example root.
59+
60+
The JSON report records baseline/candidate per-case scores and traces, per-case deltas, failure attribution with an accuracy self-check against the expected categories in `case_meta.json`, every gate check with its reason, decision, cost split into optimizer and evaluation spend, token counts, duration, prompt SHA-256 hashes, the GEPA seed, and a full config snapshot.
61+
62+
## 6. Run Modes
63+
64+
Fake mode (no credentials, deterministic):
65+
66+
```bash
67+
python examples/optimization/eval_optimize_loop/run.py --mode fake
68+
```
69+
70+
Live mode:
71+
72+
```bash
73+
# Linux / macOS
74+
export TRPC_AGENT_API_KEY=...
75+
export TRPC_AGENT_BASE_URL=...
76+
export TRPC_AGENT_MODEL_NAME=...
77+
78+
# Windows (PowerShell): $env:TRPC_AGENT_API_KEY = "..."; etc.
79+
python examples/optimization/eval_optimize_loop/run.py --mode live
80+
```
81+
82+
`fake` mode uses a deterministic fake model, fake judge, and scripted candidate so the full loop runs without API keys and with zero network calls. `live` mode uses `agent/agent.py`, creates a fresh `LlmAgent` for each call, and invokes `AgentOptimizer.optimize`.
83+
84+
Environment variables:
85+
86+
| Variable | Default | Meaning |
87+
|----------|---------|---------|
88+
| `EVAL_OPT_LOG_LEVEL` | `INFO` | Log verbosity for the pipeline logger. |
89+
| `EVAL_OPT_USD_PER_1M_TOKENS` | `1.0` | USD price per 1M tokens used to estimate live evaluation cost. |
90+
| `EVAL_OPT_CALL_TIMEOUT` | `120` | Per-call timeout (seconds) for live agent calls. |
91+
| `EVAL_OPT_CALL_ATTEMPTS` | `3` | Max attempts per live agent call (exponential backoff between retries). |
92+
| `EVAL_OPT_CALL_BACKOFF` | `1.0` | Backoff base in seconds (delay = base * 2^attempt + jitter). |
93+
94+
## 7. Cost Accounting And Its Limits
95+
96+
The audit report splits spend into optimizer cost (reported by `AgentOptimizer`) and evaluation cost (estimated from the tokens accumulated across the four baseline/candidate evaluation passes at the `EVAL_OPT_USD_PER_1M_TOKENS` rate). The `cost_budget` gate checks the total.
97+
98+
Be aware of what the gate is and is not: it is a **post-hoc audit check** — it can reject a candidate whose search cost exceeded budget, but the money is already spent by then. The in-run spend cap for live mode is `max_metric_calls` in `optimizer.sdk.json`; size it to your budget before launching a live run.
99+
100+
## 8. Customizing The Agent
101+
102+
Edit `agent/agent.py` when connecting a real business agent.
103+
104+
Key constraints:
105+
106+
- `make_call_agent(prompt_path)` must return an async function with the exact optimizer contract `async (query: str) -> str`.
107+
- `create_agent(prompt_path)` must re-read the prompt file every time so candidates written by `AgentOptimizer` take effect immediately.
108+
- `TargetPrompt.add_path("system_prompt", path)` must point to the same prompt file that the agent actually reads.
109+
- For HTTP, CLI, remote config, or multi-agent pipelines, keep the outer contract the same and replace only the bridge implementation.
110+
111+
The outer report still computes richer trace-style scoring. The SDK optimizer itself receives final-text responses through `call_agent`, so `optimizer.sdk.json` intentionally avoids metrics that require full session traces.
112+
113+
## 9. Design And Validation
114+
115+
Failure attribution is rule-based over structured signals, not case ids. Each case records final response, tool trajectory, rubric sub-scores, and expected/actual tool calls. Rubric failures map to `format_error` or `llm_rubric_not_met`; tool mismatches map to tool, parameter, spurious-call, or knowledge-recall categories.
116+
117+
The gate is validation-first. A candidate is accepted only if validation mean improves by the configured threshold, no new hard fail appears, key validation cases do not regress, train improvement does not coincide with validation loss, and cost is within budget.
118+
119+
The bundled fake candidate intentionally improves two train cases and one validation case while damaging two key validation cases. The expected sample decision is `REJECT`, demonstrating overfit rejection:
120+
121+
```bash
122+
python examples/optimization/eval_optimize_loop/run.py --mode fake
123+
```
124+
125+
```text
126+
train: 0.25 -> 0.7833
127+
validation: 0.7333 -> 0.6667
128+
decision: REJECT
129+
```
130+
131+
## 10. 方案设计说明
132+
133+
**失败归因。** 归因完全基于结构化评测信号而非 case 命名:每条 case 记录 final_response、tool_trajectory、rubric 三个子分与期望/实际工具轨迹。工具轨迹不匹配时按规则判类——期望依赖权威检索工具(`case_meta.json``authoritative_tool`)却未调用,记为知识召回不足;调全期望工具又额外多调,记为多余工具调用;首个工具同名但参数不同,记为参数错误;其余记为工具调用错误。rubric 失败按声明维度映射为格式错误或 rubric 不达标。`case_meta.json` 为每条 case 声明期望类别 `category`,报告内置归因自检,样例 4 条失败 case 全部归因正确。
134+
135+
**接受策略。** gate 以验证集为先,五项可配置检查全部通过才接受候选:验证集均分提升达到阈值、无新增 hard fail、关键 case(`key=true`)不退化、非过拟合、总成本(优化器花费加按 token 估算的评测花费)不超预算。各检查相互独立,拒绝理由逐项落盘,便于定位。
136+
137+
**防过拟合。** 训练集与验证集物理分离且启动时校验为不同文件;第四项检查专门拦截"训练集提升、验证集下降"的候选;关键 case 退化与新增 hard fail 两项提供正交的二次保险,即便总分变化很小也能拦住有害候选。随附样例即演示该场景:候选使训练集 +0.53 但验证集回落,gate 正确拒绝。
138+
139+
**产物审计。** 每次运行写入独立的 `runs/<时间戳>_<run_id>/` 目录(append-only,历史不被覆盖),含 baseline 与候选 prompt 快照、JSON/Markdown 双报告;报告记录逐 case 分数与轨迹、逐 case delta、归因统计与自检准确率、每项 gate 检查与决策理由、成本与 token 的优化器/评测拆分、耗时、GEPA 随机种子、prompt 的 SHA-256 以及完整配置快照。run_id 同时注入日志行,跨产物可对齐追溯。
140+
141+
## 11. Tests
142+
143+
The attribution rules, rubric scorer, gate checks, case diffing, and config validation are covered by IO-free unit tests:
144+
145+
```bash
146+
python -m pytest examples/optimization/eval_optimize_loop/tests -q
147+
```
148+
149+
Known limits: live mode requires SDK dependencies plus `TRPC_AGENT_API_KEY`, `TRPC_AGENT_BASE_URL`, and `TRPC_AGENT_MODEL_NAME`; no-key environments should use `--mode fake`. The live retry logic matches rate-limit errors by exception type name because provider SDK exception classes are intentionally not imported here.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"""Agent bridge package for the eval_optimize_loop example."""
Lines changed: 197 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Live agent bridge for the eval_optimize_loop example.
7+
8+
The optimizer contract is intentionally small: ``call_agent`` is an async
9+
function that accepts one user query and returns the final response text. This
10+
module re-reads the prompt file on every invocation so prompt candidates written
11+
by AgentOptimizer take effect immediately.
12+
13+
The public bridge in this file mirrors the SDK docs:
14+
15+
* ``create_agent`` builds a fresh ``LlmAgent`` from the current prompt file.
16+
* ``run_agent`` drives that agent through ``Runner`` and ``InMemorySessionService``.
17+
* ``make_call_agent`` returns the exact async callable required by
18+
``AgentOptimizer.optimize`` when a ``TargetPrompt`` is registered.
19+
"""
20+
21+
from __future__ import annotations
22+
23+
import asyncio
24+
import logging
25+
import os
26+
import random
27+
import uuid
28+
from pathlib import Path
29+
from typing import Any
30+
from typing import Awaitable
31+
from typing import Callable
32+
33+
from trpc_agent_sdk.agents import LlmAgent
34+
from trpc_agent_sdk.models import OpenAIModel
35+
from trpc_agent_sdk.runners import Runner
36+
from trpc_agent_sdk.sessions import InMemorySessionService
37+
from trpc_agent_sdk.tools import FunctionTool
38+
from trpc_agent_sdk.types import Content
39+
from trpc_agent_sdk.types import Part
40+
41+
42+
APP_NAME = "eval_optimize_loop"
43+
LOGGER = logging.getLogger("eval_optimize_loop.agent")
44+
45+
# Live-call resilience knobs. A single flaky network call must not abort the
46+
# whole pipeline after real money was already spent on earlier evaluations.
47+
CALL_TIMEOUT_SECONDS = float(os.getenv("EVAL_OPT_CALL_TIMEOUT", "120"))
48+
CALL_MAX_ATTEMPTS = int(os.getenv("EVAL_OPT_CALL_ATTEMPTS", "3"))
49+
CALL_BACKOFF_BASE_SECONDS = float(os.getenv("EVAL_OPT_CALL_BACKOFF", "1.0"))
50+
51+
52+
def is_retryable(exc: BaseException) -> bool:
53+
"""Decide whether a failed live call is worth retrying.
54+
55+
Connection problems and timeouts are always retryable. Provider SDK
56+
exception classes are not imported here (the example must stay usable
57+
without them), so rate-limit style errors are matched by type name.
58+
"""
59+
if isinstance(exc, (asyncio.TimeoutError, ConnectionError, OSError)):
60+
return True
61+
name = type(exc).__name__.lower()
62+
return "ratelimit" in name or "timeout" in name or "connection" in name
63+
64+
65+
def lookup_order(order_id: str) -> str:
66+
"""FunctionTool body used by the live ``LlmAgent`` example."""
67+
data = {
68+
"A100": "Order A100 is in transit and arrives on Friday.",
69+
"A200": "Order A200 is delivered.",
70+
}
71+
return data.get(order_id, f"No order record found for {order_id}.")
72+
73+
74+
def search_policy(topic: str) -> str:
75+
"""FunctionTool body for policy and warranty lookup examples."""
76+
topic_lower = topic.lower()
77+
if "damaged" in topic_lower or "refund" in topic_lower:
78+
return "Damaged items are eligible for a full refund within 30 days."
79+
if "model z" in topic_lower or "warranty" in topic_lower:
80+
return "Model Z has a 24-month warranty."
81+
return "No matching policy snippet was found."
82+
83+
84+
def get_model_config() -> tuple[str, str, str]:
85+
"""Read live model credentials consumed by ``OpenAIModel``."""
86+
api_key = os.getenv("TRPC_AGENT_API_KEY", "")
87+
base_url = os.getenv("TRPC_AGENT_BASE_URL", "")
88+
model_name = os.getenv("TRPC_AGENT_MODEL_NAME", "")
89+
if not api_key or not base_url or not model_name:
90+
raise ValueError(
91+
"Live mode requires TRPC_AGENT_API_KEY, TRPC_AGENT_BASE_URL, and "
92+
"TRPC_AGENT_MODEL_NAME. Use --mode fake for the no-key path."
93+
)
94+
return api_key, base_url, model_name
95+
96+
97+
def create_agent(prompt_path: Path) -> LlmAgent:
98+
"""Create a fresh ``LlmAgent`` from the current prompt file.
99+
100+
Re-reading here is the critical TargetPrompt contract: when
101+
``AgentOptimizer`` writes a candidate prompt, the next call immediately uses
102+
that candidate without restarting the process.
103+
"""
104+
api_key, base_url, model_name = get_model_config()
105+
instruction = Path(prompt_path).read_text(encoding="utf-8").strip()
106+
return LlmAgent(
107+
name="support_assistant",
108+
description="A support assistant whose system prompt is under optimization.",
109+
model=OpenAIModel(model_name=model_name, api_key=api_key, base_url=base_url),
110+
instruction=instruction,
111+
tools=[FunctionTool(lookup_order), FunctionTool(search_policy)],
112+
)
113+
114+
115+
async def run_agent_once(query: str, prompt_path: Path) -> dict[str, Any]:
116+
"""Run the live agent once and collect text, tool calls, and token usage.
117+
118+
``AgentOptimizer.optimize`` only needs final response text, but the outer
119+
issue-level report also wants key trajectory information and per-call token
120+
counts for the cost audit. This richer helper supports all of them.
121+
"""
122+
agent = create_agent(prompt_path)
123+
session_service = InMemorySessionService()
124+
runner = Runner(app_name=APP_NAME, agent=agent, session_service=session_service)
125+
session_id = str(uuid.uuid4())
126+
user_id = "optimizer"
127+
await session_service.create_session(
128+
app_name=APP_NAME,
129+
user_id=user_id,
130+
session_id=session_id,
131+
state={},
132+
)
133+
message = Content(role="user", parts=[Part.from_text(text=query)])
134+
final_text = ""
135+
tools: list[dict[str, Any]] = []
136+
tokens = 0
137+
async for event in runner.run_async(
138+
user_id=user_id,
139+
session_id=session_id,
140+
new_message=message,
141+
):
142+
usage = getattr(event, "usage_metadata", None)
143+
if usage is not None:
144+
tokens += getattr(usage, "total_token_count", None) or 0
145+
if not event.content or not event.content.parts:
146+
continue
147+
for part in event.content.parts:
148+
function_call = getattr(part, "function_call", None)
149+
if function_call is not None:
150+
tools.append(
151+
{
152+
"name": getattr(function_call, "name", None),
153+
"args": dict(getattr(function_call, "args", {}) or {}),
154+
}
155+
)
156+
if event.is_final_response():
157+
for part in event.content.parts:
158+
if getattr(part, "text", None) and not getattr(part, "thought", False):
159+
final_text += part.text
160+
return {"text": final_text.strip(), "tools": tools, "tokens": tokens}
161+
162+
163+
async def run_agent(query: str, prompt_path: Path) -> dict[str, Any]:
164+
"""Run the live agent with timeout plus exponential-backoff retry.
165+
166+
Each attempt builds a fresh agent and session, so retrying is idempotent.
167+
Non-retryable errors (see ``is_retryable``) propagate immediately.
168+
"""
169+
for attempt in range(1, CALL_MAX_ATTEMPTS + 1):
170+
try:
171+
return await asyncio.wait_for(
172+
run_agent_once(query=query, prompt_path=prompt_path),
173+
timeout=CALL_TIMEOUT_SECONDS,
174+
)
175+
except Exception as exc:
176+
if attempt == CALL_MAX_ATTEMPTS or not is_retryable(exc):
177+
raise
178+
delay = CALL_BACKOFF_BASE_SECONDS * 2 ** (attempt - 1) + random.uniform(0, 0.5)
179+
LOGGER.warning(
180+
"live agent call failed (%s: %s), retry %d/%d in %.1fs",
181+
type(exc).__name__,
182+
str(exc)[:200],
183+
attempt,
184+
CALL_MAX_ATTEMPTS - 1,
185+
delay,
186+
)
187+
await asyncio.sleep(delay)
188+
raise RuntimeError("unreachable: retry loop exited without returning or raising")
189+
190+
191+
def make_call_agent(prompt_path: Path) -> Callable[[str], Awaitable[str]]:
192+
"""Return the fixed async ``(query: str) -> str`` bridge required by GEPA."""
193+
194+
async def call_agent(query: str) -> str:
195+
return (await run_agent(query=query, prompt_path=prompt_path))["text"]
196+
197+
return call_agent
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
{
2+
"_comment": "Per-case metadata for attribution, gate checks, and fake/live trace scoring. It is kept outside evalsets so EvalSet schema validation remains clean.",
3+
"train_order_lookup_optimizable": {
4+
"category": "tool_call_error",
5+
"key": false,
6+
"rubric": "none"
7+
},
8+
"train_refund_policy_optimizable": {
9+
"category": "knowledge_recall_insufficient",
10+
"key": false,
11+
"rubric": "none",
12+
"authoritative_tool": "search_policy"
13+
},
14+
"train_json_format_ineffective": {
15+
"category": "format_error",
16+
"key": false,
17+
"rubric": "json_format"
18+
},
19+
"val_warranty_new_pass": {
20+
"category": "knowledge_recall_insufficient",
21+
"key": false,
22+
"rubric": "none",
23+
"authoritative_tool": "search_policy"
24+
},
25+
"val_smalltalk_regression": {
26+
"category": "spurious_tool_call",
27+
"key": true,
28+
"rubric": "no_tool"
29+
},
30+
"val_order_soft_degradation": {
31+
"category": "spurious_tool_call",
32+
"key": true,
33+
"rubric": "single_tool"
34+
}
35+
}

0 commit comments

Comments
 (0)