Skip to content

Commit 8d3703a

Browse files
leeyyileeyyi
committed
FEAT: 在模型请求失败时能够自动重试
1 parent 8080800 commit 8d3703a

22 files changed

Lines changed: 1519 additions & 306 deletions

‎examples/llmagent_with_model_retry/.env‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Copy this file or edit it in place before running the example.
2+
# The example uses an OpenAI-compatible endpoint.
3+
4+
TRPC_AGENT_API_KEY=
5+
TRPC_AGENT_BASE_URL=
6+
TRPC_AGENT_MODEL_NAME=
7+
8+
# Optional model retry tuning.
9+
# num_retries is the number of retry attempts in addition to the initial call.
10+
TRPC_AGENT_MODEL_RETRY_NUM_RETRIES=2
11+
TRPC_AGENT_MODEL_RETRY_INITIAL_BACKOFF=1.0
12+
TRPC_AGENT_MODEL_RETRY_MAX_BACKOFF=8.0
13+
TRPC_AGENT_MODEL_RETRY_BACKOFF_MULTIPLIER=2.0
14+
TRPC_AGENT_MODEL_RETRY_JITTER=true

‎examples/llmagent_with_model_retry/README.md‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
# LLM Agent 模型重试示例
2+
3+
本示例演示如何在模型构造时传入 `ModelRetryConfig`，让业务代码不用自己实现重试逻辑；当 LLM 请求遇到限流、超时、网络波动等瞬时错误时，SDK 会自动重试。
4+
5+
## 关键特性
6+
7+
- **按需开启重试**：只有显式传入 `ModelRetryConfig` 的模型才会启用 SDK 托管重试。
8+
- **一套配置，多种模型可用**：同一个 `ModelRetryConfig` 可以传给 OpenAI、Anthropic、LiteLLM 等模型实现，统一控制重试次数和指数退避。
9+
- **Provider 自己判断是否重试**：OpenAI、Anthropic、LiteLLM 会按各自 SDK 的错误语义、响应 header 和 HTTP status 判断是否重试。
10+
- **避免重复输出内容**：流式输出已经产生内容后如果再失败，错误会直接透出，不会重试并重复输出内容。
11+
- **优先使用服务端等待时间**：默认会读取 `Retry-After` / `retry-after-ms`，有服务端等待时间时优先使用。
12+
13+
## Agent 层级结构说明
14+
15+
本例是单 Agent 示例，重试配置绑定在模型上：
16+
17+
```text
18+
weather_retry_agent (LlmAgent)
19+
├── model: OpenAIModel(..., model_retry_config=ModelRetryConfig(...))
20+
├── tool: get_weather_report(city)
21+
└── runner: 无自定义重试逻辑
22+
```
23+
24+
关键文件：
25+
26+
- [examples/llmagent_with_model_retry/agent/agent.py](./agent/agent.py)：创建模型并注入 `ModelRetryConfig`
27+
- [examples/llmagent_with_model_retry/agent/config.py](./agent/config.py)：读取模型连接与重试环境变量
28+
- [examples/llmagent_with_model_retry/agent/tools.py](./agent/tools.py)：天气工具实现
29+
- [examples/llmagent_with_model_retry/agent/prompts.py](./agent/prompts.py)：提示词
30+
- [examples/llmagent_with_model_retry/run_agent.py](./run_agent.py)：运行入口，展示业务层无需手写重试
31+
32+
## 关键代码解释
33+
34+
### 1) 创建重试配置
35+
36+
`agent/config.py` 从环境变量构造：
37+
38+
```python
39+
from trpc_agent_sdk.configs import ExponentialBackoffConfig
40+
from trpc_agent_sdk.configs import ModelRetryConfig
41+
42+
ModelRetryConfig(
43+
num_retries=2, # 最多重试 2 次（不包含首次请求）
44+
backoff=ExponentialBackoffConfig(
45+
initial_backoff=1.0, # 首次重试前等待 1 秒
46+
max_backoff=8.0, # 单次重试等待时间最多 8 秒
47+
multiplier=2.0, # 每次失败后等待时间按 2 倍递增
48+
jitter=True, # 加入随机抖动，避免并发重试同时打到服务
49+
),
50+
)
51+
```
52+
53+
### 2) 注入模型
54+
55+
`agent/agent.py` 将配置传给模型构造器：
56+
57+
```python
58+
OpenAIModel(
59+
model_name=model_name,
60+
api_key=api_key,
61+
base_url=base_url,
62+
model_retry_config=retry_config, # 将重试配置注入模型，调用失败时按配置自动重试
63+
)
64+
```
65+
66+
### 3) 配置字段说明
67+
68+
- `num_retries`：初始请求失败后的额外重试次数，不包含第一次请求。
69+
- `initial_backoff`：第一次重试前的基础等待时间，单位秒。
70+
- `max_backoff`：单次等待时间上限，单位秒。
71+
- `multiplier`：指数退避倍数。
72+
- `jitter`：是否启用 full jitter，避免并发请求同时重试。
73+
74+
SDK 会优先尊重 provider 返回的 `Retry-After` / `retry-after-ms` 服务端建议等待时间。
75+
76+
### 4) Runner 不需要重试逻辑
77+
78+
`run_agent.py` 仍然只调用：
79+
80+
```python
81+
async for event in runner.run_async(...):
82+
...
83+
```
84+
85+
如果模型调用在产出内容前遇到可重试错误，SDK 会按 `ModelRetryConfig` 自动重试。
86+
87+
## 会重试和不会重试的场景
88+
89+
不同 provider 会按各自 SDK 的错误语义判断异常类型；对于带响应信息的错误，通用优先级如下：
90+
91+
### 会重试
92+
93+
- 响应 header `x-should-retry: true`。
94+
- HTTP status 为 `408` / `409` / `429` / `>=500`。
95+
- OpenAI / Anthropic 相关的超时、连接类瞬时错误。
96+
97+
### 不会重试
98+
99+
- 响应 header `x-should-retry: false`。
100+
- HTTP status 为其他 `4xx` 错误，例如 `400` / `401` / `403` / `404`。
101+
- 重试次数已耗尽。
102+
- 流式输出已经产生内容后才发生的错误。
103+
104+
## 环境与运行
105+
106+
### 环境要求
107+
108+
- Python 3.10+
109+
110+
### 安装步骤
111+
112+
```bash
113+
git clone https://github.com/trpc-group/trpc-agent-python.git
114+
cd trpc-agent-python
115+
python3 -m venv .venv
116+
source .venv/bin/activate
117+
pip3 install -e .
118+
```
119+
120+
### 环境变量要求
121+
122+
在 [examples/llmagent_with_model_retry/.env](./.env) 中配置（或通过 `export` 设置）：
123+
124+
```bash
125+
# 必填：模型连接配置
126+
TRPC_AGENT_API_KEY=<your-api-key>
127+
TRPC_AGENT_BASE_URL=<openai-compatible-base-url>
128+
TRPC_AGENT_MODEL_NAME=<model-name>
129+
130+
# 可选：模型重试配置；不设置时使用示例默认值
131+
TRPC_AGENT_MODEL_RETRY_NUM_RETRIES=2
132+
TRPC_AGENT_MODEL_RETRY_INITIAL_BACKOFF=1.0
133+
TRPC_AGENT_MODEL_RETRY_MAX_BACKOFF=8.0
134+
TRPC_AGENT_MODEL_RETRY_BACKOFF_MULTIPLIER=2.0
135+
TRPC_AGENT_MODEL_RETRY_JITTER=true
136+
```
137+
138+
### 运行命令
139+
140+
```bash
141+
cd examples/llmagent_with_model_retry
142+
python3 run_agent.py
143+
```
144+
145+
## 运行结果示例
146+
147+
```text
148+
Model retry enabled: {'num_retries': 2, 'backoff': {'initial_backoff': 1.0, 'max_backoff': 8.0, 'multiplier': 2.0, 'jitter': True}}
149+
Session ID: fdb9e370...
150+
User: What's the current weather in Beijing?
151+
Assistant:
152+
Invoke Tool: get_weather_report({'city': 'Beijing'})
153+
Tool Result: {'temperature': '25°C', 'condition': 'Sunny', 'humidity': '60%'}
154+
155+
# 当模型服务返回可重试错误时，SDK 会在模型层自动重试：
156+
[WARNING] Model call failed (exception=RateLimitError); retrying in 0.46s (attempt 1/2).
157+
```

‎examples/llmagent_with_model_retry/agent/__init__.py‎

Whitespace-only changes.

‎examples/llmagent_with_model_retry/agent/agent.py‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Agent module for the model retry example."""
7+
8+
from trpc_agent_sdk.agents import LlmAgent
9+
from trpc_agent_sdk.models import LLMModel
10+
from trpc_agent_sdk.models import OpenAIModel
11+
from trpc_agent_sdk.tools import FunctionTool
12+
13+
from .config import get_model_config
14+
from .config import get_model_retry_config
15+
from .prompts import INSTRUCTION
16+
from .tools import get_weather_report
17+
18+
19+
def _create_model() -> LLMModel:
20+
"""Create an OpenAI-compatible model with SDK-managed retry enabled."""
21+
api_key, base_url, model_name = get_model_config()
22+
retry_config = get_model_retry_config()
23+
print(f"Model retry enabled: {retry_config.model_dump()}")
24+
return OpenAIModel(
25+
model_name=model_name,
26+
api_key=api_key,
27+
base_url=base_url,
28+
model_retry_config=retry_config,
29+
)
30+
31+
32+
def create_agent() -> LlmAgent:
33+
"""Create a weather agent that uses model-level retry."""
34+
return LlmAgent(
35+
name="weather_retry_agent",
36+
description="A weather assistant with SDK-managed model retry enabled.",
37+
model=_create_model(),
38+
instruction=INSTRUCTION,
39+
tools=[FunctionTool(get_weather_report)],
40+
)
41+
42+
43+
root_agent = create_agent()

‎examples/llmagent_with_model_retry/agent/config.py‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Configuration helpers for the model retry example."""
7+
8+
import os
9+
10+
from trpc_agent_sdk.configs import ExponentialBackoffConfig
11+
from trpc_agent_sdk.configs import ModelRetryConfig
12+
13+
14+
def _get_float(name: str, default: float) -> float:
15+
value = os.getenv(name)
16+
if value is None or value == "":
17+
return default
18+
return float(value)
19+
20+
21+
def _get_int(name: str, default: int) -> int:
22+
value = os.getenv(name)
23+
if value is None or value == "":
24+
return default
25+
return int(value)
26+
27+
28+
def get_model_config() -> tuple[str, str, str]:
29+
"""Get model connection settings from environment variables."""
30+
api_key = os.getenv("TRPC_AGENT_API_KEY", "")
31+
base_url = os.getenv("TRPC_AGENT_BASE_URL", "")
32+
model_name = os.getenv("TRPC_AGENT_MODEL_NAME", "")
33+
if not api_key or not base_url or not model_name:
34+
raise ValueError("TRPC_AGENT_API_KEY, TRPC_AGENT_BASE_URL, and "
35+
"TRPC_AGENT_MODEL_NAME must be set in environment variables")
36+
return api_key, base_url, model_name
37+
38+
39+
def get_model_retry_config() -> ModelRetryConfig:
40+
"""Build the opt-in SDK-managed model retry configuration."""
41+
return ModelRetryConfig(
42+
num_retries=_get_int("TRPC_AGENT_MODEL_RETRY_NUM_RETRIES", 2),
43+
backoff=ExponentialBackoffConfig(
44+
initial_backoff=_get_float("TRPC_AGENT_MODEL_RETRY_INITIAL_BACKOFF", 1.0),
45+
max_backoff=_get_float("TRPC_AGENT_MODEL_RETRY_MAX_BACKOFF", 8.0),
46+
multiplier=_get_float("TRPC_AGENT_MODEL_RETRY_BACKOFF_MULTIPLIER", 2.0),
47+
jitter=os.getenv("TRPC_AGENT_MODEL_RETRY_JITTER", "true").strip().lower() in {"1", "true", "yes", "y", "on"},
48+
),
49+
)

‎examples/llmagent_with_model_retry/agent/prompts.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Prompts for the model retry example agent."""
7+
8+
INSTRUCTION = """You are a practical weather assistant.
9+
10+
When the user asks for weather, identify the city and call get_weather_report.
11+
If the city is missing, ask one short clarification question.
12+
After receiving tool results, summarize the weather clearly and mention the retry configuration only if the user asks about reliability.
13+
"""

‎examples/llmagent_with_model_retry/agent/tools.py‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Tools for the model retry example agent."""
7+
8+
9+
def get_weather_report(city: str) -> dict:
10+
"""Get weather information for the specified city."""
11+
weather_data = {
12+
"Beijing": {
13+
"temperature": "25°C",
14+
"condition": "Sunny",
15+
"humidity": "60%",
16+
},
17+
"Shanghai": {
18+
"temperature": "28°C",
19+
"condition": "Cloudy",
20+
"humidity": "70%",
21+
},
22+
"Guangzhou": {
23+
"temperature": "32°C",
24+
"condition": "Thunderstorm",
25+
"humidity": "85%",
26+
},
27+
"Shenzhen": {
28+
"temperature": "30°C",
29+
"condition": "Light rain",
30+
"humidity": "78%",
31+
},
32+
}
33+
return weather_data.get(
34+
city,
35+
{
36+
"temperature": "Unknown",
37+
"condition": "Data not available",
38+
"humidity": "Unknown",
39+
},
40+
)

0 commit comments

Comments
 (0)