Skip to content

Commit 9774bb7

Browse files
leeyyileeyyi
committed
FEAT: 在模型请求失败时能够自动重试
1 parent 32d7af9 commit 9774bb7

17 files changed

Lines changed: 1155 additions & 49 deletions

File tree

‎examples/llmagent_with_model_retry/.env‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Copy this file or edit it in place before running the example.
2+
# The example uses an OpenAI-compatible endpoint.
3+
4+
TRPC_AGENT_API_KEY=
5+
TRPC_AGENT_BASE_URL=
6+
TRPC_AGENT_MODEL_NAME=
7+
8+
# Optional model retry tuning.
9+
# num_retries is the number of retry attempts in addition to the initial call.
10+
TRPC_AGENT_MODEL_RETRY_NUM_RETRIES=2
11+
TRPC_AGENT_MODEL_RETRY_INITIAL_BACKOFF=1.0
12+
TRPC_AGENT_MODEL_RETRY_MAX_BACKOFF=8.0
13+
TRPC_AGENT_MODEL_RETRY_BACKOFF_MULTIPLIER=2.0
14+
TRPC_AGENT_MODEL_RETRY_JITTER=true
15+
TRPC_AGENT_MODEL_RETRY_RESPECT_RETRY_AFTER=true

‎examples/llmagent_with_model_retry/README.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# LLM Agent 模型重试示例
2+
3+
本示例演示如何在模型构造时传入 `ModelRetryConfig`，让业务代码不用自己实现重试逻辑；当 LLM 请求遇到限流、超时、网络波动等瞬时错误时，SDK 会自动重试。
4+
5+
## 关键特性
6+
7+
- **按需开启重试**：只有显式传入 `ModelRetryConfig` 的模型才会启用 SDK 托管重试。
8+
- **一套配置，多种模型可用**：同一个 `ModelRetryConfig` 可以传给 OpenAI、Anthropic 等模型实现，统一控制重试行为。
9+
- **只重试临时问题**：限流、超时、服务端内部错误、连接异常会在次数限制内重试。
10+
- **避免重复输出内容**：流式输出已经产生内容后如果再失败，错误会直接透出，不会重试并重复输出内容。
11+
- **优先使用服务端等待时间**：默认会读取 `Retry-After` / `retry-after-ms`，有服务端等待时间时优先使用。
12+
13+
## Agent 层级结构说明
14+
15+
本例是单 Agent 示例，重试配置绑定在模型上：
16+
17+
```text
18+
weather_retry_agent (LlmAgent)
19+
├── model: OpenAIModel(..., model_retry_config=ModelRetryConfig(...))
20+
├── tool: get_weather_report(city)
21+
└── runner: 无自定义重试逻辑
22+
```
23+
24+
关键文件：
25+
26+
- [examples/llmagent_with_model_retry/agent/agent.py](./agent/agent.py)：创建模型并注入 `ModelRetryConfig`
27+
- [examples/llmagent_with_model_retry/agent/config.py](./agent/config.py)：读取模型连接与重试环境变量
28+
- [examples/llmagent_with_model_retry/agent/tools.py](./agent/tools.py)：天气工具实现
29+
- [examples/llmagent_with_model_retry/agent/prompts.py](./agent/prompts.py)：提示词
30+
- [examples/llmagent_with_model_retry/run_agent.py](./run_agent.py)：运行入口，展示业务层无需手写重试
31+
32+
## 关键代码解释
33+
34+
### 1) 创建重试配置
35+
36+
`agent/config.py` 从环境变量构造：
37+
38+
```python
39+
ModelRetryConfig(
40+
num_retries=2,
41+
initial_backoff=1.0,
42+
max_backoff=8.0,
43+
backoff_multiplier=2.0,
44+
jitter=True,
45+
respect_retry_after=True,
46+
)
47+
```
48+
49+
### 2) 注入模型
50+
51+
`agent/agent.py` 将配置传给模型构造器：
52+
53+
```python
54+
OpenAIModel(
55+
model_name=model_name,
56+
api_key=api_key,
57+
base_url=base_url,
58+
model_retry_config=retry_config,
59+
)
60+
```
61+
62+
### 3) Runner 不需要重试逻辑
63+
64+
`run_agent.py` 仍然只调用：
65+
66+
```python
67+
async for event in runner.run_async(...):
68+
...
69+
```
70+
71+
如果模型调用在产出内容前遇到可重试错误，SDK 会按 `ModelRetryConfig` 自动重试。
72+
73+
## 会重试和不会重试的场景
74+
75+
### 会重试
76+
77+
- `429` 限流
78+
- `408` / `409` 超时或模型服务锁等待超时
79+
- `5xx` 服务端内部错误
80+
- 超时异常
81+
- 连接或传输异常
82+
83+
### 不会重试
84+
85+
- `400` 错误请求
86+
- `401` / `403` 认证或权限错误
87+
- 未识别错误类别
88+
- 重试次数已耗尽
89+
- 流式输出已经产生内容后才发生的错误
90+
91+
## 环境与运行
92+
93+
### 环境要求
94+
95+
- Python 3.10+
96+
97+
### 安装步骤
98+
99+
```bash
100+
git clone https://github.com/trpc-group/trpc-agent-python.git
101+
cd trpc-agent-python
102+
python3 -m venv .venv
103+
source .venv/bin/activate
104+
pip3 install -e .
105+
```
106+
107+
### 环境变量要求
108+
109+
在 [examples/llmagent_with_model_retry/.env](./.env) 中配置（或通过 `export` 设置）：
110+
111+
```bash
112+
# 必填：模型连接配置
113+
TRPC_AGENT_API_KEY=<your-api-key>
114+
TRPC_AGENT_BASE_URL=<openai-compatible-base-url>
115+
TRPC_AGENT_MODEL_NAME=<model-name>
116+
117+
# 可选：模型重试配置；不设置时使用示例默认值
118+
TRPC_AGENT_MODEL_RETRY_NUM_RETRIES=2
119+
TRPC_AGENT_MODEL_RETRY_INITIAL_BACKOFF=1.0
120+
TRPC_AGENT_MODEL_RETRY_MAX_BACKOFF=8.0
121+
TRPC_AGENT_MODEL_RETRY_BACKOFF_MULTIPLIER=2.0
122+
TRPC_AGENT_MODEL_RETRY_JITTER=true
123+
TRPC_AGENT_MODEL_RETRY_RESPECT_RETRY_AFTER=true
124+
```
125+
126+
### 运行命令
127+
128+
```bash
129+
cd examples/llmagent_with_model_retry
130+
python3 run_agent.py
131+
```
132+
133+
## 运行结果示例
134+
135+
```text
136+
Model retry enabled: {'num_retries': 2, 'initial_backoff': 1.0, 'max_backoff': 8.0, 'backoff_multiplier': 2.0, 'jitter': True, 'respect_retry_after': True}
137+
Session ID: fdb9e370...
138+
SDK-managed retry is configured on the model; no retry loop is needed in this runner.
139+
User: What's the current weather in Beijing?
140+
Assistant:
141+
Invoke Tool: get_weather_report({'city': 'Beijing'})
142+
Tool Result: {'temperature': '25°C', 'condition': 'Sunny', 'humidity': '60%'}
143+
144+
# 当模型服务返回可重试错误时，SDK 会在模型层自动重试：
145+
[WARNING] Model call failed (category=rate_limit); retrying in 0.46s (attempt 1/2).
146+
```
147+
148+
## 适用场景建议
149+
150+
- 在模型服务限流，或其他瞬时错误时自动重试
151+
- 需要结合服务端 `Retry-After` 控制退避等待时间。

‎examples/llmagent_with_model_retry/agent/__init__.py‎

Whitespace-only changes.

‎examples/llmagent_with_model_retry/agent/agent.py‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Agent module for the model retry example."""
7+
8+
from trpc_agent_sdk.agents import LlmAgent
9+
from trpc_agent_sdk.models import LLMModel
10+
from trpc_agent_sdk.models import OpenAIModel
11+
from trpc_agent_sdk.tools import FunctionTool
12+
13+
from .config import get_model_config
14+
from .config import get_model_retry_config
15+
from .prompts import INSTRUCTION
16+
from .tools import get_weather_report
17+
18+
19+
def _create_model() -> LLMModel:
20+
"""Create an OpenAI-compatible model with SDK-managed retry enabled."""
21+
api_key, base_url, model_name = get_model_config()
22+
retry_config = get_model_retry_config()
23+
print(f"Model retry enabled: {retry_config.model_dump()}")
24+
return OpenAIModel(
25+
model_name=model_name,
26+
api_key=api_key,
27+
base_url=base_url,
28+
model_retry_config=retry_config,
29+
)
30+
31+
32+
def create_agent() -> LlmAgent:
33+
"""Create a weather agent that uses model-level retry."""
34+
return LlmAgent(
35+
name="weather_retry_agent",
36+
description="A weather assistant with SDK-managed model retry enabled.",
37+
model=_create_model(),
38+
instruction=INSTRUCTION,
39+
tools=[FunctionTool(get_weather_report)],
40+
)
41+
42+
43+
root_agent = create_agent()

‎examples/llmagent_with_model_retry/agent/config.py‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Configuration helpers for the model retry example."""
7+
8+
import os
9+
10+
from trpc_agent_sdk.configs import ModelRetryConfig
11+
12+
13+
def _get_bool(name: str, default: bool) -> bool:
14+
value = os.getenv(name)
15+
if value is None or value == "":
16+
return default
17+
return value.strip().lower() in {"1", "true", "yes", "y", "on"}
18+
19+
20+
def _get_float(name: str, default: float) -> float:
21+
value = os.getenv(name)
22+
if value is None or value == "":
23+
return default
24+
return float(value)
25+
26+
27+
def _get_int(name: str, default: int) -> int:
28+
value = os.getenv(name)
29+
if value is None or value == "":
30+
return default
31+
return int(value)
32+
33+
34+
def get_model_config() -> tuple[str, str, str]:
35+
"""Get model connection settings from environment variables."""
36+
api_key = os.getenv("TRPC_AGENT_API_KEY", "")
37+
base_url = os.getenv("TRPC_AGENT_BASE_URL", "")
38+
model_name = os.getenv("TRPC_AGENT_MODEL_NAME", "")
39+
if not api_key or not base_url or not model_name:
40+
raise ValueError("TRPC_AGENT_API_KEY, TRPC_AGENT_BASE_URL, and "
41+
"TRPC_AGENT_MODEL_NAME must be set in environment variables")
42+
return api_key, base_url, model_name
43+
44+
45+
def get_model_retry_config() -> ModelRetryConfig:
46+
"""Build the opt-in SDK-managed model retry configuration."""
47+
return ModelRetryConfig(
48+
num_retries=_get_int("TRPC_AGENT_MODEL_RETRY_NUM_RETRIES", 2),
49+
initial_backoff=_get_float("TRPC_AGENT_MODEL_RETRY_INITIAL_BACKOFF", 1.0),
50+
max_backoff=_get_float("TRPC_AGENT_MODEL_RETRY_MAX_BACKOFF", 8.0),
51+
backoff_multiplier=_get_float("TRPC_AGENT_MODEL_RETRY_BACKOFF_MULTIPLIER", 2.0),
52+
jitter=_get_bool("TRPC_AGENT_MODEL_RETRY_JITTER", True),
53+
respect_retry_after=_get_bool("TRPC_AGENT_MODEL_RETRY_RESPECT_RETRY_AFTER", True),
54+
)

‎examples/llmagent_with_model_retry/agent/prompts.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Prompts for the model retry example agent."""
7+
8+
INSTRUCTION = """You are a practical weather assistant.
9+
10+
When the user asks for weather, identify the city and call get_weather_report.
11+
If the city is missing, ask one short clarification question.
12+
After receiving tool results, summarize the weather clearly and mention the retry configuration only if the user asks about reliability.
13+
"""

‎examples/llmagent_with_model_retry/agent/tools.py‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Tools for the model retry example agent."""
7+
8+
9+
def get_weather_report(city: str) -> dict:
10+
"""Get weather information for the specified city."""
11+
weather_data = {
12+
"Beijing": {
13+
"temperature": "25°C",
14+
"condition": "Sunny",
15+
"humidity": "60%",
16+
},
17+
"Shanghai": {
18+
"temperature": "28°C",
19+
"condition": "Cloudy",
20+
"humidity": "70%",
21+
},
22+
"Guangzhou": {
23+
"temperature": "32°C",
24+
"condition": "Thunderstorm",
25+
"humidity": "85%",
26+
},
27+
"Shenzhen": {
28+
"temperature": "30°C",
29+
"condition": "Light rain",
30+
"humidity": "78%",
31+
},
32+
}
33+
return weather_data.get(
34+
city,
35+
{
36+
"temperature": "Unknown",
37+
"condition": "Data not available",
38+
"humidity": "Unknown",
39+
},
40+
)

0 commit comments

Comments
 (0)