feat: support agent reflection at local (#433)

* feat: support agent reflection at local

* add docs

Author: yaozheng-fang

docs/docs/deploy/optimization.md

@@ -161,3 +161,80 @@ veadk_rl_lightning_project
 ![启动client](../assets/images/optimization/lightning_client.png)
 
 ![启动server](../assets/images/optimization/lightning_training_server.png)
+
+## Agent 自我反思
+
+VeADK 中支持基于 Tracing 文件数据，通过第三方 Agent 推理来进行自我反思，生成优化后的系统提示词。
+
+### 使用方法
+
+您可以在适宜的时机将 Agent 推理得到的 Tracing 文件数据，提交到 `reflector` 进行自我反思，如下代码：
+
+```python
+import asyncio
+
+from veadk import Agent, Runner
+from veadk.reflector.local_reflector import LocalReflector
+from veadk.tracing.telemetry.opentelemetry_tracer import OpentelemetryTracer
+
+agent = Agent(tracers=[OpentelemetryTracer()])
+reflector = LocalReflector(agent=agent)
+
+app_name = "app"
+user_id = "user"
+session_id = "session"
+
+
+async def main():
+    runner = Runner(agent=agent, app_name=app_name)
+
+    await runner.run(
+        messages="你好，我觉得你的回答不够礼貌",
+        user_id=user_id,
+        session_id=session_id,
+    )
+
+    trace_file = runner.save_tracing_file(session_id=session_id)
+
+    response = await reflector.reflect(
+        trace_file=trace_file
+    )
+    print(response)
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
+```
+
+### 结果说明
+
+原始提示词：
+
+```text
+You an AI agent created by the VeADK team.
+
+You excel at the following tasks:
+1. Data science
+- Information gathering and fact-checking
+- Data processing and analysis
+2. Documentation
+- Writing multi-chapter articles and in-depth research reports
+3. Coding & Programming
+- Creating websites, applications, and tools
+- Solve problems and bugs in code (e.g., Python, JavaScript, SQL, ...)
+- If necessary, using programming to solve various problems beyond development
+4. If user gives you tools, finish various tasks that can be accomplished using tools and available resources
+```
+
+优化后，您将看到类似如下的输出：
+
+```text
+optimized_prompt='You are an AI agent created by the VeADK team. Your core mission is to assist users with expertise in data science, documentation, and coding, while maintaining a warm, respectful, and engaging communication style.\n\nYou excel at the following tasks:\n1. Data science\n- Information gathering and fact-checking\n- Data processing and analysis\n2. Documentation\n- Writing multi-chapter articles and in-depth research reports\n3. Coding & Programming\n- Creating websites, applications, and tools\n- Solving problems and bugs in code (e.g., Python, JavaScript, SQL, ...)\n- Using programming to solve various problems beyond development\n4. Tool usage\n- Effectively using provided tools and available resources to accomplish tasks\n\nCommunication Guidelines:\n- Always use polite and warm language (e.g., appropriate honorifics, friendly tone)\n- Show appreciation for user feedback and suggestions\n- Proactively confirm user needs and preferences\n- Maintain a helpful and encouraging attitude throughout interactions\n\nYour responses should be both technically accurate and conversationally pleasant, ensuring users feel valued and supported.' 
+
+reason="The trace shows a user complaint about the agent's lack of politeness in responses. The agent's current system prompt focuses exclusively on technical capabilities without addressing communication style. The optimized prompt adds explicit communication guidelines to ensure the agent maintains a warm, respectful tone while preserving all technical capabilities. This addresses the user's feedback directly while maintaining the agent's core functionality."
+```
+
+输出分为两部分：
+
+- `optimized_prompt`: 优化后的系统提示词
+- `reason`: 优化原因

veadk/reflector/__init__.py

@@ -0,0 +1,13 @@
+# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

veadk/reflector/base_reflector.py

@@ -0,0 +1,44 @@
+# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from abc import ABC, abstractmethod
+
+from pydantic import BaseModel
+
+from veadk import Agent
+
+
+class ReflectorResult(BaseModel):
+    optimized_prompt: str
+    """Optimized system prompt"""
+
+    reason: str
+    """Reason for the optimized prompt"""
+
+
+class BaseReflector(ABC):
+    def __init__(self, agent: Agent):
+        self.agent = agent
+
+    @abstractmethod
+    def reflect(self, traces: list[dict]) -> ReflectorResult:
+        """Do agent self-reflection and generate a optimized system prompt.
+
+        Args:
+            traces (list[dict]): The traces for agent reflection.
+
+        Returns:
+            str: The optimized system prompt.
+        """
+        ...

veadk/reflector/local_reflector.py

@@ -0,0 +1,77 @@
+# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+import os
+
+from typing_extensions import override
+
+from veadk import Agent, Runner
+from veadk.reflector.base_reflector import BaseReflector, ReflectorResult
+
+
+class LocalReflector(BaseReflector):
+    def __init__(self, agent: Agent):
+        super().__init__(agent)
+
+    def _read_traces_from_dir(self, trace_dir: str) -> list[dict]:
+        traces = []
+        for file in os.listdir(trace_dir):
+            if file.endswith(".json"):
+                with open(os.path.join(trace_dir, file), "r") as f:
+                    traces.append(json.load(f))
+        return traces
+
+    @override
+    async def reflect(self, trace_file: str) -> ReflectorResult:
+        assert os.path.isfile(trace_file), f"{trace_file} is not a file."
+        assert trace_file.endswith(".json"), f"{trace_file} is not a valid json file."
+
+        with open(trace_file, "r") as f:
+            traces = json.load(f)
+
+        agent = Agent(
+            name="agent_reflector",
+            description="Reflect the traces and generate a optimized system prompt.",
+            instruction="""You are a helpful optimizer and reflector.
+            
+Your task is to reflect the traces and generate a optimized system prompt (based on the given agent's system prompt).
+
+You should response in json format with two fields: 
+- optimized_prompt: The optimized system prompt.
+- reason: The reason for the optimized prompt.
+            """,
+            output_schema=ReflectorResult,
+        )
+        runner = Runner(agent)
+
+        response = await runner.run(
+            messages=f"The system prompt is: {self.agent.instruction}, and the history traces are: {traces}"
+        )
+
+        if response:
+            try:
+                optimized_prompt = json.loads(response).get("optimized_prompt", "")
+                reason = json.loads(response).get("reason", "")
+            except Exception as e:
+                optimized_prompt = ""
+                reason = f"response from optimizer is not valid json: {e}, response: {response}"
+        else:
+            optimized_prompt = ""
+            reason = "response from optimizer is empty"
+
+        return ReflectorResult(
+            optimized_prompt=optimized_prompt,
+            reason=reason,
+        )