feat: implement llm guidance for repetition tool call#7388
Merged
Conversation
Contributor
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- In
_build_same_tool_guidance, the initialif streak < 3: return ""makes the finalreturnbranch unreachable, so the "By the way" guidance for low streaks can never be emitted; the threshold checks should be reordered so that streak==2 hits the intended branch and the function has a single default return.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `_build_same_tool_guidance`, the initial `if streak < 3: return ""` makes the final `return` branch unreachable, so the "By the way" guidance for low streaks can never be emitted; the threshold checks should be reordered so that streak==2 hits the intended branch and the function has a single default return.
## Individual Comments
### Comment 1
<location path="astrbot/core/agent/runners/tool_loop_agent_runner.py" line_range="440-445" />
<code_context>
+ self._same_tool_streak = 1
+ return self._same_tool_streak
+
+ def _build_same_tool_guidance(self, tool_name: str, streak: int) -> str:
+ if streak < 3:
+ return ""
+
+ if streak >= 5:
+ return (
+ "\n\n[SYSTEM NOTICE] Important: you have executed the same tool "
+ f"`{tool_name}` {streak} times consecutively. Repetition is now very "
</code_context>
<issue_to_address>
**issue (bug_risk):** The `streak < 3` early return makes the final `return` branch unreachable.
Because of the early `if streak < 3: return ""`, execution can never reach any later branch intended for `<3`, `3–4`, and `>=5` tiers. If you want three distinct behaviors, you’ll need to adjust the initial condition (e.g., `if streak < 2`) or reorder/restructure the conditionals so that all tiers are actually reachable.
</issue_to_address>
### Comment 2
<location path="tests/test_tool_loop_agent_runner.py" line_range="596-600" />
<code_context>
+ ]
+ assert len(tool_messages) == 5
+
+ tool_contents = [str(message.content) for message in tool_messages]
+ assert "same tool" not in tool_contents[0]
+ assert "By the way" in tool_contents[1]
+ assert "2 times consecutively" in tool_contents[1]
+ assert "Important" in tool_contents[2]
+ assert "3 times consecutively" in tool_contents[2]
+ assert "Important" in tool_contents[4]
+ assert "5 times consecutively" in tool_contents[4]
+ assert "very high" in tool_contents[4]
+
+
</code_context>
<issue_to_address>
**suggestion (testing):** Make assertions more resilient by checking for the full system notice markers instead of partial English substrings
These assertions rely on partial English phrases ("By the way", "Important", "very high"), which are brittle given that surrounding text is localized and copy may change. To make the tests more stable, assert on structured markers like the `[SYSTEM NOTICE]` prefix and the full streak clause (including backticked tool name and "X times consecutively"), rather than individual adjectives.
```suggestion
tool_messages = [
m for m in runner.run_context.messages if getattr(m, "role", None) == "tool"
]
assert len(tool_messages) == 5
tool_contents = [str(message.content) for message in tool_messages]
import re
# First tool message should not contain the system notice about repeating the same tool
assert "same tool" not in tool_contents[0]
# Second tool message: 2-time streak notice
assert "[SYSTEM NOTICE]" in tool_contents[1]
assert re.search(r"\[SYSTEM NOTICE].*`.+`.*2 times consecutively", tool_contents[1])
# Third tool message: 3-time streak notice
assert "[SYSTEM NOTICE]" in tool_contents[2]
assert re.search(r"\[SYSTEM NOTICE].*`.+`.*3 times consecutively", tool_contents[2])
# Fifth tool message: 5-time streak notice with very high importance
assert "[SYSTEM NOTICE]" in tool_contents[4]
assert re.search(r"\[SYSTEM NOTICE].*`.+`.*5 times consecutively", tool_contents[4])
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to track and discourage repetitive tool calls by the agent. It adds logic to monitor consecutive calls to the same tool and appends escalating system notices to the tool results to guide the LLM toward alternative strategies. Unit tests were added to verify this behavior. Additionally, a CSS constraint was removed from the provider sources panel in the dashboard. Review feedback identifies a logic error in the guidance threshold that renders the first warning level unreachable and warns of potential scrolling issues in the UI due to the CSS change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes: #7387
fixes: #6259
Modifications / 改动点
Screenshots or Test Results / 运行截图或测试结果
Checklist / 检查清单
😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
/ 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。
👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
/ 我的更改经过了良好的测试,并已在上方提供了“验证步骤”和“运行截图”。
🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in
requirements.txtandpyproject.toml./ 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到
requirements.txt和pyproject.toml文件相应位置。😮 My changes do not introduce malicious code.
/ 我的更改没有引入恶意代码。
Summary by Sourcery
Add system guidance to discourage excessive repetition of the same tool in the tool loop agent and adjust UI scrolling behavior for provider sources.
New Features:
Enhancements:
Tests: