feat(service-ai): steer the agent to visualize_data for chart requests#1821
Merged
Conversation
Small models (e.g. claude-haiku) sometimes answered a "draw a bar chart" request with a markdown TABLE — running query_data/aggregate_data and formatting the numbers — instead of calling visualize_data. This was a tool-selection problem, not a capability gap: the chart preference was buried as guideline #7 and competed with guideline #4 ("format with markdown tables"). - data-explorer-skill.ts: add a prominent "Choosing the right tool" section ABOVE the guidelines — chart intent (incl. CN terms 图表/柱状图/折线图/饼图/画图) → MUST call visualize_data; never substitute a table; reconcile the table-formatting guideline; fix duplicate guideline numbering. - visualize-data.tool.ts: strengthen the tool description to be imperative ("the ONLY tool that draws a chart… you MUST call this, not a table; if you already fetched the numbers, still call visualize_data to render them"). Prompt-only tuning — no behavior/contract change. Raises the likelihood the model reaches for visualize_data on a plain chart request without an explicit "use visualize_data" nudge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Follow-up to #1820. In live testing, a small model (claude-haiku) sometimes answered a "draw a bar chart" request with a markdown table — running
query_data/aggregate_dataand formatting the numbers — instead of callingvisualize_data. The chart still worked when explicitly told "use visualize_data", so this is a tool-selection problem, not a capability gap: the chart preference was buried as guideline #7 and competed with guideline #4 ("format with markdown tables").What
data-explorer-skill.ts— adds a prominent "Choosing the right tool" section above the guidelines: chart intent (incl. Chinese terms 图表/柱状图/折线图/饼图/画图/可视化) → MUST callvisualize_data, never substitute a table, and render even if the numbers were already fetched. Reconciles the table-formatting guideline and fixes the duplicate guideline numbering.visualize-data.tool.ts— strengthens the tool description to be imperative ("the ONLY tool that draws a chart… you MUST call this, not a table").Prompt-only tuning — no behavior/contract change, no new deps. Raises the likelihood the model reaches for
visualize_dataon a plain chart request without an explicit nudge. Inherently probabilistic (LLM tool selection), so this improves reliability rather than guaranteeing it.🤖 Generated with Claude Code