Skip to content

Commit 5296e8e

Browse files
committed
update tool call ouput summarizer to log extraction instead - identify top 5 log sections related to user objective
1 parent 948df6d commit 5296e8e

3 files changed

Lines changed: 32 additions & 22 deletions

File tree

packages/core/src/agents/build-test-agent.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -180,7 +180,7 @@ You have access to these tools:
180180
directive: `## Output Format
181181
If you decide to choose a tool to further the task, your response must *ONLY* contain the tool choice in JSON format.
182182
183-
The \`goal\` you provide is critical, as it will be passed to a separate, specialized agent that executes the command. This agent has **no access to our conversation history**. Therefore, your \`goal\` **must be a fully self-contained and actionable instruction**. It needs to include not just *what* to do, but also *why*, so the other agent understands the context.
183+
The \`goal\` you provide is critical, as it will be passed to a separate, specialized agent that executes the command. This agent has **no access to our conversation history**. Therefore, your \`goal\` **must be a fully self-contained and actionable instruction**; it must not refer to previous steps (e.g., it should not say 'specified file' or 'mentioned argument' instead those should all be directly stated). It needs to include not just *what* to do, but also *why*, so the other agent understands the context.
184184
185185
**JSON Structure:**
186186
\`\`\`json
@@ -214,7 +214,7 @@ You have access to these tools:
214214
## Output Format
215215
If you decide to choose a tool to further the task, your response must *ONLY* contain the tool choice in JSON format.
216216
217-
The \`goal\` you provide is critical, as it will be passed to a separate, specialized agent that executes the command. This agent has **no access to our conversation history**. Therefore, your \`goal\` **must be a fully self-contained and actionable instruction**. It needs to include not just *what* to do, but also *why*, so the other agent understands the context.
217+
The \`goal\` you provide is critical, as it will be passed to a separate, specialized agent that executes the command. This agent has **no access to our conversation history**. Therefore, your \`goal\` **must be a fully self-contained and actionable instruction**; it must not refer to previous steps, instead those should all be directly included in the goal. It needs to include not just *what* to do, but also *why*, so the other agent understands the context.
218218
219219
220220
**JSON Structure:**

packages/core/src/services/summarizer.ts

Lines changed: 24 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -11,26 +11,35 @@ import { StreamEventType } from '../core/geminiChat.js';
1111
import { debugLogger } from '../utils/debugLogger.js';
1212
import * as fs from 'node:fs/promises';
1313

14-
const SUMMARIZER_SYSTEM_PROMPT = `## Role
15-
You are an expert Tool Call Output Summarizer.
14+
const SUMMARIZER_SYSTEM_PROMPT = `## Role
15+
You are a Technical Log Extraction Specialist.
1616
1717
## Task Definition
18-
Your task is to summarize a given tool call output. The summary must specifically highlight how the tool call output contributes to or addresses the user's overall objective.
19-
20-
## Instruction
21-
When generating the summary, focus solely on the information within the \`toolcall\` that directly addresses or fulfills aspects of the \`objective\`. **The summary must be as exhaustive as possible, capturing every relevant detail from the tool call, and must be presented as a bulleted list.** Do not include extraneous details or interpret the tool call beyond its direct relevance to the objective.
22-
23-
## Response Format Constraints
24-
The response **must be a bulleted list** containing **up to 20 points** (maximum). Each bullet point should be concise. DO NOT include any explicit explanations, reasoning, or thinking process outside of the bullet points. Your output must ONLY be the bulleted summary.`;
18+
Your task is to identify and extract the **top 5 most relevant sections** from the tool call output that relate to the user's \`<objective>\`.
19+
20+
## Instructions
21+
1. **Prioritize Signal over Noise:** Search the logs for the 5 sections that provide the most conclusive evidence regarding the objective.
22+
- **Priority 1:** Terminal success indicators for the target (e.g., lines with \`✓\`, \`PASS\`, or \`Tests: X passed\`).
23+
- **Priority 2:** Detailed error messages or stack traces if the target failed (e.g., \`✕\`, \`AssertionError\`).
24+
- **Priority 3:** Final process summaries or exit codes.
25+
- **Lowest Priority:** Repetitive "No test files found" or "queued" messages.
26+
2. **Direct Extraction:** Provide these 5 sections as **separate code blocks**.
27+
3. **Preserve Verbatim Text:** Do not summarize, edit, or interpret the text within the blocks. Copy them exactly as they appear in the raw log.
28+
4. **Context Labels:** Before each code block, add a brief one-line label identifying the context (e.g., \`[Context: packages/core]\`).
29+
30+
## Constraints
31+
- Output exactly 5 blocks (or fewer if the total log is very short).
32+
- Do not add analysis or conclusions.
33+
- Output ONLY the labeled code blocks.
34+
`;
2535

2636
const SUMMARIZER_USER_PROMPT = `## Input
27-
User's Objective: {{objective}}
28-
Tool Call Output: {{toolcall}}
29-
30-
## Output Reminder
31-
Take a deep breath, read the instructions again, read the inputs again. Each instruction is crucial and must be executed with utmost care and attention to detail.
37+
User's Objective:
38+
{{objective}}
3239
33-
Summary:`;
40+
Tool Call Output:
41+
{{toolcall}}
42+
`;
3443

3544
export class SummarizationService {
3645
constructor() {}

understand.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -250,13 +250,14 @@ Using the codebase investigator: tell me how the ollama inference works.
250250

251251
<!-- -->
252252

253-
1. chat_history maybe shouldn't have the model's tool choice. actually this
254-
could be fine.
255-
1. Tool call service should take in the full chat history since if the model's
256-
tool choice is provided then the last mesage is just the tool choice action
257-
by the main subagent loop.
253+
1. If the tool call service makes a bad tool call that should be corrected by
254+
the tool call service itself, not be seen by the subagent.
258255
1. When tool service makes a tool call, that should be reflected in the UI. In
259256
general, subagent services should be reflected in the UI.
257+
1. If there is a cancel in the subagent right now the summarizer gets something
258+
like this
259+
`Tool Call Output: [{"text":"{\"id\":\"283a387e-a136-4443-b3cd-1267b58cb5db########0-build_and_test_agent-mchfa2#2-0\",\"name\":\"run_shell_command\",\"response\":{\"error\":\"[Operation Cancelled] User cancelled the operation.\"}}"}]`
260+
and keeps executing, it should be interrupted properly and stop inference.
260261

261262
<!-- For routing -->
262263

0 commit comments

Comments
 (0)