Skip to content

Commit 05120f3

Browse files
declan-scaleclaude
andcommitted
test(harness): add parallel-tools + tool-error conformance fixture
Per review (AGX1-373): the existing fixtures only exercise one tool at a time, so cross-channel equivalence was proven over trivially-orderable streams — it could not catch the two channels agreeing with each other while both mishandling parallel/interleaved spans or a failing tool. Adds a fixture where two tool spans are open simultaneously (one opened via the streamed Start+Done path, one via Full), close in a different order than they opened, and one returns an error. The tool error uses today's "Error: ..." content convention; once ToolResponseContent.is_error lands (AGX1-371) the fixture should also assert error status on the span. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 4a6281e commit 05120f3

1 file changed

Lines changed: 68 additions & 0 deletions

File tree

tests/lib/core/harness/conformance/test_conformance.py

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,74 @@
165165
),
166166
],
167167
),
168+
# fixture 5: parallel tool calls + a tool that errors (AGX1-373 review,
169+
# danielmillerp). The earlier fixtures only exercise one tool at a time, so
170+
# equivalence is proven over trivially-orderable streams. This stresses the
171+
# representative case: two tool spans open SIMULTANEOUSLY (p-ls opens via the
172+
# streamed Start+Done path, p-read opens via Full while p-ls is still open),
173+
# then close in a different order than they opened, and one of them returns
174+
# an error. It guards against the two channels agreeing with each other while
175+
# both mishandling interleaved/parallel spans or a failing tool.
176+
#
177+
# The tool error is represented the way the harness encodes it today — an
178+
# "Error: ..." string in ToolResponseContent.content (see
179+
# claude_agents/hooks/hooks.py post_tool_use_failure_hook). Once the deferred
180+
# ToolResponseContent.is_error field lands (AGX1-371), extend this fixture to
181+
# assert the error status propagates onto the closed tool span.
182+
Fixture(
183+
name="parallel-tools-with-error",
184+
events=[
185+
# p-ls: streamed tool_request (opens its span at Done).
186+
StreamTaskMessageStart(
187+
type="start",
188+
index=0,
189+
content=ToolRequestContent(
190+
type="tool_request",
191+
author="agent",
192+
tool_call_id="p-ls",
193+
name="Bash",
194+
arguments={"command": "ls /nope"},
195+
),
196+
),
197+
StreamTaskMessageDone(type="done", index=0),
198+
# p-read: Full tool_request opens a second span while p-ls is open.
199+
StreamTaskMessageFull(
200+
type="full",
201+
index=1,
202+
content=ToolRequestContent(
203+
type="tool_request",
204+
author="agent",
205+
tool_call_id="p-read",
206+
name="Read",
207+
arguments={"path": "/etc/hosts"},
208+
),
209+
),
210+
# p-ls errors and closes first (close order != open order).
211+
StreamTaskMessageFull(
212+
type="full",
213+
index=2,
214+
content=ToolResponseContent(
215+
type="tool_response",
216+
author="agent",
217+
tool_call_id="p-ls",
218+
name="Bash",
219+
content="Error: ls: /nope: No such file or directory",
220+
),
221+
),
222+
# p-read succeeds and closes second.
223+
StreamTaskMessageFull(
224+
type="full",
225+
index=3,
226+
content=ToolResponseContent(
227+
type="tool_response",
228+
author="agent",
229+
tool_call_id="p-read",
230+
name="Read",
231+
content="127.0.0.1 localhost",
232+
),
233+
),
234+
],
235+
),
168236
]
169237

170238
# Register all fixtures for backward-compatible use via all_fixtures()

0 commit comments

Comments
 (0)