AmazonBedrockChatGenerator + ToolInvoker don't handle multiple tool calls at once properly

[vblagoje]

Description:

When using AmazonBedrockChatGenerator with a model capable of requesting multiple tool calls in a single turn (like Anthropic Claude 3.5 Sonnet), the component fails when attempting to send the results of these multiple tool executions back to the Bedrock API.

Steps Observed:

1. The Haystack Agent calls AmazonBedrockChatGenerator.
2. The Bedrock model (anthropic.claude-3-5-sonnet-20240620-v1:0) responds with a message containing multiple distinct tool_calls requests, each with a unique ID. For example:
   • Call 1: { "id": "tooluse_TIB5o2PCQHW_fpWTOwvROA", "function": { "name": "get_weather_forecast", ... } }
   • Call 2: { "id": "tooluse_45K4_G22SEaolJj1FAD14w", "function": { "name": "qdrant-find", ... } }
3. The ToolInvoker (within the Agent) successfully receives these requests and executes the corresponding tools (get_weather_forecast and qdrant-find).
4. AmazonBedrockChatGenerator attempts to continue the conversation by sending the results of both tool executions back to the Bedrock ConverseStream API.
5. The call to client.converse_stream fails with a botocore.errorfactory.ValidationException.

Error Message:

botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: Expected toolResult blocks at messages.4.content for the following Ids: tooluse_45K4_G22SEaolJj1FAD14w, but found: tooluse_TIB5o2PCQHW_fpWTOwvROA

Analysis:

The error message clearly indicates that the Bedrock API was expecting the result associated with the qdrant-find tool call (tooluse_45K4_G22SEaolJj1FAD14w) but instead received the result associated with the get_weather_forecast tool call (tooluse_TIB5o2PCQHW_fpWTOwvROA) at that position in the message list (messages.4.content).

It seems that when AmazonBedrockChatGenerator prepares the message containing the tool results to send back to Bedrock, it is incorrectly ordering or formatting the toolResult blocks when multiple results need to be sent simultaneously. The Bedrock API requires the results to be provided and correctly matched to their corresponding toolUseId from the previous turn.

Note: Single tool calls function correctly, isolating the issue specifically to the handling of multiple concurrent tool call results.

Expected Behavior:

AmazonBedrockChatGenerator should correctly format the message sent back to the Bedrock ConverseStream API, including all toolResult blocks, correctly ordered and matched to their respective toolUseIds from the LLM's previous turn, allowing the conversation flow to continue after multiple tool executions.

Actual Behavior:

A ValidationException is raised by botocore/Bedrock, preventing the agent from processing the results of the multiple tool calls and halting execution with a PipelineRuntimeError.

Environment:

• Haystack Version: 2.13
• Haystack Amazon Bedrock Integration Version: 3.5.1
• Model: anthropic.claude-3-5-sonnet-20240620-v1:0 (via Bedrock ConverseStream API)
• Python Version: 3.10

Steps to reproduce:

• Use any small isolated pipeline example forcing multiple calls or
• Use https://github.com/vblagoje/itinerary-agent with AmazonBedrockChatGenerator and update the prompt to force multiple tool calls