Skip to content

Google GenAI: use finish_reason.tool_calls when the model produces a tool call? #2143

@anakin87

Description

@anakin87

Haystack's finish_reason is used both in StreamingChunk and ChatMessage.meta["finish_reason"] to indicate why the text generation stopped.

Google GenAI uses STOP as a general-purpose finish reason, for both normal text completions and tool calls. This differs from other providers like OpenAI, which return more specific reasons (e.g., tool_calls or similar) when a tool call is involved.

In Google GenAI integration, Haystack mirrors Google's convention by passing through STOP even for tool calls. One alternative would be to implement a logic to detect tool calls in the response and remap the finish_reason accordingly.

Initially discussed in #2142 (comment)

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions