Inconsistent `finish_reason` between streaming and non-streaming modes

[Amnah199]

📝 Summary & Motivation

The FinishReason literal introduced in StreamingChunk restricts allowed values to a predefined set. This has introduced inconsistencies between the finish_reason values in streaming vs. non-streaming modes across multiple integrations.

These inconsistencies fall into two main categories, explained below.

Case 1: Inconsistency Between StreamingChunks and Final ChatMessage

In both streaming and non-streaming modes, the final ChatMessage typically carries the original finish_reason from the model API. However, the streamed StreamingChunks use a mapped FinishReason enum instead.
This creates a mismatch between what is streamed and what appears in the final message.

Additionally, this leads to inefficient workarounds like:

• We first map the API’s finish_reason to a FinishReason enum (to comply with StreamingChunk requirements),
• And then map it back to the original string for the final ChatMessage.

Affected Integrations:

• AnthropicChatGenerator
• OllamaChatGenerator

Case 2: Inconsistency Between Streaming and Non-Streaming Modes

In some integrations, the finish_reason differs depending on whether streaming is enabled or not, even within the same component.

For example, in GoogleGenAIChatGenerator:

Streaming Mode:
The finish_reason is mapped to a FinishReason enum. This used in streaming chunks and stored in the ChatMessage.
Reference

Non-Streaming Mode:
The model's original finish_reason string is passed directly into the final ChatMessage without mapping.

This results in inconsistencies in the ChatMessage format.

Affected Integrations:

• GoogleGenAIChatGenerator
• HuggingFaceAPIChatGenerator

Checklist

Tasks

• The changes are merged in the main branch (Code + Docstrings)
• New package version declares the breaking change
• The package has been released on PyPI
• Docs at https://docs.haystack.deepset.ai/ were updated
• Integration tile on https://github.com/deepset-ai/haystack-integrations was updated
• Notebooks on https://github.com/deepset-ai/haystack-cookbook were updated (if needed)
• Tutorials on https://github.com/deepset-ai/haystack-tutorials were updated (if needed)
• Articles on https://github.com/deepset-ai/haystack-home/tree/main/content were updated (if needed)

[sjrl]

@Amnah199 Just to clarify does Anthropic actively have this issue, I thought we fixed/changed it so both cases returned the same finish reason?

[sjrl]

Also I don't know if we need to label this as a breaking change. Here is what my suggestion would be:

1. Fix the HuggingFace and Google GenAI integrations to be consistent in a non-breaking way. This could mean updating the streaming version to return a ChatMessage with the original finish reason returned by the respective provider.
2. Look into if we should add a dedicated finish_reason field to ChatMessage that matches the same finish_reason's used by StreamingChunk. I'd like to get @anakin87 's opinion on this.
3. If we do Step 2 then in all of our integrations we update them to use the new finish_reason field in ChatMessage using our defined finish reasons. And we would leave the one in meta untouched.

So I'd say the high priority part of this outline above would just be Step 1. The other two steps can come later.

[Amnah199]

@sjrl You’re right — with this PR, the ChatMessage in the Anthropic integration will now have the same finish_reason value for both streaming and non-streaming modes. That means the only remaining inconsistency is between the values in the streaming chunks and the final returned ChatMessage.

Also, as I mentioned earlier, the current workaround isn’t ideal:

To ensure consistent finish_reason values in ChatMessage, we currently map the streaming chunk’s enum value back to the original model value. This is just a workaround and not an efficient solution.

[anakin87]

Just my two cents.

I also don't like the inconsistency in the finish_reason between streaming and non-streaming mode.

In general, I would keep the finish_reason in meta and always use the mapped value.

As for breaking changes: keeping it in meta but changing the value is a bit of a borderline case, since we don't expect users to strictly rely on this metadata. We've made similar changes in the past without treating them as breaking. See deepset-ai/haystack#9272 for example.

[Amnah199]

I’ve updated the issue description for clarity. Since we’re still in the process of adding the new streaming support to some integrations, I haven’t marked those. Once we agree on a solution (let’s call it Option 1 by @sjrl ), we can apply it consistently as we continue updating the streaming logic.

[sjrl]

Closing since both sub-tasks are finished