Describe the bug
Anthropic returns streaming events where the first chunk has the input_tokens and the last chunk has the completion_tokens.
However, in this line we try to get usage only from the last chunk:
|
last_chunk_meta = chunks[-1].meta |
We need to change the conversion so that we get input tokens from the first chunk and output tokens from the last chunk.
To Reproduce
Use the AnthropicChatGenerator with streaming and look at ChatMessage.meta["usage"].
Describe your environment (please complete the following information):
- OS: [e.g. iOS]
- Haystack version:
- Integration version:
Describe the bug
Anthropic returns streaming events where the first chunk has the
input_tokensand the last chunk has thecompletion_tokens.However, in this line we try to get usage only from the last chunk:
haystack-core-integrations/integrations/anthropic/src/haystack_integrations/components/generators/anthropic/chat/chat_generator.py
Line 416 in d804dd7
We need to change the conversion so that we get input tokens from the first chunk and output tokens from the last chunk.
To Reproduce
Use the AnthropicChatGenerator with streaming and look at
ChatMessage.meta["usage"].Describe your environment (please complete the following information):