Skip to content

AnthropicChatGenerator does not get correct prompt_tokens when using streaming. #1992

Description

@mathislucka

Describe the bug
Anthropic returns streaming events where the first chunk has the input_tokens and the last chunk has the completion_tokens.

However, in this line we try to get usage only from the last chunk:

We need to change the conversion so that we get input tokens from the first chunk and output tokens from the last chunk.

To Reproduce
Use the AnthropicChatGenerator with streaming and look at ChatMessage.meta["usage"].

Describe your environment (please complete the following information):

  • OS: [e.g. iOS]
  • Haystack version:
  • Integration version:

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions