Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
For GoogleGenAIChatGenerator and other google genai chat generators, the cached token count, and other metadata is not transparently passed to the output of the function, making it impossible for our use case of calculating api price since the cached token count is not exposed.
|
usage_metadata = response.usage_metadata |
Describe the solution you'd like
A clear and concise description of what you want to happen.
Can you do something similar to OpenAIChatGenerator, where usage dict from underlying provider response is transparently passed? I understand there are intricacies involved since you ended up mapping google token metadata output to a subset of OpenAI's response shape, but ultimately we just need a way to access the cached token output data. Thank you!
https://github.com/deepset-ai/haystack/blob/1d7bc772123b4d871a641683cce7d84c8bb73533/haystack/components/generators/chat/openai.py#L586
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
For
GoogleGenAIChatGeneratorand other google genai chat generators, the cached token count, and other metadata is not transparently passed to the output of the function, making it impossible for our use case of calculating api price since the cached token count is not exposed.haystack-core-integrations/integrations/google_genai/src/haystack_integrations/components/generators/google_genai/chat/chat_generator.py
Line 311 in 051ccb8
Describe the solution you'd like
A clear and concise description of what you want to happen.
Can you do something similar to
OpenAIChatGenerator, whereusagedict from underlying provider response is transparently passed? I understand there are intricacies involved since you ended up mapping google token metadata output to a subset of OpenAI's response shape, but ultimately we just need a way to access the cached token output data. Thank you!https://github.com/deepset-ai/haystack/blob/1d7bc772123b4d871a641683cce7d84c8bb73533/haystack/components/generators/chat/openai.py#L586
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.