Describe the bug
The _transform_output method uses simple conditional to build AssistantMessage. If any tool calls are made, the text content of the message, if there is one, is dropped.
At the same time, the AssistantMessage is constructed using either content or reasoning_content field of Delta, in this order of precedence. This means that if both are defined, the reasoning content is ignored.
AssistantMessage(update.content or update.reasoning_content or "", id=chunk.id)
However, both fields can be populated at the same time. [1] In fact, it is possible to have tool_calls, content and reasoning_content set on the same Delta object.
But the initialization of AssistantMessage introduces a third problem. If the Delta object from litellm is initialized without any reasoning content, the attribute reasoning_content is deleted.
if reasoning_content is not None:
self.reasoning_content = reasoning_content
else:
# ensure default response matches OpenAI spec
del self.reasoning_content
This is a problem introduced by litellm. But it will possibly manifest here, if the content field doesn't evaluate to true.
In that case the the attempt to access reasoning_content will fail with AttributeError.
In principle, that situation is rather unlikely, as it would require the inference provider not to set reasoning, or regular content.
But it can still happen. I have seen it happen with some Gemini models running in GCP.
To Reproduce
- Create agent with
LiteLLMChatModel
- Start the agent and compare raw inference output to responses handled by agent.
Expected behavior
All content received from inference should be processed into AssistantMessage objects.
Set-up:
- Bee version: 0.1.80
- Model provider Google Gemini
Additional context
Litellm source:
[1] https://github.com/BerriAI/litellm/blob/aa587bd9d3c1f21decd590a68fd07427f1fa3653/litellm/types/utils.py#L1263
Describe the bug
The
_transform_outputmethod uses simple conditional to buildAssistantMessage. If any tool calls are made, the text content of the message, if there is one, is dropped.At the same time, the
AssistantMessageis constructed using eithercontentorreasoning_contentfield ofDelta, in this order of precedence. This means that if both are defined, the reasoning content is ignored.However, both fields can be populated at the same time. [1] In fact, it is possible to have
tool_calls,contentandreasoning_contentset on the sameDeltaobject.But the initialization of
AssistantMessageintroduces a third problem. If theDeltaobject from litellm is initialized without any reasoning content, the attributereasoning_contentis deleted.This is a problem introduced by litellm. But it will possibly manifest here, if the
contentfield doesn't evaluate to true.In that case the the attempt to access
reasoning_contentwill fail withAttributeError.In principle, that situation is rather unlikely, as it would require the inference provider not to set reasoning, or regular content.
But it can still happen. I have seen it happen with some Gemini models running in GCP.
To Reproduce
LiteLLMChatModelExpected behavior
All content received from inference should be processed into
AssistantMessageobjects.Set-up:
Additional context
Litellm source:
[1] https://github.com/BerriAI/litellm/blob/aa587bd9d3c1f21decd590a68fd07427f1fa3653/litellm/types/utils.py#L1263