Skip to content

The _transform_output method of ChatModel may discard text content or override reasoning content with possible side effect of AttributeError #1448

@jpodivin

Description

@jpodivin

Describe the bug
The _transform_output method uses simple conditional to build AssistantMessage. If any tool calls are made, the text content of the message, if there is one, is dropped.

At the same time, the AssistantMessage is constructed using either content or reasoning_content field of Delta, in this order of precedence. This means that if both are defined, the reasoning content is ignored.

AssistantMessage(update.content or update.reasoning_content or "", id=chunk.id)

However, both fields can be populated at the same time. [1] In fact, it is possible to have tool_calls, content and reasoning_content set on the same Delta object.

But the initialization of AssistantMessage introduces a third problem. If the Delta object from litellm is initialized without any reasoning content, the attribute reasoning_content is deleted.

        if reasoning_content is not None:
            self.reasoning_content = reasoning_content
        else:
            # ensure default response matches OpenAI spec
            del self.reasoning_content

This is a problem introduced by litellm. But it will possibly manifest here, if the content field doesn't evaluate to true.
In that case the the attempt to access reasoning_content will fail with AttributeError.

In principle, that situation is rather unlikely, as it would require the inference provider not to set reasoning, or regular content.
But it can still happen. I have seen it happen with some Gemini models running in GCP.

To Reproduce

  1. Create agent with LiteLLMChatModel
  2. Start the agent and compare raw inference output to responses handled by agent.

Expected behavior
All content received from inference should be processed into AssistantMessage objects.

Set-up:

  • Bee version: 0.1.80
  • Model provider Google Gemini

Additional context
Litellm source:

[1] https://github.com/BerriAI/litellm/blob/aa587bd9d3c1f21decd590a68fd07427f1fa3653/litellm/types/utils.py#L1263

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingstaleIssue has not had recent activity or appears to be solved. Stale issues will be automatically closed

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions