Skip to content

fix: fix the conversion of cohere chunks to Haystack streaming chunks#2968

Merged
anakin87 merged 17 commits intomainfrom
reasoning-cohere
Mar 17, 2026
Merged

fix: fix the conversion of cohere chunks to Haystack streaming chunks#2968
anakin87 merged 17 commits intomainfrom
reasoning-cohere

Conversation

@sjrl
Copy link
Copy Markdown
Contributor

@sjrl sjrl commented Mar 17, 2026

Related Issues

  • fixes #issue-number

Proposed Changes:

  • Finish Reason mapping for streaming chunks was incomplete and was incorrect TOOL_CALLS --> TOOL_CALL
  • StreamingChunk.start = True was incorrect for the first non-empty tool-plan-delta chunk
  • StreamingChunk.start was incorrectly set to True on tool-call-end

How did you test it?

  • Refactored streaming unit tests to use real Cohere chunks
  • Added test to make sure final ChatMessage created from streaming chunks is as expected

Also tested locally with the script

from haystack.components.agents import Agent
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.tools import Tool

from haystack_integrations.components.generators.cohere import CohereChatGenerator


# Tool Function
def calculate(expression: str) -> dict:
    try:
        result = eval(expression, {"__builtins__": {}})
        return {"result": result}
    except Exception as e:
        return {"error": str(e)}


# Tool Definition
calculator_tool = Tool(
    name="calculator",
    description="Evaluate basic math expressions.",
    parameters={
        "type": "object",
        "properties": {"expression": {"type": "string", "description": "Math expression to evaluate"}},
        "required": ["expression"],
    },
    function=calculate,
    outputs_to_state={"calc_result": {"source": "result"}},
)

# Agent Setup
agent = Agent(
    chat_generator=CohereChatGenerator(
        model="command-a-03-2025",
        # model="command-a-reasoning-08-2025",
        # generation_kwargs={"thinking": {"type": "enabled", "token_budget": 10}}
    ),
    streaming_callback=print_streaming_chunk,
    tools=[calculator_tool],
    state_schema={"calc_result": {"type": int}},
)

# Run the Agent
response = agent.run(
    messages=[
        ChatMessage.from_user(
            "What is 7 + 2? And what is 2 * 2? Make sure to use the calculator tool to compute the result."
        )
    ]
)

Which now produces the expected formatted streaming response

Screenshot 2026-03-17 at 13 53 23

Previously looked like

Screenshot 2026-03-17 at 13 53 59

Notes for the reviewer

Once this is merged I'll follow it up with a PR to resolve #2179

Checklist


@sjrl sjrl requested a review from a team as a code owner March 17, 2026 12:51
@sjrl sjrl requested review from davidsbatista and removed request for a team March 17, 2026 12:51
@github-actions github-actions Bot added integration:cohere type:documentation Improvements or additions to documentation labels Mar 17, 2026
@sjrl sjrl self-assigned this Mar 17, 2026
Copy link
Copy Markdown
Contributor

@davidsbatista davidsbatista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@anakin87 anakin87 merged commit 5f1d9e4 into main Mar 17, 2026
12 checks passed
@anakin87 anakin87 deleted the reasoning-cohere branch March 17, 2026 17:23
@anakin87
Copy link
Copy Markdown
Member

@sjrl sorry, I merged and released this as an experiment to generate a new API reference, but I did not realize it does not touch docstrings. Hope it was already OK.

@sjrl
Copy link
Copy Markdown
Contributor Author

sjrl commented Mar 17, 2026

@sjrl sorry, I merged and released this as an experiment to generate a new API reference, but I did not realize it does not touch docstrings. Hope it was already OK.

yeah no worries it was good to go!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration:cohere type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cohere - reasoning support

3 participants