Skip to content

feat: Add Chat Generator supporting OpenAI Responses API#9808

Merged
sjrl merged 51 commits into
mainfrom
openai-responses
Nov 6, 2025
Merged

feat: Add Chat Generator supporting OpenAI Responses API#9808
sjrl merged 51 commits into
mainfrom
openai-responses

Conversation

@Amnah199

@Amnah199 Amnah199 commented Sep 22, 2025

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

  • Add a chat generator that uses openai's responses endpoint.
  • The tools param in init and run methods allows openai and MCP tools besides haystack Tools.
  • Structured outputs can be enabled by passing text_format in generation_kwargs.
  • Reasoning can be enabled by setting reasoning params in generation_kwargs.
  • There are other features in Responses API like background requests etc, which are not supported in this iteration.
  • With every ToolCall sent back, the API also expects the reasoning id even if the reasoning object was empty.

How did you test it?

  • Add new unit tests to test features.
  • Add new integration tests.

Notes for the reviewer

Using OpenAI tools and MCP tools doesn’t behave like standard function calls.

  • Tool calls are usually identified by type="function_call". However, for OpenAI/MCP tools, the type value varies depending on the tool being used — for example, type="web_search_call".
  • Currently, we only return standard ResponseFunctionToolCall objects. As a result, tool calls from OpenAI/MCP tools won’t appear in the ChatMessage, so the user won’t see them.
  • This is fine, since these tool calls are handled internally by OpenAI to generate the final response. The user already receives the output.

Checklist

  • I have read the contributors guidelines and the code of conduct
  • I have updated the related issue with new insights and changes
  • I added unit tests and updated the docstrings
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I documented my code
  • I ran pre-commit hooks and fixed any issue

@github-actions github-actions Bot added the type:documentation Improvements on the docs label Sep 22, 2025
@coveralls

coveralls commented Sep 22, 2025

Copy link
Copy Markdown
Collaborator

Pull Request Test Coverage Report for Build 19132972551

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.7%) to 91.566%

Totals Coverage Status
Change from base Build 19102178311: -0.7%
Covered Lines: 13668
Relevant Lines: 14927

💛 - Coveralls

@Amnah199 Amnah199 requested a review from vblagoje October 13, 2025 07:57
@Amnah199 Amnah199 marked this pull request as ready for review October 13, 2025 10:15
@Amnah199 Amnah199 requested a review from a team as a code owner October 13, 2025 10:15
Comment thread haystack/dataclasses/chat_message.py Outdated
Comment thread haystack/components/generators/chat/openai_responses.py Outdated
Comment thread haystack/components/generators/chat/openai_responses.py Outdated
Comment thread haystack/components/generators/chat/openai_responses.py Outdated
Comment thread haystack/components/generators/chat/openai_responses.py
Comment thread haystack/components/generators/chat/openai_responses.py Outdated
Comment thread haystack/components/generators/chat/openai_responses.py Outdated
Comment thread test/components/generators/chat/test_openai_responses.py Outdated
Comment thread test/components/generators/chat/test_openai_responses.py Outdated
Comment thread test/components/generators/chat/test_openai_responses.py Outdated
Comment thread test/components/generators/chat/test_openai_responses.py Outdated
Comment thread test/components/generators/chat/test_openai_responses.py Outdated
Comment thread haystack/components/generators/chat/openai_responses.py
Comment thread haystack/components/generators/chat/openai_responses.py Outdated
@Amnah199 Amnah199 requested a review from vblagoje November 5, 2025 20:34
Comment thread haystack/components/generators/chat/openai_responses.py Outdated
Comment thread haystack/components/generators/chat/openai_responses.py Outdated
@sjrl

sjrl commented Nov 6, 2025

Copy link
Copy Markdown
Contributor

@Amnah199 can we make sure to update the docs files in docs/pydoc/config and docs/pydoc/config_docusaurus?

Comment thread haystack/components/generators/chat/openai_responses.py Outdated

@sjrl sjrl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

timeout: Optional[float] = None,
max_retries: Optional[int] = None,
tools: Optional[Union[ToolsType, list[dict]]] = None,
tools_strict: bool = False,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sjrl This value is True by default in Responses. For consistency with other chat generators I kept it false.

@sjrl sjrl merged commit 2a0a6f1 into main Nov 6, 2025
23 checks passed
@sjrl sjrl deleted the openai-responses branch November 6, 2025 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for OpenAI's Responses API

5 participants