Skip to content

feat: add reasoning token support to Mistral integration#3265

Merged
bogdankostic merged 10 commits into
deepset-ai:mainfrom
ArkaD171717:mistral-reasoning-support
May 14, 2026
Merged

feat: add reasoning token support to Mistral integration#3265
bogdankostic merged 10 commits into
deepset-ai:mainfrom
ArkaD171717:mistral-reasoning-support

Conversation

@ArkaD171717
Copy link
Copy Markdown
Contributor

@ArkaD171717 ArkaD171717 commented May 2, 2026

Mistral returns reasoning as array content blocks:
content: [{"type": "thinking", ...}, {"type": "text", ...}]

The OpenAI SDK expects content: str | None and throws a Pydantic
ValidationError when it sees this
Non-streaming calls now use with_raw_response to get the raw JSON and parse
it manually, sidestepping SDK validation entirely
Streaming still goes through the normal SDK path since it processes
token-by-token (a warning is logged that reasoning content won't be captured
during streaming)

reasoning_effort and prompt_mode are routed through extra_body since
they're Mistral-specific params that the OpenAI SDK would reject as unknown
kwargs

Multi-turn reasoning is handled the same way as the OpenRouter PR: re-inject
array content into the formatted message dicts in _prepare_api_call

Also cleaned up duplicate entries in SUPPORTED_MODELS

25 new tests covering sync, async, multi-turn, edge cases
(unexpected content types, empty messages, JSON string parsing, async empty
messages, async streaming reasoning warning)
All 16 existing tests still pass

Closes #2180

Parse Mistral array content blocks (thinking + text) for reasoning
extraction. Use with_raw_response to bypass OpenAI SDK Pydantic
validation on non-streaming calls. Route reasoning_effort and
prompt_mode through extra_body. Handle multi-turn reasoning by
re-injecting array content into formatted message dicts.

Also fixed duplicate entries in SUPPORTED_MODELS and added streaming
warning when reasoning params are active.

19 new tests covering parsing, conversion, edge cases, parameter
routing, multi-turn, async, and streaming warning.

Closes deepset-ai#2180
@ArkaD171717 ArkaD171717 requested a review from a team as a code owner May 2, 2026 09:30
@ArkaD171717 ArkaD171717 requested review from bogdankostic and removed request for a team May 2, 2026 09:30
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented May 2, 2026

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions Bot added integration:mistral type:documentation Improvements or additions to documentation labels May 2, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 2, 2026

Heads-up for maintainers

This PR is from a fork and touches integrations whose integration tests require API keys.
Those tests are skipped in CI because fork PRs don't have access to repo secrets for security reasons.

Affected integrations:

  • mistral

Please run the integration tests locally (hatch run test:integration inside each folder) before approving.

Fixes PLC0415 (import not at top level) and S110 (bare try-except-pass).
Patch _prepare_api_call to raise RuntimeError after the warning is
logged, catch with pytest.raises(RuntimeError). Avoids B017 (blind
Exception) and S110 (try-except-pass).
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 2, 2026

Coverage report (mistral)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/mistral/src/haystack_integrations/components/generators/mistral/chat
  chat_generator.py 306, 386-387, 454-455
Project Total  

This report was generated by python-coverage-comment-action

ArkaD171717 and others added 4 commits May 2, 2026 13:19
Add tests for unexpected content types, empty messages, JSON string
input to response converter, async empty messages, and async streaming
with reasoning warning.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ArkaD171717
Copy link
Copy Markdown
Contributor Author

Brought up to date

Copy link
Copy Markdown
Contributor

@bogdankostic bogdankostic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ArkaD171717, thanks for creating this PR!

Unfortunately, 9 of the integration tests are failing. Almost all of them seem to share the same root cause related to the API response: AttributeError: 'LegacyAPIResponse' object has no attribute 'json'.

FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run_response_format - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run_with_response_format_json_schema - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run_with_tools - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run_with_tools_and_response - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_pipeline_with_mistral_chat_generator - haystack.core.errors.PipelineRuntimeError: The following component failed to run:
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run_with_mixed_tools - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator_async.py::TestMistralChatGeneratorAsync::test_live_run_async - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator_async.py::TestMistralChatGeneratorAsync::test_live_run_with_tools_and_response_async - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'

Let us know if you need any help reproducing these test failures locally! Just a heads-up: running these tests requires a Mistral API key.

…ompat

LegacyAPIResponse (returned by with_raw_response in some openai SDK
versions) does not expose a .json() method, causing AttributeError on
all non-streaming calls. Switch to .text which is available on both
LegacyAPIResponse and APIResponse. Update test mocks to match.
@ArkaD171717
Copy link
Copy Markdown
Contributor Author

@bogdankostic Thanks for running the integration tests! The issue was that with_raw_response returns a LegacyAPIResponse in the SDK version pinned here which doesnt have .json(). Switched to .text which is available on both LegacyAPIResponse and APIResponse. I updated the test mocks to match and shld be fixed in the latest push.

LMK if any tests are still failing

Copy link
Copy Markdown
Contributor

@bogdankostic bogdankostic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ArkaD171717! I added two integration tests and the PR should be good to go now :)

@bogdankostic bogdankostic merged commit 53d1bab into deepset-ai:main May 14, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration:mistral type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mistral - reasoning support

3 participants