feat: add reasoning token support to Mistral integration#3265
Conversation
Parse Mistral array content blocks (thinking + text) for reasoning extraction. Use with_raw_response to bypass OpenAI SDK Pydantic validation on non-streaming calls. Route reasoning_effort and prompt_mode through extra_body. Handle multi-turn reasoning by re-injecting array content into formatted message dicts. Also fixed duplicate entries in SUPPORTED_MODELS and added streaming warning when reasoning params are active. 19 new tests covering parsing, conversion, edge cases, parameter routing, multi-turn, async, and streaming warning. Closes deepset-ai#2180
|
Heads-up for maintainers This PR is from a fork and touches integrations whose integration tests require API keys. Affected integrations:
Please run the integration tests locally ( |
Fixes PLC0415 (import not at top level) and S110 (bare try-except-pass).
Patch _prepare_api_call to raise RuntimeError after the warning is logged, catch with pytest.raises(RuntimeError). Avoids B017 (blind Exception) and S110 (try-except-pass).
Coverage report (mistral)Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
Add tests for unexpected content types, empty messages, JSON string input to response converter, async empty messages, and async streaming with reasoning warning.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Brought up to date |
bogdankostic
left a comment
There was a problem hiding this comment.
Hi @ArkaD171717, thanks for creating this PR!
Unfortunately, 9 of the integration tests are failing. Almost all of them seem to share the same root cause related to the API response: AttributeError: 'LegacyAPIResponse' object has no attribute 'json'.
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run_response_format - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run_with_response_format_json_schema - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run_with_tools - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run_with_tools_and_response - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_pipeline_with_mistral_chat_generator - haystack.core.errors.PipelineRuntimeError: The following component failed to run:
FAILED tests/test_mistral_chat_generator.py::TestMistralChatGenerator::test_live_run_with_mixed_tools - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator_async.py::TestMistralChatGeneratorAsync::test_live_run_async - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
FAILED tests/test_mistral_chat_generator_async.py::TestMistralChatGeneratorAsync::test_live_run_with_tools_and_response_async - AttributeError: 'LegacyAPIResponse' object has no attribute 'json'
Let us know if you need any help reproducing these test failures locally! Just a heads-up: running these tests requires a Mistral API key.
…ompat LegacyAPIResponse (returned by with_raw_response in some openai SDK versions) does not expose a .json() method, causing AttributeError on all non-streaming calls. Switch to .text which is available on both LegacyAPIResponse and APIResponse. Update test mocks to match.
…71717/haystack-core-integrations into mistral-reasoning-support
|
@bogdankostic Thanks for running the integration tests! The issue was that with_raw_response returns a LegacyAPIResponse in the SDK version pinned here which doesnt have .json(). Switched to .text which is available on both LegacyAPIResponse and APIResponse. I updated the test mocks to match and shld be fixed in the latest push. LMK if any tests are still failing |
bogdankostic
left a comment
There was a problem hiding this comment.
Thanks @ArkaD171717! I added two integration tests and the PR should be good to go now :)
Mistral returns reasoning as array content blocks:
content: [{"type": "thinking", ...}, {"type": "text", ...}]The OpenAI SDK expects
content: str | Noneand throws a PydanticValidationError when it sees this
Non-streaming calls now use
with_raw_responseto get the raw JSON and parseit manually, sidestepping SDK validation entirely
Streaming still goes through the normal SDK path since it processes
token-by-token (a warning is logged that reasoning content won't be captured
during streaming)
reasoning_effortandprompt_modeare routed throughextra_bodysincethey're Mistral-specific params that the OpenAI SDK would reject as unknown
kwargs
Multi-turn reasoning is handled the same way as the OpenRouter PR: re-inject
array content into the formatted message dicts in
_prepare_api_callAlso cleaned up duplicate entries in SUPPORTED_MODELS
25 new tests covering sync, async, multi-turn, edge cases
(unexpected content types, empty messages, JSON string parsing, async empty
messages, async streaming reasoning warning)
All 16 existing tests still pass
Closes #2180