Skip to content

feat(cohere): add timeout and max_retries to chat generator#2873

Merged
julian-risch merged 2 commits intodeepset-ai:mainfrom
Keyur-S-Patel:feat/cohere-chat-timeout-retries
Mar 7, 2026
Merged

feat(cohere): add timeout and max_retries to chat generator#2873
julian-risch merged 2 commits intodeepset-ai:mainfrom
Keyur-S-Patel:feat/cohere-chat-timeout-retries

Conversation

@Keyur-S-Patel
Copy link
Copy Markdown
Contributor

@Keyur-S-Patel Keyur-S-Patel commented Feb 25, 2026

Related Issues

Proposed Changes:

  • Added timeout and max_retries to CohereChatGenerator.__init__ as keyword-only params (aligned with Anthropic pattern).
  • Implemented retry behavior using provider SDK-compatible httpx transport injection:
  • Passed custom httpx_client into ClientV2 / AsyncClientV2 when max_retries is set.
  • Kept backward compatibility for extra generation kwargs (**kwargs merged into generation_kwargs).
  • Updated to_dict/from_dict expectations for timeout and max_retries.
  • Updated CohereGenerator tests to reflect current inherited behavior (generation_kwargs instead of removed model_parameters).

How did you test it?

  • hatch run test:unit tests/test_chat_generator.py
  • hatch run test:unit tests/test_generator.py
  • hatch run test:unit tests (full Cohere unit suite)

All unit tests passed on this branch.

Notes for the reviewer

  • max_retries is implemented via httpx transport retries because direct max_retries constructor support on ClientV2 is not consistent across environments/versions.
  • This PR also includes test alignment in tests/test_generator.py for current CohereGenerator behavior.

Checklist

@Keyur-S-Patel Keyur-S-Patel requested a review from a team as a code owner February 25, 2026 10:47
@Keyur-S-Patel Keyur-S-Patel requested review from julian-risch and removed request for a team February 25, 2026 10:47
@github-actions github-actions Bot added integration:cohere type:documentation Improvements or additions to documentation labels Feb 25, 2026
@Keyur-S-Patel Keyur-S-Patel force-pushed the feat/cohere-chat-timeout-retries branch from 7c13187 to f961fe7 Compare February 25, 2026 11:15
@Keyur-S-Patel
Copy link
Copy Markdown
Contributor Author

@anakin87 Can you help me close this PR, its similar to the one you reviewed here

Copy link
Copy Markdown
Contributor Author

@Keyur-S-Patel Keyur-S-Patel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gentle reminder about pr review @julian-risch

@Keyur-S-Patel
Copy link
Copy Markdown
Contributor Author

@davidsbatista can you look up this one, should be quick review

Copy link
Copy Markdown
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Keyur-S-Patel Thank you for opening this pull request. The code changes look good to me! 👍 I looked into the Cohere SDK and confirmed that when using timeout and max_retries at the same time, for example CohereChatGenerator(timeout=30.0, max_retries=5), the SDK stores the timeout passed via client_kwargs and passes it to each httpx_client.request(..., timeout=30) done with the custom client.
Thank you also for your patience with regard to this review. I'd like to assure you that we do our best to review every community PR in a timely manner. We auto-rotate reviews and there is no need to tag individual team members in PR comments. Thank you!

@julian-risch julian-risch merged commit c8278f3 into deepset-ai:main Mar 7, 2026
11 checks passed
@Keyur-S-Patel
Copy link
Copy Markdown
Contributor Author

Tysm for reviewing @julian-risch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration:cohere type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose timeout and max_retries in ChatGenerators to help work around rate limits

2 participants