feat: Add warm_up() method to ChatGenerators for tool initialization by HamidOna · Pull Request #9942 · deepset-ai/haystack

HamidOna · 2025-10-26T12:38:49Z

Related Issues

fixes Add warm_up() method to ChatGenerators for tool initialization #9907

Proposed Changes:

This PR adds the warm_up() method to all ChatGenerator components to properly initialize tools that require warm-up (database connections, model loading, etc.) before pipeline execution.

Components Modified:

OpenAIChatGenerator - Added standard warm_up() implementation with idempotency
AzureOpenAIChatGenerator - Added standard warm_up() implementation with idempotency
HuggingFaceAPIChatGenerator - Added standard warm_up() implementation with idempotency
HuggingFaceLocalChatGenerator - Enhanced existing warm_up() to also warm up tools while preserving pipeline initialization
FallbackChatGenerator - Added delegation-based warm_up() that calls warm_up() on all underlying generators

Implementation approach:

Standard components (OpenAI, Azure, HuggingFaceAPI): Call warm_up_tools() utility function
HuggingFaceLocal: Enhanced existing warm_up() method to also warm up tools after pipeline initialization
Fallback: Delegates to underlying generators using hasattr() for defensive checking
All implementations include _is_warmed_up flag for idempotency
Follows the same pattern established in Agent and ToolInvoker components

How did you test it?

Added 3 comprehensive unit tests per component (15 tests total)
Test coverage includes: single tool case, no tools case, and multiple tools case
All tests verify idempotent behavior (warm_up() can be called multiple times safely)
Tests for OpenAI, Azure, and Fallback passing locally
Tests for HuggingFaceAPI and HuggingFaceLocal created (require optional dependencies: flaky, PyTorch/TensorFlow - will be verified by CI)

Notes for the reviewer

The implementation follows the established warm_up pattern from Agent and ToolInvoker components
FallbackChatGenerator uses a delegation pattern (different from other components) because it doesn't store tools directly - it passes them through to underlying generators at runtime
HuggingFaceLocalChatGenerator already had a warm_up() method for pipeline initialization, so this PR enhances it rather than replacing it
All existing functionality preserved, only additions made

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: feat:
I documented my code
I ran pre-commit hooks and fixed any issue

- Add warm_up() method that calls warm_up_tools() - Add _is_warmed_up flag for idempotency - Import warm_up_tools from haystack.tools - Add comprehensive tests: - test_warm_up_with_tools: single tool case - test_warm_up_with_no_tools: no tools case - test_warm_up_with_multiple_tools: multiple tools case - All tests passing Part of issue deepset-ai#9907

- Add warm_up_tools import from haystack.tools.utils - Add _is_warmed_up flag for idempotency - Enhance existing warm_up() to also warm up tools - Preserve existing pipeline initialization logic - Add comprehensive tests: - test_warm_up_with_tools: single tool case - test_warm_up_with_no_tools: no tools case - test_warm_up_with_multiple_tools: multiple tools case Part of issue deepset-ai#9907

- Add warm_up() method that delegates to underlying generators - Uses hasattr check to gracefully handle generators without warm_up - Add comprehensive tests: - test_warm_up_delegates_to_generators: verify delegation works - test_warm_up_with_no_warm_up_method: handle missing warm_up gracefully - test_warm_up_mixed_generators: mix of generators with/without warm_up - All tests passing Part of issue deepset-ai#9907

vercel · 2025-10-26T12:38:55Z

@HamidOna13 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

CLAassistant · 2025-10-26T12:38:56Z

All committers have signed the CLA.

HamidOna · 2025-10-26T12:43:36Z

While implementing this feature, I discovered a potential issue with the warm_up_tools() utility function:

When a Toolset object is passed to a component's tools parameter, the warm_up_tools() function only calls Toolset.warm_up(), which is a no-op. It doesn't iterate through the individual tools inside the Toolset to warm them up.

This means if someone does:

generator = OpenAIChatGenerator(tools=[my_toolset])
generator.warm_up()

The individual tools within my_toolset won't actually be warmed up.

This is beyond the scope of this PR, but wanted to document it. Should I open a separate issue for this?

vblagoje · 2025-10-27T10:35:37Z

While implementing this feature, I discovered a potential issue with the warm_up_tools() utility function:

When a Toolset object is passed to a component's tools parameter, the warm_up_tools() function only calls Toolset.warm_up(), which is a no-op. It doesn't iterate through the individual tools inside the Toolset to warm them up.

This means if someone does:
generator = OpenAIChatGenerator(tools=[my_toolset])
generator.warm_up()
The individual tools within my_toolset won't actually be warmed up.

This is beyond the scope of this PR, but wanted to document it. Should I open a separate issue for this?

Hey @HamidOna this is a good catch. The disconnect happened due to two usage approaches for Toolset:

semantically group a few Tool instances into a Toolset (init as a list of Tool) and add them to pipeline/agent together as Toolset.
Toolset subset (e.g. MCPToolset) connects to a service/resource and manages a set of Tool instances as a group.

For the first one - yes it makes sense to call warm_up on each Tool in the Toolset. For the second - it doesn't because Toolset subclass manages/interpolates management to all tools in it. Having said all that it the argument to make default Toolset warm_up call warm_up on Tool instances does make sense because Toolset subclasses can change that behaviour to customize the warm_up as they rightly should. We'll talk internally about this and report back. Thanks for pointing this out! cc @sjrl

vblagoje · 2025-10-28T14:58:59Z

@HamidOna apologies for the delay, this PR looks good in principle. Would you please open a PR that calls warm_up for individual Tools in Toolset just as we spoke above

coveralls · 2025-10-28T15:00:08Z

Pull Request Test Coverage Report for Build 18879776280

Details

0 of 0 changed or added relevant lines in 0 files are covered.
41 unchanged lines in 5 files lost coverage.
Overall coverage increased (+0.04%) to 92.235%

Files with Coverage Reduction	New Missed Lines	%
components/generators/chat/fallback.py	3	96.15%
components/generators/chat/azure.py	4	94.03%
components/generators/chat/openai.py	5	97.34%
components/generators/chat/hugging_face_api.py	6	96.48%
components/generators/chat/hugging_face_local.py	23	86.34%

Totals
Change from base Build 18869627382:	0.04%
Covered Lines:	13494
Relevant Lines:	14630

💛 - Coveralls

vblagoje · 2025-10-28T15:06:13Z

Also @HamidOna please don't forget to sign CLA, see above. This is standard procedure for Contributor LA.

HamidOna · 2025-10-28T15:17:18Z

@HamidOna apologies for the delay, this PR looks good in principle. Would you please open a PR that calls warm_up for individual Tools in Toolset just as we spoke above

HamidOna · 2025-10-28T15:21:38Z

Also @HamidOna please don't forget to sign CLA, see above. This is standard procedure for Contributor LA.

Hey @vblagoje Thank you
I signed it but not sure why it's not reflecting

Apologies for the close/reopen - accidentally closed while replying to this

vblagoje

LG, let's wait to see what's up with this CLA server, maybe it unblocks by tomorrow. If not, we'll figure it out, I've seen this before.

HamidOna · 2025-10-28T15:41:14Z

LG, let's wait to see what's up with this CLA server, maybe it unblocks by tomorrow. If not, we'll figure it out, I've seen this before.

yah, we can wait but reflects on my side though

HamidOna13 · 2025-10-28T16:15:13Z

LG, let's wait to see what's up with this CLA server, maybe it unblocks by tomorrow. If not, we'll figure it out, I've seen this before.

yah, we can wait but reflects on my side though

@vblagoje I figured out the problem. For some reason, It also needed me to sign with my other github account

vblagoje · 2025-10-29T08:31:25Z

@@ -201,6 +202,18 @@ def __init__(  # pylint: disable=too-many-positional-arguments
        self.async_client = AsyncAzureOpenAI(
            http_client=init_http_client(self.http_client_kwargs, async_client=True), **client_args
        )
+        self._is_warmed_up = False
+
+    def warm_up(self):


Ah @HamidOna do we need this one as its parent already implements the warm_up?

Yes, Azure needs both the flag initialization and the warm_up() method because:

Azure does not call super().init() - See line 73 and the comment on lines 152-53. It explicitly skips the parent's initialization because it only needs to instantiate the Azure-specific client.

Since Azure's init() doesn't call the parent's init():

The self._is_warmed_up = False flag from OpenAI's init() is never set

Without this flag, calling any inherited warm_up() would fail with AttributeError making it a requirement for Azure

The warm_up() method implementation is identical to OpenAI's, so technically we could remove Azure's warm_up() method and let it inherit from OpenAI (since the flag is now initialized). However, It makes sense to keep both for explicitness. Let me know if you'd prefer I remove the warm_up() method from Azure and just keep the flag initialization!

Hmm, right this is a tension between DRY and explicitness you advocate. We could simply add:
self._is_warmed_up = False
in Azure init and be done with it. Let me ask my colleague @sjrl for an opinion

I'd rather reimplemt the warm up method so it's more explicit.

Ok thanks @sjrl then we are gtg here @HamidOna

🙏 @HamidOna keep these PRs coming. Great work!

@HamidOna you can, if you want, also add this method to all Chat Generators in https://github.com/deepset-ai/haystack-core-integrations/ Just lmk

@vblagoje

Related Issues: * Follows up on PR deepset-ai#9942 (feat: Add warm_up() method to ChatGenerators) * Addresses bug discovered during implementation of PR deepset-ai#9942 for issue deepset-ai#9907 Proposed Changes: The warm_up_tools() utility function was only calling warm_up() on Toolset objects themselves, but not on the individual Tool instances contained within them. This meant tools inside a Toolset were not properly initialized before use. This PR modifies warm_up_tools() to iterate through Toolsets and call warm_up() on each individual tool, in addition to calling warm_up() on the Toolset itself. Changes: - Modified warm_up_tools() in haystack/tools/utils.py to iterate through Toolsets when encountered (both as single argument and within lists) - Added iteration to call warm_up() on each individual Tool inside Toolsets - Added comprehensive test class TestWarmUpTools with 7 test cases How did you test it: - Added 7 comprehensive unit tests in test/tools/test_tools_utils.py: * test_warm_up_tools_with_none - handles None input * test_warm_up_tools_with_single_tool - single tool in list * test_warm_up_tools_with_single_toolset - KEY TEST: verifies both Toolset and individual tools are warmed * test_warm_up_tools_with_list_containing_toolset - toolset within list * test_warm_up_tools_with_multiple_toolsets - multiple toolsets * test_warm_up_tools_with_mixed_tools_and_toolsets - mixed scenarios * test_warm_up_tools_idempotency - safe to call multiple times Notes for the reviewer: I discovered this bug while implementing PR deepset-ai#9942 (for issue deepset-ai#9907). When a Toolset object is passed to a component's tools parameter, the warm_up_tools() function only calls Toolset.warm_up(), which is a no-op. It doesn't iterate through the individual tools inside the Toolset to warm them up. acknowledged by @vblagoje and @sjrl This implementation: - Modified warm_up_tools() to iterate through Toolsets and call warm_up() on each individual tool - Added comprehensive tests for Toolset warming behavior - Verified both the Toolset and its contained tools are warmed up Checklist: I have read the contributors guidelines and the code of conduct I have updated the related issue with new insights and changes I added unit tests and updated the docstrings I've used one of the conventional commit types for my PR title: fix: I documented my code I ran pre-commit hooks and fixed any issue

@vblagoje

) * fix: warm up individual tools inside Toolsets in warm_up_tools() Related Issues: * Follows up on PR #9942 (feat: Add warm_up() method to ChatGenerators) * Addresses bug discovered during implementation of PR #9942 for issue #9907 Proposed Changes: The warm_up_tools() utility function was only calling warm_up() on Toolset objects themselves, but not on the individual Tool instances contained within them. This meant tools inside a Toolset were not properly initialized before use. This PR modifies warm_up_tools() to iterate through Toolsets and call warm_up() on each individual tool, in addition to calling warm_up() on the Toolset itself. Changes: - Modified warm_up_tools() in haystack/tools/utils.py to iterate through Toolsets when encountered (both as single argument and within lists) - Added iteration to call warm_up() on each individual Tool inside Toolsets - Added comprehensive test class TestWarmUpTools with 7 test cases How did you test it: - Added 7 comprehensive unit tests in test/tools/test_tools_utils.py: * test_warm_up_tools_with_none - handles None input * test_warm_up_tools_with_single_tool - single tool in list * test_warm_up_tools_with_single_toolset - KEY TEST: verifies both Toolset and individual tools are warmed * test_warm_up_tools_with_list_containing_toolset - toolset within list * test_warm_up_tools_with_multiple_toolsets - multiple toolsets * test_warm_up_tools_with_mixed_tools_and_toolsets - mixed scenarios * test_warm_up_tools_idempotency - safe to call multiple times Notes for the reviewer: I discovered this bug while implementing PR #9942 (for issue #9907). When a Toolset object is passed to a component's tools parameter, the warm_up_tools() function only calls Toolset.warm_up(), which is a no-op. It doesn't iterate through the individual tools inside the Toolset to warm them up. acknowledged by @vblagoje and @sjrl This implementation: - Modified warm_up_tools() to iterate through Toolsets and call warm_up() on each individual tool - Added comprehensive tests for Toolset warming behavior - Verified both the Toolset and its contained tools are warmed up Checklist: I have read the contributors guidelines and the code of conduct I have updated the related issue with new insights and changes I added unit tests and updated the docstrings I've used one of the conventional commit types for my PR title: fix: I documented my code I ran pre-commit hooks and fixed any issue * added release note * refactor: move tool warm-up iteration to Toolset.warm_up() Addresses architectural feedback - moved iteration logic from warm_up_tools() to base Toolset.warm_up() for better encapsulation. Subclasses can now override warm_up() to customize initialization without breaking the contract. - Toolset.warm_up() now iterates and warms tools by default - warm_up_tools() simplified to delegate to warm_up() - Updated tests and release notes --------- Co-authored-by: HamidOna13 <abdulhamid.onawole@aizatron.com>

HamidOna13 added 6 commits October 26, 2025 07:44

docs: Add release notes for warm_up() feature

9ca93ec

HamidOna requested a review from a team as a code owner October 26, 2025 12:38

HamidOna requested review from mpangrazzi and removed request for a team October 26, 2025 12:38

github-actions Bot added topic:tests type:documentation Improvements on the docs labels Oct 26, 2025

sjrl requested review from vblagoje and removed request for mpangrazzi October 27, 2025 08:43

HamidOna closed this Oct 28, 2025

HamidOna reopened this Oct 28, 2025

vblagoje approved these changes Oct 28, 2025

View reviewed changes

vblagoje reviewed Oct 29, 2025

View reviewed changes

vblagoje merged commit 6c78f10 into deepset-ai:main Oct 29, 2025
43 of 44 checks passed

vblagoje mentioned this pull request Oct 29, 2025

fix: Add warm_up() calls to tests after ChatGenerator tool initialization changes #9965

Merged

sjrl mentioned this pull request Oct 29, 2025

Instead of throwing a RuntimeError if a Component has not been warmed up just call warm_up at runtime #9966

Closed

HamidOna mentioned this pull request Nov 1, 2025

fix: warm up individual tools inside Toolsets in warm_up_tools() #10002

Merged

6 tasks

sjrl mentioned this pull request Nov 5, 2025

feat: Add Chat Generator supporting OpenAI Responses API #9808

Merged

Uh oh!

Conversation

HamidOna commented Oct 26, 2025

Related Issues

Proposed Changes:

How did you test it?

Notes for the reviewer

Checklist

Uh oh!

vercel Bot commented Oct 26, 2025

Uh oh!

CLAassistant commented Oct 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HamidOna commented Oct 26, 2025

Uh oh!

vblagoje commented Oct 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vblagoje commented Oct 28, 2025

Uh oh!

coveralls commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 18879776280

Details

💛 - Coveralls

Uh oh!

vblagoje commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HamidOna commented Oct 28, 2025

Uh oh!

HamidOna commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vblagoje left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HamidOna commented Oct 28, 2025

Uh oh!

HamidOna13 commented Oct 28, 2025

Uh oh!

vblagoje Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

HamidOna Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

vblagoje Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

sjrl Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

vblagoje Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

vblagoje Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

vblagoje Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

CLAassistant commented Oct 26, 2025 •

edited

Loading

vblagoje commented Oct 27, 2025 •

edited

Loading

coveralls commented Oct 28, 2025 •

edited

Loading

vblagoje commented Oct 28, 2025 •

edited

Loading

HamidOna commented Oct 28, 2025 •

edited

Loading

vblagoje left a comment •

edited

Loading