ci: add workflow to check correct API reference generation

[anakin87]

Related Issues

• related to https://github.com/deepset-ai/haystack-private/issues/217

We'd like auto-generated PRs for API reference updates to be automatically approved.

Ideally, we'd like to avoid broken Docusaurus builds (in case of failure, the website won't be updated in any case).

Since checking Vercel deployment success from a PR seems very hard to do (I tried some time ago), I am bringing into Haystack a mechanism introduced some time ago in core integrations and experimental: try generating the API reference when it changes in the PR and do a lightweight check with docusaurus-mdx-checker.

(deepset-ai/haystack-core-integrations#2829; deepset-ai/haystack-experimental#437)

Proposed Changes:

• introduce a workflow that looks for changed pydoc configs or Haystack python files, builds the API references and runs docusaurus-mdx-checker on it
  • use the docstrings_checksum script to avoid building the API reference when docstring don't change

How did you test it?

Temporarily triggered the workflow on this PR (by changing configs or docstrings): https://github.com/deepset-ai/haystack/actions/runs/23156953275/job/67274135243
Works correctly

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
haystack-docs	Ready	Preview, Comment	Mar 17, 2026 9:48am

[coveralls]

Pull Request Test Coverage Report for Build 23188105850

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 92.908%

---

Totals
Change from base Build 23188036496:	0.0%
Covered Lines:	15813
Relevant Lines:	17020

---

💛 - Coveralls

[davidsbatista]

LGTM