Related Issues

• fixes https://github.com/deepset-ai/haystack-private/issues/133

Proposed Changes:

Add workflow and script to sync docs with the search index.

Workflow
Builds Docusaurus website and runs the script.

Executed daily (and manually triggered if needed).

Script

1. Collects all HTML files from the docs and reference directories for the stable Haystack version.
2. Uploads the HTML files to the deepset workspace.
   • A timestamp-based metadata field is used to track document versions in the workspace.
3. Deletes the old HTML files from the deepset workspace.
   • Since most files are overwritten during upload, only a small number of deletions is expected.
   • In case MAX_DELETIONS_SAFETY_LIMIT is exceeded, we block the deletion.

How did you test it?

Several manual runs and some others in this PR. See for example https://github.com/deepset-ai/haystack/actions/runs/19678376584/job/56365735670

Notes for the reviewer

The script is simple and aligned with our website structure.

For each run of the workflow, we basically overwrite the previous docs uploaded on deepset. I chose this approach to keep the CI lean (we don't run this workflow every time the docs change) and for simplicity.
While we could have compared git commits, with multiple documentation versions that also change at release time, the comparison logic would have become quite tricky.

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.