fix: S3Downloader - improve concurrent behavior

[anakin87]

Related Issues

• partially addresses S3Downloader cache races cause corrupt downloads under concurrent pipeline runs #3294

Proposed Changes:

• bug: _download_file writes straight to the final cache path, so a concurrent reader can observe a partial PDF mid-write -> download to a temp path and finally move the file to its expected location (no more partial files)
• other minor race condition: if file_path.is_file(): file_path.touch(). Between the condition and the touch command, another downloader running could potentially delete the file and touch recreates an empty one. -> use utime that does not create the file but only updates timestamp

How did you test it?

CI

Notes for the reviewer

Another major race condition (_cleanup_cache only protects files referenced by the current run, so it can delete files another in-flight run is about to read) is not touched by this PR because it's hard to avoid. A workaround is setting max_cache_size to a high number of files (similar to disabling cache deletion).

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[github-actions]

Coverage report (amazon_bedrock)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
integrations/amazon_bedrock/src/haystack_integrations/components/downloaders/s3
s3_downloader.py
Project Total

This report was generated by python-coverage-comment-action

[sjrl]

Thanks, looks good!