fix: Update the de/serialization with schema utils by Amnah199 · Pull Request #9526 · deepset-ai/haystack

Amnah199 · 2025-06-17T11:17:30Z

Related Issues

The current serialize function assumes the input is always a dictionary, which is problematic in flexible input scenarios (e.g., breakpoints).
When serializing a dictionary, only type: "object" is stored without detailing its structure — properties are missing or incomplete.

Proposed Changes:

Refactor serialize to support flexible value types beyond dictionaries.
When serializing dictionaries, store as type: "object", properties: "key":"value" to properly describe the schema.

How did you test it?

Updated tests
Added new tests

Notes for the reviewer

It was anticipated that these functions might need to be updated.

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
I documented my code
I ran pre-commit hooks and fixed any issue

…/deepset-ai/haystack into update_serialization_schema_util

sjrl · 2025-06-18T07:04:45Z

@Amnah199 looking really good! A few minor comments and only one slightly larger one here

coveralls · 2025-06-18T08:02:15Z

Pull Request Test Coverage Report for Build 15848630494

Details

0 of 0 changed or added relevant lines in 0 files are covered.
21 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.007%) to 90.175%

Files with Coverage Reduction	New Missed Lines	%
core/pipeline/async_pipeline.py	3	67.74%
utils/base_serialization.py	18	89.09%

Totals
Change from base Build 15846939495:	-0.007%
Covered Lines:	11610
Relevant Lines:	12875

💛 - Coveralls

…date_serialization_schema_util

…/deepset-ai/haystack into update_serialization_schema_util

sjrl · 2025-06-23T08:35:36Z

+    # Handle Haystack style objects (e.g. dataclasses and Components)
+    elif hasattr(payload, "to_dict") and callable(payload.to_dict):


This is slightly tangential, but could you open a follow up issue to add a to_dict and from_dict method for ByteStream? It's the only dataclass we regularly return from our components that's missing a to_dict and from_dict method.

Created the issue here #9543 .

…/deepset-ai/haystack into update_serialization_schema_util

sjrl

Looks good!

Update the util methods

d4e5698

github-actions Bot added topic:tests type:documentation Improvements on the docs labels Jun 17, 2025

Update tests

925f067

Amnah199 added ignore-for-release-notes PRs with this flag won't be included in the release notes. labels Jun 17, 2025

Amnah199 added 4 commits June 17, 2025 18:20

Merge branch 'main' into update_serialization_schema_util

0a31792

fix tests

9a72fc7

Merge branch 'update_serialization_schema_util' of https://github.com…

51eaf9a

…/deepset-ai/haystack into update_serialization_schema_util

schema fix

d5ce027

Amnah199 marked this pull request as ready for review June 17, 2025 22:06

Amnah199 requested a review from a team as a code owner June 17, 2025 22:06

Amnah199 requested review from davidsbatista and sjrl and removed request for a team and davidsbatista June 17, 2025 22:06

sjrl reviewed Jun 18, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py Outdated

sjrl reviewed Jun 18, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py Outdated

sjrl reviewed Jun 18, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py

sjrl reviewed Jun 18, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py Outdated

sjrl reviewed Jun 18, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py Outdated

Merge branch 'main' into update_serialization_schema_util

17ad477

Add json schema for tuples and sets

831b28d

Amnah199 added this to the 2.15.0 milestone Jun 20, 2025

Amnah199 added 4 commits June 20, 2025 13:42

Merge branch 'main' of https://github.com/deepset-ai/haystack into up…

2423b10

…date_serialization_schema_util

Add proper conversion for sets and tuples

213df05

Merge branch 'update_serialization_schema_util' of https://github.com…

cc1bcc3

…/deepset-ai/haystack into update_serialization_schema_util

Adjust typing

142a590

Amnah199 added 2 commits June 20, 2025 16:01

Linting

0a57b82

Optimize deserialization

357dbf5

Amnah199 self-assigned this Jun 22, 2025

Amnah199 added 2 commits June 23, 2025 10:23

remove TODO

734b2a7

Merge branch 'main' into update_serialization_schema_util

3760663

sjrl reviewed Jun 23, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py Outdated

sjrl reviewed Jun 23, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py Outdated

sjrl reviewed Jun 23, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py Outdated

sjrl reviewed Jun 23, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py Outdated

sjrl reviewed Jun 23, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py Outdated

sjrl reviewed Jun 23, 2025

View reviewed changes

Comment thread test/utils/test_base_serialization.py Outdated

Amnah199 added 4 commits June 23, 2025 11:23

PR comments

0fa85b5

Merge branch 'update_serialization_schema_util' of https://github.com…

8c5ce3e

…/deepset-ai/haystack into update_serialization_schema_util

PR comments

9078591

Update tests and deserialization error

5f12dd6

Amnah199 commented Jun 23, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py

Amnah199 mentioned this pull request Jun 23, 2025

feat: add to_dict/from_dict to ByteStream #9543

Closed

Support legacy deserialization

69bd12f

Amnah199 commented Jun 23, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py

sjrl reviewed Jun 23, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py Outdated

sjrl reviewed Jun 23, 2025

View reviewed changes

Comment thread haystack/utils/base_serialization.py

Amnah199 added 2 commits June 23, 2025 16:41

Update deprecating warning

822198b

Update test

1564cf7

Amnah199 mentioned this pull request Jun 23, 2025

chore: Remove backward compatibility for State deserialization #9544

Closed

sjrl approved these changes Jun 24, 2025

View reviewed changes

Merge branch 'main' into update_serialization_schema_util

552c3e4

Amnah199 merged commit 9ed0b9b into main Jun 24, 2025
17 checks passed

Amnah199 deleted the update_serialization_schema_util branch June 24, 2025 11:10

sjrl mentioned this pull request Jun 30, 2025

feat: add to_dict/from_dict to StreamingChunk #9567

Closed

		# Handle Haystack style objects (e.g. dataclasses and Components)
		elif hasattr(payload, "to_dict") and callable(payload.to_dict):

Conversation

Amnah199 commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issues

Proposed Changes:

How did you test it?

Notes for the reviewer

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sjrl commented Jun 18, 2025

Uh oh!

coveralls commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 15848630494

Details

💛 - Coveralls

Uh oh!

Uh oh!

sjrl Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

Amnah199 Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sjrl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Amnah199 commented Jun 17, 2025 •

edited

Loading

coveralls commented Jun 18, 2025 •

edited

Loading