🐛 Describe the bug
Document.from_dict() cannot correctly deserialize certain dictionaries produced by Document.to_dict() using the default flatten=True behavior.
When a document's metadata contains a key literally named "meta" alongside additional metadata keys, the serialized representation is valid, but deserialization raises a ValueError instead of reconstructing the original Document.
This breaks the expected round-trip invariant:
Document.from_dict(document.to_dict()) == document
❌ Error message
ValueError: You can pass either the 'meta' parameter or flattened metadata keys as keyword arguments, but currently you're passing both. Pass either the 'meta' parameter or flattened metadata keys.
✅ Expected behavior
Any document serialized using Document.to_dict() should successfully deserialize with Document.from_dict(), regardless of the names of metadata keys.
The following should always succeed:
restored = Document.from_dict(document.to_dict())
assert restored == document
🔁 Steps to reproduce
from haystack import Document
doc = Document(
content="Hello",
meta={
"meta": {"nested": "value"},
"author": "Alice",
},
)
serialized = doc.to_dict() # flatten=True (default)
Document.from_dict(serialized)
Current result
ValueError: You can pass either the 'meta' parameter or flattened metadata keys...
Expected result
restored = Document.from_dict(serialized)
assert restored == doc
assert restored.meta == {
"meta": {"nested": "value"},
"author": "Alice",
}
🔍 Root cause
For flattened documents, the serialized output becomes:
{
"id": "...",
"content": "Hello",
"meta": {"nested": "value"},
"author": "Alice",
}
Document.from_dict() interprets the top-level "meta" entry as the root metadata container instead of a flattened metadata key.
When the remaining flattened metadata ("author") is processed, the implementation believes both flattened and non-flattened metadata were supplied simultaneously and raises a ValueError.
📝 Related context
PR #11952 fixed the round-trip case where the metadata key "meta" contains a non-mapping value.
This issue concerns the remaining edge case where the "meta" metadata key itself contains a mapping and additional flattened metadata keys are present.
📋 Environment
| Component |
Value |
| OS |
Windows 11 |
| Haystack Version |
main |
| Commit |
1bc569f66ef2495a6ce676dcb2037d3c189769be |
☑️ Checklist
🐛 Describe the bug
Document.from_dict()cannot correctly deserialize certain dictionaries produced byDocument.to_dict()using the defaultflatten=Truebehavior.When a document's metadata contains a key literally named
"meta"alongside additional metadata keys, the serialized representation is valid, but deserialization raises aValueErrorinstead of reconstructing the originalDocument.This breaks the expected round-trip invariant:
❌ Error message
✅ Expected behavior
Any document serialized using
Document.to_dict()should successfully deserialize withDocument.from_dict(), regardless of the names of metadata keys.The following should always succeed:
🔁 Steps to reproduce
Current result
Expected result
🔍 Root cause
For flattened documents, the serialized output becomes:
{ "id": "...", "content": "Hello", "meta": {"nested": "value"}, "author": "Alice", }Document.from_dict()interprets the top-level"meta"entry as the root metadata container instead of a flattened metadata key.When the remaining flattened metadata (
"author") is processed, the implementation believes both flattened and non-flattened metadata were supplied simultaneously and raises aValueError.📝 Related context
PR #11952 fixed the round-trip case where the metadata key
"meta"contains a non-mapping value.This issue concerns the remaining edge case where the
"meta"metadata key itself contains a mapping and additional flattened metadata keys are present.📋 Environment
main1bc569f66ef2495a6ce676dcb2037d3c189769be☑️ Checklist
mainbranch