Skip to content

fix: coerce non-string metadata values to strings in MemoryFact.parse_metadata#2623

Open
handnewb wants to merge 2 commits into
vectorize-io:mainfrom
handnewb:fix/metadata-original-id-coercion
Open

fix: coerce non-string metadata values to strings in MemoryFact.parse_metadata#2623
handnewb wants to merge 2 commits into
vectorize-io:mainfrom
handnewb:fix/metadata-original-id-coercion

Conversation

@handnewb

@handnewb handnewb commented Jul 8, 2026

Copy link
Copy Markdown

Summary

Fix consolidation blocker caused by non-string metadata values in MemoryFact.

Problem

When memory_units contain metadata with non-string values (e.g., {"original_id": 348} from observation bookmarks), the consolidation worker fails with:

ValidationError: metadata.original_id — Input should be a valid string [type=string_type]

This happens because MemoryFact.metadata is typed as dict[str, str] but the parse_metadata validator doesn't coerce non-string dict values.

Root Cause

In engine/response_models.py:253, metadata: dict[str, str]. The @field_validator("metadata") method only handles str → dict coercion (for asyncpg JSONB serialization), not int/other → str coercion for individual values.

Fix

Added value coercion in parse_metadata: after JSON parsing (or on raw dict input), iterate all key-value pairs and convert both to strings with {str(k): str(val) for k, val in v.items()}. This safely handles integer, float, bool, and None values.

Impact

  • 394 out of 10,741 memory_units affected (3.7%) in a production bank
  • Consolidation was completely blocked with 6,321 pending items
  • 71 consecutive consolidation failures before workaround applied

Testing

  • Existing tests pass (no behavior change for already-string metadata)
  • Verified on bank with 10,741 MUs — consolidation resumed immediately after fix + workaround cleanup
  • Backward compatible: string values pass through unchanged

Closes #2622

handnewb and others added 2 commits July 8, 2026 07:58
…_metadata

When memory_units contain metadata with non-string values (e.g.,
{"original_id": 348} from observation bookmarks), the consolidation
worker fails with:

  ValidationError: metadata.original_id — Input should be a valid string

This happens because metadata is typed as dict[str, str] but JSONB
columns can store integers. The parse_metadata validator handles
str→dict coercion (asyncpg) but not int→str coercion for values.

Fix: coerce all dict values to strings after JSON parsing.

Fixes consolidation blocking on banks with numeric original_id values.
@benfrank241

Copy link
Copy Markdown
Member

verified this fixes #2622 — reproduced the metadata.original_id string_type error on main and confirmed the coercion resolves it (int → "12345", also handles the asyncpg JSONB-string path, strings unchanged). pushed a regression test (tests/test_response_models.py, 4 cases) since there wasn't one.

one follow-up (not for this PR): the same dict[str, str] typing is on the HTTP response models too — RecallResult.metadata and the memory-unit list/detail models in api/http.py (lines ~376 / ~612). those don't run through MemoryFact.parse_metadata, so a numeric metadata value could still trip a ValidationError on the direct HTTP recall/browse path. worth a separate pass to apply the same coercion there. this PR correctly unblocks the consolidation path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Consolidation blocked when metadata values are non-string types (e.g., integer original_id)

2 participants