FragmentMetadata.to_json() raises NotImplementedError when the fragment has stable row ID metadata (row_id_meta is not None). This makes it impossible to serialize fragment metadata via JSON when enable_stable_row_ids=True.
Minimal repro
import lance
import pyarrow as pa
uri = "/tmp/repro_to_json.lance"
ds = lance.write_dataset(pa.table({"x": [1, 2, 3]}), uri, enable_stable_row_ids=True)
ds.get_fragments()[0].metadata.to_json() # raises NotImplementedError
Error
NotImplementedError: PyRowIdMeta.asdict is not yet supported.
The traceback points to lance/fragment.py:122:
self.row_id_meta.asdict() if self.row_id_meta is not None else None
Impact
Any workflow that round-trips fragment metadata through JSON breaks when stable row IDs are enabled. For example, serializing FragmentMetadata via json.dumps(frag.to_json()) after fragment.update_columns() for later reconstruction with FragmentMetadata.from_json() no longer works. This is needed for distributed workflows where fragment metadata must be serialized for transport between processes.
Environment
- lance 5.0.0-beta.4
- Python 3.11
- Linux x86_64
FragmentMetadata.to_json()raisesNotImplementedErrorwhen the fragment has stable row ID metadata (row_id_metais not None). This makes it impossible to serialize fragment metadata via JSON whenenable_stable_row_ids=True.Minimal repro
Error
The traceback points to
lance/fragment.py:122:Impact
Any workflow that round-trips fragment metadata through JSON breaks when stable row IDs are enabled. For example, serializing
FragmentMetadataviajson.dumps(frag.to_json())afterfragment.update_columns()for later reconstruction withFragmentMetadata.from_json()no longer works. This is needed for distributed workflows where fragment metadata must be serialized for transport between processes.Environment