Apache Iceberg Rust version
0.6.0 (latest version)
Describe the bug
The V3 spec serializes partition fields and sort fields whose transform takes multiple arguments with a source-ids list instead of the single source-id. PartitionField and SortField only know source-id, so deserializing table metadata that contains any multi-argument transform field fails with missing field source-id — the whole v3 table becomes unreadable.
Per the spec, all v3 readers are required to read tables with unknown transforms, ignoring them.
For comparison, PyIceberg implemented this read tolerance in apache/iceberg-python#3630.
To Reproduce
use iceberg::spec::PartitionField;
let field: Result<PartitionField, _> = serde_json::from_str(
r#"{"source-ids": [1, 2], "field-id": 1000, "name": "m", "transform": "bucket[4]"}"#,
);
// Err: missing field `source-id` — expected the field to load with Transform::Unknown
Loading v3 table metadata containing such a partition field or sort field fails the same way.
Expected behavior
Partition and sort fields with source-ids deserialize: a single-element list normalizes onto source-id; a multi-element list keeps the ids and treats the transform as Transform::Unknown. Serialization round-trips per the spec's writer rules (single-argument transforms write only source-id, multi-argument transforms write only source-ids).
Part of #2411 (Multi-arg Transforms → metadata read).
Willingness to contribute
I can contribute a fix for this bug independently
Apache Iceberg Rust version
0.6.0 (latest version)
Describe the bug
The V3 spec serializes partition fields and sort fields whose transform takes multiple arguments with a
source-idslist instead of the singlesource-id.PartitionFieldandSortFieldonly knowsource-id, so deserializing table metadata that contains any multi-argument transform field fails withmissing field source-id— the whole v3 table becomes unreadable.Per the spec, all v3 readers are required to read tables with unknown transforms, ignoring them.
For comparison, PyIceberg implemented this read tolerance in apache/iceberg-python#3630.
To Reproduce
Loading v3 table metadata containing such a partition field or sort field fails the same way.
Expected behavior
Partition and sort fields with
source-idsdeserialize: a single-element list normalizes ontosource-id; a multi-element list keeps the ids and treats the transform asTransform::Unknown. Serialization round-trips per the spec's writer rules (single-argument transforms write onlysource-id, multi-argument transforms write onlysource-ids).Part of #2411 (Multi-arg Transforms → metadata read).
Willingness to contribute
I can contribute a fix for this bug independently