Skip to content

Commit 697891b

Browse files
authored
fix: use kLegacyPartitionDataIdStart as partition field ID baseline (#832)
FreshPartitionSpec and TableMetadataBuilder::Impl both initialized the partition field ID counter to `kInvalidPartitionFieldId` (-1), causing the first partition field to get ID **0** instead of **1000**. Partition field ID 0 collides with ManifestEntry internal fields: | Field | ID | |---|---| | `ManifestEntry.status` | 0 | | `ManifestEntry.snapshot_id` | 1 | | `ManifestEntry.data_file` | 2 | This corrupts manifest file schemas during serialization and breaks partition pruning. ## Fix Change both initializers to `kLegacyPartitionDataIdStart - 1` (999), consistent with Java Iceberg's `PARTITION_DATA_ID_START` convention and the rest of the C++ codebase (`PartitionSpec` constructor, `Unpartitioned()`, `UpdatePartitionSpec`). The first partition field now correctly gets ID **1000**. | Location | Before | After | |---|---|---| | `FreshPartitionSpec` counter | -1 | 999 | | `TableMetadataBuilder::Impl::last_partition_id` | -1 | 999 | | First partition field ID | 0 | **1000** |
1 parent 330de84 commit 697891b

2 files changed

Lines changed: 3 additions & 3 deletions

File tree

src/iceberg/table_metadata.cc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ Result<std::unique_ptr<PartitionSpec>> FreshPartitionSpec(int32_t spec_id,
7171
const Schema& fresh_schema) {
7272
std::vector<PartitionField> partition_fields;
7373
partition_fields.reserve(spec.fields().size());
74-
int32_t last_partition_field_id = PartitionSpec::kInvalidPartitionFieldId;
74+
int32_t last_partition_field_id = PartitionSpec::kLegacyPartitionDataIdStart - 1;
7575
for (auto& field : spec.fields()) {
7676
ICEBERG_ASSIGN_OR_RAISE(auto source_name,
7777
base_schema.FindColumnNameById(field.source_id()));
@@ -556,7 +556,7 @@ class TableMetadataBuilder::Impl {
556556
metadata_.last_updated_ms = kInvalidLastUpdatedMs;
557557
metadata_.last_column_id = Schema::kInvalidColumnId;
558558
metadata_.default_spec_id = PartitionSpec::kInitialSpecId;
559-
metadata_.last_partition_id = PartitionSpec::kInvalidPartitionFieldId;
559+
metadata_.last_partition_id = PartitionSpec::kLegacyPartitionDataIdStart - 1;
560560
metadata_.current_snapshot_id = kInvalidSnapshotId;
561561
metadata_.default_sort_order_id = SortOrder::kInitialSortOrderId;
562562
metadata_.next_row_id = TableMetadata::kInitialRowId;

src/iceberg/test/table_metadata_builder_test.cc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ TEST(TableMetadataTest, Make) {
125125
auto spec_fields =
126126
metadata->partition_specs[0]->fields() | std::ranges::to<std::vector>();
127127
ASSERT_EQ(1, spec_fields.size());
128-
EXPECT_EQ(PartitionSpec::kInvalidPartitionFieldId + 1, spec_fields[0].field_id());
128+
EXPECT_EQ(PartitionSpec::kLegacyPartitionDataIdStart, spec_fields[0].field_id());
129129
EXPECT_EQ(2, spec_fields[0].source_id());
130130
EXPECT_EQ("part_name", spec_fields[0].name());
131131

0 commit comments

Comments
 (0)