Commit da8eb71
committed
feat(iceberg): add delete file support to SnapshotProducer
This enables SnapshotProducer to accept and process delete files
(position deletes and equality deletes) alongside data files.
Changes:
- Add added_delete_files field to SnapshotProducer
- Add validate_added_delete_files() for delete file validation:
- Rejects delete files in V1 format
- Validates content types (PositionDeletes, EqualityDeletes)
- Requires equality_ids for equality delete files
- Validates partition spec compatibility
- Add write_delete_manifest() to write delete manifests with
ManifestContentType::Deletes
- Update manifest_file() to include delete manifests
- Update summary() to populate delete file metrics
- Enhance validate_duplicate_files() for both data and delete files
- Add comprehensive unit tests
This lays the groundwork for operations like RowDelta that need to
atomically commit both data files and delete files.1 parent 2301352 commit da8eb71
2 files changed
Lines changed: 372 additions & 27 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
| 93 | + | |
93 | 94 | | |
94 | 95 | | |
95 | 96 | | |
| |||
0 commit comments