Commit 32cda28
fix(manifest): encode day-transform partition fields with Avro date logical type (#915)
`DayTransform.Apply()` returns an `int32` value representing days since
the epoch. For manifest partition data, Iceberg implementations commonly
write `day(...)` partition fields as Avro `int` with the `date` logical
type, while readers should be able to accept both plain Avro `int` and
Avro `date`.
Previously, `DayTransform.ResultType()` returned `Int32`, which caused
`PartitionSpec.PartitionType()` to model day-transform partition fields
as plain `Int32`. As a result, Iceberg Go wrote day-partition fields as
plain Avro `int`, instead of Avro `date`, diverging from the writer
behavior of other Iceberg implementations.
This PR changes `DayTransform.ResultType()` to return `DateType`. That
lets the existing partition type and Avro schema conversion paths
naturally emit Avro `date` for day-transform partition fields.
Tests added:
- Verify that `day(ts)` partition fields are encoded with the Avro
`date` logical type.
- Verify that non-day `Int32` partition fields still use plain Avro
`int` encoding.
---------
Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com>1 parent 9af9dc5 commit 32cda28
2 files changed
Lines changed: 70 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
196 | 197 | | |
197 | 198 | | |
198 | 199 | | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
786 | 786 | | |
787 | 787 | | |
788 | 788 | | |
789 | | - | |
| 789 | + | |
790 | 790 | | |
791 | 791 | | |
792 | 792 | | |
| |||
0 commit comments