Commit 7ae534c
committed
fix(manifest): encode day-transform partition fields with Avro date logical type
DayTransform.Apply() returns int32 (days since epoch) but the Avro
encoding of a day-partition column must carry the 'date' logical type
so that Trino, Spark, and other Iceberg engines can read manifests.
Root fix: PartitionSpec.PartitionType() now overrides DayTransform's
ResultType (Int32) to DateType for the partition struct it produces.
The existing DateType branch in partitionTypeToAvroSchema then emits
DateNode (int + date logical type) automatically — no coupling between
the Avro layer and transform internals.
This also fixes DataFileStatistics.ToDataFile() in table/internal/utils.go,
which builds fieldIDToLogicalType by switching on ResultType: the
case DateType branch now fires correctly for day-partition fields.
Tests verify that day-partition fields decode as time.Time (confirming
the date logical type is applied) while plain Int32 fields decode as
int32.1 parent efe96b6 commit 7ae534c
2 files changed
Lines changed: 80 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
499 | 499 | | |
500 | 500 | | |
501 | 501 | | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
502 | 511 | | |
503 | 512 | | |
504 | 513 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
196 | 197 | | |
197 | 198 | | |
198 | 199 | | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
0 commit comments