Skip to content

Commit 3b67bd8

Browse files
committed
avro: restore compatibility docs and sqllogictest expectations
1 parent a0755b7 commit 3b67bd8

2 files changed

Lines changed: 6 additions & 10 deletions

File tree

  • datafusion/sqllogictest/test_files
  • docs/source/library-user-guide/upgrading

datafusion/sqllogictest/test_files/avro.slt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ LOCATION '../../testing/data/avro/alltypes_plain.avro';
124124

125125
statement ok
126126
CREATE EXTERNAL TABLE single_nan (
127-
mycol DOUBLE
127+
mycol FLOAT
128128
)
129129
STORED AS AVRO
130130
LOCATION '../../testing/data/avro/single_nan.avro';

docs/source/library-user-guide/upgrading/53.0.0.md

Lines changed: 5 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -473,11 +473,11 @@ Now:
473473
0 row(s) fetched.
474474
```
475475

476-
### Avro API, feature, and decoding changes
476+
### Avro API, feature, and timestamp decoding changes
477477

478478
As part of the Avro reader migration (see [#17861]), DataFusion now delegates
479-
Avro-to-Arrow type decoding to `arrow-avro` (aligned with `arrow-avro` semantics),
480-
and several Avro-related APIs/feature wiring changed:
479+
Avro-to-Arrow type decoding to `arrow-avro`, and several Avro-related
480+
APIs/feature wiring changed:
481481

482482
- `DataFusionError::AvroError` has been removed.
483483
- `From<apache_avro::Error> for DataFusionError` has been removed.
@@ -487,8 +487,7 @@ and several Avro-related APIs/feature wiring changed:
487487
- Cargo feature wiring changed:
488488
- `datafusion` crate `avro` feature no longer enables `datafusion-common/avro`
489489
- `datafusion-proto` crate `avro` feature no longer enables `datafusion-common/avro`
490-
- **Avro datatype interpretation now follows `arrow-avro` behavior.** Notable effects:
491-
- Avro `string` logical values are read as Arrow `Binary` in DataFusion Avro scans
490+
- **Avro timestamp logical type interpretation changed.** Notable effects:
492491
- Avro `timestamp-*` logical types are read as UTC timezone-aware Arrow timestamps
493492
(`Timestamp(..., Some("+00:00"))`)
494493
- Avro `local-timestamp-*` logical types remain timezone-naive
@@ -499,8 +498,7 @@ and several Avro-related APIs/feature wiring changed:
499498
- Users matching on `DataFusionError::AvroError`
500499
- Users importing `datafusion::apache_avro`
501500
- Users depending on the old `datafusion-common/avro` feature wiring
502-
- Users relying on DataFusion-specific Avro decoding behavior (especially `string`
503-
and timestamp logical types)
501+
- Users relying on previous Avro timestamp logical type behavior
504502

505503
**Migration guide:**
506504

@@ -509,8 +507,6 @@ and several Avro-related APIs/feature wiring changed:
509507
the current error surface.
510508
- If you depend on Avro feature propagation, update Cargo feature expectations:
511509
`datafusion`/`datafusion-proto` `avro` no longer enables `datafusion-common/avro`.
512-
- Review Avro table schemas and add explicit casts where needed for binary-backed
513-
string values (for example, `CAST(binary_col AS VARCHAR)`).
514510
- Validate timestamp handling where timezone semantics matter:
515511
`timestamp-*` is UTC timezone-aware, while `local-timestamp-*` is timezone-naive.
516512

0 commit comments

Comments
 (0)