Commit 3457bc2
authored
use PyArrowFileIO as default for abfs and wasb schemes (#2395)
<!--
Thanks for opening a pull request!
-->
<!-- In the case this PR will resolve an issue, please replace
${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->
# Rationale for this change
Now that [pyarrow FileIO supports
ADLS](#2111), we can update
the
[SCHEMA_TO_FILE_IO](https://github.com/apache/iceberg-python/blob/370e770383c9c129aff2fe24cc92a68981688159/pyiceberg/io/__init__.py#L307-L319)
mapping for `abfs` and `wasb` to use ARROW_FILE_IO, similar to how it’s
handled for s3
We’re keeping FsspecFileIO as the preferred default, since the
PyArrowFileIO implementation is only available in pyarrow >= 20.0.0
## Are these changes tested?
## Are there any user-facing changes?
Add mapping SCHEMA_TO_FILE_IO for `wasb` and `abfs` to `FsspecFileIO`
and `PyArrowFileIO`
<!-- In the case of user-facing changes, please add the changelog label.
-->1 parent 3eecdad commit 3457bc2
1 file changed
+4
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
313 | 313 | | |
314 | 314 | | |
315 | 315 | | |
316 | | - | |
317 | | - | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
318 | 320 | | |
319 | 321 | | |
320 | 322 | | |
| |||
0 commit comments