Commit 249c23c
authored
~(Draft until I am sure I can use this API to make FileStream behave
better)~
## Which issue does this PR close?
- part of #20529
- Needed for #21351
- Broken out of #20820
- Closes #21427
## Rationale for this change
I can get 10% faster on many ClickBench queries by reordeirng files at
runtime. You can see it all working together here:
#21351
To do do, I need to rework the FileStream so that it can reorder
operations at runtime. Eventually that will include both CPU and IO.
This PR is a step in the direction by introducing the main Morsel API
and implementing it for Parquet. The next PR
(#21342) rewrites FileStream in
terms of the Morsel API
## What changes are included in this PR?
1. Add proposed `Morsel` API
2. Rewrite Parquet opener in terms of that API
3. Add an adapter layer (back to FileOpener, so I don't have to rewrite
FileStream in the same PR)
My next PR will rewrite the FileStream to use the Morsel API
## Are these changes tested?
Yes by existing CI.
I will work on adding additional tests for just Parquet opener in a
follow on PR
## Are there any user-facing changes?
No
1 parent 6a770aa commit 249c23c
0 commit comments