feat: sort file groups by statistics during sort pushdown (Sort pushdown phase 2)#21182

Merged

adriangb merged 17 commits intoapache:mainfrom

zhuqi-lucas:feat/sort-file-groups-by-statistics

Apr 6, 2026

Commits on Apr 6, 2026

feat: sort file groups by statistics during sort pushdown

zhuqi-lucas
and
claude
committed
address review: remove unnecessary stats computation in reverse path, improve docs

zhuqi-lucas
and
claude
committed
add test: reverse path preserves file order, does not apply stats sorting

zhuqi-lucas
and
claude
committed
restore doc comment with ASCII diagrams for get_projected_output_ordering

zhuqi-lucas
and
claude
committed
docs: add non-overlapping exception to partition ordering diagram

zhuqi-lucas
and
claude
committed
remove redistribute_files_across_groups_by_statistics

zhuqi-lucas
and
claude
committed
fix: re-check ordering after statistics-based file sorting

zhuqi-lucas
and
claude
committed
fix: preserve fetch (LIMIT) when eliminating SortExec via Exact pushdown

zhuqi-lucas
and
claude
committed
docs: accurate PR description and code comments for sort pushdown

zhuqi-lucas
and
claude
committed
docs: fix PR description and code comments to accurately describe sort pushdown behavior
zhuqi-lucas
committed
perf: increase SortPreservingMergeExec prefetch buffer from 1 to 16
zhuqi-lucas
committed
fix: prevent sort elimination when files have NULLs in sort columns
zhuqi-lucas
committed
fix comments
zhuqi-lucas
committed
refactor: scope SPM prefetch buffer to sort elimination path only
zhuqi-lucas
committed
fmt
zhuqi-lucas
committed
refactor: replace SPM prefetch with BufferExec for sort elimination
zhuqi-lucas
committed
perf: increase BufferExec capacity from 8MB to 64MB for sort elimination
zhuqi-lucas
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: sort file groups by statistics during sort pushdown (Sort pushdown phase 2)#21182

feat: sort file groups by statistics during sort pushdown (Sort pushdown phase 2)#21182
adriangb merged 17 commits intoapache:mainfrom
zhuqi-lucas:feat/sort-file-groups-by-statistics

Commits on Apr 6, 2026

feat: sort file groups by statistics during sort pushdown

address review: remove unnecessary stats computation in reverse path, improve docs

add test: reverse path preserves file order, does not apply stats sorting

restore doc comment with ASCII diagrams for get_projected_output_ordering

docs: add non-overlapping exception to partition ordering diagram

remove redistribute_files_across_groups_by_statistics

fix: re-check ordering after statistics-based file sorting

fix: preserve fetch (LIMIT) when eliminating SortExec via Exact pushdown

docs: accurate PR description and code comments for sort pushdown

docs: fix PR description and code comments to accurately describe sort pushdown behavior

perf: increase SortPreservingMergeExec prefetch buffer from 1 to 16

fix: prevent sort elimination when files have NULLs in sort columns

fix comments

refactor: scope SPM prefetch buffer to sort elimination path only

fmt

refactor: replace SPM prefetch with BufferExec for sort elimination

perf: increase BufferExec capacity from 8MB to 64MB for sort elimination