Skip to content

Commit e4dd17d

Browse files
zhuqi-lucasclaude
andcommitted
fix: preserve fetch (LIMIT) when eliminating SortExec via Exact pushdown
When PushdownSort returns Exact and removes SortExec, the fetch (LIMIT) from the original SortExec was lost. DataSourceExec then reads all data instead of stopping early. Now passes sort_exec.fetch() to inner.with_fetch() so DataSourceExec gets limit=N and stops reading after N rows. Benchmark impact (6M rows, reversed file names): Q2 ORDER BY LIMIT 100: main 1237ms → 9ms (99% faster) Q4 SELECT * LIMIT 100: main 5971ms → 24ms (99.6% faster) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 46fa442 commit e4dd17d

1 file changed

Lines changed: 8 additions & 1 deletion

File tree

datafusion/physical-optimizer/src/pushdown_sort.rs

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,14 @@ impl PhysicalOptimizerRule for PushdownSort {
9999
// Each node type defines its own pushdown behavior via try_pushdown_sort()
100100
match sort_input.try_pushdown_sort(required_ordering)? {
101101
SortOrderPushdownResult::Exact { inner } => {
102-
// Data source guarantees perfect ordering - remove the Sort operator
102+
// Data source guarantees perfect ordering - remove the Sort operator.
103+
// Preserve the fetch (LIMIT) from the original SortExec so the
104+
// data source can stop reading early.
105+
let inner = if let Some(fetch) = sort_exec.fetch() {
106+
inner.with_fetch(Some(fetch)).unwrap_or(inner)
107+
} else {
108+
inner
109+
};
103110
Ok(Transformed::yes(inner))
104111
}
105112
SortOrderPushdownResult::Inexact { inner } => {

0 commit comments

Comments
 (0)