Commit 35a581c
committed
perf: increase SortPreservingMergeExec prefetch buffer from 1 to 16
When SPM reads directly from I/O-bound sources (e.g., DataSourceExec
without SortExec buffering), the merge loop stalls waiting for Parquet
I/O on each poll. Increasing the prefetch buffer lets background tasks
read ahead while the merge processes previous batches.
Local benchmark (release, 16 partitions, sort_pushdown_sorted):
Q1 full scan: Main 110ms → PR 82ms (1.3x faster)
Q3 SELECT *: Main 239ms → PR 228ms (1.05x faster)
Q2/Q4 LIMIT: 3-7ms (unchanged, already fast)1 parent 911f0dd commit 35a581c
1 file changed
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
361 | 361 | | |
362 | 362 | | |
363 | 363 | | |
364 | | - | |
| 364 | + | |
365 | 365 | | |
366 | 366 | | |
367 | 367 | | |
| |||
0 commit comments