Avoid redundant allocations in struct layout reader#8851
Conversation
Pre-size the dtype and name vectors to their final length and push the validity entry first, instead of collecting at field-count capacity and then inserting at index 0. This removes both the reallocation and the O(n) element shift caused by `insert(0, ..)`. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Merging this PR will improve performance by 11.17%
Performance Changes
Tip Curious why this is faster? Comment Comparing Footnotes
|
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: Vortex queries 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.040x ➖, 0↑ 0↓)
datafusion / parquet (0.979x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.973x ➖, 0↑ 0↓)
duckdb / parquet (1.010x ➖, 0↑ 0↓)
No file size changes detected. |
Pre-size the dtype and name vectors to their final length and push the
validityentry first, instead of collecting at field-count capacity and theninsert(0, ..).The previous code
collected eachVecat exact field-count capacity, theninsert(0, ..)forced a reallocation and an O(n) shift of every element. Both are now gone.Checks
cargo build -p vortex-layout✅🤖 Generated with Claude Code