Skip to content

Support out of order grouped aggregate accumulation#8379

Open
gatesn wants to merge 4 commits into
developfrom
ngates/grouped-aggregate
Open

Support out of order grouped aggregate accumulation#8379
gatesn wants to merge 4 commits into
developfrom
ngates/grouped-aggregate

Conversation

@gatesn

@gatesn gatesn commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Our original API for grouped aggregate functions only supported groups that we pre-sorted into complete lists. That means large groups would need to be globally sorted before we could start accumulating their internal state. This is not how most engines define their aggregate APIs.

This PR allows us to support grouped accumulation in the more traditional way by using group IDs.

Signed-off-by: "Nicholas Gates" <nick@nickgates.com>
@gatesn gatesn requested a review from onursatici June 11, 2026 21:41
@gatesn gatesn added the changelog/break A breaking API change label Jun 11, 2026
@gatesn gatesn changed the title Support dense grouped aggregate accumulation Support out of order grouped aggregate accumulation Jun 11, 2026
Signed-off-by: "Nicholas Gates" <nick@nickgates.com>
@gatesn gatesn marked this pull request as ready for review June 11, 2026 21:43
@gatesn gatesn requested a review from a team June 11, 2026 21:43
@gatesn gatesn enabled auto-merge (squash) June 11, 2026 21:43
@codspeed-hq

codspeed-hq Bot commented Jun 11, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by ×2.1

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 15 improved benchmarks
✅ 1521 untouched benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation count_i32_clustered_nulls 651.2 µs 62.5 µs ×10
Simulation sum_f64_clustered_nulls 557.1 µs 92.8 µs ×6
Simulation sum_i32_clustered_nulls 552.4 µs 92.6 µs ×6
Simulation count_varbinview 253.9 µs 50.5 µs ×5
Simulation sum_f64_all_valid 294.4 µs 79.8 µs ×3.7
Simulation sum_i32_nullable_all_valid 328.9 µs 138.1 µs ×2.4
Simulation chunked_bool_canonical_into[(1000, 10)] 35.5 µs 20.3 µs +74.93%
Simulation chunked_varbinview_canonical_into[(1000, 10)] 198.1 µs 161.7 µs +22.48%
Simulation chunked_varbinview_into_canonical[(1000, 10)] 213.5 µs 177.1 µs +20.57%
Simulation bitwise_not_vortex_buffer_mut[128] 244.4 ns 215.3 ns +13.55%
Simulation encode_varbin[(1000, 4)] 159.7 µs 142.4 µs +12.11%
Simulation encode_varbin[(1000, 8)] 160.4 µs 143.5 µs +11.73%
Simulation encode_varbin[(1000, 32)] 164.9 µs 148.7 µs +10.87%
Simulation bitwise_not_vortex_buffer_mut[1024] 304.7 ns 275.6 ns +10.58%
Simulation encode_varbin[(1000, 512)] 219.7 µs 199.3 µs +10.24%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ngates/grouped-aggregate (adae76e) with develop (8acef3a)

Open in CodSpeed

gatesn added 2 commits June 11, 2026 18:04
Signed-off-by: "Nicholas Gates" <nick@nickgates.com>
I, Nicholas Gates <nick@nickgates.com>, hereby add my Signed-off-by to this commit: 6bdcd32

I, Nicholas Gates <nick@nickgates.com>, hereby add my Signed-off-by to this commit: 9bd157f

I, Nicholas Gates <nick@nickgates.com>, hereby add my Signed-off-by to this commit: 50701b2

Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn gatesn requested a review from joseph-isaacs June 12, 2026 02:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/break A breaking API change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant