Skip to content

Commit faef4a9

Browse files
dfa1claude
andcommitted
perf(compute): scale ComputeKernelBenchmark to 100M rows (memory-bound)
1M fit in L3 on Apple silicon; at 100M each column is ~800 MB, far past cache, so the kernels are genuinely memory-bound. Writes 100 chunks of 1M (reusing the write arrays to bound heap), decodes all chunks once in setup, and each method scans the full 100M per op. Steady-state (Apple M5, ms/op) forLoop vs Compute.filter: plain 53 vs 344, FoR 56 vs 336, ALP 77 vs 464, Dict 573 vs 839; sum 67 vs 66. Two separate levers fall out: (1) the filter mask-write path is ~6x the naive loop even memory-bound (and on a plain column), so the per-element off-heap bitmap write — not decode — is the cost; (2) Dict alone decodes ~10x slower than the others (codes->values random scatter), the one place encoded-domain compare-on-codes would pay. sum already matches the loop. The Dict column surfaces as OffsetLongArray(DictLongArray) at scale (shared flat layout across aligned chunks); the type assert unwraps the offset view. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 2ac0e6c commit faef4a9

1 file changed

Lines changed: 235 additions & 127 deletions

File tree

0 commit comments

Comments
 (0)