Commit faef4a9
perf(compute): scale ComputeKernelBenchmark to 100M rows (memory-bound)
1M fit in L3 on Apple silicon; at 100M each column is ~800 MB, far past
cache, so the kernels are genuinely memory-bound. Writes 100 chunks of 1M
(reusing the write arrays to bound heap), decodes all chunks once in setup,
and each method scans the full 100M per op.
Steady-state (Apple M5, ms/op) forLoop vs Compute.filter: plain 53 vs 344,
FoR 56 vs 336, ALP 77 vs 464, Dict 573 vs 839; sum 67 vs 66. Two separate
levers fall out: (1) the filter mask-write path is ~6x the naive loop even
memory-bound (and on a plain column), so the per-element off-heap bitmap
write — not decode — is the cost; (2) Dict alone decodes ~10x slower than
the others (codes->values random scatter), the one place encoded-domain
compare-on-codes would pay. sum already matches the loop.
The Dict column surfaces as OffsetLongArray(DictLongArray) at scale (shared
flat layout across aligned chunks); the type assert unwraps the offset view.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>1 parent 2ac0e6c commit faef4a9
1 file changed
Lines changed: 235 additions & 127 deletions
0 commit comments