Commit 83bb368
perf(ilp): slice-by-8 CRC32C cuts SF append p50 ~4x
Replaces the byte-at-a-time CRC32C inner loop in the JNI implementation
with a slice-by-8 variant that consumes 8 input bytes per iteration via
eight parallel 256-entry table lookups whose results are XORed.
The seven additional tables (~7 KB of static read-only data) are derived
from the existing crc32c_table at build time using the standard
`table[k][i] = (table[k-1][i] >> 8) ^ table[0][table[k-1][i] & 0xFF]`
recurrence, which corresponds to "advance the input by one more zero
byte". They are emitted as static const initialisers — same rationale as
the original table: hard-coding sidesteps the C-memory-model pitfalls of
lazy initialisation on weakly-ordered platforms.
Measured on darwin-aarch64 with SegmentLogLatencyBenchmark at the typical
512-byte SF frame payload (warmup=50_000, measure=500_000, fsync=off):
before: min ~12_000 ns, p50 ~14_000 ns
after: min 2_625 ns, p50 3_625 ns
That collapses the per-append CRC cost from the dominant term (~85% of
p50) to a small fraction, which is what the SF store-and-forward layer
needs at high frame rates. The tail (p99/p99.9) is dominated by pwrite
syscalls and OS scheduling, not CRC, and is unchanged.
Correctness is covered by the existing Crc32cTest suite — in particular
testChainingPropertyOverManyRandomInputs (200 random buffers up to 2048
bytes, 5 random split points each) which exercises the slice-8 main loop
plus byte-at-a-time tail across every alignment offset and length class
the SF path can produce.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent af58d7d commit 83bb368
2 files changed
Lines changed: 302 additions & 12 deletions
0 commit comments