@@ -8,39 +8,87 @@ the [`benchmark/`](../benchmark) module; see its
88| Contestant | Binding | Modes |
99| ------------| ---------| -------|
1010| ** zstd-java** (this project) | FFM (no JNI) | ` byte[] ` and zero-copy ` MemorySegment ` |
11- | ** zstd-jni** (` com.github.luben ` ) | JNI | ` byte[] ` |
11+ | ** zstd-jni** (` com.github.luben ` ) | JNI | ` byte[] ` ** and ** zero-copy direct ` ByteBuffer ` |
1212| ** aircompressor** (` io.airlift:aircompressor-v3 ` ) | pure Java | caller-buffer ` byte[] ` |
1313
1414## TL;DR
1515
16- - ** Throughput:** zstd-java's ` MemorySegment ` path is fastest at small/cache-resident
17- payloads (~ 5-9% over ` byte[] ` , more over JNI). The edge ** shrinks to nothing at
18- 64 MiB** , where everything is DRAM-bandwidth bound and converges. aircompressor
19- (pure Java) trails the native bindings, ~ 2x on compress.
20- - ** Allocation:** this is the real ` MemorySegment ` win. The segment path is
21- ** allocation-free** (flat ~ 0 B/op at any size); the ` byte[] ` /JNI paths allocate
22- ~ the output size ** on every call** . At 64 MiB the ` byte[] ` path churns 67 MB/op
23- through the young generation; JNI compress churns ~ 79 MB/op. Invisible in
24- throughput, brutal under sustained load.
25-
26- The headline isn't raw speed — it's ** eliminating per-op heap allocation** , which
27- matters most exactly where throughput converges (large, bandwidth-bound payloads
28- under GC).
16+ - ** Best vs best** (our ` MemorySegment ` vs zstd-jni's own zero-copy direct
17+ ` ByteBuffer ` , same zstd 1.5.7 both sides): ** allocation is a tie** — both are
18+ ~ 0 B/op. The throughput edge is ** per-call overhead** : clearest on small,
19+ call-overhead-dominated payloads (+10–23%) and ** converging to a tie** when
20+ compute or bandwidth dominates (large decompress, 64 MiB). This is the honest
21+ FFM-vs-JNI shape — biggest where the payload is smallest. See
22+ [ Golden corpus: best vs best] ( #golden-corpus-best-vs-best ) .
23+ - ** vs the allocating ` byte[] ` APIs:** the ` MemorySegment ` path is
24+ ** allocation-free** (flat ~ 0 B/op at any size) while ` byte[] ` / JNI-` byte[] `
25+ allocate ~ the output size ** every call** (67–79 MB/op at 64 MiB). Real, but it
26+ compares our zero-copy path against their * heap* API — not their zero-copy one.
27+ The allocation win is over the convenient API, not over JNI per se.
28+
29+ The honest headline: against zstd-jni's * best* path we ** match on allocation and
30+ lead modestly on call overhead** ; against the convenient ` byte[] ` APIs the
31+ zero-copy path additionally eliminates per-op heap allocation.
2932
3033## Environment
3134
3235- Apple M5, 32 GB. P-core L2 16 MiB (Apple Silicon: shared SLC, no classic L3).
3336 The 64 MiB payload is the cache-busting case.
34- - JDK 25 (Azul). zstd-jni 1.5.7-11, aircompressor-v3 3.6, JMH 1.37.
35- - Level 3 (zstd default). Payloads are deterministic, ~ 3x-compressible text.
36-
37- ** These are a quick, low-iteration run (1 fork, 2 warmup, 3 measurement) — a
38- directional read, not publication-grade. The 64 MiB rows in particular have wide
39- intervals (few ops per iteration). Rerun with JMH defaults on the target host
40- before quoting.**
37+ - JDK 25 (Azul). zstd-jni 1.5.7-11 (bundles zstd 1.5.7, matching our build),
38+ aircompressor-v3 3.6, JMH 1.37.
39+ - Level 3 (zstd default).
40+ - ** Golden corpus** run: 3 forks × 3 warmup × 5 measurement, ` -prof gc ` , error
41+ bars are 99.9% CIs.
42+ - ** Synthetic** tables below: deterministic ~ 3x-compressible text, and a quick,
43+ low-iteration run (1 fork, 2 warmup, 3 measurement) — directional, not
44+ publication-grade; the 64 MiB rows have wide intervals. Rerun with JMH defaults
45+ before quoting those.
46+
47+ ## Golden corpus: best vs best
48+
49+ The fairest comparison: ** our best zero-copy path against zstd-jni's best
50+ zero-copy path** , both reusing a context and off-heap buffers, neither allocating
51+ per call — our ` MemorySegment ` (` compressJavaSegment ` / ` decompressJavaSegment ` )
52+ vs zstd-jni's direct-` ByteBuffer ` API (` compressJniByteBuffer ` /
53+ ` decompressJniByteBuffer ` ). Inputs are real fixtures from zstd's own
54+ [ golden corpus] ( ../third_party/zstd/tests/golden-compression ) , not synthetic
55+ text, so the small/structured cases exercise per-call boundary overhead — exactly
56+ where FFM-vs-JNI differs. Both sides link ** the same zstd 1.5.7** , so any gap is
57+ binding overhead, not codec version.
58+
59+ This run is publication-grade for the cut shown (3 forks × 3 warmup × 5
60+ measurement, ` -prof gc ` ), on the environment below.
61+
62+ ### Throughput (ops/ms, higher is better)
63+
64+ | file (size) | JavaSegment | JniByteBuffer | edge |
65+ | -------------| ------------:| --------------:| -----:|
66+ | compress ` http ` (1.2 KiB) | ** 353.6** ±3.0 | 322.1 ±22.9 | +9.8% |
67+ | compress ` large-literal ` (200 KiB) | ** 46.1** ±1.4 | 42.2 ±0.3 | +9.4% |
68+ | decompress ` http ` | ** 922.7** ±5.9 | 750.8 ±0.9 | +22.9% |
69+ | decompress ` large-literal ` | 56.1 ±0.7 | 55.6 ±0.4 | +0.9% (tie) |
70+
71+ ### Allocation (` gc.alloc.rate.norm ` , B/op)
72+
73+ | | JavaSegment | JniByteBuffer |
74+ | -| ------------:| --------------:|
75+ | every case | ~ 0.00 | ~ 0.00 |
76+
77+ ** Reading it:** we lead ~ +9–10% on compress and +23% on small decompress (the
78+ call-overhead-dominated cases), tie on large decompress (bandwidth-bound), and
79+ ** match exactly on allocation** — both genuinely zero-copy. The earlier
80+ "allocation-free vs JNI" claim only held against JNI's ` byte[] ` API; against its
81+ zero-copy path the allocation advantage is gone, and the speed edge is the
82+ expected FFM call-overhead margin, largest at the smallest payloads.
4183
4284## Throughput (ops/ms, higher is better)
4385
86+ > The tables below use the original ** synthetic** payloads and compare against
87+ > zstd-jni's * allocating* ` byte[] ` API (` zstdJni ` ), not its zero-copy path. They
88+ > show the ` MemorySegment ` -vs-` byte[] ` allocation story; for the fair
89+ > zero-copy-vs-zero-copy comparison see
90+ > [ Golden corpus: best vs best] ( #golden-corpus-best-vs-best ) above.
91+
4492### Compress
4593
4694| size | zstdJavaSegment | zstdJavaBytes | zstdJni | aircompressor |
@@ -137,8 +185,13 @@ removes entirely, and it dominates under sustained, allocation-sensitive load.
137185``` bash
138186./mvnw -q -pl benchmark -am package -DskipTests
139187
140- # throughput + allocation, all sizes
141- java -jar benchmark/target/benchmarks.jar -prof gc
188+ # golden corpus, best vs best (our MemorySegment vs zstd-jni direct ByteBuffer)
189+ java --enable-native-access=ALL-UNNAMED -jar benchmark/target/benchmarks.jar \
190+ " GoldenCorpusBenchmark.*(Segment|JniByteBuffer)" \
191+ -p file=http,large-literal-and-match-lengths -f 3 -wi 3 -i 5 -prof gc
192+
193+ # synthetic throughput + allocation, all sizes
194+ java -jar benchmark/target/benchmarks.jar CompressBenchmark DecompressBenchmark -prof gc
142195
143196# single size
144197java -jar benchmark/target/benchmarks.jar -prof gc -p size=67108864
0 commit comments