Skip to content

Commit fdb6a44

Browse files
dfa1claude
andcommitted
docs: add benchmark results (throughput + GC allocation)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 601b6ce commit fdb6a44

1 file changed

Lines changed: 112 additions & 0 deletions

File tree

docs/benchmarks.md

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# Benchmarks
2+
3+
JMH microbenchmarks comparing **zstd-java** against the common JVM zstd options,
4+
in both heap (`byte[]`) and zero-copy (`MemorySegment`) modes. Source lives in
5+
the [`benchmark/`](../benchmark) module; see its
6+
[README](../benchmark/README.md) to reproduce.
7+
8+
| Contestant | Binding | Modes |
9+
|------------|---------|-------|
10+
| **zstd-java** (this project) | FFM (no JNI) | `byte[]` and zero-copy `MemorySegment` |
11+
| **zstd-jni** (`com.github.luben`) | JNI | `byte[]` |
12+
| **aircompressor** (`io.airlift:aircompressor-v3`) | pure Java | caller-buffer `byte[]` |
13+
14+
## TL;DR
15+
16+
- **Throughput:** zstd-java's `MemorySegment` path is fastest at small/cache-resident
17+
payloads (~5-9% over `byte[]`, more over JNI). The edge **shrinks to nothing at
18+
64 MiB**, where everything is DRAM-bandwidth bound and converges. aircompressor
19+
(pure Java) trails the native bindings, ~2x on compress.
20+
- **Allocation:** this is the real `MemorySegment` win. The segment path is
21+
**allocation-free** (flat ~0 B/op at any size); the `byte[]`/JNI paths allocate
22+
~the output size **on every call**. At 64 MiB the `byte[]` path churns 67 MB/op
23+
through the young generation; JNI compress churns ~79 MB/op. Invisible in
24+
throughput, brutal under sustained load.
25+
26+
The headline isn't raw speed — it's **eliminating per-op heap allocation**, which
27+
matters most exactly where throughput converges (large, bandwidth-bound payloads
28+
under GC).
29+
30+
## Environment
31+
32+
- Apple M5, 32 GB. P-core L2 16 MiB (Apple Silicon: shared SLC, no classic L3).
33+
The 64 MiB payload is the cache-busting case.
34+
- JDK 25 (Azul). zstd-jni 1.5.7-11, aircompressor-v3 3.6, JMH 1.37.
35+
- Level 3 (zstd default). Payloads are deterministic, ~3x-compressible text.
36+
37+
**These are a quick, low-iteration run (1 fork, 2 warmup, 3 measurement) — a
38+
directional read, not publication-grade. The 64 MiB rows in particular have wide
39+
intervals (few ops per iteration). Rerun with JMH defaults on the target host
40+
before quoting.**
41+
42+
## Throughput (ops/ms, higher is better)
43+
44+
### Compress
45+
46+
| size | zstdJavaSegment | zstdJavaBytes | zstdJni | aircompressor |
47+
|------|----------------:|--------------:|--------:|--------------:|
48+
| 1 KiB | **287.7** | 278.2 | 238.3 | 148.3 |
49+
| 64 KiB | **14.34** | 13.60 | 13.46 | 8.00 |
50+
| 1 MiB | **0.927** | 0.889 | 0.906 | 0.562 |
51+
| 64 MiB | 0.014 | 0.014 | 0.013 | 0.007 |
52+
53+
### Decompress
54+
55+
| size | zstdJavaSegment | zstdJavaBytes | zstdJni | aircompressor |
56+
|------|----------------:|--------------:|--------:|--------------:|
57+
| 1 KiB | **590.7** | 560.1 | 460.6 | 477.0 |
58+
| 64 KiB | **32.3** | 29.7 | 26.6 | 26.8 |
59+
| 1 MiB | **2.15** | 2.02 | 1.82 | 1.74 |
60+
| 64 MiB | 0.029 | 0.029 | 0.028 | 0.016 |
61+
62+
At 64 MiB the segment/byte[]/jni columns are statistically indistinguishable —
63+
all bound by memory bandwidth, not API overhead.
64+
65+
## Allocation (`gc.alloc.rate.norm`, bytes/op, lower is better)
66+
67+
The decisive chart. Note the segment column is flat near zero while the others
68+
scale linearly with payload size.
69+
70+
### Compress
71+
72+
| size | zstdJavaSegment | zstdJavaBytes | zstdJni | aircompressor |
73+
|------|----------------:|--------------:|--------:|--------------:|
74+
| 1 KiB | **0.07** | 408 | 1,464 | 35,464 |
75+
| 64 KiB | **6.8** | 11,694 | 77,473 | 724,034 |
76+
| 1 MiB | **44** | 182,069 | 1,234,501 | 1,449,282 |
77+
| 64 MiB | **1,296** | 11,621,915 | 78,992,794 | 2,971,072 |
78+
79+
### Decompress
80+
81+
| size | zstdJavaSegment | zstdJavaBytes | zstdJni | aircompressor |
82+
|------|----------------:|--------------:|--------:|--------------:|
83+
| 1 KiB | **0.03** | 1,136 | 1,072 | 48 |
84+
| 64 KiB | **13** | 65,651 | 65,585 | 49 |
85+
| 1 MiB | **16** | 1,048,785 | 1,048,634 | 58 |
86+
| 64 MiB | **1,969** | 67,111,319 | 67,109,554 | 4,799 |
87+
88+
### Notes
89+
90+
- **Segment is allocation-free both directions** — the caller owns the off-heap
91+
source and destination, so nothing touches the heap.
92+
- **byte[]/JNI allocate ~the output size every call** (their APIs return a fresh
93+
array). At 64 MiB that is 11-67 MB/op.
94+
- **JNI compress allocates ~79 MB/op at 64 MiB** (~7x the others): its
95+
`compress(byte[], level)` allocates a worst-case `compressBound` buffer and then
96+
a trimmed copy — double buffering.
97+
- **aircompressor decompress is also ~0 B/op** here, because the benchmark reuses
98+
a caller-supplied destination array. A caller-buffer API kills decode allocation
99+
regardless of binding; the segment path additionally avoids the off-heap↔heap copy.
100+
aircompressor *compress* still allocates internal hash tables per call.
101+
102+
## Reproduce
103+
104+
```bash
105+
./mvnw -q -pl benchmark -am package -DskipTests
106+
107+
# throughput + allocation, all sizes
108+
java -jar benchmark/target/benchmarks.jar -prof gc
109+
110+
# single size
111+
java -jar benchmark/target/benchmarks.jar -prof gc -p size=67108864
112+
```

0 commit comments

Comments
 (0)