Skip to content

Commit b4d5a40

Browse files
dfa1claude
andauthored
docs: lead the README with examples + performance (#48)
Restructure the landing page for first-time visitors: - Quickstart: byte[] round-trip, dictionary train+compress, zero-copy MemorySegment (signatures verified against the 0.6 API) - Performance: surface the publication-grade golden-corpus numbers from docs/benchmarks.md (best-vs-best vs zstd-jni's own zero-copy path: +9-23% throughput on small payloads, allocation tie; allocation-free vs the convenient byte[] APIs) — honest ties included - Sharpen the pitch (dictionary + zero-copy, the two real differentiators) and note the JDK 25 framing (first LTS with stable FFM) - Move Install above Documentation; link the release smoke matrix as per-arch proof Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent eb8de7d commit b4d5a40

1 file changed

Lines changed: 93 additions & 15 deletions

File tree

README.md

Lines changed: 93 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -10,27 +10,90 @@
1010

1111
**zstd-java** is a Java wrapper for [Zstandard](https://github.com/facebook/zstd)
1212
built on the **Foreign Function & Memory (FFM) API** — no JNI, no `sun.misc.Unsafe`.
13-
It targets **JDK 25+** (for stable `java.lang.foreign`) and leads with the
14-
feature missing from most JVM zstd bindings: **dictionary compression**, trained
15-
straight from your own data.
13+
It targets **JDK 25+** (the first LTS with stable `java.lang.foreign`) and leads
14+
with the two features most JVM zstd bindings lack:
15+
16+
- **Dictionary compression**, trained straight from your own data — the big win on
17+
small, repetitive records (logs, market-data ticks, JSON/Avro rows, FIX messages).
18+
- A **zero-copy `MemorySegment` API** — compress/decompress off-heap buffers (an
19+
mmap'd slice in, an arena buffer out) with no heap copy and no per-call allocation.
1620

1721
> **AI-assisted development:** This project uses Claude Code for implementation —
1822
> C header mapping, test generation, docs. Architecture, API design, and all
1923
> decisions are human-driven.
2024
21-
## Documentation
25+
## Quickstart
2226

23-
The docs follow the [Diátaxis](https://diataxis.fr) framework:
27+
One-shot round-trip with `byte[]` — the convenient path:
2428

25-
| | Purpose | Start here |
26-
|---|---|---|
27-
| **[Tutorial](docs/tutorial.md)** | Learning by doing | Clean checkout → first round-trip |
28-
| **[How-to guides](docs/how-to.md)** | Solving a specific task | Hot paths, dictionaries, zero-copy, self-built lib |
29-
| **[Reference](docs/reference.md)** | Looking up facts | Platforms, API surface, symbol coverage, build |
30-
| **[Explanation](docs/explanation.md)** | Understanding the why | Why FFM + Zig, when zero-copy pays, benchmarks |
29+
```java
30+
import io.github.dfa1.zstd.Zstd;
3131

32-
Architecture decisions are recorded as [ADRs](adr/ADR.md) (MADR 3.0) — the
33-
foundational choices and their trade-offs, one file per decision.
32+
byte[] data = ...;
33+
byte[] frame = Zstd.compress(data); // or Zstd.compress(data, level)
34+
byte[] back = Zstd.decompress(frame); // size read from the frame header
35+
```
36+
37+
**Dictionary** — train on a sample of your records, then compress each one against
38+
the dictionary (huge ratio gains on small, similar messages):
39+
40+
```java
41+
import io.github.dfa1.zstd.*;
42+
import java.util.List;
43+
44+
List<byte[]> samples = ...; // representative records
45+
ZstdDictionary dict = ZstdDictionary.train(samples, 8 * 1024);
46+
47+
byte[] message = ...;
48+
try (ZstdCompressCtx cctx = new ZstdCompressCtx();
49+
ZstdDecompressCtx dctx = new ZstdDecompressCtx()) {
50+
byte[] frame = cctx.compress(message, dict);
51+
byte[] back = dctx.decompress(frame, message.length, dict);
52+
}
53+
```
54+
55+
**Zero-copy** — off-heap in, off-heap out, no `byte[]`, no per-call allocation:
56+
57+
```java
58+
import io.github.dfa1.zstd.*;
59+
import java.lang.foreign.*;
60+
61+
try (Arena arena = Arena.ofConfined();
62+
ZstdCompressCtx cctx = new ZstdCompressCtx();
63+
ZstdDecompressCtx dctx = new ZstdDecompressCtx()) {
64+
65+
MemorySegment src = ...; // e.g. an mmap'd file slice
66+
MemorySegment frame = cctx.compress(arena, src); // off-heap → off-heap
67+
MemorySegment restored = dctx.decompress(arena, frame);
68+
}
69+
```
70+
71+
Run with `--enable-native-access=ALL-UNNAMED`. Full walkthrough in the
72+
[tutorial](docs/tutorial.md); hot-path and dictionary recipes in the
73+
[how-to guides](docs/how-to.md).
74+
75+
## Performance
76+
77+
Microbenchmarks against the common JVM zstd options (JMH; Apple M5, JDK 25, all
78+
linking the same zstd 1.5.7). Full methodology and tables in
79+
[docs/benchmarks.md](docs/benchmarks.md) — including the honest ties.
80+
81+
**Best vs best** — our zero-copy `MemorySegment` path vs **zstd-jni's own**
82+
zero-copy direct-`ByteBuffer` path (golden-corpus fixtures, publication-grade run):
83+
84+
| operation (payload) | zstd-java `MemorySegment` | zstd-jni `ByteBuffer` | edge |
85+
|---|---:|---:|---:|
86+
| compress `http` (1.2 KiB) | **353.6** | 322.1 | +9.8% |
87+
| decompress `http` | **922.7** | 750.8 | +22.9% |
88+
| decompress `large-literal` (200 KiB) | 56.1 | 55.6 | tie |
89+
90+
*(throughput, ops/ms, higher is better; allocation is **~0 B/op on both** — both genuinely zero-copy)*
91+
92+
The edge is FFM's lower per-call overhead — **largest on small payloads**,
93+
converging to a tie when codec/bandwidth dominates. Against the *convenient*
94+
`byte[]` / JNI APIs (which allocate the output every call), the segment path is
95+
additionally **allocation-free**: flat ~0 B/op at any size vs MB/op that scales
96+
with the payload — no GC pressure on the hot path.
3497

3598
## Install
3699

@@ -79,11 +142,26 @@ plus only the `zstd-native-<classifier>` you target.
79142
```
80143

81144
Classifiers: `osx-aarch64`, `osx-x86_64`, `linux-x86_64`, `linux-aarch64`,
82-
`windows-x86_64`, `windows-aarch64`. Gradle and more detail in the
83-
[tutorial](docs/tutorial.md). Requires JDK 25+ and
145+
`windows-x86_64`, `windows-aarch64` — each verified on real hardware by the
146+
[release smoke matrix](.github/workflows/release-smoke.yml). Gradle and more
147+
detail in the [tutorial](docs/tutorial.md). Requires JDK 25+ and
84148
`--enable-native-access=ALL-UNNAMED` at runtime. Building from source is for
85149
contributors — see the [reference](docs/reference.md).
86150

151+
## Documentation
152+
153+
The docs follow the [Diátaxis](https://diataxis.fr) framework:
154+
155+
| | Purpose | Start here |
156+
|---|---|---|
157+
| **[Tutorial](docs/tutorial.md)** | Learning by doing | Clean checkout → first round-trip |
158+
| **[How-to guides](docs/how-to.md)** | Solving a specific task | Hot paths, dictionaries, zero-copy, self-built lib |
159+
| **[Reference](docs/reference.md)** | Looking up facts | Platforms, API surface, symbol coverage, build |
160+
| **[Explanation](docs/explanation.md)** | Understanding the why | Why FFM + Zig, when zero-copy pays, benchmarks |
161+
162+
Architecture decisions are recorded as [ADRs](adr/ADR.md) (MADR 3.0) — the
163+
foundational choices and their trade-offs, one file per decision.
164+
87165
## License
88166

89167
[BSD 3-Clause](LICENSE) — the same primary license as zstd, which is bundled

0 commit comments

Comments
 (0)