77
88** zstd-java** is a Java wrapper for [ Zstandard] ( https://github.com/facebook/zstd )
99built on the ** Foreign Function & Memory (FFM) API** — no JNI, no hand-written C.
10-
1110It targets ** JDK 25+** (for stable ` java.lang.foreign ` ) and leads with the
1211feature missing from most JVM zstd bindings: ** dictionary compression** , trained
1312straight from your own data.
@@ -16,34 +15,63 @@ straight from your own data.
1615> C header mapping, test generation, docs. Architecture, API design, and all
1716> decisions are human-driven.
1817
19- The native library is built from vendored zstd source via ** ` zig cc ` ** as a
20- drop-in C compiler. zstd is pure C with no build-system dependencies, so the
21- sources are compiled directly — no autotools, no CMake. Zig bundles clang and
22- libc for every target, enabling hermetic cross-compilation without a sysroot.
18+ ## Documentation
2319
24- ## Supported platforms
20+ The docs follow the [ Diátaxis] ( https://diataxis.fr ) framework — four kinds of
21+ documentation, each serving a different need:
2522
26- The library — ` io.github.dfa1:zstd-java ` — ships as a pure-Java module plus one
27- native artifact per platform:
23+ | | Purpose | Start here |
24+ | ---| ---| ---|
25+ | ** [ Tutorial] ( #tutorial-getting-started ) ** | Learning by doing | [ Getting started] ( #tutorial-getting-started ) |
26+ | ** [ How-to guides] ( #how-to-guides ) ** | Solving a specific task | [ Hot paths] ( #compress-on-a-hot-path ) , [ Dictionaries] ( #compress-many-small-payloads-with-a-dictionary ) , [ Zero-copy] ( #avoid-heap-copies-with-memorysegment ) , [ Self-built lib] ( #run-against-a-self-built-libzstd ) |
27+ | ** [ Reference] ( #reference ) ** | Looking up facts | [ Platforms] ( #supported-platforms ) , [ API surface] ( #api-surface ) , [ Symbol coverage] ( docs/supported.md ) , [ Build] ( #build-from-source ) |
28+ | ** [ Explanation] ( #explanation ) ** | Understanding the why | [ Why FFM + Zig] ( #why-ffm-and-zig ) , [ When zero-copy pays] ( docs/zero-copy.md ) , [ Benchmarks] ( docs/benchmarks.md ) |
2829
29- | OS | aarch64 | x86_64 |
30- | ---------| :-------:| :------:|
31- | macOS | ✅ | ✅ |
32- | Linux | ✅ | ✅ |
33- | Windows | ✅ | ✅ |
30+ ---
31+
32+ ## Tutorial: Getting started
33+
34+ This walks you from a clean checkout to your first compress/decompress round-trip.
35+
36+ ** 1. Clone with the zstd submodule and build.** You need JDK 25+, Maven, and
37+ [ Zig] ( https://ziglang.org/ ) on ` PATH ` (Zig is the C compiler for the native lib).
38+
39+ ``` bash
40+ git clone --recurse-submodules https://github.com/dfa1/zstd-java.git
41+ cd zstd-java
42+ mvn install
43+ ```
3444
35- ## Usage
45+ The build invokes ` scripts/build-zstd.sh ` , compiling ` libzstd ` from the vendored
46+ source — no autotools or CMake needed.
3647
37- ### One-shot
48+ ** 2. Write your first round-trip. **
3849
3950``` java
40- byte [] packed = Zstd . compress(" hello world" . getBytes());
41- byte [] restored = Zstd . decompress(packed); // size read from frame header
51+ import io.github.dfa1.zstd.Zstd ;
52+
53+ byte [] original = " hello world" . getBytes();
54+ byte [] packed = Zstd . compress(original);
55+ byte [] restored = Zstd . decompress(packed); // size read from the frame header
56+
57+ assert java.util. Arrays . equals(original, restored);
58+ ```
4259
43- byte [] hard = Zstd . compress(data, Zstd . maxCompressionLevel());
60+ ** 3. Run it with native access enabled.** The FFM API requires an explicit flag:
61+
62+ ``` bash
63+ java --enable-native-access=ALL-UNNAMED Demo.java
4464```
4565
46- ### Reusable contexts (hot paths)
66+ That's the whole loop. From here, pick a [ how-to guide] ( #how-to-guides ) for your
67+ actual task, or browse the [ reference] ( #reference ) .
68+
69+ ## How-to guides
70+
71+ Task-focused recipes. Each assumes you have the library on the classpath (see the
72+ [ tutorial] ( #tutorial-getting-started ) ).
73+
74+ ### Compress on a hot path
4775
4876Reuse a context to amortise native allocation across many calls:
4977
@@ -55,7 +83,10 @@ try (ZstdCompressCtx cctx = new ZstdCompressCtx().level(19);
5583}
5684```
5785
58- ### Dictionaries
86+ Pick the level explicitly with ` Zstd.maxCompressionLevel() ` /
87+ ` minCompressionLevel() ` when you need the extreme ends.
88+
89+ ### Compress many small payloads with a dictionary
5990
6091For many small, similar payloads (log lines, JSON records, protobufs), a
6192dictionary compresses each one far smaller than it could be alone. Train one on
@@ -86,7 +117,7 @@ try (ZstdCompressDict cdict = new ZstdCompressDict(dict, 19);
86117}
87118```
88119
89- ### Zero-copy ( ` MemorySegment ` )
120+ ### Avoid heap copies with ` MemorySegment `
90121
91122When your data is already off-heap — an ` mmap ` slice in, an arena buffer out —
92123use the ` MemorySegment ` overloads to skip the heap ` byte[] ` bounce entirely. FFM
@@ -103,39 +134,102 @@ try (Arena arena = Arena.ofConfined();
103134```
104135
105136There are matching ` compress(dst, src) ` / ` decompress(dst, src) ` overloads (plus
106- dictionary variants) returning the number of bytes written. See
107- [ docs/zero-copy.md ] ( docs/zero-copy.md ) for why and when this pays off .
137+ dictionary variants) returning the number of bytes written. For * why and when *
138+ this pays off, see the [ explanation ] ( docs/zero-copy.md ) .
108139
109- > Native access requires ` --enable-native-access=ALL-UNNAMED ` (or your module) on
110- > the JVM command line.
140+ ### Run against a self-built libzstd
111141
112- ## Building
113-
114- Requires JDK 25+, Maven, and [ Zig] ( https://ziglang.org/ ) on ` PATH ` .
142+ To use a ` libzstd ` you built yourself instead of the bundled one, point the
143+ loader at it:
115144
116145``` bash
117- git clone --recurse-submodules https://github.com/dfa1/zstd-java.git
118- cd zstd-java
119- mvn test
146+ java -Dzstd.lib.path=/path/to/libzstd.dylib --enable-native-access=ALL-UNNAMED ...
120147```
121148
122- The ` mvn ` build invokes ` scripts/build-zstd.sh ` , which compiles
123- ` libzstd.{dylib,so,dll} ` from the ` third_party/zstd ` submodule with ` zig cc ` . The script
124- cross-compiles any of the six targets from any host:
149+ Build any of the six targets from any host:
125150
126151``` bash
127152./scripts/build-zstd.sh < output-resources-dir> < classifier>
128153# classifier: osx-aarch64 | osx-x86_64 | linux-x86_64 | linux-aarch64
129154# | windows-x86_64 | windows-aarch64
130155```
131156
132- To run against a self-built library instead of the bundled one:
157+ ## Reference
158+
159+ ### Supported platforms
160+
161+ The library — ` io.github.dfa1:zstd-java ` — ships as a pure-Java module plus one
162+ native artifact per platform:
163+
164+ | OS | aarch64 | x86_64 |
165+ | ---------| :-------:| :------:|
166+ | macOS | ✅ | ✅ |
167+ | Linux | ✅ | ✅ |
168+ | Windows | ✅ | ✅ |
169+
170+ ### API surface
171+
172+ | Type | Role |
173+ | ---| ---|
174+ | ` Zstd ` | one-shot ` compress ` / ` decompress ` , level + version queries, ` compressBound ` , ` decompressedSize ` |
175+ | ` ZstdCompressCtx ` / ` ZstdDecompressCtx ` | reusable contexts; ` byte[] ` and ` MemorySegment ` overloads, dictionary variants |
176+ | ` ZstdDictionary ` | train (` ZDICT ` ), load, persist, query dict id |
177+ | ` ZstdCompressDict ` / ` ZstdDecompressDict ` | pre-digested dictionaries for hot paths |
178+ | ` ZstdFrame ` | frame inspection: header, sizes, dict id, skippable frames |
179+ | ` ZstdException ` / ` ZstdErrorCode ` | typed errors mapped from zstd's sentinels |
180+
181+ ### Symbol coverage
182+
183+ Which zstd C symbols are bound (and which deprecated ones are intentionally not),
184+ with a per-area breakdown and a comparison against zstd-jni:
185+ [ docs/supported.md] ( docs/supported.md ) .
186+
187+ ### Runtime requirement
188+
189+ Native access requires ` --enable-native-access=ALL-UNNAMED ` (or your module name)
190+ on the JVM command line.
191+
192+ ### Build from source
193+
194+ Requires JDK 25+, Maven, and [ Zig] ( https://ziglang.org/ ) on ` PATH ` .
133195
134196``` bash
135- -Dzstd.lib.path=/path/to/libzstd.dylib
197+ git clone --recurse-submodules https://github.com/dfa1/zstd-java.git
198+ cd zstd-java
199+ mvn test
136200```
137201
138- ## License
202+ ` scripts/build-zstd.sh ` compiles ` libzstd.{dylib,so,dll} ` from the
203+ ` third_party/zstd ` submodule (pinned to tag ` v1.5.7 ` ) with ` zig cc ` , cross-compiling
204+ any of the six targets from any host.
205+
206+ ### License
139207
140208[ BSD 3-Clause] ( LICENSE ) — the same primary license as zstd, which is bundled
141209under its BSD terms (zstd is dual BSD / GPLv2, © Meta Platforms, Inc.).
210+
211+ ## Explanation
212+
213+ ### Why FFM and Zig
214+
215+ The bindings use the ** Foreign Function & Memory API** rather than JNI: no
216+ hand-written C glue, no separate native compile step for the binding layer, and a
217+ direct path from Java to zstd's addresses — which is what makes the zero-copy
218+ ` MemorySegment ` API possible.
219+
220+ The native library itself is built from vendored zstd source via ** ` zig cc ` ** as
221+ a drop-in C compiler. zstd is pure C with no build-system dependencies, so the
222+ sources are compiled directly — no autotools, no CMake. Zig bundles clang and
223+ libc for every target, enabling hermetic cross-compilation without a sysroot:
224+ any host can build all six platform artifacts.
225+
226+ ### When zero-copy pays off
227+
228+ The ` MemorySegment ` fast path eliminates the heap ` byte[] ` bounce and the
229+ per-call allocation it implies. The reasoning, and the cases where it does and
230+ does not matter, is in [ docs/zero-copy.md] ( docs/zero-copy.md ) .
231+
232+ ### Benchmarks
233+
234+ Throughput and allocation versus zstd-jni (JNI) and aircompressor (pure Java),
235+ including an async-profiler breakdown: [ docs/benchmarks.md] ( docs/benchmarks.md ) .
0 commit comments