Skip to content

Commit 43097b1

Browse files
dfa1claude
andcommitted
docs: reorganize README by the Diátaxis framework
Split into tutorial (getting started), how-to guides (hot paths, dictionaries, zero-copy, self-built lib), reference (platforms, API surface, symbol coverage, build), and explanation (FFM+Zig, zero-copy, benchmarks), with a navigation map linking each quadrant. Content preserved, reframed by reader intent. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 70077c3 commit 43097b1

1 file changed

Lines changed: 131 additions & 37 deletions

File tree

README.md

Lines changed: 131 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@
77

88
**zstd-java** is a Java wrapper for [Zstandard](https://github.com/facebook/zstd)
99
built on the **Foreign Function & Memory (FFM) API** — no JNI, no hand-written C.
10-
1110
It targets **JDK 25+** (for stable `java.lang.foreign`) and leads with the
1211
feature missing from most JVM zstd bindings: **dictionary compression**, trained
1312
straight from your own data.
@@ -16,34 +15,63 @@ straight from your own data.
1615
> C header mapping, test generation, docs. Architecture, API design, and all
1716
> decisions are human-driven.
1817
19-
The native library is built from vendored zstd source via **`zig cc`** as a
20-
drop-in C compiler. zstd is pure C with no build-system dependencies, so the
21-
sources are compiled directly — no autotools, no CMake. Zig bundles clang and
22-
libc for every target, enabling hermetic cross-compilation without a sysroot.
18+
## Documentation
2319

24-
## Supported platforms
20+
The docs follow the [Diátaxis](https://diataxis.fr) framework — four kinds of
21+
documentation, each serving a different need:
2522

26-
The library — `io.github.dfa1:zstd-java` — ships as a pure-Java module plus one
27-
native artifact per platform:
23+
| | Purpose | Start here |
24+
|---|---|---|
25+
| **[Tutorial](#tutorial-getting-started)** | Learning by doing | [Getting started](#tutorial-getting-started) |
26+
| **[How-to guides](#how-to-guides)** | Solving a specific task | [Hot paths](#compress-on-a-hot-path), [Dictionaries](#compress-many-small-payloads-with-a-dictionary), [Zero-copy](#avoid-heap-copies-with-memorysegment), [Self-built lib](#run-against-a-self-built-libzstd) |
27+
| **[Reference](#reference)** | Looking up facts | [Platforms](#supported-platforms), [API surface](#api-surface), [Symbol coverage](docs/supported.md), [Build](#build-from-source) |
28+
| **[Explanation](#explanation)** | Understanding the why | [Why FFM + Zig](#why-ffm-and-zig), [When zero-copy pays](docs/zero-copy.md), [Benchmarks](docs/benchmarks.md) |
2829

29-
| OS | aarch64 | x86_64 |
30-
|---------|:-------:|:------:|
31-
| macOS |||
32-
| Linux |||
33-
| Windows |||
30+
---
31+
32+
## Tutorial: Getting started
33+
34+
This walks you from a clean checkout to your first compress/decompress round-trip.
35+
36+
**1. Clone with the zstd submodule and build.** You need JDK 25+, Maven, and
37+
[Zig](https://ziglang.org/) on `PATH` (Zig is the C compiler for the native lib).
38+
39+
```bash
40+
git clone --recurse-submodules https://github.com/dfa1/zstd-java.git
41+
cd zstd-java
42+
mvn install
43+
```
3444

35-
## Usage
45+
The build invokes `scripts/build-zstd.sh`, compiling `libzstd` from the vendored
46+
source — no autotools or CMake needed.
3647

37-
### One-shot
48+
**2. Write your first round-trip.**
3849

3950
```java
40-
byte[] packed = Zstd.compress("hello world".getBytes());
41-
byte[] restored = Zstd.decompress(packed); // size read from frame header
51+
import io.github.dfa1.zstd.Zstd;
52+
53+
byte[] original = "hello world".getBytes();
54+
byte[] packed = Zstd.compress(original);
55+
byte[] restored = Zstd.decompress(packed); // size read from the frame header
56+
57+
assert java.util.Arrays.equals(original, restored);
58+
```
4259

43-
byte[] hard = Zstd.compress(data, Zstd.maxCompressionLevel());
60+
**3. Run it with native access enabled.** The FFM API requires an explicit flag:
61+
62+
```bash
63+
java --enable-native-access=ALL-UNNAMED Demo.java
4464
```
4565

46-
### Reusable contexts (hot paths)
66+
That's the whole loop. From here, pick a [how-to guide](#how-to-guides) for your
67+
actual task, or browse the [reference](#reference).
68+
69+
## How-to guides
70+
71+
Task-focused recipes. Each assumes you have the library on the classpath (see the
72+
[tutorial](#tutorial-getting-started)).
73+
74+
### Compress on a hot path
4775

4876
Reuse a context to amortise native allocation across many calls:
4977

@@ -55,7 +83,10 @@ try (ZstdCompressCtx cctx = new ZstdCompressCtx().level(19);
5583
}
5684
```
5785

58-
### Dictionaries
86+
Pick the level explicitly with `Zstd.maxCompressionLevel()` /
87+
`minCompressionLevel()` when you need the extreme ends.
88+
89+
### Compress many small payloads with a dictionary
5990

6091
For many small, similar payloads (log lines, JSON records, protobufs), a
6192
dictionary compresses each one far smaller than it could be alone. Train one on
@@ -86,7 +117,7 @@ try (ZstdCompressDict cdict = new ZstdCompressDict(dict, 19);
86117
}
87118
```
88119

89-
### Zero-copy (`MemorySegment`)
120+
### Avoid heap copies with `MemorySegment`
90121

91122
When your data is already off-heap — an `mmap` slice in, an arena buffer out —
92123
use the `MemorySegment` overloads to skip the heap `byte[]` bounce entirely. FFM
@@ -103,39 +134,102 @@ try (Arena arena = Arena.ofConfined();
103134
```
104135

105136
There are matching `compress(dst, src)` / `decompress(dst, src)` overloads (plus
106-
dictionary variants) returning the number of bytes written. See
107-
[docs/zero-copy.md](docs/zero-copy.md) for why and when this pays off.
137+
dictionary variants) returning the number of bytes written. For *why and when*
138+
this pays off, see the [explanation](docs/zero-copy.md).
108139

109-
> Native access requires `--enable-native-access=ALL-UNNAMED` (or your module) on
110-
> the JVM command line.
140+
### Run against a self-built libzstd
111141

112-
## Building
113-
114-
Requires JDK 25+, Maven, and [Zig](https://ziglang.org/) on `PATH`.
142+
To use a `libzstd` you built yourself instead of the bundled one, point the
143+
loader at it:
115144

116145
```bash
117-
git clone --recurse-submodules https://github.com/dfa1/zstd-java.git
118-
cd zstd-java
119-
mvn test
146+
java -Dzstd.lib.path=/path/to/libzstd.dylib --enable-native-access=ALL-UNNAMED ...
120147
```
121148

122-
The `mvn` build invokes `scripts/build-zstd.sh`, which compiles
123-
`libzstd.{dylib,so,dll}` from the `third_party/zstd` submodule with `zig cc`. The script
124-
cross-compiles any of the six targets from any host:
149+
Build any of the six targets from any host:
125150

126151
```bash
127152
./scripts/build-zstd.sh <output-resources-dir> <classifier>
128153
# classifier: osx-aarch64 | osx-x86_64 | linux-x86_64 | linux-aarch64
129154
# | windows-x86_64 | windows-aarch64
130155
```
131156

132-
To run against a self-built library instead of the bundled one:
157+
## Reference
158+
159+
### Supported platforms
160+
161+
The library — `io.github.dfa1:zstd-java` — ships as a pure-Java module plus one
162+
native artifact per platform:
163+
164+
| OS | aarch64 | x86_64 |
165+
|---------|:-------:|:------:|
166+
| macOS |||
167+
| Linux |||
168+
| Windows |||
169+
170+
### API surface
171+
172+
| Type | Role |
173+
|---|---|
174+
| `Zstd` | one-shot `compress` / `decompress`, level + version queries, `compressBound`, `decompressedSize` |
175+
| `ZstdCompressCtx` / `ZstdDecompressCtx` | reusable contexts; `byte[]` and `MemorySegment` overloads, dictionary variants |
176+
| `ZstdDictionary` | train (`ZDICT`), load, persist, query dict id |
177+
| `ZstdCompressDict` / `ZstdDecompressDict` | pre-digested dictionaries for hot paths |
178+
| `ZstdFrame` | frame inspection: header, sizes, dict id, skippable frames |
179+
| `ZstdException` / `ZstdErrorCode` | typed errors mapped from zstd's sentinels |
180+
181+
### Symbol coverage
182+
183+
Which zstd C symbols are bound (and which deprecated ones are intentionally not),
184+
with a per-area breakdown and a comparison against zstd-jni:
185+
[docs/supported.md](docs/supported.md).
186+
187+
### Runtime requirement
188+
189+
Native access requires `--enable-native-access=ALL-UNNAMED` (or your module name)
190+
on the JVM command line.
191+
192+
### Build from source
193+
194+
Requires JDK 25+, Maven, and [Zig](https://ziglang.org/) on `PATH`.
133195

134196
```bash
135-
-Dzstd.lib.path=/path/to/libzstd.dylib
197+
git clone --recurse-submodules https://github.com/dfa1/zstd-java.git
198+
cd zstd-java
199+
mvn test
136200
```
137201

138-
## License
202+
`scripts/build-zstd.sh` compiles `libzstd.{dylib,so,dll}` from the
203+
`third_party/zstd` submodule (pinned to tag `v1.5.7`) with `zig cc`, cross-compiling
204+
any of the six targets from any host.
205+
206+
### License
139207

140208
[BSD 3-Clause](LICENSE) — the same primary license as zstd, which is bundled
141209
under its BSD terms (zstd is dual BSD / GPLv2, © Meta Platforms, Inc.).
210+
211+
## Explanation
212+
213+
### Why FFM and Zig
214+
215+
The bindings use the **Foreign Function & Memory API** rather than JNI: no
216+
hand-written C glue, no separate native compile step for the binding layer, and a
217+
direct path from Java to zstd's addresses — which is what makes the zero-copy
218+
`MemorySegment` API possible.
219+
220+
The native library itself is built from vendored zstd source via **`zig cc`** as
221+
a drop-in C compiler. zstd is pure C with no build-system dependencies, so the
222+
sources are compiled directly — no autotools, no CMake. Zig bundles clang and
223+
libc for every target, enabling hermetic cross-compilation without a sysroot:
224+
any host can build all six platform artifacts.
225+
226+
### When zero-copy pays off
227+
228+
The `MemorySegment` fast path eliminates the heap `byte[]` bounce and the
229+
per-call allocation it implies. The reasoning, and the cases where it does and
230+
does not matter, is in [docs/zero-copy.md](docs/zero-copy.md).
231+
232+
### Benchmarks
233+
234+
Throughput and allocation versus zstd-jni (JNI) and aircompressor (pure Java),
235+
including an async-profiler breakdown: [docs/benchmarks.md](docs/benchmarks.md).

0 commit comments

Comments
 (0)