Skip to content

Commit 367556e

Browse files
dfa1claude
andauthored
feat: add digested-dictionary factories; clarify dictionary lifetime in docs (#33)
Adds ZstdDictionary.compressDict(int) / compressDict() / decompressDict() so callers can write dict.compressDict(19) instead of new ZstdCompressDict(dict, 19). The factory names read fluently and signal that the result is AutoCloseable and must be closed. Updates docs/how-to.md to use the factories and to spell out the lifetime rule that the leaky idiom missed: refDictionary only *borrows* a digested dictionary, so it must be closed separately; a single context should use loadDictionary, which the context digests, owns, and frees — a stray ZstdCompressDict that is never closed is a native-memory leak. Deliberately did not add an owning refDictionary(ZstdDictionary, int) overload: loadDictionary already provides context-owned, leak-free dictionary handling for the single-context case, so the overload would only duplicate it. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent c71387e commit 367556e

4 files changed

Lines changed: 102 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,11 @@ git tags, which trigger publication to Maven Central.
2727
- `ZstdFrame.headerSize(byte[])` / `ZstdFrame.headerSize(MemorySegment)` — the size
2828
of a frame's header computed from just its leading bytes (as few as 5), without a
2929
full parse. Binds `ZSTD_frameHeaderSize`.
30+
- `ZstdDictionary.compressDict(int)` / `compressDict()` / `decompressDict()`
31+
factories for digested dictionaries, e.g. `dict.compressDict(19)` instead of
32+
`new ZstdCompressDict(dict, 19)`. They signal that the result is `AutoCloseable`
33+
and are for sharing one digest across contexts via `refDictionary`; a single
34+
context should prefer the context-owned `loadDictionary`.
3035

3136
### Changed
3237
- Every dictionary-id accessor now returns `ZstdDictionaryId` instead of `int`:

docs/how-to.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ by reference — no per-call digesting, no copy. It pairs with `reset` for a
6262
pooled, recycled context:
6363

6464
```java
65-
try (ZstdCompressDict cdict = new ZstdCompressDict(dict, 19)) {
65+
try (ZstdCompressDict cdict = dict.compressDict(19)) {
6666
// one cctx per pooled worker, all sharing the one digested dictionary
6767
try (ZstdCompressCtx cctx = new ZstdCompressCtx()) {
6868
cctx.refDictionary(cdict); // borrowed; cdict must outlive cctx
@@ -73,6 +73,13 @@ try (ZstdCompressDict cdict = new ZstdCompressDict(dict, 19)) {
7373
}
7474
```
7575

76+
`refDictionary` only borrows: the digested `cdict` is *not* tied to the context's
77+
lifetime, so it must be closed separately (hence its own try-with-resources). That
78+
is the price of sharing one digest across many contexts. If you have just **one**
79+
context, don't build a `ZstdCompressDict` at all — `loadDictionary` above digests
80+
into the context and frees it for you, and a stray, never-closed
81+
`ZstdCompressDict` is a native-memory leak.
82+
7683
A loaded or referenced dictionary stays until replaced, cleared with `null`, or
7784
dropped by a parameter `reset`. `ZstdDecompressCtx` mirrors all of this.
7885

@@ -98,8 +105,8 @@ ZstdDictionary reloaded = ZstdDictionary.of(persisted);
98105
On a hot path, digest the dictionary once to skip per-call setup:
99106

100107
```java
101-
try (ZstdCompressDict cdict = new ZstdCompressDict(dict, 19);
102-
ZstdDecompressDict ddict = new ZstdDecompressDict(dict);
108+
try (ZstdCompressDict cdict = dict.compressDict(19);
109+
ZstdDecompressDict ddict = dict.decompressDict();
103110
ZstdCompressCtx cctx = new ZstdCompressCtx();
104111
ZstdDecompressCtx dctx = new ZstdDecompressCtx()) {
105112
byte[] packed = cctx.compress(record, cdict);

zstd/src/main/java/io/github/dfa1/zstd/ZstdDictionary.java

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -297,6 +297,41 @@ public int size() {
297297
return bytes.length;
298298
}
299299

300+
/// Digests this dictionary once for compression at `level`, ready to share by
301+
/// reference across contexts with [ZstdCompressCtx#refDictionary(ZstdCompressDict)].
302+
///
303+
/// The returned dictionary owns native memory — close it when done (it is
304+
/// [AutoCloseable]). For a single context, prefer
305+
/// [ZstdCompressCtx#loadDictionary(ZstdDictionary)], which the context digests,
306+
/// owns, and frees for you.
307+
///
308+
/// @param level the compression level to fix for the digested dictionary
309+
/// @return a digested compression dictionary the caller must close
310+
public ZstdCompressDict compressDict(int level) {
311+
return new ZstdCompressDict(this, level);
312+
}
313+
314+
/// Digests this dictionary for compression at the library default level.
315+
/// Otherwise identical to [#compressDict(int)].
316+
///
317+
/// @return a digested compression dictionary the caller must close
318+
public ZstdCompressDict compressDict() {
319+
return new ZstdCompressDict(this);
320+
}
321+
322+
/// Digests this dictionary once for decompression, ready to share by reference
323+
/// across contexts with [ZstdDecompressCtx#refDictionary(ZstdDecompressDict)].
324+
///
325+
/// The returned dictionary owns native memory — close it when done (it is
326+
/// [AutoCloseable]). For a single context, prefer
327+
/// [ZstdDecompressCtx#loadDictionary(ZstdDictionary)], which the context owns
328+
/// and frees for you.
329+
///
330+
/// @return a digested decompression dictionary the caller must close
331+
public ZstdDecompressDict decompressDict() {
332+
return new ZstdDecompressDict(this);
333+
}
334+
300335
/// Internal: direct view of the bytes for native calls. Not exposed.
301336
byte[] raw() {
302337
return bytes;

zstd/src/test/java/io/github/dfa1/zstd/ZstdDictionaryTest.java

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,6 +266,58 @@ void interoperatesWithRawPath() {
266266
}
267267
}
268268

269+
@Nested
270+
class Factories {
271+
272+
@Test
273+
void compressDictFixesTheRequestedLevel() {
274+
// When a digested compress dictionary is built via the factory
275+
try (ZstdCompressDict cdict = sut.compressDict(19)) {
276+
// Then it carries that level and the dictionary's id
277+
assertThat(cdict.level()).isEqualTo(19);
278+
assertThat(cdict.id()).isEqualTo(sut.id());
279+
}
280+
}
281+
282+
@Test
283+
void compressDictDefaultsToTheLibraryLevel() {
284+
// When built without a level
285+
try (ZstdCompressDict cdict = sut.compressDict()) {
286+
// Then it uses the library default
287+
assertThat(cdict.level()).isEqualTo(Zstd.defaultCompressionLevel());
288+
}
289+
}
290+
291+
@Test
292+
void decompressDictReportsTheDictionaryId() {
293+
// When a digested decompress dictionary is built via the factory
294+
try (ZstdDecompressDict ddict = sut.decompressDict()) {
295+
// Then it carries the dictionary's id
296+
assertThat(ddict.id()).isEqualTo(sut.id());
297+
}
298+
}
299+
300+
@Test
301+
void factoryDictionariesRoundTrip() {
302+
// Given factory-built digested dictionaries
303+
byte[] record = samples.get(123);
304+
305+
byte[] restored;
306+
try (ZstdCompressCtx cctx = new ZstdCompressCtx();
307+
ZstdDecompressCtx dctx = new ZstdDecompressCtx();
308+
ZstdCompressDict cdict = sut.compressDict(19);
309+
ZstdDecompressDict ddict = sut.decompressDict()) {
310+
311+
// When round-tripped through them
312+
byte[] frame = cctx.compress(record, cdict);
313+
restored = dctx.decompress(frame, record.length, ddict);
314+
}
315+
316+
// Then the record is recovered
317+
assertThat(restored).isEqualTo(record);
318+
}
319+
}
320+
269321
@Nested
270322
class Serialisation {
271323

0 commit comments

Comments
 (0)