Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ git tags, which trigger publication to Maven Central.
- `ZstdFrame.headerSize(byte[])` / `ZstdFrame.headerSize(MemorySegment)` — the size
of a frame's header computed from just its leading bytes (as few as 5), without a
full parse. Binds `ZSTD_frameHeaderSize`.
- `ZstdDictionary.compressDict(int)` / `compressDict()` / `decompressDict()` —
factories for digested dictionaries, e.g. `dict.compressDict(19)` instead of
`new ZstdCompressDict(dict, 19)`. They signal that the result is `AutoCloseable`
and are for sharing one digest across contexts via `refDictionary`; a single
context should prefer the context-owned `loadDictionary`.

### Changed
- Every dictionary-id accessor now returns `ZstdDictionaryId` instead of `int`:
Expand Down
13 changes: 10 additions & 3 deletions docs/how-to.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ by reference — no per-call digesting, no copy. It pairs with `reset` for a
pooled, recycled context:

```java
try (ZstdCompressDict cdict = new ZstdCompressDict(dict, 19)) {
try (ZstdCompressDict cdict = dict.compressDict(19)) {
// one cctx per pooled worker, all sharing the one digested dictionary
try (ZstdCompressCtx cctx = new ZstdCompressCtx()) {
cctx.refDictionary(cdict); // borrowed; cdict must outlive cctx
Expand All @@ -73,6 +73,13 @@ try (ZstdCompressDict cdict = new ZstdCompressDict(dict, 19)) {
}
```

`refDictionary` only borrows: the digested `cdict` is *not* tied to the context's
lifetime, so it must be closed separately (hence its own try-with-resources). That
is the price of sharing one digest across many contexts. If you have just **one**
context, don't build a `ZstdCompressDict` at all — `loadDictionary` above digests
into the context and frees it for you, and a stray, never-closed
`ZstdCompressDict` is a native-memory leak.

A loaded or referenced dictionary stays until replaced, cleared with `null`, or
dropped by a parameter `reset`. `ZstdDecompressCtx` mirrors all of this.

Expand All @@ -98,8 +105,8 @@ ZstdDictionary reloaded = ZstdDictionary.of(persisted);
On a hot path, digest the dictionary once to skip per-call setup:

```java
try (ZstdCompressDict cdict = new ZstdCompressDict(dict, 19);
ZstdDecompressDict ddict = new ZstdDecompressDict(dict);
try (ZstdCompressDict cdict = dict.compressDict(19);
ZstdDecompressDict ddict = dict.decompressDict();
ZstdCompressCtx cctx = new ZstdCompressCtx();
ZstdDecompressCtx dctx = new ZstdDecompressCtx()) {
byte[] packed = cctx.compress(record, cdict);
Expand Down
35 changes: 35 additions & 0 deletions zstd/src/main/java/io/github/dfa1/zstd/ZstdDictionary.java
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,41 @@ public int size() {
return bytes.length;
}

/// Digests this dictionary once for compression at `level`, ready to share by
/// reference across contexts with [ZstdCompressCtx#refDictionary(ZstdCompressDict)].
///
/// The returned dictionary owns native memory — close it when done (it is
/// [AutoCloseable]). For a single context, prefer
/// [ZstdCompressCtx#loadDictionary(ZstdDictionary)], which the context digests,
/// owns, and frees for you.
///
/// @param level the compression level to fix for the digested dictionary
/// @return a digested compression dictionary the caller must close
public ZstdCompressDict compressDict(int level) {
return new ZstdCompressDict(this, level);
}

/// Digests this dictionary for compression at the library default level.
/// Otherwise identical to [#compressDict(int)].
///
/// @return a digested compression dictionary the caller must close
public ZstdCompressDict compressDict() {
return new ZstdCompressDict(this);
}

/// Digests this dictionary once for decompression, ready to share by reference
/// across contexts with [ZstdDecompressCtx#refDictionary(ZstdDecompressDict)].
///
/// The returned dictionary owns native memory — close it when done (it is
/// [AutoCloseable]). For a single context, prefer
/// [ZstdDecompressCtx#loadDictionary(ZstdDictionary)], which the context owns
/// and frees for you.
///
/// @return a digested decompression dictionary the caller must close
public ZstdDecompressDict decompressDict() {
return new ZstdDecompressDict(this);
}

/// Internal: direct view of the bytes for native calls. Not exposed.
byte[] raw() {
return bytes;
Expand Down
52 changes: 52 additions & 0 deletions zstd/src/test/java/io/github/dfa1/zstd/ZstdDictionaryTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,58 @@ void interoperatesWithRawPath() {
}
}

@Nested
class Factories {

@Test
void compressDictFixesTheRequestedLevel() {
// When a digested compress dictionary is built via the factory
try (ZstdCompressDict cdict = sut.compressDict(19)) {
// Then it carries that level and the dictionary's id
assertThat(cdict.level()).isEqualTo(19);
assertThat(cdict.id()).isEqualTo(sut.id());
}
}

@Test
void compressDictDefaultsToTheLibraryLevel() {
// When built without a level
try (ZstdCompressDict cdict = sut.compressDict()) {
// Then it uses the library default
assertThat(cdict.level()).isEqualTo(Zstd.defaultCompressionLevel());
}
}

@Test
void decompressDictReportsTheDictionaryId() {
// When a digested decompress dictionary is built via the factory
try (ZstdDecompressDict ddict = sut.decompressDict()) {
// Then it carries the dictionary's id
assertThat(ddict.id()).isEqualTo(sut.id());
}
}

@Test
void factoryDictionariesRoundTrip() {
// Given factory-built digested dictionaries
byte[] record = samples.get(123);

byte[] restored;
try (ZstdCompressCtx cctx = new ZstdCompressCtx();
ZstdDecompressCtx dctx = new ZstdDecompressCtx();
ZstdCompressDict cdict = sut.compressDict(19);
ZstdDecompressDict ddict = sut.decompressDict()) {

// When round-tripped through them
byte[] frame = cctx.compress(record, cdict);
restored = dctx.decompress(frame, record.length, ddict);
}

// Then the record is recovered
assertThat(restored).isEqualTo(record);
}
}

@Nested
class Serialisation {

Expand Down
Loading