Skip to content

Commit 1b243f1

Browse files
dfa1claude
andcommitted
feat: bind ZSTD_getDictID_fromDict and add ZstdDictionaryId value type
Adds Zstd.dictId(byte[]) / Zstd.dictId(MemorySegment), the core-libzstd reader for the dictionary id stamped in raw dictionary bytes — the counterpart to ZstdFrame.dictId (which reads a frame) and the no-wrapper alternative to ZstdDictionary.id() (which uses the equivalent ZDICT reader). Binds ZSTD_getDictID_fromDict. Introduces ZstdDictionaryId, a record wrapping the 32-bit id with an unsigned value(), isPresent(), and a NONE sentinel for "no id" (0). Every dictionary-id accessor now returns it instead of int: ZstdDictionary.id(), ZstdCompressDict.id(), ZstdDecompressDict.id(), ZstdFrame.dictId(...), and the ZstdFrameHeader.dictId() component. This makes the unsigned semantics and the 0-means-absent sentinel explicit at every call site. docs/supported.md: ZSTD_getDictID_fromDict now bound (Dictionary 10 -> 11/23). CHANGELOG Unreleased updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 77dc09f commit 1b243f1

15 files changed

Lines changed: 260 additions & 34 deletions

File tree

CHANGELOG.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,17 @@ git tags, which trigger publication to Maven Central.
1515
same prefix to decode. Binds `ZSTD_CCtx_refPrefix` / `ZSTD_DCtx_refPrefix`.
1616
Segment-only by design — heap callers that need a copy should use
1717
`loadDictionary` instead.
18+
- `Zstd.dictId(byte[])` / `Zstd.dictId(MemorySegment)` — read the dictionary id
19+
stamped in raw dictionary bytes without wrapping them in a `ZstdDictionary`.
20+
Binds `ZSTD_getDictID_fromDict`.
21+
- `ZstdDictionaryId` value type — a `record` wrapping the 32-bit dictionary id
22+
with an unsigned `value()`, `isPresent()`, and the `NONE` sentinel for "no id".
23+
24+
### Changed
25+
- Every dictionary-id accessor now returns `ZstdDictionaryId` instead of `int`:
26+
`ZstdDictionary.id()`, `ZstdCompressDict.id()`, `ZstdDecompressDict.id()`,
27+
`ZstdFrame.dictId(...)`, and `ZstdFrameHeader.dictId()`. The `0` sentinel is now
28+
`ZstdDictionaryId.NONE`, and the id reads as unsigned via `value()`.
1829

1930
## [0.5]
2031

docs/supported.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ rather than the deprecated `ZSTD_getDecompressedSize`.
2929
| Version | 2 / 2 | complete |
3030
| Errors | 4 / 4 | complete: name, typed `ZstdErrorCode`, and `description()` |
3131
| Reusable contexts | 6 / 8 | CCtx/DCtx create/free/compress/decompress |
32-
| Dictionary — simple | 10 / 23 | raw + digested (CDict/DDict) + dict-id queries; `_advanced`/`_byReference`/`Begin` variants not bound |
32+
| Dictionary — simple | 11 / 23 | raw + digested (CDict/DDict) + dict-id queries; `_advanced`/`_byReference`/`Begin` variants not bound |
3333
| Dictionary training (ZDICT) | 8 / 12 | trainFromBuffer, cover/fastCover optimizers, finalizeDictionary, getDictHeaderSize |
3434
| Streaming — compress | 3 / 22 | `ZstdOutputStream` (compressStream2 + buffer sizes) |
3535
| Streaming — decompress | 3 / 15 | `ZstdInputStream` (decompressStream + buffer sizes) |
@@ -141,7 +141,7 @@ sequence-producer hooks vs this library's frame-inspection and typed-error surfa
141141
| `ZSTD_createCCtx_advanced` |||
142142
| `ZSTD_createDCtx_advanced` |||
143143

144-
### Dictionary — simple (10/23)
144+
### Dictionary — simple (11/23)
145145

146146
| Symbol | Bound | zstd-jni |
147147
|---|:---:|:---:|
@@ -166,7 +166,7 @@ sequence-producer hooks vs this library's frame-inspection and typed-error surfa
166166
| `ZSTD_decompressBegin_usingDict` |||
167167
| `ZSTD_getDictID_fromCDict` |||
168168
| `ZSTD_getDictID_fromDDict` |||
169-
| `ZSTD_getDictID_fromDict` | ||
169+
| `ZSTD_getDictID_fromDict` | ||
170170
| `ZSTD_getDictID_fromFrame` |||
171171

172172
### Dictionary training, ZDICT (8/12)

integration-tests/src/test/java/io/github/dfa1/zstd/it/GoldenCorpusTest.java

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
package io.github.dfa1.zstd.it;
22

3+
import io.github.dfa1.zstd.ZstdDictionaryId;
34
import io.github.dfa1.zstd.Zstd;
45
import io.github.dfa1.zstd.ZstdCompressCtx;
56
import io.github.dfa1.zstd.ZstdDecompressCtx;
@@ -250,11 +251,11 @@ void dictIdRidesWithFrame(Path file) {
250251
}
251252

252253
// When
253-
int frameDictId = ZstdFrame.dictId(frame);
254+
ZstdDictionaryId frameDictId = ZstdFrame.dictId(frame);
254255

255256
// Then
256257
assertThat(frameDictId).isEqualTo(dict.id());
257-
assertThat(frameDictId).isNotZero();
258+
assertThat(frameDictId).isNotEqualTo(ZstdDictionaryId.NONE);
258259
}
259260
}
260261
}

integration-tests/src/test/java/io/github/dfa1/zstd/it/ZstdInteropExtrasTest.java

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package io.github.dfa1.zstd.it;
22

33
import com.github.luben.zstd.ZstdCompressCtx;
4+
import io.github.dfa1.zstd.ZstdDictionaryId;
45
import io.github.dfa1.zstd.Zstd;
56
import io.github.dfa1.zstd.ZstdDictionary;
67
import io.github.dfa1.zstd.ZstdException;
@@ -219,11 +220,11 @@ void javaReadsDictIdFromJniDictFrame() {
219220
byte[] frame = com.github.luben.zstd.Zstd.compress(record(7), jniDict);
220221

221222
// When
222-
int dictId = ZstdFrame.dictId(frame);
223+
ZstdDictionaryId dictId = ZstdFrame.dictId(frame);
223224

224225
// Then
225226
assertThat(dictId).isEqualTo(dict.id());
226-
assertThat(dictId).isNotZero();
227+
assertThat(dictId).isNotEqualTo(ZstdDictionaryId.NONE);
227228
}
228229

229230
private ZstdDictionary trainDict() {

zstd/src/main/java/io/github/dfa1/zstd/Bindings.java

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,10 @@ final class Bindings {
170170
// unsigned ZSTD_getDictID_fromDDict(const ZSTD_DDict*)
171171
static final MethodHandle GET_DICT_ID_FROM_DDICT =
172172
NativeLibrary.lookup("ZSTD_getDictID_fromDDict", FunctionDescriptor.of(JAVA_INT, ADDRESS));
173+
// unsigned ZSTD_getDictID_fromDict(const void* dict, size_t dictSize)
174+
static final MethodHandle GET_DICT_ID_FROM_DICT =
175+
NativeLibrary.lookup("ZSTD_getDictID_fromDict",
176+
FunctionDescriptor.of(JAVA_INT, ADDRESS, JAVA_LONG));
173177

174178
// ZSTD_bounds { size_t error; int lowerBound; int upperBound; } — returned by value
175179
static final MemoryLayout BOUNDS_LAYOUT =

zstd/src/main/java/io/github/dfa1/zstd/Zstd.java

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,42 @@ public static long decompressedSize(MemorySegment frame) {
111111
return size;
112112
}
113113

114+
/// Dictionary id stamped in raw dictionary `bytes`, read with the core
115+
/// `libzstd` reader.
116+
///
117+
/// This reads the id a *dictionary* records, not the id a frame references —
118+
/// for the latter use [ZstdFrame#dictId(byte[])]. It returns the same value as
119+
/// [ZstdDictionary#id()] (which uses the equivalent `ZDICT` reader); prefer this
120+
/// when you only hold raw bytes and do not want to wrap them in a
121+
/// [ZstdDictionary].
122+
///
123+
/// @param bytes raw dictionary content
124+
/// @return the dictionary id, or [ZstdDictionaryId#NONE] if `bytes` is not a standard zstd
125+
/// dictionary (for example a raw/content-only dictionary with no header)
126+
public static ZstdDictionaryId dictId(byte[] bytes) {
127+
try (Arena arena = Arena.ofConfined()) {
128+
return dictId(copyIn(arena, bytes), bytes.length);
129+
}
130+
}
131+
132+
/// Dictionary id stamped in native dictionary `bytes`, read with no copy.
133+
/// Otherwise identical to [#dictId(byte[])].
134+
///
135+
/// @param bytes native dictionary content
136+
/// @return the dictionary id, or [ZstdDictionaryId#NONE] if `bytes` is not a standard zstd dictionary
137+
public static ZstdDictionaryId dictId(MemorySegment bytes) {
138+
NativeCall.requireNative(bytes, "bytes");
139+
return dictId(bytes, bytes.byteSize());
140+
}
141+
142+
private static ZstdDictionaryId dictId(MemorySegment bytes, long size) {
143+
try {
144+
return ZstdDictionaryId.of((int) Bindings.GET_DICT_ID_FROM_DICT.invokeExact(bytes, size));
145+
} catch (Throwable t) {
146+
throw NativeCall.rethrow(t);
147+
}
148+
}
149+
114150
/// Maximum compressed size for an input of `srcSize` bytes — the buffer
115151
/// size guaranteed to never overflow during compression.
116152
///

zstd/src/main/java/io/github/dfa1/zstd/ZstdCompressDict.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -81,10 +81,10 @@ public int level() {
8181

8282
/// The dictionary id stamped into frames compressed with this dictionary.
8383
///
84-
/// @return the dictionary id, or `0` for a content-only dictionary
85-
public int id() {
84+
/// @return the dictionary id, or [ZstdDictionaryId#NONE] for a content-only dictionary
85+
public ZstdDictionaryId id() {
8686
try {
87-
return (int) Bindings.GET_DICT_ID_FROM_CDICT.invokeExact(ptr());
87+
return ZstdDictionaryId.of((int) Bindings.GET_DICT_ID_FROM_CDICT.invokeExact(ptr()));
8888
} catch (Throwable t) {
8989
throw NativeCall.rethrow(t);
9090
}

zstd/src/main/java/io/github/dfa1/zstd/ZstdDecompressDict.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,10 +51,10 @@ private static MemorySegment create(MemorySegment dict) {
5151

5252
/// The dictionary id this dictionary decodes frames for.
5353
///
54-
/// @return the dictionary id, or `0` for a content-only dictionary
55-
public int id() {
54+
/// @return the dictionary id, or [ZstdDictionaryId#NONE] for a content-only dictionary
55+
public ZstdDictionaryId id() {
5656
try {
57-
return (int) Bindings.GET_DICT_ID_FROM_DDICT.invokeExact(ptr());
57+
return ZstdDictionaryId.of((int) Bindings.GET_DICT_ID_FROM_DDICT.invokeExact(ptr()));
5858
} catch (Throwable t) {
5959
throw NativeCall.rethrow(t);
6060
}

zstd/src/main/java/io/github/dfa1/zstd/ZstdDictionary.java

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -253,13 +253,13 @@ private static ZstdDictionary toDictionary(MemorySegment dictBuf, long produced,
253253
}
254254

255255
/// The dictionary id zstd stamps into frames compressed with this dictionary,
256-
/// or `0` for a raw/content-only dictionary with no header.
256+
/// or [ZstdDictionaryId#NONE] for a raw/content-only dictionary with no header.
257257
///
258-
/// @return the dictionary id, or `0` if none
259-
public int id() {
258+
/// @return the dictionary id, or [ZstdDictionaryId#NONE] if none
259+
public ZstdDictionaryId id() {
260260
try (Arena arena = Arena.ofConfined()) {
261261
MemorySegment seg = Zstd.copyIn(arena, bytes);
262-
return (int) Bindings.ZDICT_GET_DICT_ID.invokeExact(seg, (long) bytes.length);
262+
return ZstdDictionaryId.of((int) Bindings.ZDICT_GET_DICT_ID.invokeExact(seg, (long) bytes.length));
263263
} catch (Throwable t) {
264264
throw NativeCall.rethrow(t);
265265
}
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
package io.github.dfa1.zstd;
2+
3+
/// A zstd dictionary id: the 32-bit value a dictionary stamps into the frames it
4+
/// compresses, letting a decompressor pick the matching dictionary.
5+
///
6+
/// zstd treats the id as a C `unsigned`, so the wire value can exceed
7+
/// [Integer#MAX_VALUE]. The raw 32-bit pattern is kept in [#raw()]; read it as an
8+
/// unsigned magnitude with [#value()]. The value `0` is the sentinel for "no
9+
/// dictionary id" — a raw/content-only dictionary, or a frame that records none —
10+
/// exposed as [#NONE] and tested with [#isPresent()].
11+
///
12+
/// @param raw the 32-bit id as zstd stores it, possibly negative when read as a
13+
/// signed `int`; `0` means no id
14+
public record ZstdDictionaryId(int raw) {
15+
16+
/// The absent id: no dictionary, or a dictionary with no recorded id.
17+
public static final ZstdDictionaryId NONE = new ZstdDictionaryId(0);
18+
19+
/// Wraps a raw 32-bit id, returning [#NONE] for `0`.
20+
///
21+
/// @param raw the 32-bit id as zstd stores it
22+
/// @return the wrapped id, or [#NONE] if `raw` is `0`
23+
public static ZstdDictionaryId of(int raw) {
24+
return raw == 0 ? NONE : new ZstdDictionaryId(raw);
25+
}
26+
27+
/// Whether an id is present (non-zero).
28+
///
29+
/// @return `true` unless this is [#NONE]
30+
public boolean isPresent() {
31+
return raw != 0;
32+
}
33+
34+
/// The id as an unsigned magnitude in `[0, 2^32)`, widening the raw 32-bit
35+
/// pattern without sign extension.
36+
///
37+
/// @return the unsigned id value
38+
public long value() {
39+
return Integer.toUnsignedLong(raw);
40+
}
41+
}

0 commit comments

Comments
 (0)