Commit 1ebd21b
Annotate TokenizerFactory for Java call sites (#400)
Adds `@JvmStatic` to both factory entry points on the
`TokenizerFactory` object (`fromGguf(Map)`, `fromTokenizerJson(String)`).
Same motivation as StableHloConverterFactory in the previous
commit — without the annotation, Java consumers had to navigate
through the Kotlin object's `INSTANCE` marker:
var tokenizer =
TokenizerFactory.INSTANCE.fromGguf(ggufFields);
With the annotation they get the idiomatic static form:
var tokenizer =
TokenizerFactory.fromGguf(ggufFields);
The factory is the canonical entry point for the new Qwen
byte-level BPE + SentencePiece tokenizers that landed in #463
and #464, so this is a meaningful win for Java consumers of
the upcoming 0.19.0 release — they get Qwen / Llama / Gemma /
TinyLlama tokenization without any Kotlin-specific interop
glue.
Verified across jvmTest, compileKotlinWasmJs, and macosArm64Test
for skainet-io-core — no regressions.
Third of five commits polishing the Java / JVM consumption
story for the upcoming 0.19.0 release. See #400.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent 25be9dc commit 1ebd21b
1 file changed
Lines changed: 3 additions & 0 deletions
Lines changed: 3 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
6 | 7 | | |
7 | 8 | | |
8 | 9 | | |
| |||
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
| 37 | + | |
36 | 38 | | |
37 | 39 | | |
38 | 40 | | |
| |||
57 | 59 | | |
58 | 60 | | |
59 | 61 | | |
| 62 | + | |
60 | 63 | | |
61 | 64 | | |
62 | 65 | | |
| |||
0 commit comments