You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+24Lines changed: 24 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,6 +7,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
8
8
## [Unreleased]
9
9
10
+
## [0.3.38] Optimized CJK Detokenization, Sync Grammar Parser, and Patched CUDA Graph Logs
11
+
12
+
- perf: Optimize detokenize buffer sizing for CJK-heavy outputs
13
+
- Increase the initial detokenization buffer estimate from 1x tokens to `5x + 32` bytes. Performance analysis revealed that CJK-heavy outputs often require around 4.0x–5.04x bytes per token, with small-token edge cases reaching about 6.0x, so the previous estimate frequently forced a failed `llama_detokenize` call followed by a resize and retry. Previously, this resulted in almost twice as many calls to llama_detokenize.
14
+
- This reduces avoidable detokenization retries and cuts call overhead in CJK-heavy cases, resulting in an observed ~3–5% inference performance improvement.
15
+
16
+
- patch(logger): filter out verbose noisy CUDA Graph debug logs
17
+
- Add a temporary patch in `ggml_log_callback` to suppress the noisy `CUDA Graph id %zu reused` messages generated by the underlying C++ backend. A complete logger refactoring is planned for better log control in the future.
18
+
19
+
- docs: add LlamaGrammar wiki page and Update /docs/wiki/index.md
20
+
- Now Github Wiki Online: https://github.com/JamePeng/llama-cpp-python/wiki
21
+
22
+
- feat(grammar): sync JSON schema to GBNF converter with upstream
23
+
- Allow `LlamaGrammar.from_json_schema` and `json_schema_to_gbnf` to accept both string and dict schema inputs.
24
+
- Expose `allow_fetch`, `dotall`, and `raw_pattern` arguments to the public API to match the upstream script.
25
+
- Fix missing handling for empty/unconstrained schema objects (e.g. `{"description": "..."}`) which now correctly default to accepting any value.
26
+
- Fix bug where `has_min and has_max` evaluated incorrectly when variables were zero. Replaced `!= None` with `is not None` in `_generate_min_max_int`.
27
+
- Update internal constants and regex patterns (`INVALID_RULE_CHARS_RE`, `GRAMMAR_LITERAL_ESCAPE_RE`, `GRAMMAR_RANGE_LITERAL_ESCAPE_RE`) to resolve character escaping issues.
28
+
- Update reference link to point to the new `ggml-org` organization.
29
+
30
+
- feat: Update llama.cpp to [ggml-org/llama.cpp/commit/e48034dfc9e5705248fd39dc437ca887dc55a528](https://github.com/ggml-org/llama.cpp/commit/e48034dfc9e5705248fd39dc437ca887dc55a528)
31
+
32
+
More information see: https://github.com/JamePeng/llama-cpp-python/compare/fe38cbfac424973ccd9cfaa199a64a02502af4d1...fe8657adf71856967320bfb932461d65ecf77b19
33
+
10
34
## [0.3.37] MoE CPU Offloading, O(1) Speculative Decoding, Thread-Safe Abort & New LLM Wiki
11
35
12
36
- docs: A basic new documentation system for the LLM-Wiki has been initially established.
0 commit comments