You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- This release mainly addresses several issues discovered during real-world backend deployment, especially on Windows environments with dynamic backend loading, as well as improving the robustness of MTMD, batching, and native API interactions.
Signed-off-by: JamePeng <jame_peng@sina.com>
- Add load source diagnostics for system and bundled libraries
18
+
- Improve visibility when debugging shared library loading issues
19
+
20
+
**Note**:
21
+
* For most single-DLL backends, the bin directory can still work as a fallback search path. However, some cases may fail due to missing dependencies such as `libomp140.x86_64.dll`.
22
+
* For `multi-DLL backends`, such as the `SYCL backend`, which depends on multiple DLLs (`dnnl.dll`, `tbb12.dll`, `mk_*.dll`, etc.), loading ggml-sycl.dll may fail when its dependent DLLs cannot be found, potentially resulting in an `access violation` crash.
23
+
* This update ensures that the DLL search path prioritizes /lib instead of /bin during the initial lookup stage, improving backend loading reliability.
24
+
* Special thanks to **@allanmeng** for reporting and testing the SYCL backend issue.
25
+
26
+
- fix(ggml): load ggml-base before ggml library
27
+
- Load ggml-base shared library before ggml to ensure the base
28
+
runtime dependency is initialized prior to loading the main ggml
29
+
library.
30
+
31
+
- This improves dynamic library loading reliability on platforms
32
+
where ggml depends on ggml-base during initialization.
33
+
34
+
- fix(mtmd): validate MTMD inputs before tokenization
35
+
- Add Python-side MTMD input validation before calling the native mtmd_tokenize
36
+
path. Normalize missing bitmap lists to empty lists for pure text prompts, check
37
+
that rendered media markers match decoded bitmap inputs, reject missing bitmap
38
+
entries, and validate that the media marker is available.
39
+
40
+
- Improve media placeholder mismatch errors with marker counts and marker details,
41
+
and surface mtmd_tokenize failures with richer diagnostic context including media
42
+
counts and backend support flags.
43
+
44
+
- feat(LlamaBatch): add mixed token embedding batch support
45
+
- Add optional mixed=True initialization for LlamaBatch so token+embedding rows can
46
+
be represented in a single llama_batch. Mixed batches keep the native embd buffer
47
+
from llama_batch_init and attach a Python-owned token buffer, which is cleared
48
+
before llama_batch_free() to avoid invalid ownership.
49
+
50
+
- Route token-only and embedding-only write APIs away from mixed batches, add
51
+
mixed-batch validation, and introduce add_token_embedding for EAGLE3/MTP-style
52
+
decoder inputs containing both token ids and embedding vectors.
53
+
54
+
- This prepares LlamaBatch for speculative decoding paths that require mixed
55
+
token+hidden-state inputs, especially EAGLE3 and MTP. It keeps ordinary
56
+
token-only and embedding-only APIs separated while providing a dedicated
57
+
add_token_embedding path for mixed decoder rows.
58
+
59
+
- feat(LlamaBatch): add embedding rows to LlamaBatch
60
+
- Add shared seq_id validation for token and embedding batch writes.
61
+
62
+
- Introduce embedding-buffer checks plus add_embedding and add_embeddings helpers
63
+
for embd-only llama_batch inputs, enabling decoder paths that consume external
64
+
embedding rows while keeping token writes restricted to token buffers.
65
+
66
+
- This prepares LlamaBatch for embedding-only decode paths, such as speculative
67
+
decoding feature injection or external encoder/projector outputs.
68
+
69
+
- It does not implement mixed token+embedding batches yet; those still need a
70
+
separate ownership-safe design for the token buffer.
71
+
72
+
- fix(LlamaBatch): harden LlamaBatch token writes
73
+
- Clarify llama_batch token vs embedding allocation semantics and keep future
74
+
embedding/mixed-batch support open.
75
+
76
+
- Add token-buffer checks before add_token/add_sequence, validate add_sequence
77
+
input lengths and seq_ids, and improve error messages for invalid batch
78
+
configuration.
79
+
80
+
- fix(eval): validate eval tokens before native decode
81
+
- Add token-id validation at the Llama.eval() boundary before context shifting,
82
+
batch construction, or llama_decode execution. This prevents invalid token
83
+
types, negative token ids, and out-of-vocabulary ids from reaching the native
84
+
decode path, where they may otherwise cause hard crashes instead of Python
85
+
exceptions.
86
+
87
+
- Wrap llama_decode with defensive exception handling in LlamaContext.decode() so
88
+
native exceptions are surfaced with clearer diagnostic context.
89
+
90
+
- Also include a small token preview in Llama.eval() fatal decode errors to make
91
+
backend failures easier to debug without changing the existing recoverable KV
92
+
slot handling behavior.
93
+
94
+
- fix(types): make assistant message name optional
95
+
- Mark the assistant message `name` field as `NotRequired[Optional[str]]`
96
+
to match the optional nature of assistant message metadata and avoid
97
+
requiring callers to provide `name` in typed chat completion requests.
98
+
99
+
- feat: Update llama.cpp to [ggml-org/llama.cpp/commit/e3546c7948e3af463d0b401e6421d5a4c2faf565](https://github.com/ggml-org/llama.cpp/commit/e3546c7948e3af463d0b401e6421d5a4c2faf565)
100
+
101
+
- feat: Sync llama.cpp llama/mtmd/ggml API Binding 20260711
102
+
103
+
More information see: https://github.com/JamePeng/llama-cpp-python/compare/169d5e1a43fb6ff4e5b6f5d0f26f1ec8acbd97b8...3da4c603612c3344031b32ffbeb1da1c84bb205a
0 commit comments