Skip to content

Commit 6f21d4d

Browse files
committed
docs: update mtmd chat handler import paths in README
- Update import statements for multi-modal chat handlers from llama_cpp.llama_chat_format to llama_cpp.llama_multimodal in the documentation examples. Signed-off-by: JamePeng <jame_peng@sina.com>
1 parent 46708d1 commit 6f21d4d

1 file changed

Lines changed: 17 additions & 6 deletions

File tree

README.md

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1053,11 +1053,18 @@ Below are the supported multi-modal models and their respective chat handlers (P
10531053
| [qwen3.6](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF) | `Qwen35ChatHandler` | `qwen3.6` |
10541054
| [step3-vl](https://huggingface.co/JamePeng2023/Step3-VL-10B-GGUF) | `Step3VLChatHandler` | `step3-vl` |
10551055
1056-
Then you'll need to use a custom chat handler to load the clip model and process the chat messages and images.
1056+
Then you'll need to use a custom chat handler to load the mmproj model and process the chat messages and images.
1057+
1058+
Note: Starting from 0.3.41-preview, in order to extend MTMD capabilities, multimodal-related logic has been separated from `llama_chat_format` into `llama_multimodal`.
1059+
New implementations are recommended to use the updated multimodal interfaces in `llama_multimodal`
1060+
For backward compatibility, the legacy `llama_chat_format` path is still retained and continues to support existing integrations, but may be deprecated in future versions.
1061+
Additionally, the parameter `clip_model_path` has been renamed to `mmproj_path` to better reflect its purpose and align with the underlying multimodal projection model naming convention.
1062+
The old parameter name `clip_model_path` is kept as a compatibility alias in some interfaces, but new code should use `mmproj_path` exclusively.
10571063
10581064
```python
10591065
from llama_cpp import Llama
1060-
from llama_cpp.llama_chat_format import Llava15ChatHandler
1066+
# from llama_cpp.llama_chat_format import Llava15ChatHandler
1067+
from llama_cpp.llama_multimodal import Llava15ChatHandler
10611068
10621069
model_path="path/to/llava/ggml-model-f16.gguf"
10631070
mmproj_path="path/to/llava/mmproj-model-f16.gguf"
@@ -1086,7 +1093,8 @@ You can also pull the model from the Hugging Face Hub using the `from_pretrained
10861093
10871094
```python
10881095
from llama_cpp import Llama
1089-
from llama_cpp.llama_chat_format import MoondreamChatHandler
1096+
# from llama_cpp.llama_chat_format import MoondreamChatHandler
1097+
from llama_cpp.llama_multimodal import MoondreamChatHandler
10901098
10911099
chat_handler = MoondreamChatHandler.from_pretrained(
10921100
repo_id="vikhyatk/moondream2",
@@ -1128,7 +1136,8 @@ print(response["choices"][0]["text"])
11281136
```python
11291137
# Import necessary libraries
11301138
from llama_cpp import Llama
1131-
from llama_cpp.llama_chat_format import Qwen3VLChatHandler
1139+
# from llama_cpp.llama_chat_format import Qwen3VLChatHandler
1140+
from llama_cpp.llama_multimodal import Qwen3VLChatHandler
11321141
import base64
11331142
import os
11341143
@@ -1285,7 +1294,8 @@ The `Qwen3ASRChatHandler` is specifically designed for the Qwen3 Automatic Speec
12851294
12861295
```python
12871296
from llama_cpp import Llama
1288-
from llama_cpp.llama_chat_format import Qwen3ASRChatHandler
1297+
# from llama_cpp.llama_chat_format import Qwen3ASRChatHandler
1298+
from llama_cpp.llama_multimodal import Qwen3ASRChatHandler
12891299
import base64
12901300
import os
12911301
@@ -1390,7 +1400,8 @@ Below is a complete, production-ready example demonstrating how to dynamically r
13901400
13911401
```python
13921402
from llama_cpp import Llama
1393-
from llama_cpp.llama_chat_format import Gemma4ChatHandler
1403+
# from llama_cpp.llama_chat_format import Gemma4ChatHandler
1404+
from llama_cpp.llama_multimodal import Gemma4ChatHandler
13941405
import base64
13951406
import os
13961407

0 commit comments

Comments
 (0)