|
4 | 4 |
|
5 | 5 | # Python Bindings for [`llama.cpp`](https://github.com/ggml-org/llama.cpp) |
6 | 6 |
|
7 | | -[](https://llama-cpp-python.readthedocs.io/en/latest/?badge=latest) |
8 | 7 | [](https://github.com/JamePeng/llama-cpp-python/actions/workflows/test.yaml) |
9 | 8 |  |
10 | 9 | [](https://pypi.org/project/llama-cpp-python/) |
11 | 10 | [](https://pepy.tech/projects/llama-cpp-python) |
12 | 11 | []() |
13 | 12 |
|
14 | | -Simple Python bindings for **@ggerganov's** [`llama.cpp`](https://github.com/ggml-org/llama.cpp) library. |
| 13 | +Efficiency Python bindings for **ggml-org's** [`llama.cpp`](https://github.com/ggml-org/llama.cpp) library. |
15 | 14 | This package provides: |
16 | 15 |
|
17 | 16 | - Low-level access to C API via `ctypes` interface. |
| 17 | + - [llama_cpp_lib](https://github.com/JamePeng/llama-cpp-python/blob/main/llama_cpp/llama_cpp.py) |
| 18 | + - [mtmd_cpp_lib](https://github.com/JamePeng/llama-cpp-python/blob/main/llama_cpp/mtmd_cpp.py) |
18 | 19 | - High-level Python API for text completion |
19 | | - - OpenAI-like API |
20 | | - - [LangChain compatibility](https://python.langchain.com/docs/integrations/llms/llamacpp) |
21 | | - - [LlamaIndex compatibility](https://docs.llamaindex.ai/en/stable/examples/llm/llama_2_llama_cpp.html) |
22 | | -- OpenAI compatible web server |
23 | | - - [Local Copilot replacement](https://llama-cpp-python.readthedocs.io/en/latest/server/#code-completion) |
24 | | - - [Function Calling support](https://llama-cpp-python.readthedocs.io/en/latest/server/#function-calling) |
25 | | - - [Vision API support](https://llama-cpp-python.readthedocs.io/en/latest/server/#multimodal-models) |
26 | | - - [Multiple Models](https://llama-cpp-python.readthedocs.io/en/latest/server/#configuration-and-multi-model-support) |
27 | | - |
28 | | -Documentation is available at [https://llama-cpp-python.readthedocs.io/en/latest](https://llama-cpp-python.readthedocs.io/en/latest). |
| 20 | + - OpenAI-like API and Type([llama_types.py](https://github.com/JamePeng/llama-cpp-python/blob/main/llama_cpp/llama_types.py)) |
| 21 | + - [High-level API](https://github.com/JamePeng/llama-cpp-python#high-level-api) |
| 22 | + - [Continuing Assistant Responses (Prefill)](https://github.com/JamePeng/llama-cpp-python#continuing-assistant-responses-prefill) |
| 23 | + - [Dynamic LoRA Routing & Control Vectors (Multi-Tenant Serving)](https://github.com/JamePeng/llama-cpp-python#dynamic-lora-routing--control-vectors-multi-tenant-serving) |
| 24 | + - [Dynamic LoRA Example](https://github.com/JamePeng/llama-cpp-python#dynamic-lora-example) |
| 25 | + - [Control Vector Injection (Representation Engineering)](https://github.com/JamePeng/llama-cpp-python#control-vector-injection-representation-engineering) |
| 26 | + - [Sampling Configuration & Usage (LlamaSamplingParams)](https://github.com/JamePeng/llama-cpp-python#sampling-configuration--usage-llamasamplingparams) |
| 27 | + - [Multi-modal Models Support](https://github.com/JamePeng/llama-cpp-python#multi-modal-models) |
| 28 | + - Support Models Lists |
| 29 | + - [Loading a Local Image With Qwen3VL(Thinking/Instruct)](https://github.com/JamePeng/llama-cpp-python#loading-a-local-image-with-qwen3vlthinkinginstruct) |
| 30 | + - [Comprehensive Omni MultiModal Example: Gemma-4 (Vision + Audio + Text)](https://github.com/JamePeng/llama-cpp-python#comprehensive-omni-multimodal-example-gemma-4-vision--audio--text) |
| 31 | + - [Embeddings & Reranking (GGUF)](https://github.com/JamePeng/llama-cpp-python#embeddings--reranking-gguf) |
| 32 | + - [1. Text Embeddings (Vector Search)](https://github.com/JamePeng/llama-cpp-python#1-text-embeddings-vector-search) |
| 33 | + - [2. Reranking (Cross-Encoder Scoring)](https://github.com/JamePeng/llama-cpp-python#2-reranking-cross-encoder-scoring) |
| 34 | + - [3. Normalization](https://github.com/JamePeng/llama-cpp-python#3-normalization) |
| 35 | + - [Speculative Decoding](https://github.com/JamePeng/llama-cpp-python#speculative-decoding) |
| 36 | +- [FAQ](https://github.com/JamePeng/llama-cpp-python#faq) |
| 37 | + |
| 38 | +The new documentation will be maintained in the [docs/wiki](https://github.com/JamePeng/llama-cpp-python/tree/main/docs/wiki) directory based on the LLM Wiki approach. Interested volunteers are welcome to participate in its maintenance and updates :) |
29 | 39 |
|
30 | 40 |
|
31 | 41 | ## Discussions |
|
0 commit comments