Skip to content

Commit 820d183

Browse files
committed
Update README.md
Signed-off-by: JamePeng <jame_peng@sina.com>
1 parent 7a4dd5b commit 820d183

1 file changed

Lines changed: 22 additions & 12 deletions

File tree

README.md

Lines changed: 22 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,28 +4,38 @@
44

55
# Python Bindings for [`llama.cpp`](https://github.com/ggml-org/llama.cpp)
66

7-
[![Documentation Status](https://readthedocs.org/projects/llama-cpp-python/badge/?version=latest)](https://llama-cpp-python.readthedocs.io/en/latest/?badge=latest)
87
[![Tests](https://github.com/JamePeng/llama-cpp-python/actions/workflows/test.yaml/badge.svg?branch=main)](https://github.com/JamePeng/llama-cpp-python/actions/workflows/test.yaml)
98
![GitHub Tag](https://img.shields.io/github/v/tag/JamePeng/llama-cpp-python)
109
[![PyPI - License](https://img.shields.io/pypi/l/llama-cpp-python)](https://pypi.org/project/llama-cpp-python/)
1110
[![PyPI - Downloads](https://static.pepy.tech/badge/llama-cpp-python/month)](https://pepy.tech/projects/llama-cpp-python)
1211
[![Github All Releases](https://img.shields.io/github/downloads/abetlen/llama-cpp-python/total.svg?label=Github%20Downloads)]()
1312

14-
Simple Python bindings for **@ggerganov's** [`llama.cpp`](https://github.com/ggml-org/llama.cpp) library.
13+
Efficiency Python bindings for **ggml-org's** [`llama.cpp`](https://github.com/ggml-org/llama.cpp) library.
1514
This package provides:
1615

1716
- Low-level access to C API via `ctypes` interface.
17+
- [llama_cpp_lib](https://github.com/JamePeng/llama-cpp-python/blob/main/llama_cpp/llama_cpp.py)
18+
- [mtmd_cpp_lib](https://github.com/JamePeng/llama-cpp-python/blob/main/llama_cpp/mtmd_cpp.py)
1819
- High-level Python API for text completion
19-
- OpenAI-like API
20-
- [LangChain compatibility](https://python.langchain.com/docs/integrations/llms/llamacpp)
21-
- [LlamaIndex compatibility](https://docs.llamaindex.ai/en/stable/examples/llm/llama_2_llama_cpp.html)
22-
- OpenAI compatible web server
23-
- [Local Copilot replacement](https://llama-cpp-python.readthedocs.io/en/latest/server/#code-completion)
24-
- [Function Calling support](https://llama-cpp-python.readthedocs.io/en/latest/server/#function-calling)
25-
- [Vision API support](https://llama-cpp-python.readthedocs.io/en/latest/server/#multimodal-models)
26-
- [Multiple Models](https://llama-cpp-python.readthedocs.io/en/latest/server/#configuration-and-multi-model-support)
27-
28-
Documentation is available at [https://llama-cpp-python.readthedocs.io/en/latest](https://llama-cpp-python.readthedocs.io/en/latest).
20+
- OpenAI-like API and Type([llama_types.py](https://github.com/JamePeng/llama-cpp-python/blob/main/llama_cpp/llama_types.py))
21+
- [High-level API](https://github.com/JamePeng/llama-cpp-python#high-level-api)
22+
- [Continuing Assistant Responses (Prefill)](https://github.com/JamePeng/llama-cpp-python#continuing-assistant-responses-prefill)
23+
- [Dynamic LoRA Routing & Control Vectors (Multi-Tenant Serving)](https://github.com/JamePeng/llama-cpp-python#dynamic-lora-routing--control-vectors-multi-tenant-serving)
24+
- [Dynamic LoRA Example](https://github.com/JamePeng/llama-cpp-python#dynamic-lora-example)
25+
- [Control Vector Injection (Representation Engineering)](https://github.com/JamePeng/llama-cpp-python#control-vector-injection-representation-engineering)
26+
- [Sampling Configuration & Usage (LlamaSamplingParams)](https://github.com/JamePeng/llama-cpp-python#sampling-configuration--usage-llamasamplingparams)
27+
- [Multi-modal Models Support](https://github.com/JamePeng/llama-cpp-python#multi-modal-models)
28+
- Support Models Lists
29+
- [Loading a Local Image With Qwen3VL(Thinking/Instruct)](https://github.com/JamePeng/llama-cpp-python#loading-a-local-image-with-qwen3vlthinkinginstruct)
30+
- [Comprehensive Omni MultiModal Example: Gemma-4 (Vision + Audio + Text)](https://github.com/JamePeng/llama-cpp-python#comprehensive-omni-multimodal-example-gemma-4-vision--audio--text)
31+
- [Embeddings & Reranking (GGUF)](https://github.com/JamePeng/llama-cpp-python#embeddings--reranking-gguf)
32+
- [1. Text Embeddings (Vector Search)](https://github.com/JamePeng/llama-cpp-python#1-text-embeddings-vector-search)
33+
- [2. Reranking (Cross-Encoder Scoring)](https://github.com/JamePeng/llama-cpp-python#2-reranking-cross-encoder-scoring)
34+
- [3. Normalization](https://github.com/JamePeng/llama-cpp-python#3-normalization)
35+
- [Speculative Decoding](https://github.com/JamePeng/llama-cpp-python#speculative-decoding)
36+
- [FAQ](https://github.com/JamePeng/llama-cpp-python#faq)
37+
38+
The new documentation will be maintained in the [docs/wiki](https://github.com/JamePeng/llama-cpp-python/tree/main/docs/wiki) directory based on the LLM Wiki approach. Interested volunteers are welcome to participate in its maintenance and updates :)
2939

3040

3141
## Discussions

0 commit comments

Comments
 (0)