You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: document all available backends and add "built by us" list (#10376)
Bring the Backend & Model Compatibility Table up to the full set of
backends published in backend/index.yaml (60+), organized by modality
with per-backend acceleration targets. Add an "Available Backends"
pointer and expand the backend-type list in the backends feature doc.
Update the README backend count to 60+ and add a "Backends built by us"
section listing the native C/C++/GGML engines maintained by the LocalAI
project (parakeet.cpp, voxtral.c, vibevoice.cpp, rf-detr.cpp,
locate-anything.cpp, depth-anything.cpp, LocalVQE, local-store).
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Copy file name to clipboardExpand all lines: README.md
+16-1Lines changed: 16 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -220,10 +220,25 @@ For older news and full release notes, see [GitHub Releases](https://github.com/
220
220
221
221
## Supported Backends & Acceleration
222
222
223
-
LocalAI supports **36+ backends** including llama.cpp, vLLM, transformers, whisper.cpp, diffusers, MLX, MLX-VLM, and many more. Hardware acceleration is available for **NVIDIA** (CUDA 12/13), **AMD** (ROCm), **Intel** (oneAPI/SYCL), **Apple Silicon** (Metal), **Vulkan**, and **NVIDIA Jetson** (L4T). All backends can be installed on-the-fly from the [Backend Gallery](https://localai.io/backends/).
223
+
LocalAI supports **60+ backends** including llama.cpp, vLLM, SGLang, transformers, whisper.cpp, diffusers, MLX, MLX-VLM, and many more. Hardware acceleration is available for **NVIDIA** (CUDA 12/13), **AMD** (ROCm), **Intel** (oneAPI/SYCL), **Apple Silicon** (Metal), **Vulkan**, and **NVIDIA Jetson** (L4T). All backends can be installed on-the-fly from the [Backend Gallery](https://localai.io/backends/).
224
224
225
225
See the full [Backend & Model Compatibility Table](https://localai.io/model-compatibility/) and [GPU Acceleration guide](https://localai.io/features/gpu-acceleration/).
226
226
227
+
### Backends built by us
228
+
229
+
Most backends wrap a best-in-class upstream engine. A handful of them are native C/C++/GGML engines (no Python at inference) developed and maintained by the LocalAI project itself:
230
+
231
+
| Backend | What it does |
232
+
|---------|-------------|
233
+
|[parakeet.cpp](https://github.com/mudler/parakeet.cpp)| C++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription |
234
+
|[voxtral.c](https://github.com/mudler/voxtral.c)| Voxtral Realtime 4B speech-to-text in pure C |
235
+
|[vibevoice.cpp](https://github.com/mudler/vibevoice.cpp)| Native port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization |
236
+
|[rf-detr.cpp](https://github.com/mudler/rf-detr.cpp)| Native RF-DETR object detection and instance segmentation |
237
+
|[locate-anything.cpp](https://github.com/mudler/locate-anything.cpp)| Open-vocabulary object detection and visual grounding (LocateAnything-3B) |
Copy file name to clipboardExpand all lines: docs/content/features/backends.md
+16-5Lines changed: 16 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,6 +8,12 @@ url: "/backends/"
8
8
9
9
LocalAI supports a variety of backends that can be used to run different types of AI models. There are core Backends which are included, and there are containerized applications that provide the runtime environment for specific model types, such as LLMs, diffusion models, or text-to-speech models.
10
10
11
+
## Available Backends
12
+
13
+
LocalAI ships **60+ backends** covering text generation, speech-to-text, text-to-speech, music and sound generation, image and video generation, vision and object detection, audio processing, reranking, fine-tuning, and more. Each one is published as an on-demand OCI image with the appropriate acceleration variants (CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T).
14
+
15
+
For the complete list of backends, the model families they support, and their acceleration targets, see the [Backend & Model Compatibility Table]({{%relref "reference/compatibility-table" %}}). The authoritative source is [`backend/index.yaml`](https://github.com/mudler/LocalAI/blob/master/backend/index.yaml), and the same catalog is browsable in the web UI under the **Backends** section.
16
+
11
17
## Managing Backends in the UI
12
18
13
19
The LocalAI web interface provides an intuitive way to manage your backends:
@@ -118,8 +124,13 @@ For getting started, see the available backends in LocalAI here: https://github.
118
124
119
125
LocalAI supports various types of backends:
120
126
121
-
- **LLM Backends**: For running language models
122
-
- **Diffusion Backends**: For image generation
123
-
- **TTS Backends**: For text-to-speech conversion
124
-
- **Whisper Backends**: For speech-to-text conversion
125
-
- **Sound Generation Backends**: For music and audio generation (e.g., ACE-Step)
127
+
- **LLM Backends**: For running language models (e.g., llama.cpp, vLLM, SGLang, transformers, MLX)
128
+
- **Speech-to-Text Backends**: For transcription (e.g., whisper.cpp, parakeet.cpp, faster-whisper, NeMo)
0 commit comments