Skip to content

Commit 9b57dcb

Browse files
localai-botmudler
andauthored
docs: document all available backends and add "built by us" list (#10376)
Bring the Backend & Model Compatibility Table up to the full set of backends published in backend/index.yaml (60+), organized by modality with per-backend acceleration targets. Add an "Available Backends" pointer and expand the backend-type list in the backends feature doc. Update the README backend count to 60+ and add a "Backends built by us" section listing the native C/C++/GGML engines maintained by the LocalAI project (parakeet.cpp, voxtral.c, vibevoice.cpp, rf-detr.cpp, locate-anything.cpp, depth-anything.cpp, LocalVQE, local-store). Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
1 parent 95e7149 commit 9b57dcb

3 files changed

Lines changed: 98 additions & 40 deletions

File tree

README.md

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -220,10 +220,25 @@ For older news and full release notes, see [GitHub Releases](https://github.com/
220220

221221
## Supported Backends & Acceleration
222222

223-
LocalAI supports **36+ backends** including llama.cpp, vLLM, transformers, whisper.cpp, diffusers, MLX, MLX-VLM, and many more. Hardware acceleration is available for **NVIDIA** (CUDA 12/13), **AMD** (ROCm), **Intel** (oneAPI/SYCL), **Apple Silicon** (Metal), **Vulkan**, and **NVIDIA Jetson** (L4T). All backends can be installed on-the-fly from the [Backend Gallery](https://localai.io/backends/).
223+
LocalAI supports **60+ backends** including llama.cpp, vLLM, SGLang, transformers, whisper.cpp, diffusers, MLX, MLX-VLM, and many more. Hardware acceleration is available for **NVIDIA** (CUDA 12/13), **AMD** (ROCm), **Intel** (oneAPI/SYCL), **Apple Silicon** (Metal), **Vulkan**, and **NVIDIA Jetson** (L4T). All backends can be installed on-the-fly from the [Backend Gallery](https://localai.io/backends/).
224224

225225
See the full [Backend & Model Compatibility Table](https://localai.io/model-compatibility/) and [GPU Acceleration guide](https://localai.io/features/gpu-acceleration/).
226226

227+
### Backends built by us
228+
229+
Most backends wrap a best-in-class upstream engine. A handful of them are native C/C++/GGML engines (no Python at inference) developed and maintained by the LocalAI project itself:
230+
231+
| Backend | What it does |
232+
|---------|-------------|
233+
| [parakeet.cpp](https://github.com/mudler/parakeet.cpp) | C++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription |
234+
| [voxtral.c](https://github.com/mudler/voxtral.c) | Voxtral Realtime 4B speech-to-text in pure C |
235+
| [vibevoice.cpp](https://github.com/mudler/vibevoice.cpp) | Native port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization |
236+
| [rf-detr.cpp](https://github.com/mudler/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation |
237+
| [locate-anything.cpp](https://github.com/mudler/locate-anything.cpp) | Open-vocabulary object detection and visual grounding (LocateAnything-3B) |
238+
| [depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) | Depth Anything 3 monocular metric depth + camera pose estimation |
239+
| [LocalVQE](https://github.com/localai-org/LocalVQE) | Joint acoustic echo cancellation, noise suppression, and dereverberation |
240+
| [local-store](https://github.com/mudler/LocalAI) | Local-first vector database for embeddings (shipped in-tree) |
241+
227242
## Resources
228243

229244
- [Documentation](https://localai.io/)

docs/content/features/backends.md

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@ url: "/backends/"
88

99
LocalAI supports a variety of backends that can be used to run different types of AI models. There are core Backends which are included, and there are containerized applications that provide the runtime environment for specific model types, such as LLMs, diffusion models, or text-to-speech models.
1010

11+
## Available Backends
12+
13+
LocalAI ships **60+ backends** covering text generation, speech-to-text, text-to-speech, music and sound generation, image and video generation, vision and object detection, audio processing, reranking, fine-tuning, and more. Each one is published as an on-demand OCI image with the appropriate acceleration variants (CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T).
14+
15+
For the complete list of backends, the model families they support, and their acceleration targets, see the [Backend & Model Compatibility Table]({{%relref "reference/compatibility-table" %}}). The authoritative source is [`backend/index.yaml`](https://github.com/mudler/LocalAI/blob/master/backend/index.yaml), and the same catalog is browsable in the web UI under the **Backends** section.
16+
1117
## Managing Backends in the UI
1218

1319
The LocalAI web interface provides an intuitive way to manage your backends:
@@ -118,8 +124,13 @@ For getting started, see the available backends in LocalAI here: https://github.
118124

119125
LocalAI supports various types of backends:
120126

121-
- **LLM Backends**: For running language models
122-
- **Diffusion Backends**: For image generation
123-
- **TTS Backends**: For text-to-speech conversion
124-
- **Whisper Backends**: For speech-to-text conversion
125-
- **Sound Generation Backends**: For music and audio generation (e.g., ACE-Step)
127+
- **LLM Backends**: For running language models (e.g., llama.cpp, vLLM, SGLang, transformers, MLX)
128+
- **Speech-to-Text Backends**: For transcription (e.g., whisper.cpp, parakeet.cpp, faster-whisper, NeMo)
129+
- **Text-to-Speech Backends**: For speech synthesis (e.g., piper, Kokoro, VibeVoice, Qwen3-TTS)
130+
- **Sound Generation Backends**: For music and audio generation (e.g., ACE-Step)
131+
- **Image & Video Generation Backends**: For diffusion models (e.g., stable-diffusion.cpp, diffusers)
132+
- **Vision & Detection Backends**: For object detection, segmentation, depth, and face/voice recognition (e.g., rf-detr.cpp, locate-anything.cpp, sam3.cpp, insightface)
133+
- **Audio Processing Backends**: For voice activity detection and audio enhancement (e.g., Silero VAD, LocalVQE)
134+
- **Utility Backends**: For reranking, fine-tuning, quantization, and vector storage (e.g., rerankers, TRL, local-store)
135+
136+
See the [Backend & Model Compatibility Table]({{%relref "reference/compatibility-table" %}}) for the full catalog.

0 commit comments

Comments
 (0)