Feature: Add FunASR as a self-hosted ASR backend

[LauraGPT]

Implementation status

PR #10090 adds a local FunASR backend to LocalAI's existing audio-transcription API.

The current backend:

• loads iic/SenseVoiceSmall through FunASR;
• provides explicit CPU, CUDA 11, CUDA 12, and Intel profiles;
• includes Torch and Torchaudio in profiles that install Torch;
• accepts LocalAI's language and transcription request options;
• strips SenseVoice language, emotion, and audio-event markup before returning OpenAI-compatible transcript text;
• runs through LocalAI's existing backend install, test, gRPC LoadModel, and AudioTranscription paths.

SenseVoiceSmall supports Mandarin, Cantonese, English, Japanese, and Korean. Long-audio segmentation can be composed with the separate FSMN-VAD model. Speaker diarization is not a native SenseVoice output and is not claimed by this backend.

Validation

The PR has been exercised on ind-gpu8 with:

• six focused tests;
• real CPU and CUDA SenseVoiceSmall inference;
• the actual LocalAI installer and shared unit-test helper;
• gRPC model loading and transcription;
• shell syntax, Python compilation, and diff checks.

The PR is clean and mergeable; its remaining gate is maintainer review.

[mudler]

@LauraGPT yes that'd be great, happy to help along with the implementation

[LauraGPT]

Thanks @mudler! Here's what I'm thinking for implementation:

Integration approach

Since LocalAI already has the qwen-asr importer pattern, FunASR could follow the same approach:

Option A: Gallery model + Python backend (simplest)

• A funasr.go importer that matches HF repos like FunAudioLLM/SenseVoiceSmall, FunAudioLLM/Fun-ASR-Nano-2512
• Backend uses the transformers pipeline or funasr Python package directly
• Exposes the standard /v1/audio/transcriptions endpoint

Option B: External backend (zero Go code)

• Document running funasr-server --device cuda as an external service
• LocalAI proxies to it via the existing OpenAI-compatible external backend mechanism

I'd recommend Option A since it keeps everything self-contained in LocalAI. The gallery entry would look something like:

name: sensevoice-small
description: "FunASR SenseVoice — 170x realtime ASR + emotion detection (50+ languages)"
backend: funasr
urls:
  - filename: model
    uri: huggingface://FunAudioLLM/SenseVoiceSmall
config_file:
  backend: funasr
  parameters:
    model: iic/SenseVoiceSmall

Key models to support:

• SenseVoice-Small (234M) — fastest, emotion detection, 5 languages
• Paraformer-zh (220M) — best Chinese accuracy
• Fun-ASR-Nano (800M) — highest accuracy, 31 languages (needs more VRAM)

I can start working on a PR. Would you prefer I target the importers/ pattern, or is there a different integration point you'd recommend?

[LauraGPT]

@mudler PR is up: #10088

Implements the FunASR backend following the existing qwen-asr pattern:

• Gallery importer (funasr.go): auto-detects FunAudioLLM/SenseVoiceSmall, FunAudioLLM/Fun-ASR-Nano-2512, funasr/paraformer-zh from HuggingFace
• Python gRPC backend: wraps funasr.AutoModel with VAD + speaker diarization
• Backend index entries: all platform variants (cpu/cuda12/cuda13/intel/rocm/metal/l4t)

Happy to iterate on any feedback.