diff --git a/.github/bump_deps.sh b/.github/bump_deps.sh index 28485ca922bd..5a595f156d61 100755 --- a/.github/bump_deps.sh +++ b/.github/bump_deps.sh @@ -9,7 +9,19 @@ if [ -z "$FILE" ]; then FILE="Makefile" fi -LAST_COMMIT=$(curl -s -H "Accept: application/vnd.github.VERSION.sha" "https://api.github.com/repos/$REPO/commits/$BRANCH") +# -L so a renamed/transferred upstream repo (GitHub answers 301) still +# resolves instead of handing us the redirect body, and -f so an HTTP error +# aborts the run rather than letting an error page reach sed below. +LAST_COMMIT=$(curl -sfL -H "Accept: application/vnd.github.VERSION.sha" "https://api.github.com/repos/$REPO/commits/$BRANCH") + +# Guard the sed input: anything that is not a bare 40-hex SHA (an API error +# body, an empty response) would otherwise be spliced into the Makefile pin — +# either corrupting it silently or blowing up sed with an unterminated +# expression, which is how this job failed for a renamed repo. +if ! [[ "$LAST_COMMIT" =~ ^[0-9a-f]{40}$ ]]; then + echo "Refusing to bump $VAR: expected a 40-char commit SHA for $REPO@$BRANCH, got: $LAST_COMMIT" >&2 + exit 1 +fi # Read $VAR from Makefile (only first match) set +e diff --git a/.github/workflows/bump_deps.yaml b/.github/workflows/bump_deps.yaml index c8daea24fbc1..80bf23ccd1f5 100644 --- a/.github/workflows/bump_deps.yaml +++ b/.github/workflows/bump_deps.yaml @@ -50,15 +50,15 @@ jobs: variable: "PARAKEET_VERSION" branch: "master" file: "backend/go/parakeet-cpp/Makefile" - - repository: "mudler/moss-transcribe.cpp" + - repository: "localai-org/moss-transcribe.cpp" variable: "MOSS_VERSION" branch: "master" file: "backend/go/moss-transcribe-cpp/Makefile" - - repository: "mudler/ced.cpp" + - repository: "localai-org/ced.cpp" variable: "CED_VERSION" - branch: "master" + branch: "main" file: "backend/go/ced/Makefile" - - repository: "mudler/voice-detect.cpp" + - repository: "localai-org/voice-detect.cpp" variable: "VOICEDETECT_VERSION" branch: "master" file: "backend/go/voice-detect/Makefile" @@ -90,7 +90,7 @@ jobs: variable: "SAM3_VERSION" branch: "main" file: "backend/go/sam3-cpp/Makefile" - - repository: "mudler/rf-detr.cpp" + - repository: "localai-org/rf-detr.cpp" variable: "RFDETR_VERSION" branch: "main" file: "backend/go/rfdetr-cpp/Makefile" diff --git a/README.md b/README.md index a81a7ddf19f2..5c356d0fc5a2 100644 --- a/README.md +++ b/README.md @@ -177,7 +177,7 @@ For more details, see the [Getting Started guide](https://localai.io/basics/gett ## Latest News -- **June 2026**: New native biometric backends from the LocalAI team: [voice-detect.cpp](https://github.com/mudler/voice-detect.cpp) for speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion) and [face-detect.cpp](https://github.com/mudler/face-detect.cpp) for face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace). Both are from-scratch C++/ggml engines with no Python or onnxruntime at inference, self-contained GGUF weights, bit-exact parity with the reference, and GPU cuDNN parity, replacing the heavier Python `insightface` and `speaker-recognition` backends ([PR #10441](https://github.com/mudler/LocalAI/pull/10441)). +- **June 2026**: New native biometric backends from the LocalAI team: [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) for speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion) and [face-detect.cpp](https://github.com/mudler/face-detect.cpp) for face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace). Both are from-scratch C++/ggml engines with no Python or onnxruntime at inference, self-contained GGUF weights, bit-exact parity with the reference, and GPU cuDNN parity, replacing the heavier Python `insightface` and `speaker-recognition` backends ([PR #10441](https://github.com/mudler/LocalAI/pull/10441)). - **June 2026**: New [realtime voice assistant demo](https://github.com/localai-org/localai-realtime-demo) (a tiny Go client for the Realtime API with a full talk-back voice loop and tool calling), plus [streaming of the realtime LLM / TTS / transcription pipeline stages](https://github.com/mudler/LocalAI/pull/10176) and [configurable WebRTC ICE candidates](https://github.com/mudler/LocalAI/pull/10231). - **June 2026**: Big speech push: the [parakeet.cpp](https://github.com/mudler/parakeet.cpp) ASR engine gains [NeMo-faithful segment timestamps](https://github.com/mudler/LocalAI/pull/10207), a [multilingual streaming Nemotron-3.5 model](https://github.com/mudler/LocalAI/pull/10199), [dynamic batching for concurrent transcription](https://github.com/mudler/LocalAI/pull/10112) and [CUDA graphs](https://github.com/mudler/LocalAI/pull/10273); the new [CrispASR backend](https://github.com/mudler/LocalAI/pull/10099) adds multi-architecture ASR + TTS, and [60 Piper TTS voices across 42 languages](https://github.com/mudler/LocalAI/pull/10296) land in the gallery (plus [per-request TTS instructions and params](https://github.com/mudler/LocalAI/pull/10172)). - **June 2026**: New backends and models: [locate-anything.cpp](https://github.com/mudler/LocalAI/pull/10264) for open-vocabulary object detection via ggml, [Ideogram4 image generation](https://github.com/mudler/LocalAI/pull/10201) in stablediffusion-ggml, [llama.cpp video input](https://github.com/mudler/LocalAI/pull/10216), and the [Gemma 4 QAT family with MTP speculative-decoding pairs](https://github.com/mudler/LocalAI/pull/10215). Plus an [interactive CLI chat mode](https://github.com/mudler/LocalAI/pull/10226) and [RAG source citations in agent responses](https://github.com/mudler/LocalAI/pull/10228). @@ -232,13 +232,13 @@ Most backends wrap a best-in-class upstream engine. A handful of them are native | Backend | What it does | |---------|-------------| | [parakeet.cpp](https://github.com/mudler/parakeet.cpp) | C++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription | -| [moss-transcribe.cpp](https://github.com/mudler/moss-transcribe.cpp) | C++/GGML port of OpenMOSS MOSS-Transcribe-Diarize: joint long-form transcription, speaker diarization and timestamping in a single pass | +| [moss-transcribe.cpp](https://github.com/localai-org/moss-transcribe.cpp) | C++/GGML port of OpenMOSS MOSS-Transcribe-Diarize: joint long-form transcription, speaker diarization and timestamping in a single pass | | [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) | C++/GGML port of the OpenMOSS MOSS-TTS family: text-to-speech (MOSS-TTS-Local v1.5, 48 kHz stereo) with reference-audio voice cloning, through the MOSS-Audio-Tokenizer neural codec | -| [ced.cpp](https://github.com/mudler/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition | -| [voice-detect.cpp](https://github.com/mudler/voice-detect.cpp) | Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend | +| [ced.cpp](https://github.com/localai-org/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition | +| [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) | Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend | | [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Voxtral Realtime 4B speech-to-text in pure C | | [vibevoice.cpp](https://github.com/mudler/vibevoice.cpp) | Native port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization | -| [rf-detr.cpp](https://github.com/mudler/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation | +| [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation | | [locate-anything.cpp](https://github.com/mudler/locate-anything.cpp) | Open-vocabulary object detection and visual grounding (LocateAnything-3B) | | [depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) | Depth Anything 3 monocular metric depth + camera pose estimation | | [face-detect.cpp](https://github.com/mudler/face-detect.cpp) | Face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace), replacing the Python insightface backend | diff --git a/backend/go/ced/Makefile b/backend/go/ced/Makefile index 2b15990ec2f1..bdd5e5dfeea4 100644 --- a/backend/go/ced/Makefile +++ b/backend/go/ced/Makefile @@ -10,7 +10,7 @@ # go build -o ced-grpc . CED_VERSION?=c04ac14b7992d00584d9e812c9bb6268598a6ce7 -CED_REPO?=https://github.com/mudler/ced.cpp +CED_REPO?=https://github.com/localai-org/ced.cpp GOCMD?=go GO_TAGS?= diff --git a/backend/go/depth-anything-cpp/test.sh b/backend/go/depth-anything-cpp/test.sh index f3dbcdeca556..1117f706629f 100755 --- a/backend/go/depth-anything-cpp/test.sh +++ b/backend/go/depth-anything-cpp/test.sh @@ -25,7 +25,7 @@ fi # Depth estimation needs real content; a synthetic image would be degenerate. TEST_IMAGE_DIR="$CURDIR/test-data" TEST_IMAGE_FILE="$TEST_IMAGE_DIR/test.jpg" -TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/mudler/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}" +TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/localai-org/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}" mkdir -p "$TEST_IMAGE_DIR" if [ ! -f "$TEST_IMAGE_FILE" ]; then diff --git a/backend/go/locate-anything-cpp/test.sh b/backend/go/locate-anything-cpp/test.sh index 0a088a593f3e..c210ecd493e1 100755 --- a/backend/go/locate-anything-cpp/test.sh +++ b/backend/go/locate-anything-cpp/test.sh @@ -27,7 +27,7 @@ fi # synthetic image would trivially yield zero detections. TEST_IMAGE_DIR="$CURDIR/test-data" TEST_IMAGE_FILE="$TEST_IMAGE_DIR/test.jpg" -TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/mudler/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}" +TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/localai-org/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}" mkdir -p "$TEST_IMAGE_DIR" if [ ! -f "$TEST_IMAGE_FILE" ]; then diff --git a/backend/go/moss-transcribe-cpp/Makefile b/backend/go/moss-transcribe-cpp/Makefile index 98f20ddb79d5..24b5ff071c9a 100644 --- a/backend/go/moss-transcribe-cpp/Makefile +++ b/backend/go/moss-transcribe-cpp/Makefile @@ -16,7 +16,7 @@ # clone-at-pin + cmake build so CI doesn't need a side-checkout. MOSS_VERSION?=190a569c13b4b247450f2fb3b2a431244e84833e -MOSS_REPO?=https://github.com/mudler/moss-transcribe.cpp +MOSS_REPO?=https://github.com/localai-org/moss-transcribe.cpp GOCMD?=go GO_TAGS?= diff --git a/backend/go/rfdetr-cpp/Makefile b/backend/go/rfdetr-cpp/Makefile index 448a8e78b9f1..29f2842eb98d 100644 --- a/backend/go/rfdetr-cpp/Makefile +++ b/backend/go/rfdetr-cpp/Makefile @@ -7,10 +7,10 @@ GO_TAGS?= JOBS?=$(shell nproc --ignore=1) # rt-detr.cpp (GitHub redirects the historical mudler/rt-detr.cpp to the new -# mudler/rf-detr.cpp slug). Pin to a specific commit if you need a stable +# localai-org/rf-detr.cpp slug). Pin to a specific commit if you need a stable # build; leaving this on `master` always picks up the latest C-API surface # (incl. the per-detection accessor functions used by gorfdetrcpp.go). -RFDETR_REPO?=https://github.com/mudler/rf-detr.cpp.git +RFDETR_REPO?=https://github.com/localai-org/rf-detr.cpp.git RFDETR_VERSION?=65c0ffcc9a9bc9dae38252f63d0417c9845a6cf7 ifeq ($(NATIVE),false) diff --git a/backend/go/rfdetr-cpp/test.sh b/backend/go/rfdetr-cpp/test.sh index dd30412005cb..e6d184214adb 100755 --- a/backend/go/rfdetr-cpp/test.sh +++ b/backend/go/rfdetr-cpp/test.sh @@ -34,7 +34,7 @@ fi # detections. TEST_IMAGE_DIR="$CURDIR/test-data" TEST_IMAGE_FILE="$TEST_IMAGE_DIR/test.jpg" -TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/mudler/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}" +TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/localai-org/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}" mkdir -p "$TEST_IMAGE_DIR" if [ ! -f "$TEST_IMAGE_FILE" ]; then diff --git a/backend/go/voice-detect/Makefile b/backend/go/voice-detect/Makefile index b2e1cfac98cb..064c4b5a007d 100644 --- a/backend/go/voice-detect/Makefile +++ b/backend/go/voice-detect/Makefile @@ -14,7 +14,7 @@ # not need a side-checkout. VOICEDETECT_VERSION?=1db1759572c90faef6f3a78c36b5941a096a9f89 -VOICEDETECT_REPO?=https://github.com/mudler/voice-detect.cpp +VOICEDETECT_REPO?=https://github.com/localai-org/voice-detect.cpp GOCMD?=go GO_TAGS?= diff --git a/backend/index.yaml b/backend/index.yaml index 86246cccbb78..5d6eb8d50f08 100644 --- a/backend/index.yaml +++ b/backend/index.yaml @@ -222,7 +222,7 @@ It does joint long-form transcription, speaker diarization and timestamping in a single pass, emitting a speaker-labelled, time-aligned transcript, and runs on CPU, NVIDIA CUDA, AMD ROCm/HIP, Intel SYCL, Vulkan, Apple Metal and NVIDIA Jetson (L4T) targets. urls: - - https://github.com/mudler/moss-transcribe.cpp + - https://github.com/localai-org/moss-transcribe.cpp tags: - audio-transcription - CPU @@ -253,7 +253,7 @@ /v1/audio/classification REST endpoint. It runs on CPU, NVIDIA CUDA, AMD ROCm/HIP, Intel SYCL, Vulkan and NVIDIA Jetson (L4T) targets. urls: - - https://github.com/mudler/ced.cpp + - https://github.com/localai-org/ced.cpp tags: - audio-classification - CPU @@ -287,7 +287,7 @@ gRPC rpcs and the /v1/voice/* REST endpoints. It runs on CPU, NVIDIA CUDA, AMD ROCm/HIP, Intel SYCL, Vulkan and NVIDIA Jetson (L4T) targets. urls: - - https://github.com/mudler/voice-detect.cpp + - https://github.com/localai-org/voice-detect.cpp tags: - voice-recognition - speaker-verification @@ -487,7 +487,7 @@ and returns bounding boxes, class labels, confidence scores, and (for segmentation variants) PNG-encoded per-detection masks. urls: - - https://github.com/mudler/rf-detr.cpp + - https://github.com/localai-org/rf-detr.cpp tags: - object-detection - image-segmentation diff --git a/docs/content/features/audio-classification.md b/docs/content/features/audio-classification.md index debb182fe83c..0450743406f6 100644 --- a/docs/content/features/audio-classification.md +++ b/docs/content/features/audio-classification.md @@ -7,7 +7,7 @@ url = "/features/audio-classification/" Sound-event classification (audio tagging) answers the question **"what am I hearing?"** - given an audio clip, it returns a list of scored [AudioSet](https://research.google.com/audioset/) labels (e.g. *Baby cry, infant cry*, *Glass breaking*, *Dog bark*, *Alarm*). -LocalAI exposes this through the `/v1/audio/classification` endpoint, modelled after `/v1/audio/transcriptions`. The reference backend is **[ced.cpp](https://github.com/mudler/ced.cpp)** (CED, a 527-class AudioSet tagger), a small ViT over a log-mel spectrogram ported to ggml with full PyTorch parity. Apache-2.0 weights are redistributable as GGUF. +LocalAI exposes this through the `/v1/audio/classification` endpoint, modelled after `/v1/audio/transcriptions`. The reference backend is **[ced.cpp](https://github.com/localai-org/ced.cpp)** (CED, a 527-class AudioSet tagger), a small ViT over a log-mel spectrogram ported to ggml with full PyTorch parity. Apache-2.0 weights are redistributable as GGUF. Because classification is exposed as a regular OpenAI-style endpoint, any HTTP client works - there is no Python dependency on the consumer side. diff --git a/docs/content/features/object-detection.md b/docs/content/features/object-detection.md index 0a7c4ae49049..27b7258f41c8 100644 --- a/docs/content/features/object-detection.md +++ b/docs/content/features/object-detection.md @@ -5,7 +5,7 @@ weight = 41 url = "/features/object-detection/" +++ -LocalAI supports object detection and image segmentation through various backends. This feature allows you to identify and locate objects within images with high accuracy and real-time performance. Available backends include [RF-DETR](https://github.com/roboflow/rf-detr) (Python) and [rf-detr.cpp](https://github.com/mudler/rf-detr.cpp) (native C++/ggml) for object detection and segmentation, and [sam3.cpp](https://github.com/PABannier/sam3.cpp) for image segmentation (SAM 3/2/EdgeTAM). +LocalAI supports object detection and image segmentation through various backends. This feature allows you to identify and locate objects within images with high accuracy and real-time performance. Available backends include [RF-DETR](https://github.com/roboflow/rf-detr) (Python) and [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp) (native C++/ggml) for object detection and segmentation, and [sam3.cpp](https://github.com/PABannier/sam3.cpp) for image segmentation (SAM 3/2/EdgeTAM). For detecting **faces** specifically, see the dedicated [Face Recognition](/features/face-recognition/) feature - its @@ -138,7 +138,7 @@ You can browse and install this model through the LocalAI web interface or using ### RF-DETR Native Backend (rfdetr-cpp) The `rfdetr-cpp` backend is a native C++/ggml implementation of RF-DETR -inference based on [rf-detr.cpp](https://github.com/mudler/rf-detr.cpp). It +inference based on [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp). It runs as a Go gRPC service that dlopens a per-CPU-variant shared library, so there is no Python runtime on the inference path - startup is fast and the binary is self-contained. diff --git a/docs/content/features/voice-recognition.md b/docs/content/features/voice-recognition.md index 9317c297278b..23bb16bd67e8 100644 --- a/docs/content/features/voice-recognition.md +++ b/docs/content/features/voice-recognition.md @@ -16,7 +16,7 @@ The audio analog to [Face Recognition](/features/face-recognition/), served over the same `/v1/voice/*` HTTP API by two backends: - **`voice-detect` (recommended, default).** A standalone C++/ggml - engine ([voice-detect.cpp](https://github.com/mudler/voice-detect.cpp)): + engine ([voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp)): no Python, no onnxruntime, no torch runtime. Each gallery entry is a single self-describing GGUF. This is the recommended option for new deployments. diff --git a/docs/content/reference/compatibility-table.md b/docs/content/reference/compatibility-table.md index 6072931f5a98..e7e9a2c7d60c 100644 --- a/docs/content/reference/compatibility-table.md +++ b/docs/content/reference/compatibility-table.md @@ -94,12 +94,12 @@ All backends listed here can be installed on demand from the [Backend Gallery]({ | Backend | Description | Acceleration | |---------|-------------|-------------| | [RF-DETR](https://github.com/roboflow/rf-detr) | Real-time transformer-based object detection (Python) | CPU, CUDA 12/13, Intel SYCL, Metal, Jetson L4T | -| [rf-detr.cpp](https://github.com/mudler/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation in C/C++ using GGML | CPU, CUDA 12/13, Intel SYCL, Vulkan, Jetson L4T | +| [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation in C/C++ using GGML | CPU, CUDA 12/13, Intel SYCL, Vulkan, Jetson L4T | | [locate-anything.cpp](https://github.com/mudler/locate-anything.cpp) | Open-vocabulary object detection and visual grounding (LocateAnything-3B) in C/C++ using GGML | CPU, CUDA 12/13, Intel SYCL, Vulkan, Jetson L4T | | [depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) | Depth Anything 3 monocular metric depth + camera pose in C/C++ using GGML | CPU, CUDA 12/13, Intel SYCL, Vulkan, Jetson L4T | | [sam3.cpp](https://github.com/PABannier/sam3.cpp) | Segment Anything (SAM 3/2/EdgeTAM) with text/point/box prompts in C/C++ using GGML | CPU, CUDA 12/13, Intel SYCL, Vulkan, Jetson L4T | | [face-detect.cpp](https://github.com/mudler/face-detect.cpp) | Native face detection, recognition, embedding, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace) in C/C++ using GGML | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T | -| [voice-detect.cpp](https://github.com/mudler/voice-detect.cpp) | Native speaker (voice) recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2) in C/C++ using GGML | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T | +| [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) | Native speaker (voice) recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2) in C/C++ using GGML | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T | | [insightface](https://github.com/deepinsight/insightface) | Face verification, embedding, and anti-spoofing liveness (ONNX Runtime) | CPU, CUDA 12 | | [speaker-recognition](https://speechbrain.github.io/) | Speaker (voice) recognition via SpeechBrain ECAPA-TDNN | CPU, CUDA 12, Metal | diff --git a/docs/content/whats-new.md b/docs/content/whats-new.md index a922f2348d86..a947642d0bb7 100644 --- a/docs/content/whats-new.md +++ b/docs/content/whats-new.md @@ -16,7 +16,7 @@ You can see the release notes [here](https://github.com/mudler/LocalAI/releases) - **April 2026**: [Audio Transform](/features/audio-transform/) - generic audio-in / audio-out endpoint with optional reference signal. First implementation: [LocalVQE](https://github.com/localai-org/LocalVQE) C++ backend (joint AEC + noise suppression + dereverberation, DeepVQE-style). Both batch (`POST /audio/transformations`) and bidirectional WebSocket streaming (`/audio/transformations/stream`). Studio "Transform" tab with synchronized waveform players for input / reference / output. - **April 2026**: [Face recognition backend](/features/face-recognition/) - `insightface`-powered 1:1 verification, 1:N identification, face embedding, face detection, and demographic analysis. Ships both a non-commercial `buffalo_l` model and an Apache 2.0 OpenCV Zoo alternative. - **May 2026**: [Speaker diarization](/features/audio-diarization/) - new `/v1/audio/diarization` endpoint returning "who spoke when" segments. Backed by `sherpa-onnx` (pyannote-3.0 + speaker embeddings + clustering) for pure diarization, and `vibevoice-cpp` for diarization bundled with long-form ASR. Supports `json` / `verbose_json` / `rttm` response formats. -- **June 2026**: [Sound classification](/features/audio-classification/) - new `/v1/audio/classification` endpoint for audio tagging / sound-event classification, returning scored [AudioSet](https://research.google.com/audioset/) labels (baby cry, glass breaking, alarms, ...). Backed by [ced.cpp](https://github.com/mudler/ced.cpp), a 527-class AudioSet tagger ported to ggml. +- **June 2026**: [Sound classification](/features/audio-classification/) - new `/v1/audio/classification` endpoint for audio tagging / sound-event classification, returning scored [AudioSet](https://research.google.com/audioset/) labels (baby cry, glass breaking, alarms, ...). Backed by [ced.cpp](https://github.com/localai-org/ced.cpp), a 527-class AudioSet tagger ported to ggml. - **June 2026**: [PII analyze / redact API](/features/middleware/#analyze--redact-api) - the PII detection pipeline (NER + restricted-regex pattern tiers) is now a standalone service: `POST /api/pii/analyze` returns detected entity spans and `POST /api/pii/redact` returns the sanitised text (or `400 pii_blocked`), without routing a chat request through the middleware. Events gain an `origin` (`middleware` / `proxy` / `pii_analyze` / `pii_redact`) so `/api/pii/events` can be filtered by source. - **July 2026**: [Model capabilities endpoint](/features/api-discovery/#model-capabilities) - `GET /v1/models/capabilities`, an additive superset of `/v1/models` that reports each model's `capabilities` plus its `input_modalities` / `output_modalities` (`text` / `image` / `audio` / `video`). Lets clients route attachments using inferred or explicitly declared model modalities instead of backend-name checks. - **June 2026**: Concurrent scoring and PII NER on llama.cpp - the `Score` (router classifier) and `TokenClassify` (PII NER) primitives now ride llama.cpp's server task queue instead of locking the context, so they run concurrently with chat/completion/embedding traffic and with each other. The `known_usecases` restriction that forced dedicated scorer/NER model configs on llama-cpp is lifted, repeated scoring calls reuse the prompt KV cache across candidates, and scoring inputs are no longer capped by the physical batch size.