Skip to content

Commit 5990412

Browse files
authored
docs: showcase verified community integrations (#3231)
* docs: showcase verified community integrations * docs: polish Chinese integration intro --------- Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
1 parent ea6ea1e commit 5990412

6 files changed

Lines changed: 68 additions & 36 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
</p>
2222

2323
<p align="center">
24-
<a href="#quick-start">Quick Start</a> · <a href="./examples/colab/">Colab</a> · <a href="#benchmark">Benchmark</a> · <a href="./docs/model_selection.md">Model selection</a> · <a href="./docs/migration_from_whisper.md">Migration guide</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="#model-zoo">Models</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent Integration</a> · <a href="https://modelscope.github.io/FunASR/">Docs</a> · <a href="./CONTRIBUTING.md">Contribute</a>
24+
<a href="#quick-start">Quick Start</a> · <a href="./examples/colab/">Colab</a> · <a href="#benchmark">Benchmark</a> · <a href="./docs/model_selection.md">Model selection</a> · <a href="./docs/migration_from_whisper.md">Migration guide</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/community_projects.md">Community integrations</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="#model-zoo">Models</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent Integration</a> · <a href="https://modelscope.github.io/FunASR/">Docs</a> · <a href="./CONTRIBUTING.md">Contribute</a>
2525
</p>
2626

2727
---

README_zh.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
</p>
2222

2323
<p align="center">
24-
<a href="#快速开始">快速开始</a> · <a href="./examples/colab/README_zh.md">Colab</a> · <a href="#性能评测">性能评测</a> · <a href="./docs/model_selection_zh.md">模型选择</a> · <a href="./docs/migration_from_whisper_zh.md">迁移指南</a> · <a href="./docs/use_case_showcase_zh.md">场景速览</a> · <a href="./docs/deployment_matrix_zh.md">部署选型</a> · <a href="#模型列表">模型列表</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent 集成</a> · <a href="https://modelscope.github.io/FunASR/zh/">文档</a> · <a href="./CONTRIBUTING.md">贡献</a>
24+
<a href="#快速开始">快速开始</a> · <a href="./examples/colab/README_zh.md">Colab</a> · <a href="#性能评测">性能评测</a> · <a href="./docs/model_selection_zh.md">模型选择</a> · <a href="./docs/migration_from_whisper_zh.md">迁移指南</a> · <a href="./docs/use_case_showcase_zh.md">场景速览</a> · <a href="./docs/community_projects_zh.md">社区集成</a> · <a href="./docs/deployment_matrix_zh.md">部署选型</a> · <a href="#模型列表">模型列表</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent 集成</a> · <a href="https://modelscope.github.io/FunASR/zh/">文档</a> · <a href="./CONTRIBUTING.md">贡献</a>
2525
</p>
2626

2727
---
@@ -288,7 +288,7 @@ docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-
288288
| 💬 [讨论](https://github.com/modelscope/FunASR/discussions) | 🤗 [HuggingFace](https://huggingface.co/funasr) |
289289
| 🤝 [贡献指南](./CONTRIBUTING.md) | 📈 [20k 增长计划](./docs/community_growth_20k.md) |
290290
| 🗺️ [仓库职责与路线图](./docs/repository_roles_zh.md) | 🌐 [funasr.com](https://www.funasr.com) |
291-
| 🧩 [社区项目](./docs/community_projects.md) | 💡 [使用案例](./docs/use_case_showcase.md) |
291+
| 🧩 [社区集成](./docs/community_projects_zh.md) | 💡 [使用案例](./docs/use_case_showcase_zh.md) |
292292

293293
## Star 趋势
294294

docs/community_projects.md

Lines changed: 22 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,45 +1,34 @@
1-
# FunASR Community Projects
1+
[English](./community_projects.md) | [简体中文](./community_projects_zh.md)
22

3-
This page collects maintained community integrations and experiments around FunASR models. These projects are useful for users exploring deployment paths beyond the official Python runtime, but they are not official FunASR implementations unless explicitly noted.
3+
# FunASR Community Integrations
44

5-
## Applications and workflows
5+
This page lists maintained projects where FunASR, Fun-ASR-Nano, SenseVoice, or an official FunASR model is already integrated upstream. Each entry below was checked against the project's default branch and, where available, a linked merged change.
66

7-
| Project | What it provides | Notes |
8-
|---|---|---|
9-
| [GPT-SoVITS](https://github.com/RVC-Boss/GPT-SoVITS) | Voice-dataset preparation and WebUI transcription with Fun-ASR-Nano, SenseVoice, and classic FunASR models. | The default CLI path uses Fun-ASR-Nano for Chinese, English, Japanese, Korean, and automatic language detection; Cantonese uses classic UniASR. Models download on first use. See the upstream [runtime fallback](https://github.com/RVC-Boss/GPT-SoVITS/pull/2801) and [backend documentation](https://github.com/RVC-Boss/GPT-SoVITS/pull/2803). |
10-
11-
Run dataset transcription from a GPT-SoVITS checkout:
12-
13-
```bash
14-
python tools/asr/funasr_asr.py -i <input> -o <output> -l zh
15-
```
7+
These integrations are community-maintained. Their release cadence, hardware support, and API stability are controlled by the upstream project, not by the FunASR maintainers.
168

17-
## VAD runtimes
9+
## Voice agents and applications
1810

19-
| Project | What it provides | Notes |
11+
| Project | What is integrated | Start here |
2012
|---|---|---|
21-
| [vad-burn](https://github.com/di-osc/vad-burn) | FSMN VAD inference in pure Rust with Python bindings. It supports offline VAD, streaming VAD, CPU-only inference, and automatic download of the default `iic/speech_fsmn_vad_zh-cn-16k-common-pytorch` model from ModelScope. | Community project from [#3106](https://github.com/modelscope/FunASR/issues/3106). Useful for Rust services, CLI tools, desktop apps, or Python pipelines that need FSMN VAD without depending on the original Python runtime. |
13+
| [Pipecat](https://github.com/pipecat-ai/pipecat) | A local `FunASRSTTService` backed by SenseVoice for voice and multimodal agent pipelines, with transcription and voice-agent examples. | [FunASR service docs](https://docs.pipecat.ai/api-reference/server/services/stt/funasr), [transcription example](https://github.com/pipecat-ai/pipecat/blob/main/examples/transcription/transcription-funasr.py), [voice-agent example](https://github.com/pipecat-ai/pipecat/blob/main/examples/voice/voice-funasr.py), and merged [#4844](https://github.com/pipecat-ai/pipecat/pull/4844). |
14+
| [xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) | The default local ASR provider uses FunASR with SenseVoiceSmall; recognition language can be set to `auto`, `zh`, `en`, `ja`, `ko`, or `yue`. A separate FunASR runtime-server provider is also available. | [Provider implementation](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/main/xiaozhi-server/core/providers/asr/fun_local.py), [configuration](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/main/xiaozhi-server/config.yaml), and merged [#3255](https://github.com/xinnan-tech/xiaozhi-esp32-server/pull/3255). |
15+
| [AutoSubs](https://github.com/tmoroney/auto-subs) | An on-device int8 ONNX SenseVoice engine for subtitle generation in DaVinci Resolve, Premiere, and After Effects. | [SenseVoice model notes](https://github.com/tmoroney/auto-subs#sensevoice), [engine source](https://github.com/tmoroney/auto-subs/blob/main/AutoSubs-App/src-tauri/crates/transcription-engine/src/engines/sense_voice.rs), and merged [#629](https://github.com/tmoroney/auto-subs/pull/629). |
16+
| [OmniVoice Studio](https://github.com/debpalash/OmniVoice-Studio) | Its OpenAI-compatible remote ASR backend can point dictation and dubbing workflows at a self-hosted FunASR or SenseVoice server. | [OpenAI-compatible ASR guide](https://github.com/debpalash/OmniVoice-Studio/blob/main/docs/engines/openai-compatible-asr.md) and merged [#1003](https://github.com/debpalash/OmniVoice-Studio/pull/1003). |
17+
| [GPT-SoVITS](https://github.com/RVC-Boss/GPT-SoVITS) | Dataset preparation and WebUI transcription with Fun-ASR-Nano, SenseVoice, and classic FunASR models. | [`funasr_asr.py`](https://github.com/RVC-Boss/GPT-SoVITS/blob/main/tools/asr/funasr_asr.py), runtime fallback [#2801](https://github.com/RVC-Boss/GPT-SoVITS/pull/2801), and backend documentation [#2803](https://github.com/RVC-Boss/GPT-SoVITS/pull/2803). |
2218

23-
`vad-burn` reports the following benchmark on `assets/vad_example.wav` (16 kHz mono PCM, 70.47 seconds) on a MacBook Pro M1 with the Burn Flex CPU backend:
19+
## Model serving and runtimes
2420

25-
| Mode | Avg time | RTF | Speedup |
26-
|---|---:|---:|---:|
27-
| FSMN VAD offline | 73.631 ms | 0.001045 | 957.08x |
28-
| FSMN VAD streaming, 600 ms chunks | 198.425 ms | 0.002816 | 355.15x |
29-
30-
Minimal Python usage from the project:
31-
32-
```python
33-
from vad_burn import FsmnVadModel, VadOptions
21+
| Project | What is integrated | Start here |
22+
|---|---|---|
23+
| [Xinference](https://github.com/xorbitsai/inference) | Built-in audio model specifications for SenseVoiceSmall, Fun-ASR-Nano-2512, and Fun-ASR-MLT-Nano-2512 through Xinference's unified inference API. | [Audio model specifications](https://github.com/xorbitsai/inference/blob/main/xinference/model/audio/model_spec.json) and the FunASR 1.3 compatibility update in merged [#5140](https://github.com/xorbitsai/inference/pull/5140). |
24+
| [Fun-ASR-vLLM](https://github.com/yuekaizhang/Fun-ASR-vllm) | Community vLLM inference for Fun-ASR-Nano and Fun-ASR-MLT-Nano, including batch evaluation and NVIDIA Triton deployment. | [Setup and benchmarks](https://github.com/yuekaizhang/Fun-ASR-vllm#readme) and the deterministic ASR decoding fix in merged [#20](https://github.com/yuekaizhang/Fun-ASR-vllm/pull/20). |
25+
| [vad-burn](https://github.com/di-osc/vad-burn) | FSMN VAD inference in pure Rust with Python bindings, including offline, streaming, and CPU-only modes. | [Project README](https://github.com/di-osc/vad-burn#readme) and the FunASR showcase in [#3106](https://github.com/modelscope/FunASR/issues/3106). |
3426

35-
vad = FsmnVadModel.from_modelscope()
36-
segments = vad.detect(samples, 16000, VadOptions())
27+
## Before adopting an integration
3728

38-
stream = vad.new_stream(VadOptions())
39-
for chunk in chunks:
40-
segments = stream.push(chunk, 16000)
41-
final_segments = stream.finish()
42-
```
29+
- Follow the upstream project's installation and release notes; do not assume its dependency versions match FunASR `main`.
30+
- Validate the exact model, language, audio format, and hardware path you plan to deploy.
31+
- Report application or adapter bugs to the integration project. Report reproducible core FunASR model or runtime bugs to [FunASR issues](https://github.com/modelscope/FunASR/issues).
4332

4433
## Add your project
4534

@@ -48,5 +37,5 @@ If you maintain a FunASR integration, open a [showcase issue](https://github.com
4837
- repository link and maintenance status
4938
- supported FunASR model or runtime path
5039
- install and minimal usage instructions
51-
- benchmark or validation details when available
40+
- a merged change, release, benchmark, or other reproducible validation
5241
- a note about whether the project is official, community-maintained, or experimental

docs/community_projects_zh.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
[English](./community_projects.md) | [简体中文](./community_projects_zh.md)
2+
3+
# FunASR 社区集成
4+
5+
本页收录已经在上游项目中落地、仍在维护的 FunASR、Fun-ASR-Nano、SenseVoice 或其他官方 FunASR 模型集成。下面每一项都已对照项目默认分支核验;如有对应的已合并 PR,文中也提供了链接。
6+
7+
这些集成由社区项目维护,其发布节奏、硬件支持和 API 稳定性由上游项目决定,不属于 FunASR 维护者的兼容性承诺。
8+
9+
## 语音 Agent 与应用
10+
11+
| 项目 | 已集成能力 | 从这里开始 |
12+
|---|---|---|
13+
| [Pipecat](https://github.com/pipecat-ai/pipecat) | 面向语音和多模态 Agent pipeline 的本地 `FunASRSTTService`,底层使用 SenseVoice,并提供转写与语音 Agent 示例。 | [FunASR 服务文档](https://docs.pipecat.ai/api-reference/server/services/stt/funasr)[转写示例](https://github.com/pipecat-ai/pipecat/blob/main/examples/transcription/transcription-funasr.py)[语音 Agent 示例](https://github.com/pipecat-ai/pipecat/blob/main/examples/voice/voice-funasr.py) 和已合并 [#4844](https://github.com/pipecat-ai/pipecat/pull/4844)|
14+
| [xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) | 默认本地 ASR provider 使用 FunASR + SenseVoiceSmall;识别语种可设为 `auto``zh``en``ja``ko``yue`,同时也提供独立 FunASR runtime server provider。 | [Provider 实现](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/main/xiaozhi-server/core/providers/asr/fun_local.py)[配置](https://github.com/xinnan-tech/xiaozhi-esp32-server/blob/main/main/xiaozhi-server/config.yaml) 和已合并 [#3255](https://github.com/xinnan-tech/xiaozhi-esp32-server/pull/3255)|
15+
| [AutoSubs](https://github.com/tmoroney/auto-subs) | 面向 DaVinci Resolve、Premiere 和 After Effects 字幕生成的端侧 int8 ONNX SenseVoice 引擎。 | [SenseVoice 模型说明](https://github.com/tmoroney/auto-subs#sensevoice)[引擎源码](https://github.com/tmoroney/auto-subs/blob/main/AutoSubs-App/src-tauri/crates/transcription-engine/src/engines/sense_voice.rs) 和已合并 [#629](https://github.com/tmoroney/auto-subs/pull/629)|
16+
| [OmniVoice Studio](https://github.com/debpalash/OmniVoice-Studio) | 通过 OpenAI 兼容远程 ASR backend,把听写和配音工作流连接到自托管 FunASR 或 SenseVoice 服务。 | [OpenAI 兼容 ASR 指南](https://github.com/debpalash/OmniVoice-Studio/blob/main/docs/engines/openai-compatible-asr.md) 和已合并 [#1003](https://github.com/debpalash/OmniVoice-Studio/pull/1003)|
17+
| [GPT-SoVITS](https://github.com/RVC-Boss/GPT-SoVITS) | 使用 Fun-ASR-Nano、SenseVoice 和经典 FunASR 模型完成数据集预处理与 WebUI 转写。 | [`funasr_asr.py`](https://github.com/RVC-Boss/GPT-SoVITS/blob/main/tools/asr/funasr_asr.py)、runtime fallback [#2801](https://github.com/RVC-Boss/GPT-SoVITS/pull/2801) 和 backend 文档 [#2803](https://github.com/RVC-Boss/GPT-SoVITS/pull/2803)|
18+
19+
## 模型服务与 Runtime
20+
21+
| 项目 | 已集成能力 | 从这里开始 |
22+
|---|---|---|
23+
| [Xinference](https://github.com/xorbitsai/inference) | 通过 Xinference 统一推理 API 提供 SenseVoiceSmall、Fun-ASR-Nano-2512 和 Fun-ASR-MLT-Nano-2512 内置音频模型规格。 | [音频模型规格](https://github.com/xorbitsai/inference/blob/main/xinference/model/audio/model_spec.json) 和已合并 [#5140](https://github.com/xorbitsai/inference/pull/5140) 中的 FunASR 1.3 兼容性更新。 |
24+
| [Fun-ASR-vLLM](https://github.com/yuekaizhang/Fun-ASR-vllm) | 面向 Fun-ASR-Nano 和 Fun-ASR-MLT-Nano 的社区 vLLM 推理实现,包含批量评测和 NVIDIA Triton 部署。 | [安装与 Benchmark](https://github.com/yuekaizhang/Fun-ASR-vllm#readme) 和已合并 [#20](https://github.com/yuekaizhang/Fun-ASR-vllm/pull/20) 中的确定性 ASR 解码修复。 |
25+
| [vad-burn](https://github.com/di-osc/vad-burn) | 纯 Rust FSMN VAD 推理与 Python binding,支持离线、流式和纯 CPU 模式。 | [项目 README](https://github.com/di-osc/vad-burn#readme) 和 FunASR showcase [#3106](https://github.com/modelscope/FunASR/issues/3106)|
26+
27+
## 采用社区集成前
28+
29+
- 按上游项目的安装和发布说明操作,不要假设其依赖版本与 FunASR `main` 完全一致。
30+
- 用计划部署的模型、语种、音频格式和硬件路径做一次真实验证。
31+
- 应用或 adapter 问题优先反馈给集成项目;可复现的 FunASR 核心模型或 runtime 问题反馈到 [FunASR issues](https://github.com/modelscope/FunASR/issues)
32+
33+
## 收录你的项目
34+
35+
如果你维护了 FunASR 集成,请提交 [showcase issue](https://github.com/modelscope/FunASR/issues/new?template=showcase.md),并提供:
36+
37+
- 仓库链接和维护状态
38+
- 支持的 FunASR 模型或 runtime 路径
39+
- 安装方式和最小使用示例
40+
- 已合并改动、正式发布、benchmark 或其他可复现验证
41+
- 官方、社区维护或实验性项目的明确说明

docs/use_case_showcase.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ FunASR is useful far beyond a single offline transcription command. This page co
1111
| Compare accuracy and speed | [Benchmark report](https://modelscope.github.io/FunASR/benchmark.html) | Reproduce the 184-file long-audio benchmark before choosing a model. |
1212
| Migrate from Whisper/cloud ASR | [Migration guide](./migration_from_whisper.md) | Map existing pipelines to FunASR, benchmark representative audio, and plan a safe rollout. |
1313
| Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/), [Gradio browser demo](../examples/openai_api/GRADIO.md), [client recipes](../examples/openai_api/CLIENTS.md), [JavaScript/TypeScript recipes](../examples/openai_api/JAVASCRIPT.md), and [workflow recipes](../examples/openai_api/WORKFLOWS.md) | Reuse LangChain, Dify, n8n, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. |
14+
| Reuse an existing integration | [Community integrations](./community_projects.md) | Start from verified upstream paths for voice agents, local assistants, desktop subtitles, model serving, and Rust VAD. |
1415
| Add speech input to agents | [MCP server](../examples/mcp_server/) and [voice input](../examples/voice_input/) | Connect local ASR to Claude, Cursor, and desktop agent workflows. |
1516
| Choose a deployment path | [Deployment matrix](./deployment_matrix.md) | Compare Python API, OpenAI API, Docker Compose, Kubernetes, WebSocket, vLLM, MCP, batch, subtitles, and Triton. |
1617
| Serve streaming ASR | [Runtime service docs](../runtime/readme.md) | Run WebSocket or service-mode ASR for live captioning and call-center style workloads. |

docs/use_case_showcase_zh.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ FunASR 不只是一个离线转写命令。这个页面把常见的评测、部
1111
| 对比准确率和速度 | [性能评测报告](https://modelscope.github.io/FunASR/zh/benchmark.html) | 选型前先查看 184 条长音频评测结果。 |
1212
| 从 Whisper/云端 ASR 迁移 | [迁移指南](./migration_from_whisper_zh.md) | 将现有流水线映射到 FunASR,用代表性音频评测并规划安全上线。 |
1313
| 搭建私有语音 API | [OpenAI 兼容 API 示例](../examples/openai_api/README_zh.md)[Gradio 浏览器 Demo](../examples/openai_api/GRADIO_zh.md)[客户端配方](../examples/openai_api/CLIENTS.md)[JavaScript/TypeScript 配方](../examples/openai_api/JAVASCRIPT_zh.md)[工作流配方](../examples/openai_api/WORKFLOWS_zh.md) | 复用 LangChain、Dify、n8n、AutoGen 等 OpenAI 风格客户端,音频不出内网。 |
14+
| 复用已有生态集成 | [社区集成](./community_projects_zh.md) | 从已验证的上游路径开始接入语音 Agent、本地助手、桌面字幕、模型服务和 Rust VAD。 |
1415
| 给 Agent 增加语音输入 | [MCP 服务](../examples/mcp_server/)[语音输入示例](../examples/voice_input/) | 将本地 ASR 接入 Claude、Cursor 和桌面 Agent 工作流。 |
1516
| 选择部署路径 | [部署选型表](./deployment_matrix_zh.md) | 对比 Python API、OpenAI API、Docker Compose、Kubernetes、WebSocket、vLLM、MCP、批处理、字幕和 Triton。 |
1617
| 部署流式 ASR | [Runtime 服务文档](../runtime/readme_cn.md) | 面向实时字幕、客服、会议等低延迟场景。 |

0 commit comments

Comments
 (0)