Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/test-llamacpp-downloader.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: Test llama.cpp downloader

on:
workflow_dispatch:
pull_request:
paths:
- 'README.md'
- 'README_zh.md'
- 'runtime/llama.cpp/download-funasr-model.sh'
- 'runtime/llama.cpp/tests/test_download_funasr_model.sh'
- '.github/workflows/test-llamacpp-downloader.yml'
push:
branches:
- main
paths:
- 'README.md'
- 'README_zh.md'
- 'runtime/llama.cpp/download-funasr-model.sh'
- 'runtime/llama.cpp/tests/test_download_funasr_model.sh'
- '.github/workflows/test-llamacpp-downloader.yml'

permissions:
contents: read

jobs:
contract:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run downloader and README contract tests
run: bash runtime/llama.cpp/tests/test_download_funasr_model.sh
13 changes: 10 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,13 +246,20 @@ docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-
Run **SenseVoice / Paraformer / Fun-ASR-Nano** as a **single self-contained binary** on CPU and edge devices — this is to FunASR what [whisper.cpp](https://github.com/ggml-org/whisper.cpp) is to Whisper, but with **~3× lower CER than whisper.cpp on Chinese**. Built-in FSMN-VAD, no Python at runtime.

```bash
# 1) Grab a prebuilt binary from Releases (Linux / macOS / Windows), then:
# Linux / macOS: run from the extracted release directory
bash download-funasr-model.sh sensevoice ./gguf # or: paraformer | nano
llama-funasr-sensevoice -m ./gguf/SenseVoiceSmall-f16.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
# → 欢迎大家来体验达摩院推出的语音识别模型
```

**Prebuilt binaries:** [Releases](../../releases) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/)
```powershell
# Windows PowerShell: run from the extracted archive root (with the `hf` CLI installed)
hf download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir .\gguf
hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav
```

**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/)

[OpenAI API example →](./examples/openai_api/) · [Gradio demo →](./examples/openai_api/GRADIO.md) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript recipes →](./examples/openai_api/JAVASCRIPT.md) · [Kubernetes template →](./examples/openai_api/kubernetes/) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Postman collection →](./examples/openai_api/POSTMAN.md) · [OpenAPI spec →](./examples/openai_api/OPENAPI.md) · [Security guide →](./examples/openai_api/SECURITY.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)

Expand Down
20 changes: 19 additions & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,25 @@ curl http://localhost:8000/v1/audio/transcriptions \
docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12
```

> **CPU / 边缘部署(无需 GPU、无需 Python):** 用 **llama.cpp / GGUF** 跑 Fun-ASR-Nano / SenseVoice / Paraformer —— 单个自包含二进制,对标 whisper.cpp。详见 [runtime/llama.cpp/](./runtime/llama.cpp/)。
### CPU / 边缘部署 - llama.cpp / GGUF(无需 GPU、无需 Python)

在 CPU 和边缘设备上用单个自包含二进制运行 **SenseVoice / Paraformer / Fun-ASR-Nano**,无需 Python 运行环境,并内置 FSMN-VAD。

```bash
# Linux / macOS:在解压后的发布目录中执行
bash download-funasr-model.sh sensevoice ./gguf # 也可使用 paraformer 或 nano
./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
# -> 欢迎大家来体验达摩院推出的语音识别模型
```

```powershell
# Windows PowerShell:在解压根目录执行(需已安装 `hf` CLI)
hf download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir .\gguf
hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav
```

**预编译二进制:** [Releases](https://github.com/modelscope/FunASR/releases) · **下载与快速开始:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF 模型:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测:** [runtime/llama.cpp/](./runtime/llama.cpp/)

[OpenAI API 示例 →](./examples/openai_api/README_zh.md) · [Gradio Demo →](./examples/openai_api/GRADIO_zh.md) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript 配方 →](./examples/openai_api/JAVASCRIPT_zh.md) · [Kubernetes 模板 →](./examples/openai_api/kubernetes/README_zh.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS_zh.md) · [Postman 集合 →](./examples/openai_api/POSTMAN_zh.md) · [OpenAPI 规范 →](./examples/openai_api/OPENAPI_zh.md) · [安全指南 →](./examples/openai_api/SECURITY_zh.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)

Expand Down
69 changes: 69 additions & 0 deletions runtime/llama.cpp/tests/test_download_funasr_model.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
set -euo pipefail

ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
REPO_ROOT=$(cd "$ROOT/../.." && pwd)
SCRIPT="$ROOT/download-funasr-model.sh"
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
Expand Down Expand Up @@ -62,6 +63,8 @@ bash "$SCRIPT" sensevoice "$TMP/out" >/dev/null
assert_log "download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir $TMP/out"
assert_log "download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir $TMP/out"
assert_no_log "--include *.gguf"
DEFAULT_SENSEVOICE_MODEL=$(awk '$1 == "download" && $2 == "FunAudioLLM/SenseVoiceSmall-GGUF" { print $3; exit }' "$HF_LOG")
[[ -n "$DEFAULT_SENSEVOICE_MODEL" ]]

reset_case
bash "$SCRIPT" paraformer "$TMP/out" f16 >/dev/null
Expand Down Expand Up @@ -106,4 +109,70 @@ if HF_SKIP_CREATE=1 bash "$SCRIPT" sensevoice "$TMP/out" >"$TMP/stdout" 2>"$TMP/
fi
grep -F "no GGUF files found in $TMP/out" "$TMP/stderr" >/dev/null

assert_readme_quickstart() {
local readme=$1
local section_prefix=$2
local section
local quickstart
local powershell

if ! section=$(awk -v prefix="$section_prefix" '
!found && index($0, prefix) == 1 { found = 1 }
found && /^---$/ { exit }
found { print }
END { if (!found) exit 1 }
' "$readme"); then
printf 'missing CPU/edge section in %s\n' "$readme" >&2
exit 1
fi

if ! quickstart=$(printf '%s\n' "$section" | awk '
!seen && /^```bash$/ { seen = 1; in_block = 1; next }
in_block && /^```$/ { complete = 1; exit }
in_block { print }
END { if (!seen || !complete) exit 1 }
'); then
printf 'missing Bash quickstart block in %s\n' "$readme" >&2
exit 1
fi

if ! powershell=$(printf '%s\n' "$section" | awk '
!seen && /^```powershell$/ { seen = 1; in_block = 1; next }
in_block && /^```$/ { complete = 1; exit }
in_block { print }
END { if (!seen || !complete) exit 1 }
'); then
printf 'missing PowerShell quickstart block in %s\n' "$readme" >&2
exit 1
fi

if ! grep -Eq '^bash download-funasr-model\.sh sensevoice \./gguf([[:space:]]+#.*)?$' <<<"$quickstart"; then
printf 'missing default SenseVoice download command in %s\n' "$readme" >&2
exit 1
fi
if ! grep -Fx "./llama-funasr-sensevoice -m ./gguf/$DEFAULT_SENSEVOICE_MODEL --vad ./gguf/fsmn-vad.gguf -a audio.wav" <<<"$quickstart" >/dev/null; then
printf 'README command does not run the default sensevoice-small-q8.gguf in %s\n' "$readme" >&2
exit 1
fi
if ! grep -Fx "hf download FunAudioLLM/SenseVoiceSmall-GGUF $DEFAULT_SENSEVOICE_MODEL --local-dir .\\gguf" <<<"$powershell" >/dev/null; then
printf 'missing Windows SenseVoice download command in %s\n' "$readme" >&2
exit 1
fi
if ! grep -Fx 'hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf' <<<"$powershell" >/dev/null; then
printf 'missing Windows FSMN-VAD download command in %s\n' "$readme" >&2
exit 1
fi
if ! grep -Fx ".\\llama-funasr-sensevoice.exe -m .\\gguf\\$DEFAULT_SENSEVOICE_MODEL --vad .\\gguf\\fsmn-vad.gguf -a audio.wav" <<<"$powershell" >/dev/null; then
printf 'missing Windows PowerShell quickstart in %s\n' "$readme" >&2
exit 1
fi
if ! grep -F '[Releases](https://github.com/modelscope/FunASR/releases)' <<<"$section" >/dev/null; then
printf 'non-portable Releases link in %s\n' "$readme" >&2
exit 1
fi
}

assert_readme_quickstart "$REPO_ROOT/README.md" '### CPU / Edge'
assert_readme_quickstart "$REPO_ROOT/README_zh.md" '### CPU / 边缘部署'

echo "download-funasr-model contract tests passed"
Loading