diff --git a/.github/workflows/test-llamacpp-downloader.yml b/.github/workflows/test-llamacpp-downloader.yml new file mode 100644 index 000000000..eee54d3ee --- /dev/null +++ b/.github/workflows/test-llamacpp-downloader.yml @@ -0,0 +1,31 @@ +name: Test llama.cpp downloader + +on: + workflow_dispatch: + pull_request: + paths: + - 'README.md' + - 'README_zh.md' + - 'runtime/llama.cpp/download-funasr-model.sh' + - 'runtime/llama.cpp/tests/test_download_funasr_model.sh' + - '.github/workflows/test-llamacpp-downloader.yml' + push: + branches: + - main + paths: + - 'README.md' + - 'README_zh.md' + - 'runtime/llama.cpp/download-funasr-model.sh' + - 'runtime/llama.cpp/tests/test_download_funasr_model.sh' + - '.github/workflows/test-llamacpp-downloader.yml' + +permissions: + contents: read + +jobs: + contract: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Run downloader and README contract tests + run: bash runtime/llama.cpp/tests/test_download_funasr_model.sh diff --git a/README.md b/README.md index 6968b7944..6da05300d 100644 --- a/README.md +++ b/README.md @@ -246,13 +246,20 @@ docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime- Run **SenseVoice / Paraformer / Fun-ASR-Nano** as a **single self-contained binary** on CPU and edge devices — this is to FunASR what [whisper.cpp](https://github.com/ggml-org/whisper.cpp) is to Whisper, but with **~3× lower CER than whisper.cpp on Chinese**. Built-in FSMN-VAD, no Python at runtime. ```bash -# 1) Grab a prebuilt binary from Releases (Linux / macOS / Windows), then: +# Linux / macOS: run from the extracted release directory bash download-funasr-model.sh sensevoice ./gguf # or: paraformer | nano -llama-funasr-sensevoice -m ./gguf/SenseVoiceSmall-f16.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav +./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav # → 欢迎大家来体验达摩院推出的语音识别模型 ``` -**Prebuilt binaries:** [Releases](../../releases) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/) +```powershell +# Windows PowerShell: run from the extracted archive root (with the `hf` CLI installed) +hf download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir .\gguf +hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf +.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav +``` + +**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/) [OpenAI API example →](./examples/openai_api/) · [Gradio demo →](./examples/openai_api/GRADIO.md) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript recipes →](./examples/openai_api/JAVASCRIPT.md) · [Kubernetes template →](./examples/openai_api/kubernetes/) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Postman collection →](./examples/openai_api/POSTMAN.md) · [OpenAPI spec →](./examples/openai_api/OPENAPI.md) · [Security guide →](./examples/openai_api/SECURITY.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html) diff --git a/README_zh.md b/README_zh.md index 88bdb7c5b..dd2385189 100644 --- a/README_zh.md +++ b/README_zh.md @@ -274,7 +274,25 @@ curl http://localhost:8000/v1/audio/transcriptions \ docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12 ``` -> **CPU / 边缘部署(无需 GPU、无需 Python):** 用 **llama.cpp / GGUF** 跑 Fun-ASR-Nano / SenseVoice / Paraformer —— 单个自包含二进制,对标 whisper.cpp。详见 [runtime/llama.cpp/](./runtime/llama.cpp/)。 +### CPU / 边缘部署 - llama.cpp / GGUF(无需 GPU、无需 Python) + +在 CPU 和边缘设备上用单个自包含二进制运行 **SenseVoice / Paraformer / Fun-ASR-Nano**,无需 Python 运行环境,并内置 FSMN-VAD。 + +```bash +# Linux / macOS:在解压后的发布目录中执行 +bash download-funasr-model.sh sensevoice ./gguf # 也可使用 paraformer 或 nano +./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav +# -> 欢迎大家来体验达摩院推出的语音识别模型 +``` + +```powershell +# Windows PowerShell:在解压根目录执行(需已安装 `hf` CLI) +hf download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir .\gguf +hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf +.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav +``` + +**预编译二进制:** [Releases](https://github.com/modelscope/FunASR/releases) · **下载与快速开始:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF 模型:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测:** [runtime/llama.cpp/](./runtime/llama.cpp/) [OpenAI API 示例 →](./examples/openai_api/README_zh.md) · [Gradio Demo →](./examples/openai_api/GRADIO_zh.md) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript 配方 →](./examples/openai_api/JAVASCRIPT_zh.md) · [Kubernetes 模板 →](./examples/openai_api/kubernetes/README_zh.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS_zh.md) · [Postman 集合 →](./examples/openai_api/POSTMAN_zh.md) · [OpenAPI 规范 →](./examples/openai_api/OPENAPI_zh.md) · [安全指南 →](./examples/openai_api/SECURITY_zh.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html) diff --git a/runtime/llama.cpp/tests/test_download_funasr_model.sh b/runtime/llama.cpp/tests/test_download_funasr_model.sh index 48bb01b1d..2bd9e8a6c 100755 --- a/runtime/llama.cpp/tests/test_download_funasr_model.sh +++ b/runtime/llama.cpp/tests/test_download_funasr_model.sh @@ -2,6 +2,7 @@ set -euo pipefail ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) +REPO_ROOT=$(cd "$ROOT/../.." && pwd) SCRIPT="$ROOT/download-funasr-model.sh" TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT @@ -62,6 +63,8 @@ bash "$SCRIPT" sensevoice "$TMP/out" >/dev/null assert_log "download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir $TMP/out" assert_log "download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir $TMP/out" assert_no_log "--include *.gguf" +DEFAULT_SENSEVOICE_MODEL=$(awk '$1 == "download" && $2 == "FunAudioLLM/SenseVoiceSmall-GGUF" { print $3; exit }' "$HF_LOG") +[[ -n "$DEFAULT_SENSEVOICE_MODEL" ]] reset_case bash "$SCRIPT" paraformer "$TMP/out" f16 >/dev/null @@ -106,4 +109,70 @@ if HF_SKIP_CREATE=1 bash "$SCRIPT" sensevoice "$TMP/out" >"$TMP/stdout" 2>"$TMP/ fi grep -F "no GGUF files found in $TMP/out" "$TMP/stderr" >/dev/null +assert_readme_quickstart() { + local readme=$1 + local section_prefix=$2 + local section + local quickstart + local powershell + + if ! section=$(awk -v prefix="$section_prefix" ' + !found && index($0, prefix) == 1 { found = 1 } + found && /^---$/ { exit } + found { print } + END { if (!found) exit 1 } + ' "$readme"); then + printf 'missing CPU/edge section in %s\n' "$readme" >&2 + exit 1 + fi + + if ! quickstart=$(printf '%s\n' "$section" | awk ' + !seen && /^```bash$/ { seen = 1; in_block = 1; next } + in_block && /^```$/ { complete = 1; exit } + in_block { print } + END { if (!seen || !complete) exit 1 } + '); then + printf 'missing Bash quickstart block in %s\n' "$readme" >&2 + exit 1 + fi + + if ! powershell=$(printf '%s\n' "$section" | awk ' + !seen && /^```powershell$/ { seen = 1; in_block = 1; next } + in_block && /^```$/ { complete = 1; exit } + in_block { print } + END { if (!seen || !complete) exit 1 } + '); then + printf 'missing PowerShell quickstart block in %s\n' "$readme" >&2 + exit 1 + fi + + if ! grep -Eq '^bash download-funasr-model\.sh sensevoice \./gguf([[:space:]]+#.*)?$' <<<"$quickstart"; then + printf 'missing default SenseVoice download command in %s\n' "$readme" >&2 + exit 1 + fi + if ! grep -Fx "./llama-funasr-sensevoice -m ./gguf/$DEFAULT_SENSEVOICE_MODEL --vad ./gguf/fsmn-vad.gguf -a audio.wav" <<<"$quickstart" >/dev/null; then + printf 'README command does not run the default sensevoice-small-q8.gguf in %s\n' "$readme" >&2 + exit 1 + fi + if ! grep -Fx "hf download FunAudioLLM/SenseVoiceSmall-GGUF $DEFAULT_SENSEVOICE_MODEL --local-dir .\\gguf" <<<"$powershell" >/dev/null; then + printf 'missing Windows SenseVoice download command in %s\n' "$readme" >&2 + exit 1 + fi + if ! grep -Fx 'hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf' <<<"$powershell" >/dev/null; then + printf 'missing Windows FSMN-VAD download command in %s\n' "$readme" >&2 + exit 1 + fi + if ! grep -Fx ".\\llama-funasr-sensevoice.exe -m .\\gguf\\$DEFAULT_SENSEVOICE_MODEL --vad .\\gguf\\fsmn-vad.gguf -a audio.wav" <<<"$powershell" >/dev/null; then + printf 'missing Windows PowerShell quickstart in %s\n' "$readme" >&2 + exit 1 + fi + if ! grep -F '[Releases](https://github.com/modelscope/FunASR/releases)' <<<"$section" >/dev/null; then + printf 'non-portable Releases link in %s\n' "$readme" >&2 + exit 1 + fi +} + +assert_readme_quickstart "$REPO_ROOT/README.md" '### CPU / Edge' +assert_readme_quickstart "$REPO_ROOT/README_zh.md" '### CPU / 边缘部署' + echo "download-funasr-model contract tests passed"