Skip to content

Commit 8cc3bc5

Browse files
Merge pull request #2895 from ArmDeveloperEcosystem/main
production update
2 parents a6219dd + 0c14f81 commit 8cc3bc5

56 files changed

Lines changed: 2455 additions & 878 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.wordlist.txt

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5644,4 +5644,16 @@ Numbat
56445644
SKUs
56455645
asct
56465646
geminicli
5647-
passwordless
5647+
passwordless
5648+
AWQ
5649+
Coqui
5650+
GPTQ
5651+
PortAudio
5652+
PyAudio
5653+
Riva
5654+
UX
5655+
actionability
5656+
customizations
5657+
pyaudio
5658+
sounddevice
5659+
webrtcvad
Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
---
2-
title: Learn about offline voice assistants
2+
title: Build an offline voice assistant with whisper and vLLM
33
weight: 2
44

55
### FIXED, DO NOT MODIFY
66
layout: learningpathall
77
---
88

9-
## Why build an offline voice assistant?
9+
## Benefits of running a voice assistant offline
1010

1111
Voice-based AI assistants are becoming essential in customer support, productivity tools, and embedded interfaces. For example, a retail kiosk might need to answer product-related questions verbally without relying on internet access. However, many of these systems depend heavily on cloud services for speech recognition and language understanding, raising concerns around latency, cost, and data privacy.
1212

@@ -16,16 +16,16 @@ You avoid unpredictable latency caused by network fluctuations, prevent sensitiv
1616

1717
By combining local speech-to-text (STT) with a locally hosted large language model (LLM), you gain complete control over the pipeline and eliminate API dependencies. You can experiment, customize, and scale without relying on external services.
1818

19-
## What are some common development challenges?
19+
## Challenges of building a local voice assistant
2020

2121
While the benefits are clear, building a local voice assistant involves several engineering challenges.
2222

2323
Real-time audio segmentation requires reliably identifying when users start and stop speaking, accounting for natural pauses and background noise. Timing mismatches between STT and LLM components can cause delayed responses or repeated input, reducing conversational quality. You also need to balance CPU/GPU workloads to keep the pipeline responsive without overloading resources or blocking audio capture.
2424

25-
## Why use Arm and DGX Spark?
25+
## Why run offline voice AI on Arm-based DGX Spark?
2626

27-
Arm-powered platforms like [DGX Spark](https://www.nvidia.com/en-gb/products/workstations/dgx-spark/) allow efficient parallelism: use CPU cores for audio preprocessing and whisper inference, while offloading LLM reasoning to powerful GPUs. This architecture balances throughput and energy efficiencyideal for private, on-premises AI workloads. To understand the CPU and GPU architecture of DGX Spark, refer to [Unlock quantized LLM performance on Arm-based NVIDIA DGX Spark](/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/).
27+
Arm-powered platforms like [DGX Spark](https://www.nvidia.com/en-gb/products/workstations/dgx-spark/) allow efficient parallelism: use CPU cores for audio preprocessing and whisper inference, while offloading LLM reasoning to powerful GPUs. This architecture balances throughput and energy efficiency-ideal for private, on-premises AI workloads. To understand the CPU and GPU architecture of DGX Spark, refer to [Unlock quantized LLM performance on Arm-based NVIDIA DGX Spark](/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/).
2828

2929
DGX Spark also supports standard USB interfaces, making it easy to connect consumer-grade microphones for development or deployment. This makes it viable for edge inference and desktop-style prototyping.
3030

31-
In this Learning Path, youll build a complete, offline voice chatbot prototype using PyAudio, faster-whisper, and vLLM on an Arm-based systemresulting in a fully functional assistant that runs entirely on local hardware with no internet dependency.
31+
In this Learning Path, you'll build a complete, offline voice chatbot prototype using PyAudio, faster-whisper, and vLLM on an Arm-based system-resulting in a fully functional assistant that runs entirely on local hardware with no internet dependency.

content/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/2_setup.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,11 @@ weight: 3
66
layout: learningpathall
77
---
88

9-
[Faster‑whisper](https://github.com/SYSTRAN/faster-whisper) is a high‑performance reimplementation of OpenAI Whisper, designed to significantly reduce transcription latency and memory usage. It is well suited for local and real‑time speech‑to‑text (STT) pipelines, especially when running on CPU‑only systems or hybrid CPU/GPU environments.
9+
## Set up faster-whisper for offline speech recognition
1010

11-
You'll use faster‑whisper as the STT engine to convert raw microphone input into structured text. At this stage, the goal is to install faster‑whisper correctly and verify that it can transcribe audio reliably. Detailed tuning and integration are covered in later sections.
11+
[faster-whisper](https://github.com/SYSTRAN/faster-whisper) is a high-performance re-implementation of OpenAI Whisper, designed to significantly reduce transcription latency and memory usage. It's well suited for local and real-time speech-to-text (STT) pipelines, especially when running on CPU-only systems or hybrid CPU/GPU environments.
12+
13+
You'll use faster-whisper as the STT engine to convert raw microphone input into structured text. At this stage, the goal is to install faster-whisper correctly and verify that it can transcribe audio reliably. Detailed tuning and integration are covered in later sections.
1214

1315
### Install build dependencies
1416

@@ -22,11 +24,11 @@ sudo apt install python3.12 python3.12-venv python3.12-dev -y
2224
sudo apt install gcc portaudio19-dev ffmpeg -y
2325
```
2426

25-
## Create and activate Python environment
27+
## Create and activate a Python environment
2628

2729
In particular, [pyaudio](https://pypi.org/project/PyAudio/) (used for real-time microphone capture) depends on the PortAudio library and the Python C API. These must match the version of Python you're using.
2830

29-
Now that the system libraries are in place and audio input is verified, it's time to set up an isolated Python environment for your voice assistant project. This will prevent dependency conflicts and make your installation reproducible.
31+
Set up an isolated Python environment for your voice assistant project to prevent dependency conflicts and make your installation reproducible.
3032

3133
```bash
3234
python3.12 -m venv va_env
@@ -53,7 +55,7 @@ pip install requests webrtcvad sounddevice==0.5.3
5355
```
5456

5557
{{% notice Note %}}
56-
While sounddevice==0.5.4 is available, it introduces callback-related errors during audio stream cleanup that may confuse beginners.
58+
While sounddevice==0.5.4 is available, it introduces callback-related errors during audio stream cleanup that can confuse beginners.
5759
Use sounddevice==0.5.3, which is stable and avoids these warnings.
5860
{{% /notice %}}
5961

@@ -162,7 +164,7 @@ Recording for 10 seconds...
162164

163165
{{% notice Note %}}
164166
To stop the script, press Ctrl+C during any transcription loop. The current 10-second recording completes and transcribes before the program exits cleanly.
165-
Avoid using Ctrl+Z, which suspends the process instead of terminating it.
167+
Don't use Ctrl+Z, which suspends the process instead of terminating it.
166168
{{% /notice %}}
167169

168170

@@ -189,7 +191,7 @@ pip install sounddevice==0.5.3
189191

190192
You can record audio without errors, but nothing is played back.
191193

192-
Verify that your USB microphone or headset is selected as the default input/output device. Also ensure the system volume is not muted.
194+
Ensure that your USB microphone or headset is selected as the default input/output device. Also check that the system volume isn't muted.
193195

194196
**Fix:** List all available audio devices:
195197

content/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/3_fasterwhisper.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ weight: 4
66
layout: learningpathall
77
---
88

9+
## Build a CPU-based speech-to-text engine
10+
911
In this section, you'll build a real-time speech-to-text (STT) pipeline using only the CPU. Starting from a basic 10-second recorder, you'll incrementally add noise filtering, sentence segmentation, and parallel audio processing to achieve a transcription engine for Arm-based systems like DGX Spark.
1012

1113
You'll start from a minimal loop and iterate toward a multithreaded, VAD-enhanced STT engine.
@@ -104,7 +106,7 @@ When you speak to the device, the output is similar to:
104106

105107
{{% notice Note %}}
106108
faster-whisper supports many models like tiny, base, small, medium and large-v1/2/3.
107-
Check the [GitHub repository](https://github.com/openai/whisper?tab=readme-ov-file#available-models-and-languages) for more model details.
109+
See the [GitHub repository](https://github.com/openai/whisper?tab=readme-ov-file#available-models-and-languages) for more model details.
108110
{{% /notice %}}
109111

110112

@@ -238,15 +240,15 @@ When you say a long sentence with multiple clauses, the output is similar to:
238240
Segment done.
239241
```
240242

241-
The result is a smoother and more accurate voice UXparticularly important when integrating with downstream LLMs in later sections.
243+
The result is a smoother and more accurate voice UX - particularly important when integrating with downstream LLMs in later sections.
242244

243245
### Demo: Real-time speech transcription on Arm CPU with faster-whisper
244246

245-
This demo shows the real-time transcription pipeline in action, running on an Arm-based DGX Spark system. Using a USB microphone and the faster-whisper model (`medium.en`), the system records voice input, processes it on the CPU, and returns accurate transcriptions with timestampsall without relying on cloud services.
247+
This demo shows the real-time transcription pipeline in action, running on an Arm-based DGX Spark system. Using a USB microphone and the faster-whisper model (`medium.en`), the system records voice input, processes it on the CPU, and returns accurate transcriptions with timestamps - all without relying on cloud services.
246248

247249
Notice the clean terminal output and low latency, demonstrating how the pipeline is optimized for local, real-time voice recognition on resource-efficient hardware.
248250

249-
![Real-time speech transcription demo with volume visualization#center](fasterwhipser_demo1.gif "Figure 1: Real-time speech transcription with audio volume bar")
251+
![Real-time speech transcription demo with volume visualization alt-txt#center](fasterwhipser_demo1.gif "Real-time speech transcription with audio volume bar")
250252

251253
The device runs audio capture and transcription in parallel. Use `threading.Thread` to collect audio without blocking, store audio frames in a `queue.Queue`, and in the main thread, poll for new data and run STT.
252254

content/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/3a_segmentation.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,9 @@ weight: 5
66
layout: learningpathall
77
---
88

9-
After applying the previous steps—model upgrade, VAD, smart turn detection, and multi-threaded audio collection—you now have a high-quality, CPU-based local speech-to-text system.
9+
## Optimize speech segmentation for your environment
10+
11+
After applying the previous steps-model upgrade, VAD, smart turn detection, and multi-threaded audio collection - you now have a high-quality, CPU-based local speech-to-text system.
1012

1113
At this stage, the core pipeline is complete. What remains is fine-tuning: adapting the system to your environment, microphone setup, and speaking style. This flexibility is one of the key advantages of a fully local STT pipeline.
1214

@@ -42,7 +44,7 @@ Adjust this setting based on background noise and microphone quality.
4244

4345
### Tuning `MIN_SPEECH_SEC` and `SILENCE_LIMIT_SEC`
4446

45-
- `MIN_SPEECH_SEC`: This parameter defines the minimum duration of detected speech required before a segment is considered valid. Use this to filter out very short utterances such as false starts or background chatter.
47+
- `MIN_SPEECH_SEC`: This parameter defines the minimum duration of detected speech needed before a segment is considered valid. Use this to filter out very short utterances such as false starts or background chatter.
4648
- Lower values: More responsive, but may capture incomplete phrases or noise
4749
- Higher values: More stable sentences, but slower response
4850

@@ -58,7 +60,7 @@ Based on practical experiments, the following presets provide a good starting po
5860
|----------------------|----------------------|-------------------------|-------------------|
5961
| Short command phrases | 0.8 | 0.6 | Optimized for quick voice commands such as "yes", "next", or "stop". Prioritizes responsiveness over sentence completeness. |
6062
| Natural conversational speech | 1.0 | 1.0 | Balanced settings for everyday dialogue with natural pauses between phrases. |
61-
| Long-form explanations (for example, tutorials) | 2.0 | 2.0 | Designed for longer sentences and structured explanations, reducing the risk of premature segmentation. |
63+
| Long-form explanations such as tutorials | 2.0 | 2.0 | Designed for longer sentences and structured explanations, reducing the risk of premature segmentation. |
6264

6365
## Apply these settings
6466

0 commit comments

Comments
 (0)