Skip to content

Add whisper-stream-pcm: stdin/pipe PCM streaming with VAD support#3653

Open
rmorse wants to merge 11 commits intoggml-org:masterfrom
rmorse:stream-pcm
Open

Add whisper-stream-pcm: stdin/pipe PCM streaming with VAD support#3653
rmorse wants to merge 11 commits intoggml-org:masterfrom
rmorse:stream-pcm

Conversation

@rmorse
Copy link
Copy Markdown

@rmorse rmorse commented Feb 7, 2026

Summary

  • Adds whisper-stream-pcm, a new example binary for streaming raw PCM audio via stdin/pipe for real-time transcription — no WAV headers or SDL2 required
  • Supports both simple (energy-based) and Silero VAD for speech segmentation
  • Accepts 32-bit float or 16-bit signed integer PCM (16kHz, mono)

Usage

whisper-stream-pcm -m models/ggml-base.en.bin

With Silero VAD:

whisper-stream-pcm -m models/ggml-base.en.bin --vad --vad-model models/silero-vad.bin

Using FFMPEG to test/pipe in the data

ffmpeg -i audio.wav -f f32le -ar 16000 -ac 1 pipe:1 | whisper-stream-pcm -m models/ggml-base.en.bin --vad --vad-model models/silero-vad.bin

Changes

  • examples/stream-pcm/ — new example binary (stream-pcm.cpp, CMakeLists.txt, README.md)
  • examples/CMakeLists.txt — register stream-pcm example
  • tests/test-stream-pcm.cpp — test for stdin/pipe PCM streaming
  • tests/CMakeLists.txt — register test
  • README.md — mention stream-pcm in examples list

Test plan

  • whisper-stream-pcm builds on Linux/macOS/Windows
  • Piping PCM audio via ffmpeg produces transcription output
  • --vad (simple) and --vad --vad-model (Silero) both work correctly
  • ctest passes test-stream-pcm

@ggerganov
Copy link
Copy Markdown
Member

Thanks, this looks interesting. Would you mind rebasing on the latest master so we can run the CI

@rmorse
Copy link
Copy Markdown
Author

rmorse commented Mar 16, 2026

@ggerganov done :) Thanks for considering this.

After double checking, there were some upstream warnings / patterns which I didn't change for stream-pcm but did find and fix a casting issue along the way.

@ggerganov
Copy link
Copy Markdown
Member

Ok, let's see how the builds go.

I'm interested in adding such example. Need to find some time to play with this. Hopefully other people would provide some additional feedback in the meantime.

@rmorse
Copy link
Copy Markdown
Author

rmorse commented Mar 16, 2026

I managed to a build a release against the last stable release and seemed to work well (on windows at least)

Just patching that CI issue now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants