Skip to content

granite-speech.md

Latest commit

 

History

History
113 lines (91 loc) · 5.64 KB

granite-speech.md

File metadata and controls

113 lines (91 loc) · 5.64 KB
Raw

Granite Speech 4 / 4.1

IBM's Granite Speech family ported to transcribe.cpp. A Conformer audio encoder with block-local Shaw attention, a windowed projector, and the Granite-4.0-1b LLM as the text head. Variants differ in (a) the LLM decoding mode — autoregressive (AR) versus single-pass non-autoregressive (NAR) editor — and (b) which extras the head exposes: translation, word-level timestamps, etc.

For the architecture deep-dive, validation contract, and porting notes see the per-decoder family docs: docs/porting/families/granite.md covers the AR variants and docs/porting/families/granite_nar.md covers the NAR variant. The split exists because the two share an encoder but their decoder pipelines are structurally different; from a user perspective they are one family of GGUFs.

Choosing a variant

  • Best WER, slowest decode. granite-speech-4.1-2b — 4.1 generation LLM head, autoregressive; 1.31% BF16 / 1.32% Q8_0 WER on LibriSpeech test-clean. Improved punctuation and casing over 4.0-1b.
  • Most features. granite-speech-4.1-2b-plus — same 4.1 base plus word-level timestamps (parsed from the model's [T:N] centisecond markers). Encoder concatenates a mid-layer and the final layer (cat_hidden_layers=[3]), doubling projector K/V width.
  • Fastest, ASR-only. granite-speech-4.1-2b-nar — non-autoregressive editor: one forward pass through encoder + projector + bidirectional LLM + CTC decode replaces the AR token loop. ~1.5× faster than the AR siblings on Metal at Q8_0. No translation, no timestamps. 1.29% WER matches the upstream model card exactly.
  • Smallest 4.0-generation model. granite-4.0-1b-speech — the 4.0 baseline. Same architecture as 4.1-2b. 1.42% BF16 / 1.44% Q8_0 WER on LibriSpeech test-clean; covers Japanese in addition to en/fr/de/es/pt — as does the base granite-speech-4.1-2b. The granite-speech-4.1-2b-plus and granite-speech-4.1-2b-nar variants drop Japanese (en/fr/de/es/pt only).

All variants

WER is on LibriSpeech test-clean for the Q8_0 preset, measured by transcribe.cpp's WER pipeline. See each per-variant doc for the full quant matrix.

Variant Decode mode Params Q8_0 size WER (Q8_0) Languages Extras Doc
granite-4.0-1b-speech AR (audio-LLM) ~3B† 2.56 GB 1.44% en, fr, de, es, pt, ja translate (en ↔ ASR langs; en → it/zh) granite-4.0-1b-speech.md
granite-speech-4.1-2b AR (audio-LLM) ~3B† 2.56 GB 1.32% en, fr, de, es, pt, ja translate (en ↔ ASR langs; en → it/zh) granite-speech-4.1-2b.md
granite-speech-4.1-2b-plus AR (audio-LLM) ~3B† 2.35 GB 1.50% en, fr, de, es, pt word timestamps (ASR only) granite-speech-4.1-2b-plus.md
granite-speech-4.1-2b-nar NAR (editor) ~3B† 2.33 GB 1.29% en, fr, de, es, pt (ASR only) granite-speech-4.1-2b-nar.md

† Parameter counts include the Conformer audio encoder, the projector, and the Granite-4.0-1b text LM. The "1b" / "2b" in IBM's variant names refers to the speech-front-end LoRA delta size, not the fused-stack total.

Pre-built GGUFs for every variant and quant are hosted under handy-computer on Hugging Face; each per-variant doc has direct download links.

Input limits

Every variant accepts up to about 6 minutes of 16 kHz mono audio per call — the Granite-4.0-1b LLM's 4,096-token context is the binding limit. Longer audio is rejected up front with TRANSCRIBE_ERR_INPUT_TOO_LONG rather than silently truncated; split it into shorter segments. (The autoregressive variants also return TRANSCRIBE_ERR_OUTPUT_TRUNCATED if a transcript itself runs into the budget; the NAR editor produces its output in one pass and cannot.) See the input-length contract.

Quick start

Pick a variant and run:

cmake -B build
cmake --build build

build/bin/transcribe-cli \
  -m models/granite-speech-4.1-2b-nar/granite-speech-4.1-2b-nar-Q8_0.gguf \
  samples/jfk.wav

The repo doesn't ship the GGUFs — pull them from the corresponding handy-computer/<variant>-gguf repo on Hugging Face, or convert from the upstream IBM checkpoint via the per-variant doc's reproduction section.

Capabilities

All variants:

  • Transcription of 16 kHz mono WAV input across the variant's supported languages.

Translation (granite-4.0-1b-speech, granite-speech-4.1-2b):

  • Translation between English and each ASR language in either direction (en ↔ fr, en ↔ de, en ↔ es, en ↔ pt, en ↔ ja), plus English-to-Italian and English-to-Mandarin (--target-language it / zh). There is no direct fr↔de etc. — translation always involves English on one side. Use --translate --target-language <bcp47>; the source language is inferred from the audio. The -plus variant is ASR-only and does not translate.

Plus only (granite-speech-4.1-2b-plus):

  • Word-level timestamps via --timestamps word — structured per-word start/end times, parsed from the model's [T:N] centisecond markers.

NAR only (granite-speech-4.1-2b-nar):

  • Single-pass non-autoregressive decode — fastest of the four.

What's not exposed by the v1 transcribe.cpp runtime: speaker diarization (advertised on -plus), keyword/hotword biasing (advertised on AR variants), real-time streaming, VAD. See the per-variant docs for status.