Skip to content

granite-4.0-1b-speech.md

Latest commit

 

History

History
180 lines (131 loc) · 6.61 KB

granite-4.0-1b-speech.md

File metadata and controls

180 lines (131 loc) · 6.61 KB
Raw

Granite Speech 4.0-1b

IBM's ibm-granite/granite-4.0-1b-speech ported to transcribe.cpp. An audio-LLM: a Conformer encoder with block-local Shaw attention, a BLIP-2 Q-Former projector, and the Granite-4.0-1b-base LLM as an autoregressive decoder.

What it's for

Offline multilingual speech-to-text covering English plus French, German, Spanish, Portuguese, and Japanese. The model takes a 16 kHz mono WAV and produces a transcript.

Translation pairs: English ↔ French, English ↔ German, English ↔ Spanish, English ↔ Portuguese, English ↔ Japanese, plus English-to-Italian and English-to-Mandarin. Always via English — there is no direct fr↔de, fr↔es, etc. Pass the target language as a BCP-47 code via --translate --target-language <code>; the source language is inferred from the audio.

See IBM's model card for training data, intended use, and upstream evaluation methodology.

Licensed Apache-2.0. Ported from upstream commit bd87ab8, pinned 2026-05-17.

Download

Quantization Download Size WER (LibriSpeech test-clean)
BF16 granite-4.0-1b-speech-BF16.gguf 4.63 GB 1.42%
F16 granite-4.0-1b-speech-F16.gguf 4.63 GB 1.42%
Q8_0 granite-4.0-1b-speech-Q8_0.gguf 2.56 GB 1.44%
Q6_K granite-4.0-1b-speech-Q6_K.gguf 2.02 GB 1.41%
Q5_K_M granite-4.0-1b-speech-Q5_K_M.gguf 1.83 GB 1.42%
Q4_K_M granite-4.0-1b-speech-Q4_K_M.gguf 1.60 GB 1.48%

WER measured on the full LibriSpeech test-clean split (2620 utterances) with greedy decoding. The BF16 reference baseline (transformers, re-run locally with the model-card prompt USER: <|audio|>can you transcribe the speech into a written format?\n ASSISTANT:) is 1.42%, matching IBM's published Open ASR Leaderboard number exactly. Text normalizer: Whisper EnglishTextNormalizer, the same normalizer Open ASR Leaderboard uses.

Quick Start

cmake -B build
cmake --build build

build/bin/transcribe-cli \
  -m models/granite-4.0-1b-speech/granite-4.0-1b-speech-Q8_0.gguf \
  samples/jfk.wav

If your audio is not already 16 kHz mono WAV, convert it first:

ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav

Translation (granite uses one chat template, parameterized by target language):

build/bin/transcribe-cli \
  -m models/granite-4.0-1b-speech/granite-4.0-1b-speech-Q8_0.gguf \
  --translate --target-language de \
  samples/jfk.wav

Performance

Cells are wall-clock latency, with speedup over realtime in parentheses.

Apple M4 Max

Mean over 3 iterations after 1 warmup.

Metal

Sample Q4_K_M Q8_0
jfk (11.0s) 254 ms (43×) 284 ms (39×)
dots (35.3s) 928 ms (38×) 1.05 s (34×)

CPU

Sample Q4_K_M Q8_0
jfk (11.0s) 1.48 s (7.4×) 1.80 s (6.1×)
dots (35.3s) 5.15 s (6.9×) 5.96 s (5.9×)

macOS 26.4, transcribe.cpp de05c43.

Apple M4

Mean over 5 iterations after 2 warmups. Q8_0.

Backend Sample Q8_0
Metal jfk (11.0s) 959 ms (11×)
CPU jfk (11.0s) 2.44 s (5×)

macOS 26.1, transcribe.cpp 275332d.

AMD Ryzen 7 PRO 4750U (Vega 8 iGPU)

Mean over 3 iterations after 1 warmup.

Vulkan (RADV)

Sample Q4_K_M Q8_0
jfk (11.0s) 3.47 s (3.2×) 3.71 s (3.0×)
dots (35.3s) 11.37 s (3.1×) 12.30 s (2.9×)

CPU

Sample Q4_K_M Q8_0
jfk (11.0s) 5.21 s (2.1×) 6.72 s (1.6×)
dots (35.3s) 18.08 s (1.9×) 24.21 s (1.5×)

Linux 6.18 (Fedora 43), transcribe.cpp dbe5814.

Capabilities

Capability Status
Transcribe (English) Yes
Transcribe (fr/de/es/pt/ja) Yes
Translate (en↔ASR, en→it/zh) Yes (--translate --target-language <bcp47>)
Word-level timestamps No (use the -plus variant)
Speaker diarization No (upstream supports via prompt; not exposed in v1 of transcribe.cpp)

Numerical Validation

Tensor-level parity with the transformers reference on samples/jfk.wav. Per-tensor max_abs / mean_abs budgets in tests/tolerances/granite.json. Drift is dominated by BF16 reduction-order noise in the 40-layer LLM stack plus a localized band at the last Shaw block-local attention window boundary in enc.block.15.out. No structural deltas vs the reference.

Reproduction

Convert

uv run --project scripts/envs/granite \
  scripts/convert-granite.py ibm-granite/granite-4.0-1b-speech \
  --repo-id ibm-granite/granite-4.0-1b-speech

Quantize

for PRESET in F16 Q8_0 Q6_K Q5_K_M Q4_K_M; do
  build/bin/transcribe-quantize \
    models/granite-4.0-1b-speech/granite-4.0-1b-speech-BF16.gguf \
    models/granite-4.0-1b-speech/granite-4.0-1b-speech-${PRESET}.gguf \
    --quant ${PRESET}
done

Validate

uv run scripts/validate.py all --family granite --variant granite-4.0-1b-speech

Reproduce WER

uv run scripts/wer/run.py \
  --model models/granite-4.0-1b-speech/granite-4.0-1b-speech-BF16.gguf \
  --manifest samples/wer/test-clean.manifest.jsonl \
  --out reports/wer/granite-4.0-1b-speech-BF16.test-clean.jsonl
uv run scripts/wer/score.py reports/wer/granite-4.0-1b-speech-BF16.test-clean.jsonl