Skip to content

canary-1b-flash.md

Latest commit

 

History

History
151 lines (113 loc) · 5.33 KB

canary-1b-flash.md

File metadata and controls

151 lines (113 loc) · 5.33 KB
Raw

Canary 1B Flash

NVIDIA's nvidia/canary-1b-flash ported to transcribe.cpp. An 883M-parameter multitask AED with a 32-layer FastConformer encoder and a 4-layer Transformer decoder.

What it's for

Offline multilingual speech-to-text and translation. The model takes a 16 kHz mono WAV and produces a transcript. Supports:

  • ASR in English, German, Spanish, and French (with explicit language hint).
  • Translation between English and German, Spanish, or French (both directions).

See NVIDIA's model card for training data, intended use, and upstream evaluation methodology.

Licensed CC-BY-4.0. Ported from upstream commit a9a55e0, pinned 2026-05-08.

Download

Quantization Download Size WER (LibriSpeech test-clean)
F32 canary-1b-flash-F32.gguf 3.3 GB 1.62%
F16 canary-1b-flash-F16.gguf 1.7 GB 1.62%
Q8_0 canary-1b-flash-Q8_0.gguf 1.0 GB 1.62%
Q6_K canary-1b-flash-Q6_K.gguf 818 MB 1.65%
Q5_K_M canary-1b-flash-Q5_K_M.gguf 734 MB 1.64%
Q4_K_M canary-1b-flash-Q4_K_M.gguf 646 MB 1.59%

WER is measured on the full LibriSpeech test-clean split (2620 utterances) with greedy decoding and no external LM. F32 reference baseline: 1.62%. NVIDIA's self-reported number on the upstream model card is 1.48%

Quick Start

cmake -B build
cmake --build build

# ASR (English)
build/bin/transcribe-cli \
  -m models/canary-1b-flash/canary-1b-flash-Q8_0.gguf \
  -l en \
  samples/jfk.wav

# Translation (English audio → German text)
build/bin/transcribe-cli \
  -m models/canary-1b-flash/canary-1b-flash-Q8_0.gguf \
  --task translate \
  -l en --target-language de \
  samples/jfk.wav

If your audio is not already 16 kHz mono WAV, convert it first:

ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav

CLI flags specific to canary:

  • --pnc / --no-pnc — punctuation & capitalization (default on).
  • -l <code> — source language code (en, de, es, fr).
  • --task translate + --target-language <code> — switch to translation mode.

Performance

Cells are wall-clock latency (mean over 3 iterations after 1 warmup), with speedup over realtime in parentheses. Units: ms below 1 s, s above (2 decimal places).

Apple M4 Max

Backend Sample Q8_0 Q4_K_M
Metal jfk (11.0s) 106.6 ms (103.2×) 111.3 ms (98.8×)
Metal dots (35.3s) 368.6 ms (95.9×) 349.2 ms (101.2×)
CPU jfk (11.0s) 518.3 ms (21.2×) 430.1 ms (25.6×)
CPU dots (35.3s) 1.79 s (19.7×) 1.51 s (23.4×)

macOS 26.4.1, transcribe.cpp 19b3b87.

AMD Ryzen 7 PRO 4750U

Backend Sample Q8_0 Q4_K_M
Vulkan jfk (11.0s) 763.5 ms (14.4×) 704.2 ms (15.6×)
Vulkan dots (35.3s) 2.46 s (14.4×) 2.31 s (15.3×)
CPU jfk (11.0s) 1.49 s (7.4×) 1.13 s (9.7×)
CPU dots (35.3s) 5.48 s (6.4×) 4.51 s (7.8×)

Fedora Linux 43, transcribe.cpp 4d44530. Vulkan device: AMD Radeon Graphics (RADV RENOIR).

Benchmark reproduction:

uv run scripts/bench/run.py \
  --models canary-1b-flash \
  --quants q8_0,q4_k_m \
  --samples jfk,dots \
  --backends metal,cpu,vulkan \
  --iters 3 --warmup 1 \
  --name canary-1b-flash-publication

Numerical Validation

transcribe.cpp is validated tensor-by-tensor against NeMo on samples/jfk.wav. All checkpointed tensors fall within family tolerance and the F32 transcript matches the NeMo reference. Last validated at commit db53eda.

Field Value
Reference NeMo, nvidia/canary-1b-flash
Dump script scripts/dump_reference_canary_nemo.py
Manifest tests/golden/canary/canary-1b-flash.manifest.json
Tolerances tests/tolerances/canary.json
Command uv run scripts/validate.py all --family canary --variant canary-1b-flash

For the full porting writeup, see docs/porting/families/canary.md.

Reproduction

Convert

uv run --project scripts/envs/canary \
  scripts/convert-canary.py nvidia/canary-1b-flash --repo-id nvidia/canary-1b-flash

Quantize

uv run scripts/quantize-all.py models/canary-1b-flash/canary-1b-flash-F32.gguf

Validate

uv run scripts/validate.py all --family canary --variant canary-1b-flash