Add speech-LM text-to-speech sample (Qwen3-TTS, Compiled Model host loop)

[john-rocky]

What

A text-to-speech sample for the speech-LM model family: Qwen3-TTS-12Hz-0.6B-Base (Apache-2.0, 10 languages, 3-second voice cloning) running fully on device as three LiteRT graphs orchestrated by a host-side loop with the Compiled Model API. It complements the Matcha-TTS sample (#162): Matcha covers the compact acoustic-model family, this covers the modern codec-LM family (talker LM → 12.5 Hz × 16-codebook audio tokens → neural codec decoder → 24 kHz PCM).

compiled_model_api/text_to_speech_lm/

• python/ — pipeline library + CLI. Models auto-download from litert-community/Qwen3-TTS-12Hz-0.6B-Base (~1.4 GB default int4 configuration).
• kotlin_cpu/android/ — Android app (Kotlin, Compiled Model API on CPU): byte-level BPE tokenizer with a startup self-test against reference token ids, memory-mapped fp16 embedding tables, signature-based talker prefill/decode with ping-pong KV-cache buffers (no per-step cache copies), the 17-invoke MTP inner loop, chunked codec decode, AudioTrack playback. install_to_device.sh fetches the models from Hugging Face and pushes them to the app's filesDir.
• conversion/ — full conversion pipeline: talker checkpoint synthesis (the stock litert-torch LLM exporter applies unchanged), MTP decode-step graph authoring, codec decoder export, host-table extraction, and voice enrollment.

cd compiled_model_api/text_to_speech_lm/python
pip install -r requirements.txt
python synthesize.py --text "Hello from LiteRT running fully on device." --output hello.wav

Why a host-side loop

Qwen3-TTS's decode loop cannot be expressed by the LiteRT-LM Engine today: each step aggregates 16 codebook embeddings plus streamed per-frame text conditioning, needs the hidden state alongside logits (it seeds a 15-step inner AR loop over a second transformer with per-step lm_heads), and outputs audio rather than text. The host loop in this sample is a reference for what an Engine-level audio-LM session would absorb.

Verification

• Talker: the TTS multimodal RoPE provably reduces to standard RoPE (bit-exact at torch level); .tflite decode sweep correlation 1.0, top-1 100%.
• MTP: 15/15 greedy tokens vs the reference inner loop.
• Codec decoder: correlation 1.0 (max abs diff 1.8e-5).
• End to end (fp32 talker, greedy): token-for-token identical codes to the PyTorch reference, waveform correlation 1.000000; ASR round-trip returns the input sentence. The int4 talker output also transcribes exactly.
• Android app: device-verified on Pixel 8a (tokenizer self-test passes on device, including Japanese input).

Performance

Per 80 ms audio frame	M4 Max CPU (Python)	Pixel 8a (Kotlin)
Talker decode	45–50 ms	~60 ms
MTP inner loop (17 invokes)	~148 ms	~370 ms
Codec decode (amortized)	~10 ms	~98 ms
Total	~205 ms → RTF ≈ 2.5	~530 ms → RTF ≈ 6.7

Pixel 8a end to end: 4.16 s of audio in 28.0 s. The MTP inner loop is the known optimization target (single unrolled graph + calibrated low-bit weights).