Skip to content

Commit 8107365

Browse files
authored
fix(fun_asr_nano): warn when vLLM dtype=fp16 (degraded output) (#2980)
Fun-ASR-Nano vLLM inference produces degraded/garbage transcription in fp16 due to numerical overflow in the audio embedding path (reproduced: degenerate repeated tokens). bf16 and fp32 are correct. Add a clear warning recommending bf16 (or fp32 on V100-class GPUs without bf16) so users do not hit silent garbage output. Refs #2976. Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
1 parent e6ab4c3 commit 8107365

1 file changed

Lines changed: 7 additions & 0 deletions

File tree

funasr/models/fun_asr_nano/inference_vllm.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -175,6 +175,13 @@ def __init__(
175175
self.device = device
176176
self.dtype = dtype
177177
self.torch_dtype = dtype_map.get(dtype, torch.bfloat16)
178+
if self.torch_dtype == torch.float16:
179+
logger.warning(
180+
"dtype='fp16' can produce degraded or garbage transcription for "
181+
"Fun-ASR-Nano (numerical overflow in the audio embedding path). "
182+
"Use dtype='bf16' (recommended) or dtype='fp32'. On GPUs without "
183+
"bfloat16 support (e.g. NVIDIA V100), use 'fp32'."
184+
)
178185
self.model_dir = model_dir
179186

180187
# Step 1: Prepare LLM weights for vLLM (extract from model.pt if needed)

0 commit comments

Comments
 (0)