Skip to content

Commit 2499159

Browse files
authored
feat(AutoModelVLLM): warn when a single audio input is too long to decode in one pass (#3033)
Fun-ASR-Nano is a segment-level (LLM-)ASR model. Feeding very long audio (e.g. a 2-minute meeting recording) directly to AutoModelVLLM decodes it in a single pass, which hits max_new_tokens long before the audio ends -> the user gets a silently truncated transcript with no error (verified: a 126s clip returns ~half the text of the VAD-segmented decode, 328 vs 680 chars). Add a one-time warning when an input exceeds a safe length, pointing users to VAD pre-segmentation or the high-level AutoModel (which segments automatically). This does not change any output; it just makes the truncation visible instead of silent. Refs #3031 (the catastrophic repetition reported there was the old hardcoded repetition_penalty=1.3 crash in prompt-embeds mode, already fixed in #2974/1.3.10; the remaining issue on long audio is this silent truncation). Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
1 parent f17ed67 commit 2499159

1 file changed

Lines changed: 39 additions & 0 deletions

File tree

funasr/auto/auto_model_vllm.py

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,8 +296,47 @@ def generate(self, inputs, **kwargs):
296296
Returns:
297297
List of result dicts with "key" and "text" fields.
298298
"""
299+
self._warn_if_audio_too_long(inputs)
299300
return self._engine.generate(inputs, **kwargs)
300301

302+
def _warn_if_audio_too_long(self, inputs, max_safe_sec=40.0):
303+
"""Warn (once) if a single audio input is long enough to be truncated.
304+
305+
Fun-ASR-Nano is a segment-level (LLM-)ASR model. Decoding very long
306+
audio in a single pass can silently truncate or degrade the output -- the
307+
decode hits ``max_new_tokens`` long before the audio ends, so the user
308+
gets a partial transcript with no error. The right usage is to
309+
pre-segment with VAD; this warning points users there instead of letting
310+
them get a silently truncated result. It does not change the output.
311+
"""
312+
if getattr(self, "_warned_audio_too_long", False):
313+
return
314+
items = inputs if isinstance(inputs, (list, tuple)) else [inputs]
315+
for item in items:
316+
duration = None
317+
try:
318+
if isinstance(item, str):
319+
import soundfile as sf
320+
321+
duration = sf.info(item).duration
322+
elif isinstance(item, np.ndarray):
323+
duration = item.shape[-1] / 16000.0
324+
elif isinstance(item, torch.Tensor):
325+
duration = item.shape[-1] / 16000.0
326+
except Exception:
327+
continue
328+
if duration is not None and duration > max_safe_sec:
329+
logger.warning(
330+
"AutoModelVLLM received a %.0fs audio input. Fun-ASR-Nano is a "
331+
"segment-level model; decoding very long audio in a single pass can "
332+
"truncate or degrade the result. Pre-segment with VAD and pass the "
333+
"segments, or use the high-level `funasr.AutoModel(model=..., "
334+
'vad_model="fsmn-vad")`, which segments long audio automatically.',
335+
duration,
336+
)
337+
self._warned_audio_too_long = True
338+
break
339+
301340
@classmethod
302341
def supported_models(cls):
303342
"""Return dict of model types and their vLLM support status."""

0 commit comments

Comments
 (0)