Skip to content

Fix crash when creating tensor from empty buffer in whisper#454

Open
kidq330 wants to merge 2 commits intoelixir-nx:mainfrom
kidq330:main
Open

Fix crash when creating tensor from empty buffer in whisper#454
kidq330 wants to merge 2 commits intoelixir-nx:mainfrom
kidq330:main

Conversation

@kidq330
Copy link
Copy Markdown

@kidq330 kidq330 commented Apr 21, 2026

Hi, I've run into an issue when my audio tensor is exactly aligned with the chunk length for the Whisper serving, here's an MRE script:

Mix.install(
  [
    {:bumblebee, "~> 0.6"},
    {:exla, "~> 0.10"}
  ],
  config: [nx: [default_backend: EXLA.Backend]]
)

hf_repo = "openai/whisper-tiny"

{:ok, whisper} = Bumblebee.load_model({:hf, hf_repo})
{:ok, featurizer} = Bumblebee.load_featurizer({:hf, hf_repo})
{:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, hf_repo})
{:ok, generation_config} = Bumblebee.load_generation_config({:hf, hf_repo})

serving =
  Bumblebee.Audio.speech_to_text_whisper(
    whisper,
    featurizer,
    tokenizer,
    generation_config,
    stream: false,
    chunk_num_seconds: 10,
    context_num_seconds: 0,
    defn_options: [compiler: EXLA]
  )

sample_rate = 16_000
chunk_num_seconds = 10
audio = Nx.broadcast(0.0, {chunk_num_seconds * sample_rate})

serving
|> Nx.Serving.run(audio)
|> Enum.to_list()

Output (same with e.g. chunk_num_seconds = 20):

** (ArgumentError) no tensors were given to concatenate
    (nx 0.10.0) lib/nx.ex:14684: Nx.concatenate/2
    (bumblebee 0.6.3) lib/bumblebee/audio/speech_to_text_whisper.ex:320: anonymous fn/1 in Bumblebee.Audio.SpeechToTextWhisper.chunk_input/4
    (elixir 1.19.5) lib/stream.ex:982: Stream.do_transform/5
    (elixir 1.19.5) lib/stream.ex:1773: Enumerable.Stream.do_each/4
    (elixir 1.19.5) lib/enum.ex:4570: Enum.map_reduce/3
    (nx 0.10.0) lib/nx/serving.ex:697: Nx.Serving.run/2
    examples/mre.exs:33: (file)

I've only run into this issue when disabling overlapping with context_num_seconds: 0, since it would be hard to align the windows length with the default value of 1/6th of the chunk length.

Comment thread lib/bumblebee/audio/speech_to_text_whisper.ex
Copy link
Copy Markdown
Member

@jonatanklosko jonatanklosko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants