Fix IndexError in HunyuanVideo I2V pipeline

[kaixuanliu]

@yiyixuxu @asomoza, pls help review, thx!

[kaixuanliu]

Problem
HunyuanVideoImageToVideoPipeline fails with IndexError: index -1 is out of bounds for dimension 1 with size 0 when using latest version transformers.

Cause
The code searches for [double_return_token_id] (271) representing \n\n to locate the assistant section. In latest transformers, Llama tokenizers no longer produce a separate token for \n\n, returning an empty tensor and causing reshape to fail.

Fix
Added fallback using <|end_header_id|> position + 1 when the double newline token is not found:

[kaixuanliu]

Here is the sample code to reproduce:

import torch
from diffusers import HunyuanVideoImageToVideoPipeline, HunyuanVideoTransformer3DModel
from diffusers.utils import load_image, export_to_video

# Available checkpoints: "hunyuanvideo-community/HunyuanVideo-I2V" and "hunyuanvideo-community/HunyuanVideo-I2V-33ch"
model_id = "hunyuanvideo-community/HunyuanVideo-I2V"
transformer = HunyuanVideoTransformer3DModel.from_pretrained(
    model_id, subfolder="transformer", torch_dtype=torch.bfloat16
)
pipe = HunyuanVideoImageToVideoPipeline.from_pretrained(
    model_id, transformer=transformer, torch_dtype=torch.float16
)
pipe.vae.enable_tiling()
pipe.to("cuda")

prompt = "A man with short gray hair plays a red electric guitar."
image = load_image(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png"
)

output = pipe(image=image, prompt=prompt).frames[0]
export_to_video(output, "output.mp4", fps=15)

[kaixuanliu]

Hi, @yiyixuxu @asomoza, can you help review this PR? Thx!

[kaixuanliu]

@DN6 , Hi, can you help review? Thx

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[kaixuanliu]

@DN6 Hi, Can we merge this PR?

[yiyixuxu]

thanks for the PR!
i left one comment

[yiyixuxu]

should we just replace the hard coded numbers with this logic instead? regardless the transformer version?

[kaixuanliu]

Thx for the advice. Have updated the code.

[kaixuanliu]

Hi, @yiyixuxu @DN6 Can you help review it again? Thx!

[yiyixuxu]

thanks!
Do you know which version of Transformeer introduced this behavior?
Would you be able to run a test with an older transnfomer version to make sure we don't break backward?

[yiyixuxu]

should we change the name of this variable now?

[kaixuanliu]

Oh, yes. Thx for the advice~ Done for it.

[kaixuanliu]

It should be an incompatible bug from transformers 5.0.0.

• transformers 4.57.6 OK
• transformers 5.0.0 Failed

And when I use this fix with both transformers 4.57.6 and 5.0.0, it works well.

[yiyixuxu]

thanks so much for working on this! @kaixuanliu