Support Export of LTX-Video 0.9.1

[Yash-Vijay29]

What does this PR do?

Adds support for to successfully convert LTX-Video 0.9.1 (a timestep_conditioning enabled version of LTX Video)
to be exported to IR graph. previously timestep_conditioning was not supported in exports and caused dummy timestep_conditioning to produce tensor a mismatch.

Created tiny-random-ltx-video-0.9.1 for github CI and testing.

Added inference and conversion tests.

TESTED HF VS OPTIMUM ACCURACY:

first ran :

wwb --base-model Lightricks/LTX-Video-0.9.1 --gt-data video_gen_test_ts/gt.csv --model-type text-to-video --hf --decode-timestep 0.05 --decode-noise-scale 0.025 --num-samples 10

then

wwb --target-model ltx-video-0.9.1-ov --gt-data video_gen_test_ts/gt.csv --model-type text-to-video --output ltx_video_genai_ts --decode-timestep 0.05 --decode-noise-scale 0.025 --num-samples 10

Accuracy: 0.9844858
Metrics_per_question:
metrics.csv

Before submitting

• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[likholat]

Please add tests for tiny-random-ltx-0.9.1-video model conversion and inference.
You can refer to similar tests that already exist for the supported LTX model:

repo:huggingface/optimum-intel ltx-video path:/^tests\/openvino\//

[Yash-Vijay29]

Please add tests for tiny-random-ltx-0.9.1-video model conversion and inference. You can refer to similar tests that already exist for the supported LTX model:

repo:huggingface/optimum-intel ltx-video path:/^tests\/openvino\//

done. Please review.

[likholat]

@rkazants please take a look

[rkazants]

@altnnatra, please take a look

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Yash-Vijay29]

@rkazants
Altered only Ltx pipeline in model patcher, tests seem to pass for the new test file i made,

Tried exporting the test model and it exports, let me know if any other changes.

Please rerun checks! thank you

[anatyrova]

Tests cannot be collected and export fails in environments with huggingface_hub>=1.0 due to ImportError: cannot import name 'HfFolder' in modeling_utils.py (pre-existing issue in main, not introduced by this PR). Could you pls confirm which version of huggingface_hub you tested with, @Yash-Vijay29 ?

[Yash-Vijay29]

Tests cannot be collected and export fails in environments with huggingface_hub>=1.0 due to ImportError: cannot import name 'HfFolder' in modeling_utils.py (pre-existing issue in main, not introduced by this PR). Could you pls confirm which version of huggingface_hub you tested with, @Yash-Vijay29 ?

@anatyrova
0.36.2 was the version of huggingface_hub i got in my setup

[anatyrova]

The branch is based on an older commit of main where HfFolder was still used in modeling_utils.py. This has since been fixed in main. I think, rebase onto the current main will resolve the import error and unblock validation. Please try this.

[Yash-Vijay29]

@anatyrova updated branch to latest commit, exporting model and "python -m pytest tests/openvino/test_ltx_export_config.py -v" passes

huggingface_hub used is still 0.36.2 (it auto downloaded that one)

Try running again! thank you!

[anatyrova]

• export in float16 run successfully on CPU but the vid shows artifacts
• inference run successfully

[Yash-Vijay29]

• export in float16 run successfully on CPU but the vid shows artifacts

  • inference run successfully

Oh. I tested in fp32 format for the accuracy result of 0.9845.
perhaps more degradation in fp16?

[anatyrova]

Was your accuracy result (0.9845) measured on the fp32 export?

[Yash-Vijay29]

Was your accuracy result (0.9845) measured on the fp32 export?

Yes it was

[anatyrova]

i tried fp32, it looked best (vid applied).

also int8, int4 do work.

https://github.com/user-attachments/assets/5b15b597-8090-4b05-8fef-2d371cb08cc1 (prompt='A cute cat climbing up a staircase, smooth motion, detailed fur, cinematic lighting')

need to confirm with @rkazants

[Yash-Vijay29]

@anatyrova what version of diffusers is CI using?
its unable to process the tiny-random-ltx-video-0.9.1 's
"timestep_scale_multiplier", the one I used for testing was diffusers 0.37.1
and the tests that failed on the CI have passed on my VM

I think the diffusers on the CI is a bit old? and doesnt have the "timestep_scale_multiplier"
in it

[anatyrova]

i reviewed the pr for the case if it creates any regression for the prev LTX-video that we already have (m_id Lightricks/LTX-Video)
all in all, pr 1652 does not create a regression for the existing LTX.

1. i compared first
   GT — HF PyTorch on main branch
   Target — OV export on pr-1652 branch
2. then i compared
   GT — HF PyTorch on main
   Target — OV export on main

similarity results (wwb) were almost identical, both with 0.838127 (main) and 0.83812696 (pr) similarity comp. to OG model. @rkazants