Can I use same Diffuser Logic to inference Wan2.2 and will Causvid Lora work with wan22 ?

[mayankagrawal10198]

Hi, Can I use same wan21 diffuser logic to run wan22 ? Please refer this https://huggingface.co/docs/diffusers/main/en/api/pipelines/wan

As I will load model something like this and store it in local dir

# Create directory if it doesn't exist
Path(local_path).mkdir(parents=True, exist_ok=True)

# Download the model
# Wan2.2-I2V-A14B-Diffusers
snapshot_download(
    repo_id=model_id,
    local_dir=local_path,
    local_dir_use_symlinks=False,
    resume_download=True
)

logger.info(f"✅ Successfully downloaded {model_id}")
return True

Then will load the model and run the pipline like this

Load image encoder

    logger.info("Loading image encoder...")
    self.image_encoder = CLIPVisionModel.from_pretrained(
        local_model_path,
        subfolder="image_encoder",
        torch_dtype=torch.float32,
        local_files_only=self.local_files_only,
        low_cpu_mem_usage=True
    )
    
    # Load VAE
    logger.info("Loading VAE...")
    self.vae = AutoencoderKLWan.from_pretrained(
        local_model_path,
        subfolder="vae",
        torch_dtype=torch.float32,
        local_files_only=self.local_files_only,
        low_cpu_mem_usage=True
    )
    
    # Load main pipeline
    logger.info("Loading main pipeline...")
    self.pipe = WanImageToVideoPipeline.from_pretrained(
        local_model_path,
        vae=self.vae,
        image_encoder=self.image_encoder,
        torch_dtype=self.torch_dtype,
        local_files_only=self.local_files_only,
        low_cpu_mem_usage=True
    )
    
    # Load CausVid LoRA (if enabled)
    if self.enable_lora:
        logger.info(f"Loading CausVid LoRA: {self.lora_path}/{self.lora_filename}")
        try:
            # Load LoRA weights following the reference pattern
            self.pipe.load_lora_weights(
                self.lora_path, 
                weight_name=self.lora_filename,
                adapter_name=self.lora_adapter_name
            )
            # Set adapter with strength as a float (not in a list)
            self.pipe.set_adapters(self.lora_adapter_name, self.lora_strength)
            logger.info(f"LoRA loaded successfully with strength: {self.lora_strength}")
            self.lora_loaded_successfully = True
        except Exception as e:
            logger.warning(f"Failed to load LoRA: {e}")
            logger.warning("Continuing without LoRA...")
            self.enable_lora = False
            self.lora_loaded_successfully = False
            
            #Run Wan22
 with torch.no_grad():
            output = self.pipe(
                image=image,
                prompt=prompt,
                negative_prompt=negative_prompt,
                height=height,
                width=width,
                num_frames=num_frames,
                guidance_scale=guidance_scale,
                num_inference_steps=num_inference_steps
            ).frames[0]

[asomoza]

Hi, in diffusers WAN 2.2 and WAN 2.1 share the same API, so any code for inference and model loading will work for both.

About the Causvid lora, you will have to test it, I read that some loras do work and other don't , I have read a lot of people that say that the light2xv lora work but haven't read anyone saying something about Causvid.

[mayankagrawal10198]

Hi, @asomoza got this error when used same diffuser API ->
No file named pytorch_model.bin, model.safetensor, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory.

I guess they have to figure out to load high noise and then low noise model sequentially

[asomoza]

first I'm going to assume you installed from main since we still haven't had a diffusers release so it only works if you install from source.

second, when people have that error, 99.9% of the time is because they use custom code with some local models that don't work, so to test if it's a problem with diffusers or with your own custom code, you should test with a huggingface repo-id instead of doing a local snapshot or manually downloading the model. But the error you're posting is because diffusers can't find the necessary model files to load them.

To answer what I think is your question, the pipeline automatically manages the low noise and high noise transformer models, you don't have to do anything special if you use the diffusers API.