Extremely slow editing of image using Instruct pix2pix in Diffusers

[alexd725]

Hi guys.

Cuda 12.0.
Memory: 16GB, Tesla T4. nproc output: 4

Im struggling with using timbrooks/instruct-pix2pix model.
The loading time of this model is less than 3 seconds.

But editing the image takes about 20 seconds. I have seen people get around 3s, how is that even achievable?

Here is my code

begin = time.time()
model_id = "timbrooks/instruct-pix2pix"
pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(model_id, torch_dtype=torch.float16, safety_checker=None)
print(f"Load             : {time.time()-begin}")
pipe.to("cuda")
print(f"Load+cuda        : {time.time()-begin}")
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
print(f"Load+scheduler   : {time.time()-begin}")
pipe.enable_xformers_memory_efficient_attention()
print(f"Load+cuda+xfomers: {time.time()-begin}")

# Apply effect
images = pipe(text_prompt, image=src_image, num_inference_steps=10).images
print(f"Load+image       : {time.time()-begin}")
image = images[0]

And here is console log.

Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:01<00:00,  4.39it/s]
Load             : 1.5197396278381348
Load+cuda        : 2.2462241649627686
Load+scheduler   : 2.2477033138275146
Load+cuda+xfomers: 2.2915024757385254
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:11<00:00,  1.11s/it]
Load+image       : 16.677665948867798

Editing image takes 16 seconds here. How can i speed it up?
Any help would be appreciated! Thanks in advance.
If you are open to detailed discussion, pls message me 'alex725.dev@gmail.com'

[alexd725]

Its taking about 60 seconds with 50 steps.

[tolgacangoz]

You can compile the UNet like this:

pipe.unet = torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True)

Except for the first inference, this can help. Also, see the documentation.

[alexd725]

@StandardAI, Thanks for your help. But it doesnot help to speed up.
I think there is a main issue in my environemnt or code, but not sure what it is. Normally the editing speed is less than 4s on others without having additional operation.

[tolgacangoz]

I agree. Colab's T4 seems pretty fast compared to yours.

[alexd725]

Amazing, It looks pretty fast.
Could you please let me know how i can use it?

[alexd725]

Why mine is so slower than the Colab'sT4?

[alexd725]

I found the way. Its taking 1 second now. Im really appreciate for your help.

[asomoza]

compiling the unet takes longer the first time, but is faster after that but that's only useful if you're not unloading the unet or stopping the script with just one inference.

What I meant is to use the command:

diffusers-cli env

in the console and paste the results of your environment, you should get something like this:

- `diffusers` version: 0.27.0
- Platform: Linux-6.1.58+-x86_64-with-glibc2.35
- Python version: 3.10.12
- PyTorch version (GPU?): 2.2.1+cu121 (True)
- Huggingface_hub version: 0.20.3
- Transformers version: 4.38.2
- Accelerate version: 0.28.0
- xFormers version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

[alexd725]

I tried with it several times. While testing, i noticed that it takes longer in case of big file. why it looks pretty fast in Colab's T4 is because the image was resized to 512 x 512.
Is there any way to speed up to edit image in case of big file?

[asomoza]

yeah that was one key difference in the colab, the bigger the generation size is, the more time it will take and this is for all things related to stable diffusion and AI in general.

There's only one caveat, if the image is really big and you're using all your VRAM it could swap to use RAM instead and that would make it really slow, don't know if that is what is happening in your case without more information.

[alexd725]

Thank you very much, @asomoza

[alexd725]

@asomoza , Im using 2.2.1 for pytorch

[tolgacangoz]

Maybe, this blog post helps further.

[sayakpaul]

Also, the compilation time would vary from hardware to hardware as the optimisation code path varies quite a bit. So, this isn't too unexpected IMO. torch.compile() and SDPA (scaled dot product attention) both tend to perform well together on relatively newer chips like 3090, 4090, etc.

[alexd725]

Thanks, I got it.