[UNet2DConditionModel] I don't want to perform conditional embedding on the UNet network

[island443]

predict the noise residual

model_pred = unet(noisy_latents, timesteps, encoder_hidden_states).sample

I don't want to perform conditional embedding on the UNet network,It means not passing in the encoder_hidden_state parameter when predicting noise.What should I do? Do I need to modify the [UNet2DConditionModel] source code?

[tolgacangoz]

Does UNet2DModel help?

[island443]

Thank you for your answer, I see that pre trained stable diffusion models are all using UNet2DConditionModel instead of UNet2DModel. Can I use UNet2DModel for latents space?
This seems to require retraining rather than fine-tuning？

[tolgacangoz]

If Stable Diffusion models' UNet2DConditionModel have trained with encoder_hidden_states parameter, then I think we have to use it, otherwise it would be another network. Does LDMPipeline help?

[island443]

LDMPipeline has been very helpful to me. Thank you very much again and wish you every success。