Pytorch 2.2 releases flash attention v2

[toilaluan]

Today, pytorch released version 2.2, which include flash attention v2 for scaled_dot_product_attention
Do it help speed up diffusers/stable_diffusion?
https://pytorch.org/blog/pytorch2-2/?utm_content=280360799&utm_medium=social&utm_source=linkedin&hss_channel=lcp-78618366

[sayakpaul]

You can benchmark it. We provide some benchmarking utilities here: https://github.com/huggingface/diffusers/tree/main/benchmarks