You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Compilation is slow the first time, but once compiled, it is significantly faster. Try to only use the compiled pipeline on the same type of inference operations. Calling the compiled pipeline on a different image size retriggers compilation which is slow and inefficient.
152
152
153
-
### Compilation on shape changes
153
+
### Dynamic shape compilation
154
154
155
-
`torch.compile()` maintains a stack of "guards" for the shapes and conditions it sees when it is triggered. When that is violated, the compiler triggers recompilation. This means that if a model was compiled on the 1024x1024 resolution, for example, it will trigger recompilation if it is called on a different resolution.
155
+
> [!TIP]
156
+
> Make sure to always use the nightly version of PyTorch for better support.
157
+
158
+
`torch.compile` keeps track of input shapes and conditions, and if these are different, it recompiles the model. For example, if a model is compiled on a 1024x1024 resolution image and used on an image with a different resolution, it triggers recompilation.
156
159
157
-
In these cases, it's beneficial to compile with `dynamic=True`:
160
+
To avoid recompilation, add `dynamic=True` to try and generate a more dynamic kernel to avoid recompilation when conditions change.
@@ -163,9 +166,11 @@ In these cases, it's beneficial to compile with `dynamic=True`:
163
166
)
164
167
```
165
168
166
-
Make sure to always use the nightly version of PyTorch for this. Specifying `use_duck_shape` to be `False` instructs the compiler if it should use the same symbolic variable to represent input sizes that are the same. For more details, check out [this comment](https://github.com/huggingface/diffusers/pull/11327#discussion_r2047659790).
169
+
Specifying `use_duck_shape=False` instructs the compiler if it should use the same symbolic variable to represent input sizes that are the same. For more details, check out this [comment](https://github.com/huggingface/diffusers/pull/11327#discussion_r2047659790).
170
+
171
+
Not all models may benefit from dynamic compilation out of the box and may require changes. Refer to this [PR](https://github.com/huggingface/diffusers/pull/11297/) that improved the [`AuraFlowPipeline`] implementation to benefit from dynamic compilation.
167
172
168
-
All models might not benefit from this out of the box and may require changes. Refer to [this PR](https://github.com/huggingface/diffusers/pull/11297/) that improved the implementation of [`AuraFlowPipeline`] to benefit from compilation with `dynamic=True`. Feel free to open an issue if dynamic compilation doesn't work expected for a model inside Diffusers.
173
+
Feel free to open an issue if dynamic compilation doesn't work as expected for a Diffusers model.
0 commit comments