You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Remote inference provides access to an [Inference Endpoint](https://huggingface.co/docs/inference-endpoints/index) to offload local generation requirements for decoding and encoding.
@@ -10,51 +10,296 @@ an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express o
10
10
specific language governing permissions and limitations under the License.
11
11
-->
12
12
13
-
# Hybrid Inference
13
+
# Remote inference
14
14
15
-
**Empowering local AI builders with Hybrid Inference**
15
+
> [!TIP]
16
+
> This is currently an experimental feature, and if you have any feedback, please feel free to leave it [here](https://github.com/huggingface/diffusers/issues/new?template=remote-vae-pilot-feedback.yml).
16
17
18
+
Remote inference offloads the decoding and encoding process to a remote endpoint to relax the memory requirements for local inference with large models. This feature is powered by [Inference Endpoints](https://huggingface.co/docs/inference-endpoints/index). Refer to the table below for the supported models and endpoint.
17
19
18
-
> [!TIP]
19
-
> Hybrid Inference is an [experimental feature](https://huggingface.co/blog/remote_vae).
20
-
> Feedback can be provided [here](https://github.com/huggingface/diffusers/issues/new?template=remote-vae-pilot-feedback.yml).
This guide will show you how to encode and decode latents with remote inference.
28
+
29
+
## Encoding
30
+
31
+
Encoding converts images and videos into latent representations. Refer to the table below for the supported VAEs.
32
+
33
+
Pass an image to [`~utils.remote_encode`] to encode it. The specific `scaling_factor` and `shift_factor` values for each model can be found in the [Remote inference](../hybrid_inference/api_reference) API reference.
34
+
35
+
```py
36
+
import torch
37
+
from diffusers import FluxPipeline
38
+
from diffusers.utils import load_image
39
+
from diffusers.utils.remote_utils import remote_encode
Decoding converts latent representations back into images or videos. Refer to the table below for the available and supported VAEs.
64
+
65
+
Set the output type to `"latent"` in the pipeline and set the `vae` to `None`. Pass the latents to the [`~utils.remote_decode`] function. For Flux, the latents are packed so the `height` and `width` also need to be passed. The specific `scaling_factor` and `shift_factor` values for each model can be found in the [Remote inference](../hybrid_inference/api_reference) API reference.
66
+
67
+
<hfoptionsid="decode">
68
+
<hfoptionid="Flux">
69
+
70
+
```py
71
+
from diffusers import FluxPipeline
72
+
73
+
pipeline = FluxPipeline.from_pretrained(
74
+
"black-forest-labs/FLUX.1-schnell",
75
+
torch_dtype=torch.bfloat16,
76
+
vae=None,
77
+
device_map="cuda"
78
+
)
79
+
80
+
prompt ="""
81
+
A photorealistic Apollo-era photograph of a cat in a small astronaut suit with a bubble helmet, standing on the Moon and holding a flagpole planted in the dusty lunar soil. The flag shows a colorful paw-print emblem. Earth glows in the black sky above the stark gray surface, with sharp shadows and high-contrast lighting like vintage NASA photos.
"A grainy Apollo-era style photograph of a cat in a snug astronaut suit with a bubble helmet, standing on the lunar surface and gripping a flag with a paw-print emblem. The gray Moon landscape stretches behind it, Earth glowing vividly in the black sky, shadows crisp and high-contrast.",
170
+
"A vintage 1960s sci-fi pulp magazine cover illustration of a heroic cat astronaut planting a flag on the Moon. Bold, saturated colors, exaggerated space gear, playful typography floating in the background, Earth painted in bright blues and greens.",
171
+
"A hyper-detailed cinematic shot of a cat astronaut on the Moon holding a fluttering flag, fur visible through the helmet glass, lunar dust scattering under its feet. The vastness of space and Earth in the distance create an epic, awe-inspiring tone.",
172
+
"A colorful cartoon drawing of a happy cat wearing a chunky, oversized spacesuit, proudly holding a flag with a big paw print on it. The Moon’s surface is simplified with craters drawn like doodles, and Earth in the sky has a smiling face.",
173
+
"A monochrome 1969-style press photo of a “first cat on the Moon” moment. The cat, in a tiny astronaut suit, stands by a planted flag, with grainy textures, scratches, and a blurred Earth in the background, mimicking old archival space photos."
The tables demonstrate the memory requirements for encoding and decoding with Stable Diffusion v1.5 and SDXL on different GPUs.
21
206
207
+
For the majority of these GPUs, the memory usage dictates whether other models (text encoders, UNet/transformer) need to be offloaded or required tiled encoding. The latter two techniques increases inference time and impacts quality.
The documentation is organized into three sections:
302
+
## Resources
57
303
58
-
***VAE Decode** Learn the basics of how to use VAE Decode with Hybrid Inference.
59
-
***VAE Encode** Learn the basics of how to use VAE Encode with Hybrid Inference.
60
-
***API Reference** Dive into task-specific settings and parameters.
304
+
- Remote inference is also supported in [SD.Next](https://github.com/vladmandic/sdnext) and [ComfyUI-HFRemoteVae](https://github.com/kijai/ComfyUI-HFRemoteVae).
305
+
- Refer to the [Remote VAEs for decoding with Inference Endpoints](https://huggingface.co/blog/remote_vae) blog post to learn more.
0 commit comments