Skip to content

lora.md

Latest commit

 

History

History
125 lines (76 loc) · 5.82 KB

lora.md

File metadata and controls

125 lines (76 loc) · 5.82 KB

LoRA

LoRA is a fast and lightweight training method that inserts and trains a significantly smaller number of parameters instead of all the model parameters. This produces a smaller file (~100 MBs) and makes it easier to quickly train a model to learn a new concept. LoRA weights are typically loaded into the denoiser, text encoder or both. The denoiser usually corresponds to a UNet ([UNet2DConditionModel], for example) or a Transformer ([SD3Transformer2DModel], for example). There are several classes for loading LoRA weights:

  • [StableDiffusionLoraLoaderMixin] provides functions for loading and unloading, fusing and unfusing, enabling and disabling, and more functions for managing LoRA weights. This class can be used with any model.
  • [StableDiffusionXLLoraLoaderMixin] is a Stable Diffusion (SDXL) version of the [StableDiffusionLoraLoaderMixin] class for loading and saving LoRA weights. It can only be used with the SDXL model.
  • [SD3LoraLoaderMixin] provides similar functions for Stable Diffusion 3.
  • [FluxLoraLoaderMixin] provides similar functions for Flux.
  • [CogVideoXLoraLoaderMixin] provides similar functions for CogVideoX.
  • [Mochi1LoraLoaderMixin] provides similar functions for Mochi.
  • [AuraFlowLoraLoaderMixin] provides similar functions for AuraFlow.
  • [LTXVideoLoraLoaderMixin] provides similar functions for LTX-Video.
  • [SanaLoraLoaderMixin] provides similar functions for Sana.
  • [HunyuanVideoLoraLoaderMixin] provides similar functions for HunyuanVideo.
  • [Lumina2LoraLoaderMixin] provides similar functions for Lumina2.
  • [WanLoraLoaderMixin] provides similar functions for Wan.
  • [SkyReelsV2LoraLoaderMixin] provides similar functions for SkyReels-V2.
  • [CogView4LoraLoaderMixin] provides similar functions for CogView4.
  • [AmusedLoraLoaderMixin] is for the [AmusedPipeline].
  • [HiDreamImageLoraLoaderMixin] provides similar functions for HiDream Image
  • [QwenImageLoraLoaderMixin] provides similar functions for Qwen Image.
  • [ZImageLoraLoaderMixin] provides similar functions for Z-Image.
  • [Flux2LoraLoaderMixin] provides similar functions for Flux2.
  • [LoraBaseMixin] provides a base class with several utility methods to fuse, unfuse, unload, LoRAs and more.

Tip

To learn more about how to load LoRA weights, see the LoRA loading guide.

LoraBaseMixin

[[autodoc]] loaders.lora_base.LoraBaseMixin

StableDiffusionLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.StableDiffusionLoraLoaderMixin

StableDiffusionXLLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.StableDiffusionXLLoraLoaderMixin

SD3LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.SD3LoraLoaderMixin

FluxLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.FluxLoraLoaderMixin

Flux2LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.Flux2LoraLoaderMixin

CogVideoXLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.CogVideoXLoraLoaderMixin

Mochi1LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.Mochi1LoraLoaderMixin

AuraFlowLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.AuraFlowLoraLoaderMixin

LTXVideoLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.LTXVideoLoraLoaderMixin

SanaLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.SanaLoraLoaderMixin

HunyuanVideoLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.HunyuanVideoLoraLoaderMixin

Lumina2LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.Lumina2LoraLoaderMixin

CogView4LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.CogView4LoraLoaderMixin

WanLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.WanLoraLoaderMixin

SkyReelsV2LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.SkyReelsV2LoraLoaderMixin

AmusedLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.AmusedLoraLoaderMixin

HiDreamImageLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.HiDreamImageLoraLoaderMixin

QwenImageLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.QwenImageLoraLoaderMixin

ZImageLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.ZImageLoraLoaderMixin

KandinskyLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.KandinskyLoraLoaderMixin

LoraBaseMixin

[[autodoc]] loaders.lora_base.LoraBaseMixin