|
4 | 4 | [output.webm](https://user-images.githubusercontent.com/59846140/230748413-fe91e90b-94b9-49ea-97ec-250469ee9472.webm) |
5 | 5 |
|
6 | 6 | ### Updates |
| 7 | +- **2023-7-12**: You can now train a LoRA that is compatibile with the [webui extension](https://github.com/kabachuha/sd-webui-text2video)! See instructions [here.](https://github.com/ExponentialML/Text-To-Video-Finetuning/edit/feat/stable_lora/README.md#training-a-lora) |
7 | 8 | - **2023-4-17**: You can now convert your trained models from diffusers to `.ckpt` format for A111 webui. Thanks @kabachuha! |
8 | 9 | - **2023-4-8**: LoRA Training released! Checkout `configs/v2/lora_training_config.yaml` for instructions. |
9 | 10 | - **2023-4-8**: Version 2 is released! |
@@ -46,15 +47,13 @@ It is **highly recommended** to install >= Torch 2.0. This way, you don't have t |
46 | 47 |
|
47 | 48 | If you don't have Xformers enabled, you can follow the instructions here: https://github.com/facebookresearch/xformers |
48 | 49 |
|
49 | | - |
50 | 50 | Recommended to use a RTX 3090, but you should be able to train on GPUs with <= 16GB ram with: |
51 | 51 | - Validation turned off. |
52 | 52 | - Xformers or Torch 2.0 Scaled Dot-Product Attention |
53 | 53 | - Gradient checkpointing enabled. |
54 | 54 | - Resolution of 256. |
55 | 55 | - Enable all LoRA options. |
56 | 56 |
|
57 | | - |
58 | 57 | ## Running inference |
59 | 58 | The `inference.py` script can be used to render videos with trained checkpoints. |
60 | 59 |
|
@@ -164,6 +163,18 @@ Then, follow each line and configure it for your specific use case. |
164 | 163 |
|
165 | 164 | The instructions should be clear enough to get you up and running with your dataset, but feel free to ask any questions in the discussion board. |
166 | 165 |
|
| 166 | +## Training a LoRA |
| 167 | +You can also train a LoRA that is both compatible with the webui extension.. By default it's set to 'cloneofsimo', which was the first LoRA implementation for Stable Diffusion. |
| 168 | +This version you can use in the `inference.py` file in this repository. It is **not** compatible with the webui. |
| 169 | + |
| 170 | +To use a LoRA with the webui, change the `lora_version` to "stable_lora" in your config. This will train an [A1111 webui extension](https://github.com/kabachuha/sd-webui-text2video) compatibile LoRA. |
| 171 | +You can get started at `configs/v2/stable_lora_config.yaml` and edit it from there. During and after training, LoRAs will be saved in your outputs directory with the prefix `_webui`. |
| 172 | + |
| 173 | +### What you cannot do: |
| 174 | +- Use LoRA files that were made for SD image models in other trainers. |
| 175 | +- Use 'cloneofsimo' LoRAs in another project (unless you build it or create a PR) |
| 176 | +- Merge LoRA weights together (yet). |
| 177 | + |
167 | 178 | ## Finetune. |
168 | 179 | ```python |
169 | 180 | python train.py --config ./configs/v2/train_config.yaml |
|
0 commit comments