Skip to content
This repository was archived by the owner on Dec 14, 2023. It is now read-only.

Commit 51d910b

Browse files
Update README.md
1 parent 082c88d commit 51d910b

1 file changed

Lines changed: 13 additions & 2 deletions

File tree

README.md

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
[output.webm](https://user-images.githubusercontent.com/59846140/230748413-fe91e90b-94b9-49ea-97ec-250469ee9472.webm)
55

66
### Updates
7+
- **2023-7-12**: You can now train a LoRA that is compatibile with the [webui extension](https://github.com/kabachuha/sd-webui-text2video)! See instructions [here.](https://github.com/ExponentialML/Text-To-Video-Finetuning/edit/feat/stable_lora/README.md#training-a-lora)
78
- **2023-4-17**: You can now convert your trained models from diffusers to `.ckpt` format for A111 webui. Thanks @kabachuha!
89
- **2023-4-8**: LoRA Training released! Checkout `configs/v2/lora_training_config.yaml` for instructions.
910
- **2023-4-8**: Version 2 is released!
@@ -46,15 +47,13 @@ It is **highly recommended** to install >= Torch 2.0. This way, you don't have t
4647

4748
If you don't have Xformers enabled, you can follow the instructions here: https://github.com/facebookresearch/xformers
4849

49-
5050
Recommended to use a RTX 3090, but you should be able to train on GPUs with <= 16GB ram with:
5151
- Validation turned off.
5252
- Xformers or Torch 2.0 Scaled Dot-Product Attention
5353
- Gradient checkpointing enabled.
5454
- Resolution of 256.
5555
- Enable all LoRA options.
5656

57-
5857
## Running inference
5958
The `inference.py` script can be used to render videos with trained checkpoints.
6059

@@ -164,6 +163,18 @@ Then, follow each line and configure it for your specific use case.
164163

165164
The instructions should be clear enough to get you up and running with your dataset, but feel free to ask any questions in the discussion board.
166165

166+
## Training a LoRA
167+
You can also train a LoRA that is both compatible with the webui extension.. By default it's set to 'cloneofsimo', which was the first LoRA implementation for Stable Diffusion.
168+
This version you can use in the `inference.py` file in this repository. It is **not** compatible with the webui.
169+
170+
To use a LoRA with the webui, change the `lora_version` to "stable_lora" in your config. This will train an [A1111 webui extension](https://github.com/kabachuha/sd-webui-text2video) compatibile LoRA.
171+
You can get started at `configs/v2/stable_lora_config.yaml` and edit it from there. During and after training, LoRAs will be saved in your outputs directory with the prefix `_webui`.
172+
173+
### What you cannot do:
174+
- Use LoRA files that were made for SD image models in other trainers.
175+
- Use 'cloneofsimo' LoRAs in another project (unless you build it or create a PR)
176+
- Merge LoRA weights together (yet).
177+
167178
## Finetune.
168179
```python
169180
python train.py --config ./configs/v2/train_config.yaml

0 commit comments

Comments
 (0)