Skip to content
This repository was archived by the owner on Dec 14, 2023. It is now read-only.

Commit f35881d

Browse files
Update README.md
1 parent 2a09bb7 commit f35881d

1 file changed

Lines changed: 23 additions & 3 deletions

File tree

README.md

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,18 @@
1+
<div align="center" width="100" height="100" >
2+
<img src="https://github.com/ExponentialML/Text-To-Video-Finetuning/assets/59846140/184f0dce-b77a-45d7-b24d-1814e5b9c314" />
3+
<div align="center" style="font-style: italic;" >
4+
<i>Video Credit: dotsimulate</i>
5+
</div>
6+
<div align="center" style="font-style: italic;" >
7+
<i>Model: Zeroscope XL</i>
8+
</div>
9+
</div>
10+
11+
12+
113
# Text-To-Video-Finetuning
214
## Finetune ModelScope's Text To Video model using Diffusers 🧨
315

4-
[output.webm](https://user-images.githubusercontent.com/59846140/230748413-fe91e90b-94b9-49ea-97ec-250469ee9472.webm)
5-
616
### Updates
717
- **2023-7-12**: You can now train a LoRA that is compatibile with the [webui extension](https://github.com/kabachuha/sd-webui-text2video)! See instructions [here.](https://github.com/ExponentialML/Text-To-Video-Finetuning/edit/feat/stable_lora/README.md#training-a-lora)
818
- **2023-4-17**: You can now convert your trained models from diffusers to `.ckpt` format for A111 webui. Thanks @kabachuha!
@@ -22,6 +32,15 @@ git lfs install
2232
git clone https://huggingface.co/damo-vilab/text-to-video-ms-1.7b ./models/model_scope_diffusers/
2333
```
2434

35+
## Other Models
36+
Alternatively, you can train starting from other models made by the community.
37+
38+
| Contributer |Model Name | Link |
39+
| -------------- | ------------ | --------------------------------------------------- |
40+
| cerspense | ZeroScope | https://huggingface.co/cerspense/zeroscope_v2_576w |
41+
| cameduru | Potat1 | https://huggingface.co/camenduru/potat1 |
42+
| strangeman3107 | animov-512x | https://huggingface.co/strangeman3107/animov-512x |
43+
2544
### Create Conda Environment (Optional)
2645
It is recommended to install Anaconda.
2746

@@ -52,7 +71,8 @@ Recommended to use a RTX 3090, but you should be able to train on GPUs with <= 1
5271
- Xformers or Torch 2.0 Scaled Dot-Product Attention
5372
- Gradient checkpointing enabled.
5473
- Resolution of 256.
55-
- Enable all LoRA options.
74+
- Hybrid LoRA training.
75+
- Training only using LoRA with ranks between 4-16.
5676

5777
## Running inference
5878
The `inference.py` script can be used to render videos with trained checkpoints.

0 commit comments

Comments
 (0)