Skip to content

[feat] JoyAI-JoyImage-Edit support #13444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Moran232 wants to merge 41 commits into
base: main
Choose a base branch
from
Open

[feat] JoyAI-JoyImage-Edit support #13444

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
54afa65
[feat] JoyAI-JoyImage-Edit support
Apr 2, 2026
8459759
[fix] remove rearrange
Apr 14, 2026
e6e6df5
[refactor] two pass when do cfg
Apr 14, 2026
f557113
[refactor] remove repa, use wantimetextembeding, refactor modulate code
Apr 14, 2026
d397b68
[refactor] Joyimage Attention refactor
Apr 14, 2026
9d78e4e
remove vae tiling and autocast
Apr 14, 2026
cc9d134
[fix] remove einops from setup.py
Apr 20, 2026
001f7d3
[refactor] Refactor JoyImageEditPipeline to use explicit arguments in…
Apr 20, 2026
657b4b0
[fix] remove deprecated method decode_latents
Apr 20, 2026
19027dd
[refactor] refactor the image pre-processing logic into a separate Va…
Apr 20, 2026
0a06f33
[refactor] add JoyImageAttention to align with Attention + AttnProces…
Apr 20, 2026
02d947c
[refactor] simplify bucket logic in JoyImageEditImageProcessor by rep…
Apr 23, 2026
2b7fb86
[fix] remove leftover training-only parameters
Apr 25, 2026
79c48a8
[fix] add layerwise casting and fp32 module patterns to JoyImageTrans…
Apr 27, 2026
98cee97
[test] add JoyImageEditPipeline fast tests and JoyImageEditTransforme…
Apr 27, 2026
a716050
[fix] fix some pipeline args to support batch inference
Apr 27, 2026
320bde7
[fix] duplicate images to match batch size when fewer images than pro…
Apr 27, 2026
3ed6ca9
[fix] remove no longer used config parameters
Apr 28, 2026
92f4d85
Merge branch 'main' into joyimage_edit
dg845 Apr 28, 2026
261613b
Apply style fixes
github-actions[bot] Apr 28, 2026
f364da3
[fix] remove unused dataclass and rewrite helpers as inline functions
Apr 28, 2026
c7bd284
[fix] make dummy objects for JoyImageEdit
Apr 28, 2026
e45e1ad
[fix] allow test_torch_compile_repeated_blocks to pass
Apr 28, 2026
aeaa334
[fix] add examples on JoyImageEditPipeline
Apr 28, 2026
ce4a3d9
fix code style issues with ruff and black
Apr 28, 2026
82e5cd2
Apply style fixes
github-actions[bot] Apr 29, 2026
844f3f1
[fix] change default num_inference_steps to 40
Apr 29, 2026
fd29a73
[fix] use forward hook to extract pre-norm hidden states for transfor…
Apr 29, 2026
3a6b658
[fix] change the assert to ValueError in pipeline
Apr 29, 2026
76c1647
[fix] rename JoyImageTransformer3DModel to JoyImageEditTransformer3DM…
Apr 29, 2026
deb5d4f
[fix] support gradient checkpointing
Apr 29, 2026
d8c7c0e
[refactor] simplify RoPE utilities, inline helpers, copy WanTimeTextI…
Apr 29, 2026
203494e
[fix] remove _get_text_encoder_ckpt and qwen_processor
Apr 29, 2026
8408f55
[fix] change nn.RMSNorm to FP32LayerNorm
Apr 29, 2026
88579cc
[fix] small fixes for suggestions given by Claude
Apr 29, 2026
ea52935
Merge branch 'main' into joyimage_edit
dg845 Apr 30, 2026
9d9ef52
[refactor] build model using from _pretained instead of config
Apr 30, 2026
87b5383
[refactor] auto-wrap prompt and support text-to-image in JoyImage Edi…
Apr 30, 2026
cf61b0a
make style, make quality and make fix-copies
Apr 30, 2026
dbfbb59
[test] small fix to use vocab_size=1024
Apr 30, 2026
756904e
Merge branch 'main' into joyimage_edit
dg845 Apr 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
366 changes: 366 additions & 0 deletions scripts/convert_joyimage_edit_to_diffusers.py
View file
Open in desktop

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions src/diffusers/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,7 @@
"HunyuanVideoFramepackTransformer3DModel",
"HunyuanVideoTransformer3DModel",
"I2VGenXLUNet",
"JoyImageEditTransformer3DModel",
"Kandinsky3UNet",
"Kandinsky5Transformer3DModel",
"LatteTransformer3DModel",
Expand Down Expand Up @@ -575,7 +576,9 @@
"IFPipeline",
"IFSuperResolutionPipeline",
"ImageTextPipelineOutput",
"JoyImageEditPipeline",
"Kandinsky3Img2ImgPipeline",
"JoyImageEditPipelineOutput",
"Kandinsky3Pipeline",
"Kandinsky5I2IPipeline",
"Kandinsky5I2VPipeline",
Expand Down Expand Up @@ -1063,6 +1066,7 @@
HunyuanVideoFramepackTransformer3DModel,
HunyuanVideoTransformer3DModel,
I2VGenXLUNet,
JoyImageEditTransformer3DModel,
Kandinsky3UNet,
Kandinsky5Transformer3DModel,
LatteTransformer3DModel,
Expand Down Expand Up @@ -1362,6 +1366,8 @@
IFPipeline,
IFSuperResolutionPipeline,
ImageTextPipelineOutput,
JoyImageEditPipeline,
JoyImageEditPipelineOutput,
Kandinsky3Img2ImgPipeline,
Kandinsky3Pipeline,
Kandinsky5I2IPipeline,
Expand Down
4 changes: 4 additions & 0 deletions src/diffusers/models/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,9 @@
_import_structure["transformers.transformer_hunyuan_video15"] = ["HunyuanVideo15Transformer3DModel"]
_import_structure["transformers.transformer_hunyuan_video_framepack"] = ["HunyuanVideoFramepackTransformer3DModel"]
_import_structure["transformers.transformer_hunyuanimage"] = ["HunyuanImageTransformer2DModel"]
_import_structure["transformers.transformer_joyimage"] = [
"JoyImageEditTransformer3DModel",
]
_import_structure["transformers.transformer_kandinsky"] = ["Kandinsky5Transformer3DModel"]
_import_structure["transformers.transformer_longcat_audio_dit"] = ["LongCatAudioDiTTransformer"]
_import_structure["transformers.transformer_longcat_image"] = ["LongCatImageTransformer2DModel"]
Expand Down Expand Up @@ -234,6 +237,7 @@
HunyuanVideo15Transformer3DModel,
HunyuanVideoFramepackTransformer3DModel,
HunyuanVideoTransformer3DModel,
JoyImageEditTransformer3DModel,
Kandinsky5Transformer3DModel,
LatteTransformer3DModel,
LongCatAudioDiTTransformer,
Expand Down
1 change: 1 addition & 0 deletions src/diffusers/models/transformers/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
from .transformer_hunyuan_video15 import HunyuanVideo15Transformer3DModel
from .transformer_hunyuan_video_framepack import HunyuanVideoFramepackTransformer3DModel
from .transformer_hunyuanimage import HunyuanImageTransformer2DModel
from .transformer_joyimage import JoyImageEditTransformer3DModel
from .transformer_kandinsky import Kandinsky5Transformer3DModel
from .transformer_longcat_audio_dit import LongCatAudioDiTTransformer
from .transformer_longcat_image import LongCatImageTransformer2DModel
Expand Down
Loading
Loading