Fix Flux IP-Adapter masks, CFG embeds, and modular helper coupling#13708
Open
taivu1998 wants to merge 1 commit intohuggingface:mainfrom
Open
Fix Flux IP-Adapter masks, CFG embeds, and modular helper coupling#13708taivu1998 wants to merge 1 commit intohuggingface:mainfrom
taivu1998 wants to merge 1 commit intohuggingface:mainfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Addresses #13584 by fixing several Flux pipeline and model issues found in the Flux review:
ip_adapter_masksinFluxIPAdapterAttnProcessorinstead of accepting and silently ignoring them.negative_prompt_embedsandnegative_pooled_prompt_embedsto Flux img2img, inpaint, and ControlNet pipelines.vae_scale_factor * 2.Root cause
The affected Flux code paths had drifted across pipeline variants:
ip_adapter_masks, but the processor never consumed them.negative_prompt, even though their input validation allowed negative embeddings.height % self.vae_scale_factor * 2, which Python parses as(height % self.vae_scale_factor) * 2.Validation
Ran the focused regression suite for the changed behavior:
uvx ruff format src/diffusers/models/transformers/transformer_flux.py src/diffusers/modular_pipelines/flux/before_denoise.py src/diffusers/modular_pipelines/flux/decoders.py src/diffusers/modular_pipelines/flux/encoders.py src/diffusers/modular_pipelines/flux/inputs.py src/diffusers/modular_pipelines/flux/pipeline_helpers.py src/diffusers/pipelines/flux/pipeline_flux.py src/diffusers/pipelines/flux/pipeline_flux_controlnet.py src/diffusers/pipelines/flux/pipeline_flux_controlnet_image_to_image.py src/diffusers/pipelines/flux/pipeline_flux_img2img.py src/diffusers/pipelines/flux/pipeline_flux_inpaint.py src/diffusers/pipelines/flux/pipeline_flux_kontext.py src/diffusers/pipelines/flux/pipeline_flux_prior_redux.py tests/models/transformers/test_models_transformer_flux.py tests/modular_pipelines/flux/test_modular_pipeline_flux.py tests/pipelines/controlnet_flux/test_controlnet_flux.py tests/pipelines/controlnet_flux/test_controlnet_flux_img2img.py tests/pipelines/flux/test_pipeline_flux.py tests/pipelines/flux/test_pipeline_flux_img2img.py tests/pipelines/flux/test_pipeline_flux_inpaint.py tests/pipelines/flux/test_pipeline_flux_kontext.py tests/pipelines/flux/test_pipeline_flux_redux.py uvx ruff check src/diffusers/models/transformers/transformer_flux.py src/diffusers/modular_pipelines/flux/before_denoise.py src/diffusers/modular_pipelines/flux/decoders.py src/diffusers/modular_pipelines/flux/encoders.py src/diffusers/modular_pipelines/flux/inputs.py src/diffusers/modular_pipelines/flux/pipeline_helpers.py src/diffusers/pipelines/flux/pipeline_flux.py src/diffusers/pipelines/flux/pipeline_flux_controlnet.py src/diffusers/pipelines/flux/pipeline_flux_controlnet_image_to_image.py src/diffusers/pipelines/flux/pipeline_flux_img2img.py src/diffusers/pipelines/flux/pipeline_flux_inpaint.py src/diffusers/pipelines/flux/pipeline_flux_kontext.py src/diffusers/pipelines/flux/pipeline_flux_prior_redux.py tests/models/transformers/test_models_transformer_flux.py tests/modular_pipelines/flux/test_modular_pipeline_flux.py tests/pipelines/controlnet_flux/test_controlnet_flux.py tests/pipelines/controlnet_flux/test_controlnet_flux_img2img.py tests/pipelines/flux/test_pipeline_flux.py tests/pipelines/flux/test_pipeline_flux_img2img.py tests/pipelines/flux/test_pipeline_flux_inpaint.py tests/pipelines/flux/test_pipeline_flux_kontext.py tests/pipelines/flux/test_pipeline_flux_redux.py git diff --check uv run --python 3.10 python -m py_compile src/diffusers/models/transformers/transformer_flux.py src/diffusers/modular_pipelines/flux/before_denoise.py src/diffusers/modular_pipelines/flux/decoders.py src/diffusers/modular_pipelines/flux/encoders.py src/diffusers/modular_pipelines/flux/inputs.py src/diffusers/modular_pipelines/flux/pipeline_helpers.py src/diffusers/pipelines/flux/pipeline_flux.py src/diffusers/pipelines/flux/pipeline_flux_controlnet.py src/diffusers/pipelines/flux/pipeline_flux_controlnet_image_to_image.py src/diffusers/pipelines/flux/pipeline_flux_img2img.py src/diffusers/pipelines/flux/pipeline_flux_inpaint.py src/diffusers/pipelines/flux/pipeline_flux_kontext.py src/diffusers/pipelines/flux/pipeline_flux_prior_redux.py tests/models/transformers/test_models_transformer_flux.py tests/modular_pipelines/flux/test_modular_pipeline_flux.py tests/pipelines/controlnet_flux/test_controlnet_flux.py tests/pipelines/controlnet_flux/test_controlnet_flux_img2img.py tests/pipelines/flux/test_pipeline_flux.py tests/pipelines/flux/test_pipeline_flux_img2img.py tests/pipelines/flux/test_pipeline_flux_inpaint.py tests/pipelines/flux/test_pipeline_flux_kontext.py tests/pipelines/flux/test_pipeline_flux_redux.py uv run --python 3.10 python -m pytest tests/models/transformers/test_models_transformer_flux.py tests/pipelines/flux/test_pipeline_flux_redux.py tests/modular_pipelines/flux/test_modular_pipeline_flux.py tests/pipelines/flux/test_pipeline_flux.py tests/pipelines/flux/test_pipeline_flux_kontext.py tests/pipelines/controlnet_flux/test_controlnet_flux_img2img.py tests/pipelines/flux/test_pipeline_flux_img2img.py tests/pipelines/flux/test_pipeline_flux_inpaint.py tests/pipelines/controlnet_flux/test_controlnet_flux.py -k "ip_adapter_masks_are_applied or ip_adapter_masks_validate_num_images or ip_adapter_no_mask_keeps_3d_hidden_state_support or FluxReduxFastTests or modular_blocks_do_not_import or latent_helpers_roundtrip or negative_prompt_embeds_shape_mismatch or dimension_check_uses_packed_latent_multiple or true_cfg_with_negative_prompt_embeds"Targeted pytest result:
Notes
This PR adds focused fast regression coverage for the fixed behavior. The broader slow/nightly real-checkpoint coverage discussed in the issue is intentionally left for a separate CI-focused PR because it spans many Flux variants and checkpoint/runtime choices.