Fix Krea2/Qwen-Image flat samples: use a VAE with non-zero quant convs#919
Open
skydam wants to merge 1 commit into
Open
Fix Krea2/Qwen-Image flat samples: use a VAE with non-zero quant convs#919skydam wants to merge 1 commit into
skydam wants to merge 1 commit into
Conversation
The "Qwen/Qwen-Image" VAE checkpoint ships all-zero quant_conv and post_quant_conv weights. AutoencoderKLQwenImage applies them on both encode and decode, which zeros the latent path: encode returns pure noise and decode returns a constant flat field. Every Krea2 sample comes out as the same muddy-brown image and training silently learns against noise targets. "Qwen/Qwen-Image-2512" is the same VAE (byte-identical encoder/decoder weights) with the quant convs populated, so switch the default to it. Also guard _load_vae against any VAE with all-zero quant convs so this fails loudly instead of wasting training hours. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I've never submitted something like this before and i want to be upfront: Claude Code came up with the solution, but i was getting these brown sample images. Claude Code says it's the VAE.
Problem
Training Krea2 (raw or turbo) produces sample previews that are a uniform
muddy-brown field — identical for every prompt, step, and run (the output JPEGs
are byte-for-byte identical). FLUX / z-image are unaffected.
Root cause
Krea2 defaults its VAE to
Qwen/Qwen-Image, loaded viaAutoencoderKLQwenImage.In the current
mainrevision of that repo, the VAE'squant_convandpost_quant_convweights are all zero. The pinned diffusers commit appliesthese convs on both encode and decode, which zeros the latent path:
silently learns against noise targets, not just broken previews.
Qwen/Qwen-Image-2512is the same VAE (byte-identical encoder/decoder weights)with the quant convs populated.
extensions_built_in/diffusion_models/qwen_image/ qwen_image.pyloads the sameQwen/Qwen-ImageVAE, so that arch is likelyaffected too.
Fix
Qwen/Qwen-Image-2512._load_vaeto raise on an all-zeroquant_convso this fails loudlyinstead of silently wasting training hours.
Minimal repro (no DiT needed)