Add SeFi-Image pipeline

[jmliu206]

What does this PR do?

Adds Diffusers support for SeFi-Image, including:

• SeFiTransformer2DModel
• SeFiPipeline
• a conversion script for original SeFi checkpoints
• API docs and ToC entries
• fast model and pipeline tests

The pipeline supports Qwen3-VL prompt encoding, SeFi dual-time denoising, classifier-free guidance for non-turbo checkpoints, turbo step validation, and model CPU offload.

Converted checkpoints

• SeFi-Image-1B-Base-diffusers
• SeFi-Image-1B-turbo-diffusers
• SeFi-Image-2B-Base-diffusers
• SeFi-Image-2B-turbo-diffusers
• SeFi-Image-5B-Base-diffusers
• SeFi-Image-5B-turbo-diffusers
• SeFi-Image-5B-RL-diffusers

The original SeFi-Image checkpoint repositories remain unchanged.

Review follow-up

• Flattened the Flux2-based transformer and made conversion from the original backbone.* keys strict.
• Removed the unsupported LoRA decorator and simplified the pipeline fallbacks, chat-template handling, rotary access, copied helpers, and VAE path selection.
• Made the scheduler conversion explicit and fixed the callback timestep value.

Before submitting

• Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
  • Did you read the Coding with AI agents guide?
  • Did you self-review the diff against .ai/review-rules.md?
• Did you read the contributor guideline?
• Did you read our philosophy doc?
• Was this discussed/approved via a GitHub issue or the forum?
  • Add SeFi-Image pipeline #14086
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?
• Are you the author (or part of the team) of the model/pipeline?

Who can review?

Potential reviewers for pipelines/models: @yiyixuxu @dg845 @asomoza

[jmliu206]

@yiyixuxu @dg845 @asomoza — SeFi-Image modular port following #14086, ready for review. Happy to adjust anything I missed.

[vladmandic]

@yiyixuxu @sayakpaul @asomoza @dg845 gentle ping - this pr is waiting for review for 3 weeks now?
@jmliu206 please upload converted models, its not reasonable to expect user to download original model, run conversion offline and then load it

[sergereview]

🤗 Serge says:

Solid, well-tested first cut of the SeFi-Image pipeline, but there are several deviations from the project's coding rules (no defensive code, no unused params, # Copied from for adapted helpers) and a structural question about how the Flux2 backbone is reused.

Structure / conventions

• SeFiTransformer2DModel instantiates a full Flux2Transformer2DModel as self.backbone, mutates time_guidance_embed to nn.Identity(), and then reaches into its internals (x_embedder, context_embedder, pos_embed, transformer_blocks, norm_out, proj_out, the three modulation modules) to reimplement the forward. This diverges from the single-file / # Copied from model conventions in .ai/models.md. Consider importing the Flux2 block classes directly (with # Copied from where appropriate) instead of wrapping the whole model and deleting one of its submodules. As written, the checkpoint will also carry dead time_guidance_embed params until the nn.Identity() swap, and self.out_channels on the SeFi model is set but never used.
• Several pipeline helpers (_prepare_text_ids, _prepare_latent_ids, _pack_latents, _unpatchify_latents) appear adapted from pipelines/flux2/pipeline_flux2.py but lack # Copied from headers, so make fix-copies won't keep them in sync. _prepare_text_ids also carries an unused t_coord parameter (never passed by any caller).

Defensive code (repo rule: no defensive/fallback code)

• SeFiPipeline.__init__ re-applies X if X is None else default for semantic_channels, texture_vae_name, default_guidance_scale, default_num_inference_steps, text_encoder_hidden_layers, max_sequence_length even though the signature already supplies non-None defaults. This is dead code unless a caller explicitly passes None.
• _build_chat_text wraps apply_chat_template in a try/except TypeError fallback, and _align_text_encoder_rotary_dtype uses try/except StopIteration plus hasattr/getattr chains — both are the kind of "just in case" fallbacks the guidelines ask to drop in favor of clear expected inputs / concise errors.

LoRA decorator without the mixin

• SeFiTransformer2DModel.forward is decorated with @apply_lora_scale("joint_attention_kwargs"), but the class does not inherit PeftAdapterMixin, so scale_lora_layers/unscale_lora_layers have nothing to act on. Either add the mixin (and wire up LoRA loading) or drop the decorator so the intent isn't misleading.

Correctness question

• _timesteps_and_sigmas indexes directly into scheduler.timesteps/scheduler.sigmas at their training resolution and the pipeline never calls scheduler.set_timesteps. Since FlowMatchEulerDiscreteScheduler defaults to shift=3.0, that scheduler-side shift compounds with the pipeline's own timestep_shift_alpha. Please confirm the converted scheduler config uses shift=1.0 (or otherwise document/justify the compounding), since the fast tests don't exercise real checkpoint schedules.

Minor

• texture_vae_config_path has two branches returning the identical path (root / "vae" / "config.json"); collapse them.
• _current_timestep is set to a sigma value (base_sigmas_schedule[i]) and handed to callback_on_step_end as the timestep argument — slightly misleading naming.

Tests are structured correctly against the current tester mixins and cover inference, turbo validation, CFG batching, and the rotary-dtype alignment path, which is good.

serge v0.1.0 · model: claude-opus-4-8 · 32 LLM turns · 34 tool calls · 209.8s · 2083251 in / 12464 out tokens

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[jmliu206]

Thanks for the review. I addressed each inline point and replied in the corresponding thread.

I also uploaded the standalone Diffusers conversions for the remaining variants:

• 2B Base
• 2B turbo
• 5B Base
• 5B turbo
• 5B RL

The original SeFi-Image checkpoint repositories were not modified.

[jmliu206]

@vladmandic The standalone Diffusers-format checkpoints are now available and can be loaded directly with SeFiPipeline.from_pretrained:

• https://huggingface.co/SeFi-Image/SeFi-Image-1B-Base-diffusers
• https://huggingface.co/SeFi-Image/SeFi-Image-1B-turbo-diffusers
• https://huggingface.co/SeFi-Image/SeFi-Image-2B-Base-diffusers
• https://huggingface.co/SeFi-Image/SeFi-Image-2B-turbo-diffusers
• https://huggingface.co/SeFi-Image/SeFi-Image-5B-Base-diffusers
• https://huggingface.co/SeFi-Image/SeFi-Image-5B-turbo-diffusers
• https://huggingface.co/SeFi-Image/SeFi-Image-5B-RL-diffusers