[discrete diffusion] Add dflash pipeline

[kashif]

What does this PR do?

Adds DFlashPipeline and DFlashTokenDiffusionScheduler to diffusers -- extracted from #12911 to keep the scope manageable.

DFlash is a speculative decoding framework that uses a lightweight block diffusion model as the draft. The draft generates a full block of tokens in one forward pass; the target LLM verifies them in parallel. This gives 6x+ lossless speedup vs. standard autoregressive decoding (paper: https://huggingface.co/papers/2602.06036).

What's included:

• DFlashPipeline -- prefill on target, draft a block, verify with the scheduler, commit accepted prefix + resample, repeat
• DFlashTokenDiffusionScheduler -- verification step: samples target posterior, computes acceptance length, returns next token at first rejection
• Tests for both pipeline and scheduler
• Docs and discrete diffusion examples (sampling, training, data regen)

Pretrained pairs from the z-lab/dflash collection work out of the box. Canonical pair: z-lab/Qwen3-8B-DFlash-b16 + Qwen/Qwen3-8B.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@yiyixuxu @dg845

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[dg845]

Hi, when I try out the example in src/diffusers/pipelines/dflash/pipeline_dflash.py:

import torch
from diffusers import DFlashPipeline
from transformers import AutoModel, AutoModelForCausalLM, AutoTokenizer

draft = AutoModel.from_pretrained(
    "z-lab/Qwen3-8B-DFlash-b16", trust_remote_code=True, dtype=torch.bfloat16
)
target = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")

pipe = DFlashPipeline(draft_model=draft, target_model=target, tokenizer=tokenizer)
out = pipe(prompt="How many positive whole-number divisors does 196 have?")
print(out.texts[0])

I get the following output:

<think>
Okay, so I need to figure out how many positive whole-number divisors 196 has. Hmm, divisors... right. Divisors are numbers that divide into 196 without leaving a remainder. So, I guess I need to find all the numbers that can divide 196 evenly.

First, maybe I should start by recalling how to find the number of divisors. I think there's a method involving prime factorization. Let me remember... if you break down a number into its prime factors, then you can use the exponents of those primes to calculate the total number of divisors.

Let me try that. So, first step: find the prime factors of 196. Let me start dividing 196 by the smallest prime numbers.

196 is even, so it's divisible by 2. Let me divide 196 by 2.

196 ÷ 2 = 98.

Okay, so 2 is a prime factor. Now, 98 is also even, so divide by 2 again.

98 ÷ 2 = 49.

So now we have 2 × 2 × 49. Now, 49 is not even, so let's check the next prime number, which is 3. Does 3 divide into 49? Let me check: 3 × 16 is 48, so 49 - 48 is 1. So no, 3 doesn't divide into 49.

Next prime number is 5. 49 ends with a 9, so it's not divisible by 5. Next is 7. Let me try dividing

Is the output being cut off expected?

[dg845]

Also, I saw that huggingface/transformers#45846 was closed, will the KV cache work correctly without it? DFlashTokenDiffusionScheduler has some cache methods (e.g. snapshot_cache, restore_cache, etc.) but I'm not sure if they need the transformers PR to work.

[kashif]

thanks @dg845 checking

[Copilot]

Pull request overview

Adds experimental DFlash support to diffusers by introducing a DFlash speculative-decoding pipeline (draft diffusion model + target causal LM) and a companion token-diffusion scheduler, along with tests, docs, and example scripts under examples/discrete_diffusion.

Changes:

• Introduces DFlashPipeline and DFlashTokenDiffusionScheduler (plus outputs) and wires them into lazy imports/dummy objects.
• Adds unit tests for the new scheduler and pipeline behavior (including callback/progress-bar regression checks).
• Adds documentation pages and discrete-diffusion example scripts (sampling, training, and data regeneration).

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/diffusers/schedulers/scheduling_dflash_token_diffusion.py	New scheduler implementing DFlash verification (posterior sampling + acceptance length).
src/diffusers/schedulers/__init__.py	Exposes the new scheduler via lazy import structure.
src/diffusers/pipelines/dflash/pipeline_dflash.py	New DFlash pipeline implementing block-wise speculative decoding.
src/diffusers/pipelines/dflash/__init__.py	Adds lazy import + optional dependency gating for the DFlash pipeline module.
src/diffusers/pipelines/__init__.py	Exposes DFlashPipeline at the pipelines package level.
src/diffusers/__init__.py	Exposes the new scheduler and pipeline at the top-level diffusers namespace.
src/diffusers/utils/dummy_pt_objects.py	Adds dummy (torch-only) objects for the new scheduler types.
src/diffusers/utils/dummy_torch_and_transformers_objects.py	Adds dummy objects for the new pipeline types (torch+transformers).
tests/schedulers/test_scheduler_dflash_token_diffusion.py	New scheduler unit tests (timesteps, sampling, step outputs, stop condition).
tests/pipelines/dflash/test_dflash.py	New pipeline unit tests + regression tests (callbacks/progress-bar).
tests/pipelines/dflash/__init__.py	Adds the test package directory for DFlash pipeline tests.
docs/source/en/api/schedulers/dflash_token_diffusion.md	New scheduler API docs page.
docs/source/en/api/pipelines/dflash.md	New pipeline API docs page.
docs/source/en/_toctree.yml	Registers the new DFlash docs pages in the docs navigation.
examples/discrete_diffusion/train_dflash.py	Example training script for DFlash draft models.
examples/discrete_diffusion/sample_dflash.py	Example sampling script using DFlashPipeline.
examples/discrete_diffusion/regenerate_dflash_data.py	Example async helper to regenerate training data via an OpenAI-compatible server.
examples/discrete_diffusion/README.md	Adds DFlash usage + training/data-prep documentation to the discrete diffusion examples README.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kashif]

@dg845 yep, expected -- you hit the default max_new_tokens=2048 and Qwen3's <think> trace runs well over that. Pass chat_template_kwargs={"enable_thinking": False} to skip thinking mode, or bump max_new_tokens to 8k+ for full traces. Updated the docs to call this out.

[kashif]

@dg845 that transformers PR was for hybrid-attention targets (Qwen3.5's gated-delta-net). For plain Qwen3-8B (full-attention), DynamicCache.crop() works fine and the closed PR doesn't affect anything. The snapshot/restore cache methods were planned for hybrid-attention support but aren't in this PR -- updated the docs to say hybrid targets aren't supported yet.