Skip to content

Commit 5d8a8bf

Browse files
fix(wan): WanModel.patchify flatten + return grid size tuple
Fixes #1063. WanModel.forward (and the parallel usp_dit_forward in diffsynth/utils/xfuser/xdit_context_parallel.py) both unpack patchify's return value as `x, (f, h, w) = self.patchify(x)`, but patchify only returns `x` -- and that `x` is a 5D (B, dim, f, h, w) tensor straight out of Conv3d, not the (B, f*h*w, dim) sequence the transformer blocks expect. Net result: every Wan training run crashes at the first forward call with `ValueError: not enough values to unpack (expected 2, got 1)`. The block loop never executes. The fix: extract (f, h, w) from the Conv3d output shape, flatten the 3D spatial-temporal grid into a token sequence (B, f*h*w, dim) ready for the transformer blocks, return (x, (f, h, w)) to match what forward and unpatchify expect. Verified end-to-end with a synthetic 8-layer WanModel on the same forward path (input shape (1, 16, 4, 8, 8) -> output shape (1, 16, 4, 8, 8) round-trip), forward + backward both complete, loss gradient propagates through all blocks.
1 parent bf5faf6 commit 5d8a8bf

1 file changed

Lines changed: 7 additions & 1 deletion

File tree

diffsynth/models/wan_video_dit.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -498,7 +498,13 @@ def patchify(self, x: torch.Tensor, control_camera_latents_input: Optional[torch
498498
y_camera = self.control_adapter(control_camera_latents_input)
499499
x = [u + v for u, v in zip(x, y_camera)]
500500
x = x[0].unsqueeze(0)
501-
return x
501+
# After Conv3d the tensor is (B, dim, f, h, w). The forward expects
502+
# (x, (f, h, w)) where x has been flattened to (B, f*h*w, dim) ready
503+
# for the transformer blocks; the grid size tuple drives the RoPE
504+
# frequency cat and the unpatchify rearrange at the end.
505+
f, h, w = x.shape[-3], x.shape[-2], x.shape[-1]
506+
x = x.flatten(2).transpose(1, 2)
507+
return x, (f, h, w)
502508

503509
def unpatchify(self, x: torch.Tensor, grid_size: torch.Tensor):
504510
return rearrange(

0 commit comments

Comments
 (0)