[tests] Fix controlnet tests

[sayakpaul]

What does this PR do?

Fix https://github.com/huggingface/diffusers/actions/runs/25780617099/job/75722330514?pr=13733

[sayakpaul]

@claude-2-serge could you perform a review?

[github-actions]

🤗 Serge says:

Straightforward fix switching four controlnet test files from float16 to float32 for dummy/fast tests, with updated expected output slices. The changes are consistent and correct.

Minor issue

• In test_controlnet_flux.py, there is one remaining float16 at line 201 inside test_flux_image_output_shape that was not updated. The pipe in that test runs in float32 (no dtype override), but the control_image tensor is still created as float16. This likely doesn't cause a failure since the test only checks output shape, but it's inconsistent with the rest of the fix and could mask a dtype-mismatch issue.

5 LLM turns · 7 tool calls · 30.6s · 48948 in / 1398 out tokens

[github-actions]

Nit: there's another dtype=torch.float16 at line 201 in test_flux_image_output_shape that wasn't updated. The pipe in that test defaults to float32, so the control_image will be float16 while the rest of the pipeline is float32. It probably doesn't break anything since that test only checks output shape, but for consistency it should likely be torch.float32 as well.

[dg845]

Thanks for the PR! The change from FP16 to FP32 looks good to me, and the tests look fixed as well.

But why does switching to FP32 fix the underlying issue(s) that caused the tests to fail in the first place? I see that full inference in the slow tests uses BF16 (Flux) or FP16 (Hunyuan, SD3); if there really is an underlying issue affecting FP16/BF16 ControlNet inference specifically, it seems like the (fast) tests won't catch it.

[sayakpaul]

They are not fixing the assertion problem (the assertion slices are being changed). It's just a safeguard to avoid numerical problems in CPUs where our fast tests run.