Need help on loading SD3.5-Medium, don't quite understand the github issue mentioned in the error output

[CaledoniaProject]

I'm trying to load SD3.5-Medium with a script,

import os
import json
import torch

from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-3.5-medium", 
    torch_dtype=torch.bfloat16
)
pipe = pipe.to('cuda')
pipe.enable_xformers_memory_efficient_attention()
pipe.enable_model_cpu_offload()

And it failed

# python3 main.py
/home/diffusers/venv/lib/python3.10/site-packages/diffusers/models/transformers/transformer_2d.py:34: FutureWarning: `Transformer2DModelOutput` is deprecated and will be removed in version 1.0.0. Importing `Transformer2DModelOutput` from `diffusers.models.transformer_2d` is deprecated and this will be removed in a future version. Please use `from diffusers.models.modeling_outputs import Transformer2DModelOutput`, instead.
  deprecate("Transformer2DModelOutput", "1.0.0", deprecation_message)
Loading pipeline components...:   0%|                                                                                                                           | 0/9 [00:00<?, ?it/s]The config attributes {'dual_attention_layers': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], 'qk_norm': 'rms_norm'} were passed to SD3Transformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Loading pipeline components...:  11%|████████████▊                                                                                                      | 1/9 [00:04<00:39,  4.90s/it]
Traceback (most recent call last):
  File "/home/diffusers/main.py", line 12, in <module>
    pipe = DiffusionPipeline.from_pretrained(
  File "/home/diffusers/venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/diffusers/venv/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 881, in from_pretrained
    loaded_sub_model = load_sub_model(
  File "/home/diffusers/venv/lib/python3.10/site-packages/diffusers/pipelines/pipeline_loading_utils.py", line 703, in load_sub_model
    loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs)
  File "/home/diffusers/venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/diffusers/venv/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 757, in from_pretrained
    unexpected_keys = load_model_dict_into_meta(
  File "/home/diffusers/venv/lib/python3.10/site-packages/diffusers/models/model_loading_utils.py", line 154, in load_model_dict_into_meta
    raise ValueError(
ValueError: Cannot load /root/.cache/huggingface/hub/models--stabilityai--stable-diffusion-3.5-medium/snapshots/b940f670f0eda2d07fbb75229e779da1ad11eb80/transformer because transformer_blocks.0.norm1.linear.bias expected shape tensor(..., device='meta', size=(9216,)), but got torch.Size([13824]). If you want to instead overwrite randomly initialized weights, please make sure to pass both `low_cpu_mem_usage=False` and `ignore_mismatched_sizes=True`. For more information, see also: https://github.com/huggingface/diffusers/issues/1619#issuecomment-1345604389 as an example.

It asks me to enable

    ignore_mismatched_sizes=True,
    low_cpu_mem_usage=False,

But I needed to offload it to CPU, do they conflicts with pipe.enable_model_cpu_offload()?

[CaledoniaProject]

Resolved with another pipeline. Closing