Skip to content

fix(peft_utils): raise actionable ValueError when rank_dict is empty in get_peft_kwargs#13759

Open
dparikh79 wants to merge 1 commit into
huggingface:mainfrom
dparikh79:fix/diffusers-get-peft-kwargs-empty-rank-dict
Open

fix(peft_utils): raise actionable ValueError when rank_dict is empty in get_peft_kwargs#13759
dparikh79 wants to merge 1 commit into
huggingface:mainfrom
dparikh79:fix/diffusers-get-peft-kwargs-empty-rank-dict

Conversation

@dparikh79
Copy link
Copy Markdown

@dparikh79 dparikh79 commented May 16, 2026

Summary

Refs huggingface/peft#3238 (the issue was filed against peft, but the failure path is here in diffusers).

get_peft_kwargs is reached from every diffusers LoRA loader (_load_lora_into_text_encoder, load_lora_into_unet, load_lora_into_transformer, etc.). Each caller walks model.named_modules() and probes the state_dict for {module_name}.lora_B.weight keys, building a rank_dict. When that loop matches nothing (typical when the saved keys carry an extra/missing prefix vs the target model, an adapter-name infix between lora_B and weight, or a non-diffusers serialization format that was not converted upstream), rank_dict arrives at get_peft_kwargs empty.

The first statement of the function is:

r = lora_alpha = list(rank_dict.values())[0]

which raises IndexError: list index out of range with no signal about which mismatch is actually responsible. The traceback ends inside get_peft_kwargs with no way to tell whether you have a prefix problem, an adapter-name infix problem, a format-conversion problem, or just a state_dict that genuinely has no LoRA weights at all.

This PR raises a ValueError ahead of the indexing, with a message that names the common causes and shows a sample of the state_dict keys. Future bug reports surface the actual mismatch shape in the user's report instead of a cryptic IndexError.

Scope

  • One file change: src/diffusers/utils/peft_utils.py, function get_peft_kwargs, ~20 added lines (a guard + diagnostic message).
  • No behaviour change on the happy path: when rank_dict is non-empty, the existing first line runs unchanged.
  • New file: tests/others/test_peft_utils.py with three focused unit tests (empty rank_dict raises, None state_dict is safe, fast-path unchanged).

This is intentionally a defensive surface-area fix only. The underlying SDXL/SD3.5 key-mismatch in #3238 is a separate concern (likely in convert_state_dict_to_peft or the per-pipeline _load_lora_into_text_encoder rank-discovery loops); diagnosing it requires reproducing the user's specific state_dict layout. Surfacing the actionable error here is the prerequisite for collecting that data from future reporters.

Test plan

  • tests/others/test_peft_utils.py (new):
    • test_empty_rank_dict_raises_actionable_value_error; exercises the new path; asserts ValueError, asserts message names the underlying mismatch and includes a state_dict key sample.
    • test_empty_rank_dict_with_none_state_dict_is_safe; covers the peft_state_dict=None defensive branch.
    • test_non_empty_rank_dict_unchanged; smoke-tests the happy path (r=4, lora_alpha=4, no rank_pattern, expected target_modules).
  • All three pass locally via direct sys.path import without needing the full diffusers install (the function is pure utility code).
  • Did not run the full diffusers test suite locally (heavy install with torch + transformers + accelerate); relying on CI.

AI Assistance Disclosure

Investigation, the patch, and the tests were drafted with Claude assistance. I read the call graph from each caller (_load_lora_into_text_encoder, load_lora_into_unet) into get_peft_kwargs to confirm rank_dict can legitimately be empty under valid-input + key-mismatch conditions, and that the change preserves the happy path bitwise. I am the human contributor accountable for this PR.

…in get_peft_kwargs

`get_peft_kwargs` is reached after callers (e.g.
`_load_lora_into_text_encoder` and the various
`load_lora_into_{unet,transformer}` flows) walk the model's
`named_modules` and probe the state_dict for
`{module}.lora_B.weight` keys. When the saved LoRA state_dict
carries an extra or missing prefix versus the target model
(e.g. `text_model.encoder.*` vs `encoder.*`), an adapter-name
infix between `lora_B` and `weight` (e.g. `.default_0.`), or a
non-diffusers serialization format that was not converted
upstream, none of the probed keys are present in the state_dict
and `rank_dict` arrives here empty.

The first statement of the function was
`r = lora_alpha = list(rank_dict.values())[0]`, which then
raised an unhelpful `IndexError: list index out of range` with
no signal about which mismatch was actually responsible (see
huggingface/peft#3238 for a real-world report against SDXL +
peft 0.19; the report was filed against peft but the failure
path is in diffusers).

Raise a `ValueError` whose message names the common causes and
shows a sample of the state_dict keys, so the next user with a
key-mismatch can diagnose their specific case from the
exception alone rather than from a 5-frame deep IndexError.

Adds focused tests for the new error path, the safe-on-None
behavior, and the unchanged fast-path.

Refs: huggingface/peft#3238
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M PR with diff < 200 LOC tests utils

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant