fix(save_and_load): save base_layer.bias for bias="lora_only"

[Anai-Guo]

What

LoraConfig(bias="lora_only") correctly marks a targeted layer's bias as trainable, but get_peft_model_state_dict() / save_pretrained() silently drop it, so a reloaded adapter does not reproduce the trained outputs.

Fixes #3306

Why

For a targeted layer the original module is wrapped, and its bias lives at <module>.base_layer.bias. The bias-key reconstruction was:

bias_name = k.split("lora_")[0] + "bias"   # -> base_model.model.proj.bias

but the real trained key is base_model.model.proj.base_layer.bias, so the lookup never matched and the bias was excluded from the saved state dict. (bias="all" works because it keys off the substring "bias" directly.)

Fix

Look up <prefix>base_layer.bias (current tuner-layer structure) in addition to the legacy <prefix>bias. The same one-line pattern was applied to the identical bias="boft_only" branch for BOFT.

Test

Added TestLoraInitialization::test_lora_only_bias_is_saved_and_reloaded, which perturbs all trainable params (incl. the base_layer bias), asserts the bias key is present in both get_peft_model_state_dict() output and the saved adapter_model.safetensors, and checks that reloading reproduces the pre-save output.

Reproducer from the issue now round-trips with max diff 0.0 (was 1.21).

🤖 Generated with Claude Code

[githubnemo]

Let's move the test around a bit, it doesn't make sense to have that in test_initializations.py since this is not about initializations at all. I propose to create a new test in test_custom_models.py similar to how test_disable_adapter_with_bias_warns works by adding the bias attribute. But we can keep the basic idea of your test. If we find more methods that are buggy this way, we don't have to fix those right now, instead we can mark them as xfail.

It would also be good to test that the bias is not in the state dict when the options isn't set. This would fail right now, I think.