FIX Call super().__post_init__() in CPTConfig and RandLoraConfig#3365
Conversation
CPTConfig and RandLoraConfig override __post_init__ without calling super().__post_init__(), so PeftConfig.__post_init__ never runs for them. As a result the peft_version field is left as None (instead of being auto-filled) and the task_type validation in the base class is skipped. Every other tuner config that overrides __post_init__ calls super(); add the call (first, matching the convention) so these two behave the same and record peft_version in their saved adapter_config.json.
BenjaminBossan
left a comment
There was a problem hiding this comment.
Thanks for catching this omission.
I checked why our tests didn't catch this and found that ALL_CONFIG_CLASSES in tests/tst_config.py doesn't include those two (there might be others that are missing). Could you please add them to the test matrix?
Follow-up to adding the missing configs to ALL_CONFIG_CLASSES: - CPTConfig requires task_type=CAUSAL_LM to construct and only supports that task type, so pin it in mandatory_kwargs and skip it in the generic valid/invalid task_type tests (it's still covered by the round-trip tests, which is what would have caught the original __post_init__ omission). - OFTConfig's from_pretrained back-compat guard fires before the generic load path, so skip it in the four from_pretrained-based tests. All other matrix tests now exercise both configs.
|
Thanks! Added them. Diffing Two needed a little handling so the generic tests stay green:
Locally: |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
BenjaminBossan
left a comment
There was a problem hiding this comment.
Thanks for taking care of syncing the config tests. The PR LGTM.
What does this PR do?
CPTConfigandRandLoraConfigoverride__post_init__but do not callsuper().__post_init__(), soPeftConfig.__post_init__never runs for them. That base method:peft_version(if self.peft_version is None: self.peft_version = self._get_peft_version()), andtask_type.As a result,
CPTConfig(...).peft_versionandRandLoraConfig(...).peft_versionare left asNone, so the savedadapter_config.jsonfor CPT / RandLoRA adapters does not record the PEFT version that every other adapter type records. The basetask_typevalidation is also skipped.Of the tuner configs that override
__post_init__, all but these two callsuper().__post_init__(). This PR adds the call (first, matching the convention) so CPT and RandLoRA behave the same.How it was verified
Before:
CPTConfig(task_type="CAUSAL_LM").peft_version→NoneRandLoraConfig().peft_version→NoneAfter:
'0.19.2.dev0@UNKNOWN', and the value round-trips throughsave_pretrained/from_pretrained.__post_init__logic is unaffected (CPT still setsis_prompt_learning,num_virtual_tokens, …; RandLoRA still setspeft_typeand thetarget_modulesset).tests/test_cpt.pyandtests/test_randlora.pypass (15 passed).ruff checkpasses on both files.Note: #3227 also edits
randlora/config.py(thesave_projectionfield default), on different lines — no overlap with this change.AI assistance
AI assistance was used to prepare this change. I reviewed every changed line, verified the behavior as described above, and take responsibility for the patch.