Merge pull request #14 from foundation-model-stack/lchu-ibm-patch-1

lchu6 · web-flow · commit 3266e79d3d6e · 2024-01-18T21:15:01.000-05:00
fix full param tuning
diff --git a/tuning/sft_trainer.py b/tuning/sft_trainer.py
@@ -155,7 +155,7 @@ def train(
         peft_config=peft_config,
     )
 
-    if run_distributed:
+    if run_distributed and peft_config is not None:
         trainer.accelerator.state.fsdp_plugin.auto_wrap_policy = fsdp_auto_wrap_policy(model)
     trainer.train()
 

Original file line number	Diff line number	Diff line change
`@@ -155,7 +155,7 @@ def train(`
`155`	`155`	`peft_config=peft_config,`
`156`	`156`	`)`
`157`	`157`
`158`		`- if run_distributed:`
	`158`	`+ if run_distributed and peft_config is not None:`
`159`	`159`	`trainer.accelerator.state.fsdp_plugin.auto_wrap_policy = fsdp_auto_wrap_policy(model)`
`160`	`160`	`trainer.train()`
`161`	`161`