File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -197,7 +197,7 @@ optimizer:
197197# Distributed training
198198distributed :
199199 backend : nccl
200- nvfsdp :
200+ mfsdp :
201201 enable : true
202202 sharding_strategy : zero3
203203
@@ -242,7 +242,7 @@ training:
242242 num_train_steps: 100 # Enough steps for stable metrics
243243
244244wandb:
245- name: "esm2_nvfsdp_benchmark "
245+ name: "esm2_mfsdp_benchmark "
246246 tags: ["L1", "benchmark", "performance"]
247247` ` `
248248
@@ -411,7 +411,7 @@ docker run --rm -it --gpus all my_recipe pytest -v .
411411
412412For reference implementations, examine existing recipes :
413413
414- - **`esm2_native_te_nvfsdp /`**: Comprehensive example showing vanilla PyTorch with TE and nvFSDP
414+ - **`esm2_native_te_mfsdp /`**: Comprehensive example showing vanilla PyTorch with TE and megatron-fsdp
415415- **`amplify_accelerate_fp8/`**: HuggingFace Accelerate integration with FP8 training
416416- **`geneformer_lightning_context_parallel/`**: PyTorch Lightning with context parallelism for long sequences
417417
You can’t perform that action at this time.
0 commit comments