Skip to content

Commit aeee4d2

Browse files
authored
docs: fix typos and errors in training-backends.md (#2287)
Signed-off-by: Terry Kong <terryk@nvidia.com>
1 parent 043775c commit aeee4d2

1 file changed

Lines changed: 2 additions & 6 deletions

File tree

docs/design-docs/training-backends.md

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -35,13 +35,9 @@ _Note_: When using Megatron, the optimizer and learning rate schedule are config
3535
### DTensor Backend
3636
To enable DTensor (FSDP2) training:
3737

38-
1. Set `policy.dtensor_config.enabled=True`.
38+
1. Set `policy.dtensor_cfg.enabled=True`.
3939
2. Refer to [examples/configs/grpo_math_1B.yaml](../../examples/configs/grpo_math_1B.yaml) for a configuration example.
4040

41-
## Backend Priority
42-
43-
**Megatron takes precedence over DTensor.** If both backends are enabled simultaneously (`policy.megatron_cfg.enabled=True` and `policy.dtensor_config.enabled=True`), the Megatron backend will be used.
44-
4541
## Configuration Examples
4642

4743
For comprehensive examples of each algorithm and backend, see the [examples/configs/recipes/llm](https://github.com/NVIDIA-NeMo/RL/tree/main/examples/configs/recipes/llm) folder. This directory contains ready-to-use configurations for various supported combinations.
@@ -72,7 +68,7 @@ export HF_HOME="/shared/nfs/huggingface"
7268

7369
### Best Practices ###
7470

75-
- **Mount in checkpoint directory**: If you are using Docker, make sure the Megatron checkpoint path is covered by `-v`/`--mount`. Similarly, if you are using SLURM+pyxis, ensure `--container-mounts` includes this path.
71+
- **Mount the checkpoint directory**: If you are using Docker, make sure the Megatron checkpoint path is covered by `-v`/`--mount`. Similarly, if you are using SLURM+pyxis, ensure `--container-mounts` includes this path.
7672
- **Use shared storage**: Ensure the checkpoint directory is accessible from all nodes (e.g., NFS, shared filesystem).
7773
- **Prefer HF_HOME**: If you already have `HF_HOME` mounted across nodes, this reduces the number of environment variables to manage.
7874
- **Sufficient space**: Ensure adequate disk space for the converted model checkpoints.

0 commit comments

Comments
 (0)