Skip to content

Commit 48c74bd

Browse files
Update docs
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
1 parent a4ad1b8 commit 48c74bd

2 files changed

Lines changed: 31 additions & 5 deletions

File tree

examples/megatron_bridge/README.md

Lines changed: 30 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,33 @@ This directory contains examples of using Model Optimizer with [NeMo Megatron-Br
1818

1919
Running these examples requires many additional dependencies to be installed (e.g., Megatron-Bridge, Megatron-core, etc.), hence we strongly recommend directly using the NeMo container (e.g., `nvcr.io/nvidia/nemo:26.02`) which has all the dependencies installed.
2020

21-
To get the latest ModelOpt features and examples, you can mount your latest ModelOpt cloned repository to the container at `/opt/Megatron-Bridge/3rdparty/Model-Optimizer` or pull the latest changes once inside the docker container (`cd /opt/Megatron-Bridge/3rdparty/Model-Optimizer && git checkout main && git pull`).
21+
To get the latest ModelOpt features and examples scripts, mount your Model-Optimizer repo to the container.
22+
23+
```bash
24+
export MODELOPT_DIR=${PWD}/Model-Optimizer # or set to your local Model-Optimizer repository path if you have cloned it
25+
if [ ! -d "${MODELOPT_DIR}" ]; then
26+
git clone https://github.com/NVIDIA/Model-Optimizer.git ${MODELOPT_DIR}
27+
fi
28+
29+
export DOCKER_IMAGE=nvcr.io/nvidia/nemo:26.02
30+
docker run \
31+
--gpus all \
32+
--shm-size=16GB \
33+
--net=host \
34+
--ulimit memlock=-1 \
35+
--rm -it \
36+
-v ${MODELOPT_DIR}:/opt/Model-Optimizer \
37+
-v ${MODELOPT_DIR}/modelopt:/opt/venv/lib/python3.12/site-packages/modelopt \
38+
-w /opt/Model-Optimizer/examples/megatron_bridge \
39+
${DOCKER_IMAGE} bash
40+
```
41+
42+
Once inside the container, you need to login with your HuggingFace token to download gated datasets / models.
43+
Note that the default dataset for pruning and quantization is [`nemotron-post-training-dataset-v2`](https://huggingface.co/datasets/nvidia/Nemotron-Post-Training-Dataset-v2), which is gated.
44+
45+
```bash
46+
huggingface-cli login --token <your token>
47+
```
2248

2349
## Pruning
2450

@@ -30,7 +56,7 @@ Example usage to prune Qwen3-8B to 6B on 2-GPUs (Pipeline Parallelism = 2) while
3056
top-10 candidates are evaluated for MMLU score (5% sampled data) to select the best model.
3157

3258
```bash
33-
torchrun --nproc_per_node 2 /opt/Megatron-Bridge/3rdparty/Model-Optimizer/examples/megatron_bridge/prune_minitron.py \
59+
torchrun --nproc_per_node 2 prune_minitron.py \
3460
--hf_model_name_or_path Qwen/Qwen3-8B \
3561
--prune_target_params 6e9 \
3662
--hparams_to_skip num_attention_heads \
@@ -41,7 +67,7 @@ Example usage for manually pruning to a specific architecture using following de
4167
1024 samples from [`nemotron-post-training-dataset-v2`](https://huggingface.co/datasets/nvidia/Nemotron-Post-Training-Dataset-v2) for calibration.
4268

4369
```bash
44-
torchrun --nproc_per_node 2 /opt/Megatron-Bridge/3rdparty/Model-Optimizer/examples/megatron_bridge/prune_minitron.py \
70+
torchrun --nproc_per_node 2 prune_minitron.py \
4571
--hf_model_name_or_path Qwen/Qwen3-8B \
4672
--prune_export_config '{"hidden_size": 3584, "ffn_hidden_size": 9216}' \
4773
--output_hf_path /tmp/Qwen3-8B-Pruned-6B-manual
@@ -50,7 +76,7 @@ torchrun --nproc_per_node 2 /opt/Megatron-Bridge/3rdparty/Model-Optimizer/exampl
5076
To see the full usage for advanced configurations, run:
5177

5278
```bash
53-
torchrun --nproc_per_node 1 /opt/Megatron-Bridge/3rdparty/Model-Optimizer/examples/megatron_bridge/prune_minitron.py --help
79+
torchrun --nproc_per_node 1 prune_minitron.py --help
5480
```
5581

5682
> [!TIP]

examples/megatron_bridge/prune_minitron.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
--output_hf_path /tmp/Qwen3-8B-Pruned-6B
2828
2929
To see the full usage for advanced configurations, run:
30-
python prune_minitron.py --help
30+
torchrun --nproc_per_node 1 prune_minitron.py --help
3131
"""
3232

3333
import argparse

0 commit comments

Comments
 (0)