@@ -218,9 +218,9 @@ uv run python examples/conversion/hf_to_megatron_generate_text.py \
218218
219219| Model | Mode | TP | PP | EP | Total GPUs | Use Case |
220220| -------| ------| ----| ----| ----| -----------:| ----------|
221- | ** Qwen3-30B-A3B** | Pretrain | 1 | 1 | 8 | 8 | Pre-training (single node ) |
222- | ** Qwen3-30B-A3B** | Full SFT | 1 | 1 | 8 | 8 | Full supervised finetuning |
223- | ** Qwen3-30B-A3B** | LoRA/DoRA | 1 | 1 | 8 | 8 | PEFT finetuning (single node) |
221+ | ** Qwen3-30B-A3B** | Pretrain | 1 | 1 | 16 | 16 | Pre-training (2 nodes ) |
222+ | ** Qwen3-30B-A3B** | Full SFT | 1 | 1 | 16 | 16 | Full supervised finetuning (2 nodes) |
223+ | ** Qwen3-30B-A3B** | LoRA/DoRA | 4 | 1 | 4 | 4 | PEFT finetuning |
224224| ** Qwen3-235B-A22B** | Pretrain | 2 | 8 | 32 | 512 | Pre-training (64 nodes) |
225225| ** Qwen3-235B-A22B** | Full SFT | 2 | 8 | 32 | 512 | Full supervised finetuning (64 nodes) |
226226| ** Qwen3-235B-A22B** | LoRA/DoRA | 2 | 8 | 32 | 512 | PEFT finetuning (64 nodes) |
@@ -239,7 +239,7 @@ config = qwen3_30b_a3b_pretrain_config(
239239 train_iters = 500_000 ,
240240 global_batch_size = 2048 ,
241241 seq_length = 4096 ,
242- # Uses TP=1, PP=1, EP=8 (8 GPUs) automatically
242+ # Uses TP=1, PP=1, EP=16 (16 GPUs) automatically
243243)
244244```
245245
@@ -272,7 +272,7 @@ config = qwen3_30b_a3b_sft_config(
272272 train_iters = 1000 ,
273273 global_batch_size = 64 ,
274274 finetune_lr = 5e-6 ,
275- # Uses TP=1, PP=1, EP=8 (8 GPUs) automatically
275+ # Uses TP=1, PP=1, EP=16 (16 GPUs) automatically
276276)
277277```
278278
@@ -288,7 +288,7 @@ config = qwen3_30b_a3b_peft_config(
288288 train_iters = 1000 ,
289289 global_batch_size = 128 ,
290290 finetune_lr = 1e-4 ,
291- # Uses TP=1 , PP=1, EP=8 (8 GPUs) automatically
291+ # Uses TP=4 , PP=1, EP=4 (4 GPUs) automatically
292292)
293293```
294294
0 commit comments