Skip to content

Commit 1e72989

Browse files
Merge pull request #3805 from AI-Hypercomputer:chengnuojin-fix-2dfsdp
PiperOrigin-RevId: 910198099
2 parents 9767436 + f40620f commit 1e72989

1 file changed

Lines changed: 2 additions & 5 deletions

File tree

src/maxtext/configs/models/deepseek3-671b-2dfsdp.yml

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -71,9 +71,7 @@ logical_axis_rules: [
7171
['activation_stage', 'stage'],
7272
['embed', ['fsdp']],
7373
['embed_moe', ['fsdp']],
74-
['embed_vocab', ['fsdp']],
75-
['embed_no_exp', ['fsdp']],
76-
['embed_no_exp_moe', ['fsdp']],
74+
['embed_vocab', ['fsdp', 'fsdp_transpose']],
7775
['q_lora', ['fsdp']],
7876
['kv_lora', ['fsdp']],
7977
['layers', 'stage'],
@@ -83,7 +81,6 @@ logical_axis_rules: [
8381
['kv_heads', ['fsdp_transpose', 'expert']],
8482
['heads', ['fsdp_transpose', 'expert']],
8583
['mlp', ['fsdp_transpose', 'expert']],
86-
['mlp_only_fsdp_transpose', ['fsdp_transpose']],
87-
['mlp_only_tensor', ['expert']],
84+
['mlp_moe', ['fsdp_transpose', 'expert']],
8885
['diloco', 'diloco'],
8986
]

0 commit comments

Comments
 (0)