Skip to content

Commit ec47f96

Browse files
committed
update
1 parent 6f01998 commit ec47f96

1 file changed

Lines changed: 9 additions & 7 deletions

File tree

src/maxtext/configs/models/deepseek3-671b-batchsplit.yml

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -75,12 +75,14 @@ logical_axis_rules: [
7575
['q_lora', ['fsdp']],
7676
['kv_lora', ['fsdp']],
7777
['layers', 'stage'],
78-
['q_lora_up_proj', ['fsdp_transpose', 'expert']],
79-
['kv_lora_up_proj', ['fsdp_transpose', 'expert']],
80-
['q_heads', ['fsdp_transpose', 'expert']],
81-
['kv_heads', ['fsdp_transpose', 'expert']],
82-
['heads', ['fsdp_transpose', 'expert']],
83-
['mlp', ['fsdp_transpose', 'expert']],
78+
['q_lora_up_proj', ['fsdp_transpose']],
79+
['kv_lora_up_proj', ['fsdp_transpose']],
80+
['q_heads', ['fsdp_transpose']],
81+
['kv_heads', ['fsdp_transpose']],
82+
['heads', ['fsdp_transpose']],
83+
['mlp', ['fsdp_transpose']],
8484
['mlp_only_fsdp_transpose', ['fsdp_transpose']],
85-
['mlp_only_tensor', ['expert']],
85+
['expert_only', ['expert']],
86+
['fsdp_transpose_only', ['fsdp_transpose']],
87+
['fsdp_transpose_and_expert', ['fsdp_transpose', 'expert']],
8688
]

0 commit comments

Comments
 (0)