Skip to content

Commit 828597c

Browse files
Moving 'activation_vocab' sharding rule with single axis to the top
Without this change, the automatic sharding resolution fails to pick up the correct sharding for activation_vocab when tensor_sequence is used.
1 parent dbc7eda commit 828597c

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

src/MaxText/configs/base.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -378,9 +378,9 @@ logical_axis_rules: [
378378
['activation_kv_batch', ['data', 'fsdp', 'fsdp_transpose', 'expert']],
379379
['activation_kv_batch_no_exp', ['data', 'fsdp', 'fsdp_transpose']],
380380
['activation_kv_head_dim', ['tensor', 'tensor_transpose', 'tensor_sequence']],
381+
['activation_vocab', 'tensor_sequence'],
381382
['activation_vocab', ['tensor', 'tensor_transpose', 'sequence', 'tensor_sequence']],
382383
['activation_vocab', ['tensor', 'tensor_transpose']],
383-
['activation_vocab', 'tensor_sequence'],
384384
['activation_vocab', ['sequence','context']],
385385
['activation_stage', 'stage'],
386386
['activation_exp', ['expert']],

0 commit comments

Comments
 (0)