Skip to content

Commit 9532cba

Browse files
committed
Roll back hyena_mixer diffs
Signed-off-by: John St. John <jstjohn@nvidia.com>
1 parent 49be647 commit 9532cba

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

  • bionemo-recipes/recipes/evo2_megatron/src/bionemo/evo2/models/megatron/hyena

bionemo-recipes/recipes/evo2_megatron/src/bionemo/evo2/models/megatron/hyena/hyena_mixer.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,9 @@ def __init__(
119119
self.fast_conv_mixer = self.hyena_config.fast_conv_mixer
120120

121121
self.use_subquadratic_ops = self.transformer_config.use_subquadratic_ops
122-
self.use_fused_b2b_causal_conv1d = self.use_subquadratic_ops
122+
# TODO: Re-enable B2BCausalConv1dModule for short/medium Hyena layers once
123+
# subquadratic-ops updates it to support causal_conv1d 1.6+ semantics.
124+
self.use_fused_b2b_causal_conv1d = False
123125

124126
# Per attention head and per partition values.
125127
assert torch.distributed.is_initialized()

0 commit comments

Comments
 (0)