Skip to content

Commit 717fb6e

Browse files
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
1 parent d5bf1af commit 717fb6e

1 file changed

Lines changed: 1 addition & 5 deletions

File tree

  • transformer_engine/pytorch/attention/dot_product_attention

transformer_engine/pytorch/attention/dot_product_attention/utils.py

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -464,11 +464,7 @@ def get_attention_backend(
464464
# On SM90, prefer FA3 over FA4 when FA3 is available.
465465
# FA3 is more mature on Hopper; FA4's SM90 backward has limitations
466466
# (MLA, non-standard head dims, SplitKV).
467-
if (
468-
use_flash_attention_4
469-
and use_flash_attention_3
470-
and device_compute_capability == (9, 0)
471-
):
467+
if use_flash_attention_4 and use_flash_attention_3 and device_compute_capability == (9, 0):
472468
if FlashAttentionUtils.v4_is_installed:
473469
logger.debug("Disabling FlashAttention 4 to prefer FlashAttention 3 on SM90")
474470
use_flash_attention_4 = False

0 commit comments

Comments
 (0)