Skip to content

add tileN = 8,16 for SM120 blockscale GEMM.#3292

Merged
hwu36 merged 2 commits into
NVIDIA:mainfrom
bzhng-development:brayden/sm120-tile-n-16
Jun 26, 2026
Merged

add tileN = 8,16 for SM120 blockscale GEMM.#3292
hwu36 merged 2 commits into
NVIDIA:mainfrom
bzhng-development:brayden/sm120-tile-n-16

Conversation

@b8zhong

@b8zhong b8zhong commented Jun 2, 2026

Copy link
Copy Markdown
Contributor
image

It will be for use with SwapAB.

@b8zhong

b8zhong commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Hi @depaulmillz , I was wondering if you could take a look at this PR? Since I noticed you were the last one to add TileN = 32. Thanks!

@depaulmillz

Copy link
Copy Markdown
Contributor

Awesome. Have you been able to try with group GEMM as well?

@b8zhong

b8zhong commented Jun 3, 2026

Copy link
Copy Markdown
Contributor Author

@depaulmillz Yes. Technically, it works (this PR is also compatible with group GEMM changes as well). But for example when testing on two common cases, DSR1 TP = 8 and Qwen-3 MoE TP = 1, the speedup can only be 3-5% for BS = 1. So it's faster (as expected), but not by much.

@depaulmillz

Copy link
Copy Markdown
Contributor

It looks like you will need to add an assertion to prevent compiling ping-pong with MMA_N=8 which will expect a (2,2,1) layout shape for the MMA. I saw some ref check errors when testing the MR on pingpong MMA_N=8 kernels due to this.

@b8zhong

b8zhong commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

@depaulmillz Just added a guard for it. Could you take a try now?

@b8zhong

b8zhong commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Hi @depaulmillz , just bumping this PR for visibility. Thanks a lot!

@depaulmillz

Copy link
Copy Markdown
Contributor

@hwu36 can we merge this?

@hwu36 hwu36 merged commit e8ecfad into NVIDIA:main Jun 26, 2026
@b8zhong

b8zhong commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @depaulmillz @hwu36 !

@b8zhong b8zhong deleted the brayden/sm120-tile-n-16 branch June 26, 2026 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants