fix(ck-tile): guard TE GEMM against numerically-wrong CShuffle configs (#9684)#9686
Draft
ozturkosu wants to merge 1 commit into
Draft
fix(ck-tile): guard TE GEMM against numerically-wrong CShuffle configs (#9684)#9686ozturkosu wants to merge 1 commit into
ozturkosu wants to merge 1 commit into
Conversation
#9684) The Tile Engine GEMM instance builder emitted cshuffle kernels whose per-wave repeat is odd (>1) and paired with a 32-wide warp tile (e.g. tile_m=192 / warp_m=2 / warp_tile_m=32, MRepeat=3). These compile and pass the epilogue's divisibility static_asserts but the CShuffle LDS store is mis-raked, so they return numerically WRONG results at runtime -- GPU-verified on gfx942 via the op's built-in CPU validation (26 such configs fail; all other non-power-of-two repeats, incl. MRepeat=3 with warp_tile_m=16 and even repeats 6/12, pass). Skip exactly that signature during instance enumeration so the TE stops shipping broken kernels. Add a unit test for the gate. Refs #9684
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the numerically-incorrect CShuffle batched/standard GEMM kernels tracked in #9684. The Tile Engine GEMM instance builder emitted cshuffle kernels whose per-wave repeat is odd (>1) and paired with a 32-wide warp tile in that dimension. They compile and pass the epilogue's divisibility
static_asserts, but the CShuffle LDS store is mis-raked and the kernels return numerically wrong results at runtime.Fixes #9684
Motivation
Found during the batched-GEMM TE→dispatcher bridge parity audit (#9306). GPU verification on gfx942 built all 116 non-power-of-2-repeat cshuffle/192 configs and ran the op's built-in CPU validation (
verify=1):tile_m=192+ waves2x2x1+ warp_tile32x32x{8,16}(MRepeat = 192/(2*32) = 3), across compv3/compv4/mem and intra/interwave.warp_tile_m=16and even repeats like 6 and 12.So the failure is specific to odd repeat (>1) + 32-wide warp tile, not "non-power-of-2 repeat" in general.
Design note
The guard is applied once, during instance enumeration in
_get_sampled_kernel_list(), via a small_cshuffle_store_ok(m_repeat, n_repeat, warp_tile_m, warp_tile_n)helper. Only the cshuffle epilogue is gated; the default epilogue stores directly and is unaffected. This is the minimal, precise signature — it stops emitting exactly the 26 broken configs and keeps all correct ones.Changes
tile_engine/ops/gemm/gemm_instance_builder.py: add_cshuffle_store_okhelper + guard in the enumeration loop.tile_engine/ops/gemm/test_gemm_instance_builder.py: unit tests for the gate.Test plan
python3 -m pytest tile_engine/ops/gemm/test_gemm_instance_builder.py(4 passed)verify=1; the 90 kept configs pass)Related
users/muozturk/ck/bridge_batched_gemm, PR feat(ck-tile): batched GEMM TE to dispatcher bridge #9306).