Skip to content

Commit 1751c6f

Browse files
committed
[None][perf] Replace DSV4 O_b split-K with CuTe DSL
Replace the dedicated C++/THOP O_b split-K GEMM with a CuTe DSL implementation built on the persistent SM100 block-scaled GEMM. Consume packed K128 UE8M0 scales directly, issue runtime-N tcgen05 MMA, and preserve BF16 split-major partials for the existing mHC fused reduction. Tune the packed-scale, dynamic-N, scheduler, and epilogue paths against current DeepGEMM. Dispatch SK2 through M=128, use CuTe SK1 for supported M>=256 shapes, and retain DeepGEMM as the unsupported-shape fallback. Add tactic, dispatch, packed-scale, and split-partial correctness coverage while removing the obsolete C++ kernel and registration. Signed-off-by: Mingyang Hao <200044211+mingyangHao@users.noreply.github.com>
1 parent f2f16e7 commit 1751c6f

13 files changed

Lines changed: 1916 additions & 1388 deletions

File tree

cpp/tensorrt_llm/kernels/mhcKernels/CMakeLists.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
# this directory (bench_*.cu, probe_*.cu, profile_*.cu, test_*.cu) are
2020
# standalone binaries with their own main() and must NOT be compiled into the
2121
# trtllm shared library.
22-
set(SRC_CU dsv4Fp8SplitKGemm.cu mhcKernels.cu mhcFusedHcKernel.cu)
22+
set(SRC_CU mhcKernels.cu mhcFusedHcKernel.cu)
2323
add_library(mhcKernels_src OBJECT ${SRC_CU})
2424

2525
set_property(TARGET mhcKernels_src PROPERTY POSITION_INDEPENDENT_CODE ON)
@@ -28,7 +28,7 @@ target_compile_options(mhcKernels_src
2828
PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--use_fast_math>)
2929

3030
# Expose the DeepGEMM SM100 MMA/TMA building blocks used by the fused
31-
# post-mapping kernels and DSV4 O_b split-K GEMM.
31+
# post-mapping kernels.
3232
target_include_directories(
3333
mhcKernels_src
3434
PRIVATE ${CMAKE_BINARY_DIR}/_deps/deepgemm-src/deep_gemm/include)

cpp/tensorrt_llm/kernels/mhcKernels/dsv4Fp8SplitKGemm.cu

Lines changed: 0 additions & 208 deletions
This file was deleted.

0 commit comments

Comments
 (0)