Skip to content

Commit 4a95dff

Browse files
authored
Add cooperative matrix dispatch for quantized linear
Differential Revision: D110336871 Pull Request resolved: #21069
1 parent 666ba0b commit 4a95dff

14 files changed

Lines changed: 2036 additions & 73 deletions

backends/vulkan/runtime/graph/ComputeGraph.h

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -686,6 +686,14 @@ class ComputeGraph final {
686686
return context_->adapter_ptr()->device_type() == vkapi::DeviceType::MALI;
687687
}
688688

689+
// AMD-RDNA GPUs (Samsung Xclipse, AMD Radeon). There is no DeviceType for
690+
// AMD, so this matches on the driver-reported device name; both casings are
691+
// checked since the string casing varies by driver.
692+
inline bool device_is_amd() {
693+
return device_name_contains("Xclipse") || device_name_contains("xclipse") ||
694+
device_name_contains("Radeon") || device_name_contains("radeon");
695+
}
696+
689697
const std::string& device_name() {
690698
return context()->adapter_ptr()->device_name();
691699
}

backends/vulkan/runtime/graph/ops/glsl/linear_dq8ca_qw_coopmat.glsl

Lines changed: 588 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Copyright (c) Meta Platforms, Inc. and affiliates.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the BSD-style license found in the
5+
# LICENSE file in the root directory of this source tree.
6+
7+
# coopmat<int8> x coopmat<int8> -> coopmat<int32> variant of the
8+
# dynamically-quantized-activation linear tiled shader (INT4 group-symmetric
9+
# weight).
10+
# WEIGHT_NBITS=4 -> linear_dq8ca_q4gsw_coopmat (INT4 group-symmetric)
11+
# Requires the VK_COMPONENT_TYPE_SINT8_KHR cooperative matrix property to be
12+
# enumerated on the device.
13+
# Loop structure follows the double-buffered reference (coopmat_mm_ref) at
14+
# a 128x64 tile with K-step 32, 4 subgroups x 64 threads. The reference's
15+
# subgroup-32 layout is NOT used — the Xclipse PAL compiler crashes in
16+
# vkCreateComputePipelines when int8 WMMA is compiled at forced subgroup
17+
# size 32 (fp16 WMMA at 32 is fine; see linear_qw_coopmat).
18+
19+
linear_dq8ca_qw_coopmat:
20+
parameter_names_with_default_values:
21+
PRECISION: highp
22+
HAS_BIAS: false
23+
WEIGHT_NBITS: 4
24+
WEIGHT_STORAGE: texture2d
25+
MMA_M: 16
26+
MMA_N: 16
27+
MMA_K: 16
28+
WG_TILE_M: 128
29+
WG_TILE_N: 64
30+
WG_TILE_K: 32
31+
SG_GRID_X: 2
32+
SG_GRID_Y: 2
33+
SUBGROUP_SIZE: 64
34+
shader_variants:
35+
- NAME: linear_dq8ca_q4gsw_coopmat_buffer_texture2d_half
36+
WEIGHT_NBITS: 4
37+
WEIGHT_STORAGE: texture2d
38+
- NAME: linear_dq8ca_q4gsw_coopmat_buffer_buffer_half
39+
WEIGHT_NBITS: 4
40+
WEIGHT_STORAGE: buffer

0 commit comments

Comments
 (0)