Commit 564af66
power/bgemm: add BFloat16-in, BFloat16-out GEMM kernel for POWER10
Add BGEMM (BF16 input → BF16 output) for POWER10 by reusing the
existing SBGEMM kernel infrastructure. A -DBGEMM compile flag switches
only the store path; the xvbf16ger2pp MMA instruction and BF16 packing
routines are shared with SBGEMM unchanged.
Changes
-------
kernel/power/KERNEL.POWER10
- Register BGEMM kernel and copy-routine targets, reusing the
sbgemm_ncopy/tcopy sources (packing layout is identical).
kernel/power/sbgemm_kernel_power10.c
- Under BGEMM: force v4sf_t to float so accumulators stay in
float32; add STORE4_BF16/STORE2_BF16 macros (read BF16 C, widen,
apply alpha*acc, convert back via xvcvspbf16, store); add
f32_to_bf16_scalar() for m&1/n&1 tails; add BGEMM variants of
all SAVE_ACC macros covering the full m/n tile hierarchy.
param.h
- Add BGEMM_DEFAULT_UNROLL_M=16, UNROLL_N=8, P/Q/R blocking
parameters for POWER10, matching the 16x8 kernel tile.
Unit test (utest/test_extensions/test_bgemm.c)
- Uses SBGEMM as a trusted reference. 21 test cases cover all four
transpose combinations, all m/n remainder paths, odd-k, alpha=0,
beta=0/1, and the m>=32 fast path. Tolerance 0.01 to account for
the one extra BF16 rounding on the BGEMM store.
Performance (POWER10, single-threaded, Transa=N Transb=N)
----------------------------------------------------------
Step=1 (sizes 1–200):
- Sizes 1–80: baseline ~12–131 MFlops (scalar fallback);
patch 200–46,000 MFlops (up to ~350,000x faster).
- Sizes 81–200: patch 1x–35x faster with 16x8 MMA tile engaged.
Step=8 (sizes 8–1024):
- Baseline peaks at ~5,200 MFlops; patch sustains 40,000–760,000
MFlops for sizes 384–960. Average speedup ~100x.
Signed-off-by: Amrita H S <amritahs@linux.vnet.ibm.com>1 parent 0b54dd1 commit 564af66
5 files changed
Lines changed: 644 additions & 25 deletions
File tree
- kernel/power
- utest
- test_extensions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
6 | 22 | | |
7 | 23 | | |
8 | 24 | | |
| |||
0 commit comments