Skip to content

Commit c959cd1

Browse files
mudlerlocalai-org-maint-bot
authored andcommitted
fix(buun-llama-cpp): shim cudaMemcpy{To,From}Symbol + WARP_SIZE on fwht128 shuffles
Two more hipblas-only build failures in buun's fattn.cu, fixed under the same patches/ infrastructure: 1. cudaMemcpyToSymbol / cudaMemcpyFromSymbol — buun's Q² calibration + TCQ codebook upload paths call the symbol variants of cudaMemcpy. ggml/src/ggml-cuda/vendors/hip.h aliases every other cudaMemcpy* name (cudaMemcpy, cudaMemcpyAsync, cudaMemcpy2DAsync, …) but the symbol pair was never added. 15+ "use of undeclared identifier" errors across fattn.cu lines 40, 54, 74-76, 94, 100-101, 371, 883, 905, 954, 976, 1449, 1463. Add the two missing aliases alongside the existing memcpy block. 2. __shfl_xor_sync fwht128 calls — same 3-arg omission pattern as the earlier argmax top-K fix. Lines 512 (ggml_cuda_fwht128 intra-warp butterfly) and 536 (fwht128_store_half neighbor fetch) drop the width argument that hip.h:33 requires. Add WARP_SIZE. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
1 parent a7552cf commit c959cd1

2 files changed

Lines changed: 60 additions & 0 deletions

File tree

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
Subject: [PATCH] ggml-cuda/vendors/hip: alias cudaMemcpy{To,From}Symbol to hip counterparts
2+
3+
Buun's Q² calibration + TCQ codebook upload paths in fattn.cu use
4+
cudaMemcpyToSymbol / cudaMemcpyFromSymbol. The HIP-compat header in
5+
ggml/src/ggml-cuda/vendors/hip.h already aliases the scalar cudaMemcpy
6+
family (cudaMemcpy, cudaMemcpyAsync, cudaMemcpy2DAsync, …) but is
7+
missing the symbol variants. Building with hipcc therefore fails with
8+
15+ "use of undeclared identifier 'cudaMemcpyToSymbol'" errors.
9+
10+
Add the two missing aliases alongside the existing memcpy block. HIP
11+
provides hipMemcpy{To,From}Symbol with the same signature as CUDA's
12+
equivalents, so this is a straight name substitution.
13+
14+
--- a/ggml/src/ggml-cuda/vendors/hip.h
15+
+++ b/ggml/src/ggml-cuda/vendors/hip.h
16+
@@ -85,6 +85,8 @@
17+
#define cudaMemcpyDeviceToDevice hipMemcpyDeviceToDevice
18+
#define cudaMemcpyDeviceToHost hipMemcpyDeviceToHost
19+
#define cudaMemcpyHostToDevice hipMemcpyHostToDevice
20+
+#define cudaMemcpyToSymbol hipMemcpyToSymbol
21+
+#define cudaMemcpyFromSymbol hipMemcpyFromSymbol
22+
#define cudaMemcpyKind hipMemcpyKind
23+
#define cudaMemset hipMemset
24+
#define cudaMemsetAsync hipMemsetAsync
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
Subject: [PATCH] ggml-cuda/fattn: pass WARP_SIZE to fwht128 __shfl_xor_sync calls
2+
3+
Same issue as the argmax top-K fix: two __shfl_xor_sync call sites in
4+
the FWHT-128 butterfly kernels (ggml_cuda_fwht128 and fwht128_store_half)
5+
use the 3-arg CUDA form and omit the `width` argument that the HIP
6+
function-like macro in vendors/hip.h:33 requires. Hipcc fails with:
7+
8+
fattn.cu:512: too few arguments provided to function-like macro
9+
invocation
10+
note: macro '__shfl_xor_sync' defined here:
11+
#define __shfl_xor_sync(mask, var, laneMask, width) \
12+
__shfl_xor(var, laneMask, width)
13+
14+
Add WARP_SIZE to both calls. CUDA codegen is unchanged (warpSize is the
15+
default); HIP now matches the macro arity.
16+
17+
--- a/ggml/src/ggml-cuda/fattn.cu
18+
+++ b/ggml/src/ggml-cuda/fattn.cu
19+
@@ -509,7 +509,7 @@
20+
// Intra-warp passes: shuffle xor with stride h, no smem, no sync.
21+
#pragma unroll
22+
for (int h = 1; h <= 16; h *= 2) {
23+
- const float other = __shfl_xor_sync(0xFFFFFFFF, val, h);
24+
+ const float other = __shfl_xor_sync(0xFFFFFFFF, val, h, WARP_SIZE);
25+
val = (tid & h) ? (other - val) : (val + other);
26+
}
27+
28+
@@ -533,7 +533,7 @@
29+
static __device__ __forceinline__ void fwht128_store_half(
30+
float val, half * dst_base) {
31+
const int tid = threadIdx.x;
32+
- const float neighbor = __shfl_xor_sync(0xFFFFFFFF, val, 1);
33+
+ const float neighbor = __shfl_xor_sync(0xFFFFFFFF, val, 1, WARP_SIZE);
34+
if ((tid & 1) == 0) {
35+
const half2 packed = __floats2half2_rn(val, neighbor);
36+
*((half2 *)(dst_base + tid)) = packed;

0 commit comments

Comments
 (0)