Skip to content

Commit bfed130

Browse files
committed
fix storage
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
1 parent aefc6bf commit bfed130

File tree

1 file changed

+4
-1
lines changed
  • bitsandbytes/backends/cpu

1 file changed

+4
-1
lines changed

bitsandbytes/backends/cpu/ops.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -265,12 +265,15 @@ def _(
265265
code: torch.Tensor,
266266
blocksize: int,
267267
) -> torch.Tensor:
268-
assert B.dtype == torch.uint8, "Only support uint8 qweight"
268+
if B.dtype != torch.uint8:
269+
B = B.view(torch.uint8)
269270
dtype = A.dtype
270271
quant_type = "fp4" if code[1] > 0 else "nf4"
271272
# cpu fused op only support bf16 for now.
272273
if dtype != torch.bfloat16:
273274
A = A.to(torch.bfloat16)
275+
if absmax.dtype != torch.bfloat16:
276+
absmax = absmax.to(torch.bfloat16)
274277

275278
final_out_shape = (*A.shape[:-1], shapeB[0])
276279
A = A.reshape(-1, A.shape[-1])

0 commit comments

Comments
 (0)