I assume this works without ONLY_CBLAS=ON, and is only an issue if you disable the BLAS interface.

Interestingly, if you do that, you'll miss out on GEMV forwarding. Depending on your use case, it might be quite a performance boost:

@Mousius it does look as if the restrictive AND NOT ONLY_CBLAS may have been added (by you, apparently ?) due to the "inexplicably" missing sbstobf16, at least neither the equivalent Makefile.system nor the actual interface/gemm.c has such a restriction.

@martin-frbg yip, I added the limiter. It was actually because it calls gemv behind the scenes and that didn't exist in ONLY_CBLAS - only cblas_gemv.

I was just giving a friendly heads up, using the new bgemm is likely someone experimenting with ML workloads and a lot of the transformer models actually end up using the gemv proxy.

Oh right, I need to make sure that GEMV is hooked up correctly now

Interestingly, if you do that, you'll miss out on GEMV forwarding. Depending on your use case, it might be quite a performance boost:

I'll try poking around with these settings. Thanks for the info. The matrix sizes are predetermined, so direct calls to cblas_*gemv are also applicable.

using the new bgemm is likely someone experimenting with ML workloads and a lot of the transformer models actually end up using the gemv proxy

Yeah, I'm testing it in an LLM inference engine.

I have successfully wired up the attention layer in full bfloat16, but the performance is ~60x worse than float32 right now, as my CPU lacks AVX512 BF16 extensions.

So this is a proof-of-concept experiment which may be useful years later (when most CPUs being sold have BF16 extensions)

@Mousius all forwarding should be pointing to GEMV i.e. the driver code, not gemv the interface function, so I think the only unresolved references to sgemv et al., if any, should be coming from LAPACK and perhaps the tests. (What's a bit annoying is that currently the CMake build will attempt to build the utest even with ONLY_CBLAS, which means the build
will fail before reaching ctest. And the few bfloat16-related tests we have are all in test, relying on the BLAS interface. Also we seem to have the same problem regarding the lack of sbstobf16 with ONLY_CBLAS in the Makefile build)