Further performance improvements to non-transposed [SD]GEMV kernels for A64FX and Neoverse V1.

[iha-taisei]

close #5210

This pull request proposes a patch for issue #5210.
I have implemented a loop unrolling in the kernel of the non-transposed [SD]GEMV for A64FX and Neoverse V1.
This PullRequest improves performance by 1.7x for A64FX and 2x for Neoverse V1 compared to v0.3.29.

[annop-w]

@iha-taisei Thank you very much for your contribution. Could we please then remove gemv_n_sve.c as well ? since it will no longer be used. I think we should try keeping the number of kernels low.

[annop-w]

@iha-taisei I have just benchmarked the gemv_n_sve_v1x3.c against the NEON kernel in #5225 on NEOVERSEV2 and the SVE version wins slightly.

So, would you mind adding gemv_n_sve_v1x3.c in kernel/arm64/KERNEL.NEOVERSEN2 as well please ? I will close #5225 in favor of this one. Thank you.

[martin-frbg]

@iha-taisei Thank you very much for your contribution. Could we please then remove gemv_n_sve.c as well ? since it will no longer be used. I think we should try keeping the number of kernels low.

I'm not convinced that we need to remove kernel files simply because they are (currently) not in use by any hardware

[abhishek-iitmadras]

Hi @annop-w

Can we use gemv_n_sve_v1x3.c for KERNEL.ARMV8SVE, like we have already for [S/D]GEMVTKERNEL with patch #5215?

cc @iha-taisei

[iha-taisei]

Hi @annop-w

So, would you mind adding gemv_n_sve_v1x3.c in kernel/arm64/KERNEL.NEOVERSEN2 as well please ? I will close #5225 in favor of this one. Thank you.

Yes, you can add the kernel to KERNEL.NEOVERSEN2, although I haven't evaluated it on such CPUs.

[annop-w]

@iha-taisei

Yes, you can add the kernel to KERNEL.NEOVERSEN2, although I haven't evaluated it on such CPUs.

I have results for NEOVERSEV2, which currently uses the same settings as NEOVERSEN2 for DYNAMIC_ARCH, in my above comment. I have not benchmarked on N2 but I believe the result will hold as well and we will see speedup.

[annop-w]

@martin-frbg

I'm not convinced that we need to remove kernel files simply because they are (currently) not in use by any hardware

From a quick look, the kernel gemv_n_sve.c in #5157 is very similar to the one in this PR (the same no. of unrolling). That's why I suggested removing the old one. In fact, it is interesting to try to understand why this one performs significantly better.

[annop-w]

@abhishek-iitmadras

Can we use gemv_n_sve_v1x3.c for KERNEL.ARMV8SVE, like we have already for [S/D]GEMVTKERNEL with patch #5215?

Yes, but I have not tried benchmarking on those CORTEX-As and -Xs. But, seeing how this new SVE kernel outperforms the assembly one on V1 and V2, I expect the same on those cores perhaps.

[martin-frbg]

@abhishek-iitmadras

Can we use gemv_n_sve_v1x3.c for KERNEL.ARMV8SVE, like we have already for [S/D]GEMVTKERNEL with patch #5215?

Yes, but I have not tried benchmarking on those CORTEX-As and -Xs. But, seeing how this new SVE kernel outperforms the assembly one on V1 and V2, I expect the same on those cores perhaps.

I can benchmark on a Pixel8, if we can agree that it is an underrated supercomputer (and if I can find the time and energy for non-trivial work again)