Use fld.d/fst.d in PROLOGUE/EPILOGUE in LOONGSON3R5 GEMM

[martin-frbg]

same underlying issue of potential data loss as detailed in #4872 - this is most likely the proper fix for the spurious utest errors seen in #4805

[azuresky01]

This new fix does not work yet. On Debian sid loong64 with gcc 14.2.0:

Building command: CC=gcc FC=gfortran make
The results:

...
TEST 103/107 potrf:smoketest_trivial [FAIL]
  ERR: test_potrs.c:535  U s(0,0) difference: 1.19209e-07
TEST 104/107 potrf:bug_695 [OK]
TEST 105/107 kernel_regress:skx_avx [OK]
TEST 106/107 fork:safety [OK]
TEST 107/107 fork:safety_after_fork_in_parent [OK]
RESULTS: 107 tests (106 ok, 1 failed, 0 skipped) ran in 201 ms
make[1]: *** [Makefile:86: run_test] Error 1
...

The problem remains.

[martin-frbg]

Ah, sad. Guess it was a bit premature to remove the pragma, though it seemed to work well without it on Loongson's 3C5000 in the gcc compile farm (using both gcc14.1 and a git snapshot of 15)

[azuresky01]

My CPU is 3A6000. By the way, the problem appears only when building with gmake. cmake builds seem always fine before/after patches.

[martin-frbg]

cmake build probably uses -O0 when no CMAKE_BUILD_TYPE is specified, and -O3 when building with setting "Release", while gmake build defaults to -O2
As far as I know 3A6000 should use same kernels as 3C5000 (basically the build system only distinguishes between cpu models that support LASX, LSX or neither), but I can only test on the 3C5000. I'll just re-enable the pragma in test_potrs.c in a minute

[martin-frbg]

curiously I do not get the error on 3C5000 even with gcc 14.2.0 (built from source) and the changes from this PR. (without them, the fault appears on each run of utest, with them I can do 100 runs of utest without a single failure). Probably the difference is in the version of binutils used, but can you please check what value of CORE gets put into your Makefile.conf (should be LOONGSON3R5) just to be sure ?

[azuresky01]

Currently the version of binutils on Debian sid loong64 is 2.43-2. And yes the value of CORE in my Makefile.conf is LOONGSON3R5, see below:

OSNAME=Linux
ARCH=loongarch64
C_COMPILER=GCC
BINARY32=
BINARY64=1
CEXTRALIB= -L/usr/lib/gcc/loongarch64-linux-gnu/14 -L/usr/lib/gcc/loongarch64-linux-gnu/14/../../../loongarch64-linux-gnu -L/usr/lib/gcc/loongarch64-linux-gnu/14/../../.. -L/lib/loongarch64-linux-gnu -L/usr/lib/loongarch64-linux-gnu  -lc 
F_COMPILER=GFORTRAN
FC=gfortran
BU=_
FEXTRALIB= -L/usr/lib/gcc/loongarch64-linux-gnu/14 -L/usr/lib/gcc/loongarch64-linux-gnu/14/../../../loongarch64-linux-gnu -L/usr/lib/gcc/loongarch64-linux-gnu/14/../../.. -L/lib/loongarch64-linux-gnu -L/usr/lib/loongarch64-linux-gnu  -lgfortran -lm -lm -lc 
CORE=LOONGSON3R5
LIBCORE=loongson3r5
NUM_CORES=8
MAKEFLAGS += -j 8
...

[azuresky01]

And I report that on Loong Arch Linux with your new fix the build can complete without errors. But when I run ./utest/openblas_utest the problem still exists. On Loong Arch linux the version of binutils is 2.42-1 and the version of gcc is 14.0.1 20240421 (experimental).

[martin-frbg]

strange - host "cfarm400.cfarm.net" where I am running my tests claims to be "Debian trixie/sid", kernel version 6.1.0rc7+ and has binutils 2.43 installed

[azuresky01]

My Debian unstable system is similar but it has a newer kernel:

OS: Debian GNU/Linux trixie/sid loongarch64
Host: Loongson-3A6000-HV-7A2000-1w-V0.1-EVB
Kernel: 6.9.7-loongarch-00002-gfbf7212b7ab3
...

And my apt source is:
deb [arch=loong64] http://ftp.ports.debian.org/debian-ports unstable main contrib non-free non-free-firmware

[azuresky01]

@martin-frbg

update: I recently tried to build OpenBLAS 0.3.30/develop with different gcc compilers (13.4/14.2/14.3/15.1/15.2), and found that the pragma in utest/test_potrs.c can be safely removed. The tested system/host/Kernel are:

OS: Debian GNU/Linux forky/sid loongarch64
Host: Loongson-3A6000-HV-7A2000-1w-V0.1-EVB
Kernel: Linux 6.12.38+deb13-loong64

However, when building OpenBLAS version 0.3.29 or earlier with gcc>=14, the pragma in utest/test_potrs.c is still needed.

[martin-frbg]

Interesting, thanks. The only related change in 0.3.30 would seem to be #5248, which addressed precision issues in several level 1 BLAS functions