Skip to content

Commit 03d3342

Browse files
authored
Merge pull request #39 from opensolvers/site/benchmarks-blis-gromacs
Sync BLIS and GROMACS Force benchmarks
2 parents 9f6207c + 9b5728e commit 03d3342

7 files changed

Lines changed: 162 additions & 27 deletions

File tree

README.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,13 +29,14 @@ Each RISC-V board exposes several compute paths. We benchmark and tune them inde
2929
| **Custom** | Custom ISA extensions beyond standard RVV | X60 **IME** / **XsmtVdot** (`smt.vmadot`) int8 on [RV2](boards/RV2.html) / [F3](boards/F3.html); [ONNX Runtime](apps/onnx.html) int4 via MLAS |
3030
| **GPU** | Integrated Imagination GPUs (OpenCL / Vulkan) | **IMG BXE-4-32 MC1** on [VisionFive 2](boards/VisionFive2.html) (JH7110); **IMG BXE-2-32** on [RV2](boards/RV2.html) / [F3](boards/F3.html) (K1) — present on silicon, **GPGPU benchmarks not yet** |
3131

32-
Recent highlights on the Orange Pi RV2 (SpaceMiT X60, RVV): fixing an OpenBLAS `gemv_n` bug restores correctness across [BLAS](scientific-libs/blas.html), [LAPACK](scientific-libs/lapack.html), [ELPA](scientific-libs/elpa.html), [ScaLAPACK](scientific-libs/scalapack.html), [HPL](apps/hpl.html), and [Quantum ESPRESSO](apps/qe.html). [FFTW `r5v`](scientific-libs/fftw.html) wins **1.06–1.60×** in isolation but **~0%** inside a real QE SCF (`FFTW_ESTIMATE`); [GROMACS](apps/gromacs.html) sees **1.23×** on isolated `PME 3D-FFT`. ONNX `accuracy_level=4` unlocks **9–10×** int4 decode — [ONNX Runtime](apps/onnx.html) / [MLAS](scientific-libs/mlas.html).
32+
Recent highlights on the Orange Pi RV2 (SpaceMiT X60, RVV): fixing an OpenBLAS `gemv_n` bug restores correctness across [BLAS](scientific-libs/blas.html), [LAPACK](scientific-libs/lapack.html), [ELPA](scientific-libs/elpa.html), [ScaLAPACK](scientific-libs/scalapack.html), [HPL](apps/hpl.html), and [Quantum ESPRESSO](apps/qe.html). [BLIS](scientific-libs/blis.html) RVV assembly beats patched OpenBLAS **~1.29×** on single-thread DGEMM (N=4096). [FFTW `r5v`](scientific-libs/fftw.html) wins **1.06–1.60×** in isolation but **~0%** inside a real QE SCF; [GROMACS](apps/gromacs.html) sees **1.23×** on isolated `PME 3D-FFT` and **3.31×** whole-app with a hand-written RVV `Force` backend. ONNX `accuracy_level=4` unlocks **9–10×** int4 decode — [ONNX Runtime](apps/onnx.html) / [MLAS](scientific-libs/mlas.html).
3333

3434
## Scientific libs
3535

3636
Library-level probes — performance *and* numerical correctness:
3737

3838
- **[BLAS](scientific-libs/blas.html)** — OpenBLAS improvements (U74 kernel, X60 `gemv_n` / TRSM fixes) and [`OpenBLAS/`](https://github.com/opensolvers/benchmarks/tree/main/OpenBLAS) verification (`bench_dgemm`, `difftest`, `verify_ctrsm`)
39+
- **[BLIS](scientific-libs/blis.html)** — FLAME BLIS `rv64iv` RVV vs patched OpenBLAS; **1.29×** DGEMM at N=4096 (1 thread)
3940
- **[NumPy](scientific-libs/numpy.html)**`bench_blas.py` DGEMM and `eigvalsh` through the SciPy stack
4041
- **[LAPACK](scientific-libs/lapack.html)** — LAPACK path via NumPy `eigvalsh`
4142
- **[ELPA](scientific-libs/elpa.html)** — dense eigensolver (CP2K / VASP class workloads)
@@ -50,12 +51,12 @@ End-to-end application benchmarks on the same boards and EESSI toolchain:
5051
- **[HPL](apps/hpl.html)** — High Performance Linpack; cross-board summary and A/B configs from [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks)
5152
- **[Quantum ESPRESSO](apps/qe.html)** — plane-wave DFT SCF (`pw.x`); whole-application BLAS backend A/B with per-routine timers
5253
- **[ONNX Runtime](apps/onnx.html)** — int4 `MatMulNBits` LLM decode; `accuracy_level=4` unlocks X60 IME (**9–10×**)
53-
- **[GROMACS](apps/gromacs.html)** — PME MD; FFT-axis A/B via `libfftw3f` `LD_PRELOAD` (**1.23×** on `PME 3D-FFT`)
54+
- **[GROMACS](apps/gromacs.html)** — PME MD; FFT-axis **1.23×** on `PME 3D-FFT`; RVV `Force` backend **3.31×** whole-app
5455

5556
## Boards
5657

5758
- **[StarFive VisionFive 2](boards/VisionFive2.html)** — JH7110 SoC, 4× SiFive U74 (`rv64gc`). U74 OpenBLAS tuning: HPL **3.13 → 5.28 GFLOP/s**.
58-
- **[Orange Pi RV2](boards/RV2.html)** — SpaceMiT K1, 8× X60 (RVV). Fixed OpenBLAS: HPL **FAILED (`nan`) → 10.53 GFLOP/s**; ELPA **34.81 s** (vs 54.92 s scalar).
59+
- **[Orange Pi RV2](boards/RV2.html)** — SpaceMiT K1, 8× X60 (RVV). Fixed OpenBLAS: HPL **FAILED (`nan`) → 10.53 GFLOP/s**; BLIS DGEMM **1.29×** vs OpenBLAS; GROMACS Force **3.31×**; ELPA **34.81 s** (vs 54.92 s scalar).
5960
- **[Banana Pi F3](boards/F3.html)** — same K1 / X60 SoC, **3.7 GB RAM**. HPL **FAILED (`nan`) → 11.52 GFLOP/s**; NumPy DGEMM up to **17.51 GFLOP/s** on patched RVV.
6061

6162
Use the menu above to jump to a board, app, or scientific lib page.

_config.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,8 @@ navigation_apps:
3737
navigation_scientific_libs:
3838
- title: BLAS
3939
url: /scientific-libs/blas.html
40+
- title: BLIS
41+
url: /scientific-libs/blis.html
4042
- title: NumPy
4143
url: /scientific-libs/numpy.html
4244
- title: LAPACK

apps/gromacs.md

Lines changed: 78 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,63 +1,121 @@
11
# GROMACS
22

3-
[GROMACS](https://www.gromacs.org/) `mdrun` PME molecular dynamics — a **whole-application FFT backend A/B**. Swaps only the single-precision FFTW library (`libfftw3f.so.3`) under one unchanged binary via `LD_PRELOAD`; GROMACS comes from the EESSI module (`GROMACS/2026.2-foss-2025b`).
3+
[GROMACS](https://www.gromacs.org/) `mdrun` PME molecular dynamics on the SpaceMiT X60 — two independent backend axes measured end-to-end:
44

5-
Benchmark source: [opensolvers/benchmarks/gromacs](https://github.com/opensolvers/benchmarks/tree/main/gromacs)`gen-water-box.sh` and `run-gmx-fft-ab.sh`.
5+
1. **FFT axis** — swap single-precision `libfftw3f.so.3` via `LD_PRELOAD` (stock EESSI build, `SIMD: None`)
6+
2. **Force axis** — hand-written **`impl_riscv_rvv/`** SIMD backend (`SIMD: RISCV_RVV`) or upstream 2026.3 autovec 1×1 kernel
67

7-
This is the **FFT-axis** companion to [Quantum ESPRESSO](qe.html)'s BLAS-axis probe. FFT microbench numbers: [FFTW](../scientific-libs/fftw.html).
8+
Benchmark source: [opensolvers/benchmarks/gromacs](https://github.com/opensolvers/benchmarks/tree/main/gromacs)`gen-water-box.sh`, `run-gmx-fft-ab.sh`, and [`rvv-backend/`](https://github.com/opensolvers/benchmarks/tree/main/gromacs/rvv-backend) (patch + `apply-and-build.sh`).
89

9-
## Why GROMACS as an FFT probe
10+
FFT microbench numbers: [FFTW](../scientific-libs/fftw.html). BLAS-axis counterpart: [Quantum ESPRESSO](qe.html).
1011

11-
PME splits long-range electrostatics into a real-space **`Force`** term and a reciprocal-space **mesh** part whose cost is a pair of 3D FFTs per step. On this build GROMACS reports **`SIMD instructions: None`** — force kernels are scalar — so swapping `libfftw3f` moves only the `PME 3D-FFT` / `PME mesh` rows in GROMACS's cycle table; `Force`, neighbor search, and constraints are built-in controls.
12+
---
1213

13-
## Correctness — RVV FFT reproduces scalar MD
14+
## FFT axis — PME 3D-FFT (`run-gmx-fft-ab.sh`)
1415

15-
Same `mdrun`, same `md.tpr`, `libfftw3f` swapped (17.2k-atom SPC water, 2000 steps):
16+
Swaps only `libfftw3f` under one unchanged EESSI `GROMACS/2026.2-foss-2025b` binary. BLAS pinned to scalar OpenBLAS; serial (`-ntmpi 1 -ntomp 1`) by design to isolate FFT.
17+
18+
On the stock build GROMACS reports **`SIMD instructions: None`** — force kernels are scalar, so `PME 3D-FFT` / `PME mesh` are the swapped rows; `Force` (90% of runtime) is the built-in control.
19+
20+
### Correctness — RVV FFT reproduces scalar MD
21+
22+
Same `mdrun`, same `md.tpr`, 17.2k-atom SPC water, 2000 steps:
1623

1724
| FFT backend | Avg potential energy | Δ vs scalar |
1825
| ----------- | -------------------: | ----------: |
1926
| Scalar (`libfftw3f`, ~944 KB) | **−236755 kJ/mol** ||
2027
| r5v / RVV (`libfftw3f`, ~14 MB, 282k RVV instr) | **−236592 kJ/mol** | **0.069%** |
2128

22-
Physically the same trajectory — expected single-precision rounding between codelet sets.
29+
### Performance — isolated FFT step (Orange Pi RV2)
2330

24-
## Performance — SPC water PME (Orange Pi RV2)
25-
26-
Serial, 48×48×48 PME grid. BLAS pinned to scalar OpenBLAS via FlexiBLAS; only `LD_PRELOAD` swaps FFT:
31+
Serial, 48×48×48 PME grid. GROMACS cycle accounting (WALL):
2732

2833
| Activity | Scalar | r5v (RVV) | Speedup | Kind |
2934
| -------- | -----: | --------: | ------: | ---- |
3035
| **`PME 3D-FFT`** | 21.095 s | 17.152 s | **1.23×** | **FFT (swapped axis)** |
3136
| `PME mesh` (total) | 73.281 s | 69.792 s | 1.05× | FFT + spread/gather/solve |
32-
| `PME solve Elec` | 9.116 s | 9.139 s | 1.00× | scalar (control) |
33-
| **`Force`** | 821.35 s | 819.97 s | 1.00× | **scalar kernels (90% of run)** |
34-
| `Neighbor search` | 6.731 s | 6.714 s | 1.00× | scalar (control) |
37+
| **`Force`** | 821.35 s | 819.97 s | 1.00× | scalar control (90%) |
38+
| `Neighbor search` | 6.731 s | 6.714 s | 1.00× | scalar control |
39+
40+
RVV wins **~1.23×** on the isolated 3D-FFT step; **`Force` confirms the A/B** (821.35 vs 819.97 s, 0.17% noise). Whole-app effect is small because `Force` dominates.
41+
42+
---
43+
44+
## Force axis — RVV SIMD backend (`rvv-backend/`)
45+
46+
The scalar `Force` term that dilutes the FFT swap is itself optimizable. [`rvv-backend/`](https://github.com/opensolvers/benchmarks/tree/main/gromacs/rvv-backend) ships a hand-written **`impl_riscv_rvv/`** backend for GROMACS **2026.2** (~1545 lines, float/mixed precision) — ARM SVE as the VLA template.
47+
48+
Built on the X60 under EESSI GCC 14.3 with `GMX_SIMD=RISCV_RVV`. `gmx --version` reports **`SIMD instructions: RISCV_RVV`**; float SIMD unit tests **158/158** pass.
49+
50+
Same serial protocol (SPC water PME, `md.tpr`, 2000 steps, `-ntmpi 1 -ntomp 1 -nb cpu -pme cpu`):
51+
52+
| Metric | Scalar (`SIMD: None`) | RVV (`SIMD: RISCV_RVV`) | Speedup |
53+
| ------ | --------------------: | ------------------------: | ------: |
54+
| **`Force` wall** | 821.35 s | **187.37 s** | **4.38×** |
55+
| Total wall | 910.33 s | **275.40 s** | **3.31×** |
56+
| Performance | 0.380 ns/day | **1.256 ns/day** | **3.31×** |
57+
| Avg potential energy | −236755 kJ/mol | −236653 kJ/mol | Δ 0.043% (correct) |
3558

36-
The **`Force` row confirms the A/B** — 821.35 vs 819.97 s (0.17%, noise). RVV wins **~1.23×** on the isolated 3D-FFT step, but `Force` dominates the whole run.
59+
**The profile inverts:** with `Force` cut ~4.4×, PME mesh becomes the leading cost (28% vs 8% of runtime) — so the FFT axis matters more after vectorizing `Force`.
3760

38-
## Where GROMACS sits on the dilution spectrum
61+
Status: captured as a patch for reproduction; **no upstream PR yet**. Double precision not ported.
62+
63+
---
64+
65+
## Force axis — three levers compared
66+
67+
All measured on Orange Pi RV2, same SPC water system, 2000 steps, 1 core (unless noted):
68+
69+
| Force lever | GROMACS | Mechanism | `Force` wall | ns/day | vs scalar |
70+
| ----------- | ------- | --------- | -----------: | -----: | --------: |
71+
| Scalar 4×4 (baseline) | 2026.2 | none (`SIMD: None`) | 821.35 s | 0.380 | 1.00× |
72+
| GCC `-march=…v` 4×4 autovec | 2026.2 | compiler on 4×4 kernel | ~836 s* | 0.372 | **0.98× (slower)** |
73+
| **Upstream 1×1 autovec** | **2026.3** | clang `#pragma`, `GMX_NBNXN_PLAINC_1X1=1` | **480.14 s** | **0.543** | **1.43×** |
74+
| **Hand-written `impl_riscv_rvv/`** | **2026.2** | `GMX_SIMD=RISCV_RVV` | **187.37 s** | **1.256** | **3.31×** |
75+
76+
<sub>*4×4 autovec Force scaled from 1000-step run; ns/day directly comparable.</sub>
77+
78+
The **2026.3 autovec 1×1** path (clang ≥ 19, `GMX_ENABLE_NBNXM_CPU_VECTORIZATION=on`) is a real win — **1.74× whole-app** vs the same binary's default 4×4 — but the hand-written backend is still **2.31× faster** whole-app (1.256 vs 0.543 ns/day). Autovec is VLEN-agnostic and upstream-native; the dedicated backend targets fixed VLEN=256 (`-mrvv-vector-bits=zvl`).
79+
80+
**Free lever:** threading — serial A/B wastes 7 of 8 cores. Same system at `-ntomp 8` reaches **~2.60 ns/day** (~6.9× vs serial baseline).
81+
82+
---
83+
84+
## Backend-dilution spectrum
3985

4086
| Probe | Axis | Isolated speedup | Whole-app effect |
4187
| ----- | ---- | ---------------: | ---------------- |
4288
| [OpenBLAS verification](../scientific-libs/blas.html#verification) | BLAS | ~2.3× | (pure kernel) |
43-
| [QE](qe.html) (DFT SCF) | BLAS | ~1.5–2.0× on BLAS routines | ~1.2–1.3× (FFT half untouched) |
89+
| [BLIS](../scientific-libs/blis.html) vs OpenBLAS | BLAS | ~1.29× (1T, N=4096) | (microbench) |
90+
| [QE](qe.html) (DFT SCF) | BLAS | ~1.5–2.0× on BLAS routines | ~1.2–1.3× (FFT untouched) |
4491
| [QE + FFTW](../scientific-libs/fftw.html) | FFT | ~1.02× on `fftw` | **~0%** (`FFTW_ESTIMATE`) |
45-
| **GROMACS** (this page) | **FFT** | **~1.23× on `PME 3D-FFT`** | small (`Force` = 90%) |
92+
| **GROMACS FFT** (this page) | FFT | **~1.23× on `PME 3D-FFT`** | small (`Force` = 90%) |
93+
| **GROMACS Force** (`rvv-backend/`) | **SIMD** | **~4.38× on `Force`** | **~3.31×** |
4694

47-
GROMACS is the mirror of QE: QE speeds up BLAS and is diluted by FFT; GROMACS speeds up FFT and is diluted by scalar force kernels.
95+
GROMACS FFT is the mirror of QE: QE speeds up BLAS and is diluted by FFT; the FFT swap is diluted by scalar `Force`. Vectorizing `Force` moves the whole app — then PME/FFT becomes the next target.
96+
97+
---
4898

4999
## Gotchas
50100

51-
- GROMACS mixed precision links **`libfftw3f.so.3`** (single precision), not the double-precision `libfftw3.so.3` from the QE/FFTW work.
101+
- GROMACS mixed precision links **`libfftw3f.so.3`** (single precision), not double-precision `libfftw3.so.3` from QE/FFTW work.
52102
- Fixed `-nsteps` keeps FFT call count identical between backends (4002 here).
53103
- RVV `libfftw3f` build is slow to compile (~2 h on one X60); scalar ~15 min.
104+
- GCC `-march=rv64gcv_zvl256b` on stock 4×4 `Force` emits 164k RVV instrs but runs **~2% slower** — gather-heavy kernel defeats autovec.
105+
- 2026.3 autovec 1×1 needs **clang ≥ 19** and runtime **`GMX_NBNXN_PLAINC_1X1=1`**; confirm via `Using plain-C-1x1` in `md.log`.
106+
- RVV-backend builds need GCC 14.3's `libstdc++` (`CXXABI_1.3.15`) on `LD_LIBRARY_PATH` at runtime.
54107

55108
## Reproduce
56109

57110
```bash
111+
# FFT axis
58112
module load GROMACS/2026.2-foss-2025b
59113
./gen-water-box.sh
60114
bash run-gmx-fft-ab.sh v1
115+
116+
# Force axis (hand-written RVV backend)
117+
cd gromacs/rvv-backend
118+
./apply-and-build.sh /path/to/gromacs-2026.2
61119
```
62120

63121
**Toolchain:** GROMACS 2026.2 / foss-2025b (EESSI), FFTW 3.3.10 `libfftw3f`, Orange Pi RV2 (SpaceMiT X60).

boards/RV2.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,9 +52,22 @@ The fix backports the upstream `gemv_n` correction from OpenBLAS ≥ 0.3.31 ([Op
5252

5353
Full walkthrough: [EESSI/docs#819](https://github.com/EESSI/docs/pull/819)*Chasing a NaN: correct RVV HPL on a RISC-V SpaceMiT X60 via EESSI*.
5454

55+
## BLIS vs OpenBLAS
56+
57+
See [BLIS](../scientific-libs/blis.html) — FLAME BLIS `rv64iv` with hand-written RVV assembly vs patched OpenBLAS on the same `bench_dgemm.c`. Single-thread DGEMM at N=4096: **2.95 vs 2.28 GFLOP/s** (**1.29×** BLIS); 8-thread OpenBLAS still leads at large N. TRSM: **2400 cases, 0 fails** on BLIS.
58+
5559
## FFTW RVV
5660

57-
See [FFTW](../scientific-libs/fftw.html) — r5v wins **1.06–1.60×** in `tests/bench`, but **~0%** end-to-end in [Quantum ESPRESSO](../apps/qe.html) (`FFTW_ESTIMATE`). [GROMACS](../apps/gromacs.html) FFT swap: **1.23×** on isolated `PME 3D-FFT`.
61+
See [FFTW](../scientific-libs/fftw.html) — r5v wins **1.06–1.60×** in `tests/bench`, but **~0%** end-to-end in [Quantum ESPRESSO](../apps/qe.html) (`FFTW_ESTIMATE`).
62+
63+
## GROMACS
64+
65+
See [GROMACS](../apps/gromacs.html) — two axes on this board:
66+
67+
| Axis | Result |
68+
| ---- | ------ |
69+
| **FFT** (`libfftw3f` swap) | **1.23×** on isolated `PME 3D-FFT`; ~0% whole-app (`Force` = 90%) |
70+
| **Force** (`rvv-backend/`, `SIMD: RISCV_RVV`) | **4.38×** on `Force`, **3.31×** whole-app (0.380 → **1.256 ns/day**) |
5871

5972
## ScaLAPACK
6073

scientific-libs/blas.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Improvements to **OpenBLAS 0.3.30** on RISC-V boards — built via EasyBuild, deployed through [EESSI](https://www.eessi.io/) with **FlexiBLAS** runtime swapping.
44

5-
Verification microbenchmarks live in [opensolvers/benchmarks/OpenBLAS](https://github.com/opensolvers/benchmarks/tree/main/OpenBLAS) (`bench_dgemm`, `difftest`, `verify_ctrsm`). Stack probes: [NumPy](numpy.html) (`bench_blas.py`). Full repo: [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks).
5+
Verification microbenchmarks live in [opensolvers/benchmarks/OpenBLAS](https://github.com/opensolvers/benchmarks/tree/main/OpenBLAS) (`bench_dgemm`, `difftest`, `verify_ctrsm`). **Alternative BLAS:** [BLIS](blis.html) RVV vs OpenBLAS DGEMM A/B on X60. Stack probes: [NumPy](numpy.html) (`bench_blas.py`). Full repo: [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks).
66

77
**Base stack:** GCC 14.3.0, OpenBLAS 0.3.30, EESSI `2025.06-001` ([`dev.eessi.io/riscv`](https://www.eessi.io/docs/repositories/dev.eessi.io-riscv/)).
88

scientific-libs/blis.md

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# BLIS
2+
3+
[BLIS](https://github.com/flame/blis) (BLAS-like Library Instantiation Software, FLAME group) on RISC-V — a **vector-vs-vector** DGEMM comparison against patched RVV **OpenBLAS** on the SpaceMiT X60.
4+
5+
BLIS ships hand-written RVV assembly level-3 microkernels under [`kernels/rviv/3/`](https://github.com/flame/blis/tree/master/kernels/rviv/3) (dynamic VLEN via `get_vlenb()`), selected by the **`rv64iv`** config target. This is not scalar-vs-vector — both libraries use RVV.
6+
7+
Benchmark source: [opensolvers/benchmarks/BLIS](https://github.com/opensolvers/benchmarks/tree/main/BLIS)`build-blis.sh`, `run-ab.sh`, shared `bench_dgemm.c`, `verify_ctrsm.c`.
8+
9+
See also [BLAS (OpenBLAS)](blas.html) for the OpenBLAS fixes and verification suite that define the baseline here.
10+
11+
## Why link, not FlexiBLAS
12+
13+
Other BLAS-axis benchmarks ([HPL](../apps/hpl.html), [NumPy](numpy.html), [QE](../apps/qe.html)) swap backends at runtime via FlexiBLAS. **FlexiBLAS is not installed on the RV2**, so this A/B links the same unchanged `bench_dgemm.c` against each library in turn — still one variable (the BLAS implementation), identical `-O3 -march=rv64imafdcv_zvl256b`.
14+
15+
## Build & run
16+
17+
```bash
18+
./build-blis.sh # ~10–20 min native; installs to $HOME/blis-install
19+
BLIS_PREFIX=$HOME/blis-install \
20+
OPENBLAS_LIB=$HOME/trsm-pr5830/libopenblas.a \
21+
./run-ab.sh
22+
```
23+
24+
Configure with **`rv64iv`** (RVV 1.0, dynamic VLEN=256) and **`--enable-threading=openmp`** — without OpenMP, DGEMM stays single-threaded (~2.7 GFLOP/s regardless of thread count). Do **not** use `sifive_rvv` (defaults to VLEN=128).
25+
26+
**Toolchain:** EESSI GCC **14.3.0** forced ahead of compat GCC 13.4.0; link with `-L$GCC14/lib -B$GCC14/lib` (see benchmarks README for the `libgomp.spec` trap).
27+
28+
## Correctness
29+
30+
| Check | Result |
31+
| ----- | ------ |
32+
| `verify_ctrsm` (BLIS `rv64iv`) | **2400 cases, 0 fails**, worst residual **2.55×10⁻⁷** (1- and 8-thread) |
33+
| `C[0]` in DGEMM sweep | **245.24** identical for BLIS and OpenBLAS at every size — no NaN |
34+
35+
BLIS TRSM passes cleanly on X60 — unlike the stock OpenBLAS `_rvv_v1` TRSM VLEN bug caught by [`verify_ctrsm` in OpenBLAS/](https://github.com/opensolvers/benchmarks/tree/main/OpenBLAS).
36+
37+
## Performance — Orange Pi RV2 (X60, VLEN=256)
38+
39+
BLIS `061c2eb` (`rv64iv`, OpenMP) vs patched RVV OpenBLAS `0.3.33.dev` (`zvl128bp`), EESSI GCC 14.3.0. Square DGEMM, 3 reps, best GFLOP/s:
40+
41+
| Threads | N | BLIS | OpenBLAS | BLIS / OpenBLAS |
42+
| ------: | --: | ---: | -------: | --------------: |
43+
| 1 | 1024 | 1.99 | 2.13 | 0.93× |
44+
| 1 | 2048 | 2.73 | 2.25 | **1.21×** |
45+
| 1 | 4096 | 2.95 | 2.28 | **1.29×** |
46+
| 8 | 1024 | 8.93 | 10.13 | 0.88× |
47+
| 8 | 2048 | 9.60 | 10.83 | 0.89× |
48+
| 8 | 4096 | 9.55 | 11.94 | 0.80× |
49+
50+
### Takeaways
51+
52+
- **Single thread, large N:** BLIS's RVV assembly microkernel **beats OpenBLAS by ~20–30%** once packing is amortized (N ≥ 2048). At N=1024 the two are within noise.
53+
- **8 threads:** OpenBLAS scales slightly better (BLIS **0.80–0.89×**). Both get ~3.5–5× from 8 cores; BLIS's OpenMP path leaves headroom vs OpenBLAS threading.
54+
- **Correctness first:** both backends numerically identical on DGEMM; BLIS TRSM verified independently.
55+
56+
## References
57+
58+
- BLIS RISC-V config: [`config_registry`](https://github.com/flame/blis/blob/master/config_registry) (`rv64iv`, `sifive_rvv`)
59+
- RVV assembly kernels: [`kernels/rviv/3/`](https://github.com/flame/blis/tree/master/kernels/rviv/3)
60+
- Prior RISC-V BLIS work: PR [#737](https://github.com/flame/blis/pull/737) (X280), [#832](https://github.com/flame/blis/pull/832), [#868](https://github.com/flame/blis/pull/868) (SG2042)

0 commit comments

Comments
 (0)