Skip to content

Commit b8c5747

Browse files
authored
Merge pull request #37 from opensolvers/site/benchmarks-openblas-sync
Sync site with benchmarks PR #18 (OpenBLAS/)
2 parents a1900b8 + 5ea8160 commit b8c5747

10 files changed

Lines changed: 71 additions & 73 deletions

File tree

README.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,7 @@ Recent highlights on the Orange Pi RV2 (SpaceMiT X60, RVV): fixing an OpenBLAS `
3535

3636
Library-level probes — performance *and* numerical correctness:
3737

38-
- **[BLAS](scientific-libs/blas.html)** — OpenBLAS improvements (U74 kernel, X60 `gemv_n` fix)
39-
- **[DGEMM](scientific-libs/dgemm.html)**`bench_dgemm` + `difftest` performance and correctness probes
38+
- **[BLAS](scientific-libs/blas.html)** — OpenBLAS improvements (U74 kernel, X60 `gemv_n` / TRSM fixes) and [`OpenBLAS/`](https://github.com/opensolvers/benchmarks/tree/main/OpenBLAS) verification (`bench_dgemm`, `difftest`, `verify_ctrsm`)
4039
- **[NumPy](scientific-libs/numpy.html)**`bench_blas.py` DGEMM and `eigvalsh` through the SciPy stack
4140
- **[LAPACK](scientific-libs/lapack.html)** — LAPACK path via NumPy `eigvalsh`
4241
- **[ELPA](scientific-libs/elpa.html)** — dense eigensolver (CP2K / VASP class workloads)

_config.yml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,6 @@ navigation_apps:
3737
navigation_scientific_libs:
3838
- title: BLAS
3939
url: /scientific-libs/blas.html
40-
- title: DGEMM
41-
url: /scientific-libs/dgemm.html
4240
- title: NumPy
4341
url: /scientific-libs/numpy.html
4442
- title: LAPACK

apps/gromacs.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ The **`Force` row confirms the A/B** — 821.35 vs 819.97 s (0.17%, noise). RVV
3939

4040
| Probe | Axis | Isolated speedup | Whole-app effect |
4141
| ----- | ---- | ---------------: | ---------------- |
42-
| [DGEMM](../scientific-libs/dgemm.html) | BLAS | ~2.3× | (pure kernel) |
42+
| [OpenBLAS verification](../scientific-libs/blas.html#verification) | BLAS | ~2.3× | (pure kernel) |
4343
| [QE](qe.html) (DFT SCF) | BLAS | ~1.5–2.0× on BLAS routines | ~1.2–1.3× (FFT half untouched) |
4444
| [QE + FFTW](../scientific-libs/fftw.html) | FFT | ~1.02× on `fftw` | **~0%** (`FFTW_ESTIMATE`) |
4545
| **GROMACS** (this page) | **FFT** | **~1.23× on `PME 3D-FFT`** | small (`Force` = 90%) |

apps/qe.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Benchmark source: [opensolvers/benchmarks/qe](https://github.com/opensolvers/ben
66

77
## Why QE as a probe
88

9-
[`dgemm`](../scientific-libs/dgemm.html) isolates BLAS-3; [HPL](hpl.html) and [ELPA](../scientific-libs/elpa.html) probe single HPC solvers; [FFTW](../scientific-libs/fftw.html) covers the FFT half. A full DFT SCF mixes level-3 GEMM (`calbec`, subspace rotation), dense LAPACK diagonalization, latency-bound BLAS-2, MPI, **and a large FFT fraction** — so it shows both whether a buggy vector BLAS breaks a production code, and what fraction of a real run each backend swap actually moves.
9+
[OpenBLAS verification](../scientific-libs/blas.html#verification) isolates BLAS-3; [HPL](hpl.html) and [ELPA](../scientific-libs/elpa.html) probe single HPC solvers; [FFTW](../scientific-libs/fftw.html) covers the FFT half. A full DFT SCF mixes level-3 GEMM (`calbec`, subspace rotation), dense LAPACK diagonalization, latency-bound BLAS-2, MPI, **and a large FFT fraction** — so it shows both whether a buggy vector BLAS breaks a production code, and what fraction of a real run each backend swap actually moves.
1010

1111
## Correctness — stock RVV `gemv_n` breaks a real DFT SCF
1212

@@ -56,7 +56,7 @@ Same X60, patched RVV vs scalar:
5656

5757
| Probe | Speedup | Why |
5858
| ----- | ------: | --- |
59-
| [DGEMM](../scientific-libs/dgemm.html) (pure level-3) | ~2.3× | all BLAS-3 |
59+
| [OpenBLAS verification](../scientific-libs/blas.html#verification) (pure level-3) | ~2.3× | all BLAS-3 |
6060
| [HPL](hpl.html) (Linpack) | ~1.8× | BLAS-3 + `dgemv` panel factorization |
6161
| [ELPA](../scientific-libs/elpa.html) (eigensolver) | ~1.58× | BLAS-3 + BLAS-2 tridiagonalization |
6262
| **QE** (full DFT SCF) | **~1.2–1.3×** | BLAS + ~40–50% FFT + MPI |
@@ -76,4 +76,4 @@ RVV_LIB=/path/to/patched/libopenblas.so ./run-qe-ab.sh si-scf.in 4
7676
RVV_LIB=/path/to/patched/libopenblas.so ./run-perf-ab.sh si-super-64.in 4
7777
```
7878

79-
**Toolchain:** QuantumESPRESSO 7.5 / foss-2025b (EESSI), FlexiBLAS 3.4.5, OpenBLAS 0.3.30. Patched vector backend = OpenBLAS 0.3.30 with the RISC-V `gemv_n` NaN fix backported — same build used by [DGEMM](../scientific-libs/dgemm.html), [HPL](hpl.html), and [ELPA](../scientific-libs/elpa.html).
79+
**Toolchain:** QuantumESPRESSO 7.5 / foss-2025b (EESSI), FlexiBLAS 3.4.5, OpenBLAS 0.3.30. Patched vector backend = OpenBLAS 0.3.30 with the RISC-V `gemv_n` NaN fix backported — same build used by [OpenBLAS verification](../scientific-libs/blas.html#verification), [HPL](hpl.html), and [ELPA](../scientific-libs/elpa.html).

boards/F3.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,8 @@ See also the [HPL app overview](../apps/hpl.html).
3434

3535
| Probe | Scalar | Patched RVV | Speedup |
3636
| ----- | ------ | ----------- | ------- |
37-
| [DGEMM](../scientific-libs/dgemm.html) N=2048, 1 core | 1.26 GFLOP/s | 2.96 GFLOP/s | 2.35× |
38-
| [DGEMM](../scientific-libs/dgemm.html) N=4096, 8 threads || 17.71 GFLOP/s ||
37+
| [DGEMM](../scientific-libs/blas.html#verification) N=2048, 1 core | 1.26 GFLOP/s | 2.96 GFLOP/s | 2.35× |
38+
| [DGEMM](../scientific-libs/blas.html#verification) N=4096, 8 threads || 17.71 GFLOP/s ||
3939
| [NumPy](../scientific-libs/numpy.html) DGEMM N=4096 | 4.91 GFLOP/s | 17.51 GFLOP/s | 3.6× |
4040
| [NumPy](../scientific-libs/numpy.html) `eigvalsh` N=2048 | 9.59 s | 5.94 s | 1.6× |
4141
| ELPA `na=3000` | 50.42 s | **34.83 s** | 1.45× |

scientific-libs/blas.md

Lines changed: 57 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Improvements to **OpenBLAS 0.3.30** on RISC-V boards — built via EasyBuild, deployed through [EESSI](https://www.eessi.io/) with **FlexiBLAS** runtime swapping.
44

5-
Verification and A/B probes: [DGEMM](dgemm.html) (`bench_dgemm`, `difftest`), [NumPy](numpy.html) (`bench_blas.py`). Source repo: [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks).
5+
Verification microbenchmarks live in [opensolvers/benchmarks/OpenBLAS](https://github.com/opensolvers/benchmarks/tree/main/OpenBLAS) (`bench_dgemm`, `difftest`, `verify_ctrsm`). Stack probes: [NumPy](numpy.html) (`bench_blas.py`). Full repo: [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks).
66

77
**Base stack:** GCC 14.3.0, OpenBLAS 0.3.30, EESSI `2025.06-001` ([`dev.eessi.io/riscv`](https://www.eessi.io/docs/repositories/dev.eessi.io-riscv/)).
88

@@ -11,8 +11,8 @@ Verification and A/B probes: [DGEMM](dgemm.html) (`bench_dgemm`, `difftest`), [N
1111
| Board / CPU | Problem (stock 0.3.30) | Fix | Result |
1212
| ----------- | ---------------------- | --- | ------ |
1313
| [VisionFive 2](../boards/VisionFive2.html) — SiFive **U74** (scalar) | No U74 kernel; falls back to generic `RISCV64_GENERIC` C `2×2` GEMM | New **4×4 DGEMM micro-kernel** in RV64 assembly, `TARGET=U74` | Single-core DGEMM **~1.4 → 1.77 GFLOP/s**; 4-core DGEMM **6.31 GFLOP/s**; [HPL **1.69×**](../apps/hpl.html) |
14-
| [Orange Pi RV2](../boards/RV2.html) — SpacemiT **X60** (RVV VLEN=256) | RVV `gemv_n` zeroes an **uninitialized** vector register → `dgemv` returns NaN | Backport upstream `gemv_n` fix; `TARGET=RISCV64_ZVL256B` | [DGEMM](dgemm.html) 2.3×; [NumPy](numpy.html) DGEMM 2.4×; [HPL](../apps/hpl.html) **10.53 GFLOP/s** |
15-
| [Banana Pi F3](../boards/F3.html) — same K1 / X60 SoC | Same RVV `gemv_n` bug as RV2 | Same [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) fix | [DGEMM](dgemm.html) 2.35×; [NumPy](numpy.html) DGEMM **3.6×**; HPL **11.52 GFLOP/s** |
14+
| [Orange Pi RV2](../boards/RV2.html) — SpacemiT **X60** (RVV VLEN=256) | RVV `gemv_n` zeroes an **uninitialized** vector register → `dgemv` returns NaN | Backport upstream `gemv_n` fix; `TARGET=RISCV64_ZVL256B` | Verification 2.3×; [NumPy](numpy.html) DGEMM 2.4×; [HPL](../apps/hpl.html) **10.53 GFLOP/s** |
15+
| [Banana Pi F3](../boards/F3.html) — same K1 / X60 SoC | Same RVV `gemv_n` bug as RV2 | Same [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) fix | Verification 2.35×; [NumPy](numpy.html) DGEMM **3.6×**; HPL **11.52 GFLOP/s** |
1616

1717
## Packages
1818

@@ -23,6 +23,60 @@ Verification and A/B probes: [DGEMM](dgemm.html) (`bench_dgemm`, `difftest`), [N
2323

2424
Build with `eb --from-pr <num> --robot` into an **EESSI-extend** user install, then `flexiblas add` / `flexiblas default` — no downstream rebuild.
2525

26+
## Verification {#verification}
27+
28+
Small, self-contained programs in [opensolvers/benchmarks/OpenBLAS](https://github.com/opensolvers/benchmarks/tree/main/OpenBLAS) for **BLAS performance and per-routine correctness** — used to isolate broken RVV kernels before trusting downstream apps.
29+
30+
| File | Purpose |
31+
| ---- | ------- |
32+
| [`bench_dgemm.c`](https://github.com/opensolvers/benchmarks/blob/main/OpenBLAS/bench_dgemm.c) | Times square `C = A×B` (3 reps), reports GFLOP/s, prints `C[0]` |
33+
| [`difftest.c`](https://github.com/opensolvers/benchmarks/blob/main/OpenBLAS/difftest.c) | `dlopen`s a BLAS `.so`, runs level-1/2/3 routines, reports `sum` / NaN counts |
34+
| [`verify_ctrsm.c`](https://github.com/opensolvers/benchmarks/blob/main/OpenBLAS/verify_ctrsm.c) | Full-parameter TRSM correctness sweep (localizes [OpenBLAS#5928](https://github.com/OpenMathLib/OpenBLAS/pull/5928) VLEN bug in `_rvv_v1` kernels) |
35+
36+
Both switch backends at runtime via FlexiBLAS or `OPENBLAS_CORETYPE` — no recompile.
37+
38+
```bash
39+
gcc -O2 bench_dgemm.c -o bench_dgemm -lflexiblas
40+
gcc -O2 difftest.c -o difftest -ldl -lm
41+
42+
OPENBLAS_NUM_THREADS=8 ./bench_dgemm 4096
43+
./difftest /path/to/libopenblas.so
44+
```
45+
46+
### Correctness — `difftest` (X60, RV2 & F3)
47+
48+
Bit-identical on [Orange Pi RV2](../boards/RV2.html) and [Banana Pi F3](../boards/F3.html):
49+
50+
| Backend | `dgemv` NaN | `dgemm` NaN | `dtrsm` NaN | `dgemv` sum |
51+
| ------- | ----------- | ----------- | ----------- | ----------- |
52+
| Stock EESSI, default RVV | **192** | 0 | 0 | 198.94 (wrong) |
53+
| Forced scalar | 0 | 0 | 0 | 42.06549 (reference) |
54+
| Patched RVV (`gemv_n` fix) | 0 | 0 | 0 | 42.06549 (matches) |
55+
56+
Fault is in **`dgemv` only** — plain `dgemm` and `dtrsm` look fine on the broken `gemv_n` build, which is why [HPL](../apps/hpl.html) and [Quantum ESPRESSO](../apps/qe.html) can fail while a GEMM micro-benchmark passes.
57+
58+
A second bug — RVV `_rvv_v1` TRSM kernels not VLEN-agnostic ([OpenBLAS#5928](https://github.com/OpenMathLib/OpenBLAS/pull/5928)) — is caught by `verify_ctrsm` on `ZVL128B` builds where `GEMM_UNROLL_M ≠ VSETVL_MAX`.
59+
60+
### Performance — `bench_dgemm`
61+
62+
#### Orange Pi RV2 (1 core, N=2048)
63+
64+
| Backend | GFLOP/s | `C[0]` |
65+
| ------- | ------- | ------ |
66+
| Scalar | 1.16 | 245.24 |
67+
| Patched RVV | 2.62 | 245.24 |
68+
69+
**2.3×** faster, numerically identical.
70+
71+
#### Banana Pi F3 (cross-board)
72+
73+
| Backend | GFLOP/s | `C[0]` |
74+
| ------- | ------- | ------ |
75+
| Scalar | 1.26 | 245.24 |
76+
| Patched RVV | 2.96 | 245.24 |
77+
78+
**2.35×** at 1 core. Threaded (8 cores, N=4096): **17.71 GFLOP/s** on patched RVV.
79+
2680
## Notes
2781

2882
- **U74** — performance kernel; stock OpenBLAS works but leaves FP throughput on the table.

scientific-libs/dgemm.md

Lines changed: 0 additions & 54 deletions
This file was deleted.

scientific-libs/lapack.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
LAPACK performance and correctness probed via NumPy **`eigvalsh`** (`dsyevd`) — the LAPACK-heavy half of the [NumPy benchmark](numpy.html).
44

5-
Benchmark source: [opensolvers/benchmarks/numpy](https://github.com/opensolvers/benchmarks/tree/main/numpy) (`bench_blas.py`). For DGEMM (`A @ B`) see the [NumPy](numpy.html) or [DGEMM](dgemm.html) pages.
5+
Benchmark source: [opensolvers/benchmarks/numpy](https://github.com/opensolvers/benchmarks/tree/main/numpy) (`bench_blas.py`). For DGEMM (`A @ B`) see the [NumPy](numpy.html) or [BLAS verification](blas.html#verification) sections.
66

77
## Orange Pi RV2 (8 threads)
88

@@ -12,7 +12,7 @@ Patched RVV OpenBLAS via FlexiBLAS (`SciPy-bundle` module, [easyconfigs#26444](h
1212
| ------ | ------ | ----------- | ------- |
1313
| EIGH (`eigvalsh`, N=2048) | 10.54 s | 6.72 s | **1.6×** |
1414

15-
Stock EESSI default RVV: `eigvalsh` returns NaN / fails to converge. See [DGEMM `difftest`](dgemm.html).
15+
Stock EESSI default RVV: `eigvalsh` returns NaN / fails to converge. See [`difftest`](blas.html#verification).
1616

1717
## Banana Pi F3 (cross-board)
1818

scientific-libs/numpy.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ A **SciPy-stack probe** for BLAS and LAPACK through NumPy — from [opensolvers/
77
| DGEMM | `A @ B` | `dgemm` | GFLOP/s |
88
| EIGH | `np.linalg.eigvalsh(S)` | `dsyevd` | seconds |
99

10-
Unlike raw [DGEMM](dgemm.html), `eigvalsh` exercises BLAS-2 tridiagonalization — a correctness gate for the same `gemv_n` bug that breaks [HPL](../apps/hpl.html) and [ELPA](elpa.html).
10+
Unlike raw [OpenBLAS verification](blas.html#verification), `eigvalsh` exercises BLAS-2 tridiagonalization — a correctness gate for the same `gemv_n` bug that breaks [HPL](../apps/hpl.html) and [ELPA](elpa.html).
1111

1212
```bash
1313
module load SciPy-bundle/2025.07-gfbf-2025b
@@ -33,6 +33,6 @@ Both patched results finite. See [LAPACK](lapack.html) for the LAPACK angle.
3333
| DGEMM N=4096 | 4.91 GFLOP/s | **17.51 GFLOP/s** | **3.6×** |
3434
| EIGH N=2048 | 9.59 s | 5.94 s | **1.6×** |
3535

36-
**Stock unpatched RVV:** `eigvalsh` raises `LinAlgError: Eigenvalues did not converge`; DGEMM alone stays finite at 10.94 GFLOP/s (only `gemv` is broken — see [DGEMM difftest](dgemm.html)).
36+
**Stock unpatched RVV:** `eigvalsh` raises `LinAlgError: Eigenvalues did not converge`; DGEMM alone stays finite at 10.94 GFLOP/s (only `gemv` is broken — see [`difftest`](blas.html#verification)).
3737

3838
Eigensolver speedup (~1.6×) is lower than DGEMM (~2.4–3.6×) because `eigvalsh` mixes BLAS-3 with latency-bound BLAS-2 — same pattern as [ELPA](elpa.html).

website.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Context file for structural and content decisions on [opensolvers.com](https://w
2020

2121
1. **Home** · **Videos** · **YouTube** (external)
2222
2. **Apps** — HPL, Quantum ESPRESSO, ONNX Runtime, GROMACS
23-
3. **Scientific libs** — BLAS, DGEMM, NumPy, LAPACK, ELPA, MLAS, FFTW, ScaLAPACK
23+
3. **Scientific libs** — BLAS (incl. OpenBLAS verification), NumPy, LAPACK, ELPA, MLAS, FFTW, ScaLAPACK
2424
4. **Boards** — VisionFive 2, OrangePi RV2, BananaPi F3
2525

2626
Nav config: `_config.yml` (`navigation`, `navigation_boards`, `navigation_apps`, `navigation_scientific_libs`). Rendered in `_includes/header.html`. Cayman theme requires `_layouts/default.html` override to include the header.
@@ -44,7 +44,7 @@ Catalog: `_data/videos.yml` (newest first). Rendered by `_includes/video-grid.ht
4444

4545
| Source | Used for |
4646
|--------|----------|
47-
| [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks) | BLAS (`dgemm/`), HPL (`hpl/`), ELPA/ScaLAPACK (`elpa/`, `scalapack/`), QE/FFTW (`qe/`, `fftw/`), GROMACS (`gromacs/`), ONNX/MLAS (`onnx/`), NumPy (`numpy/`) |
47+
| [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks) | OpenBLAS (`OpenBLAS/`), HPL (`hpl/`), ELPA/ScaLAPACK (`elpa/`, `scalapack/`), QE/FFTW (`qe/`, `fftw/`), GROMACS (`gromacs/`), ONNX/MLAS (`onnx/`), NumPy (`numpy/`) |
4848
| [EESSI/docs#818](https://github.com/EESSI/docs/pull/818) | VisionFive 2 / U74 OpenBLAS + HPL |
4949
| [EESSI/docs#819](https://github.com/EESSI/docs/pull/819) | Orange Pi RV2 / X60 RVV `gemv_n` fix + HPL |
5050
| [easyconfigs#26436](https://github.com/easybuilders/easybuild-easyconfigs/pull/26436) | U74 OpenBLAS package |
@@ -60,6 +60,7 @@ Catalog: `_data/videos.yml` (newest first). Rendered by `_includes/video-grid.ht
6060

6161
| Date | Decision |
6262
|------|----------|
63+
| 2026-07-19 | Sync benchmarks PR #18: merge OpenBLAS verification into BLAS page; remove `dgemm.html`; fix GitHub links |
6364
| 2026-07-18 | Videos page + `_data/videos.yml`; U74 YouTube link; SEO (`url`, JSON-LD, robots.txt, page descriptions) |
6465
| 2026-07-18 | Per-board compute-backend SVGs; fix invalid UTF-8 in VisionFive 2 / K1 diagrams |
6566
| 2026-07-18 | U74 video production moved to private repo `opensolvers/u74-video` (removed from site) |

0 commit comments

Comments
 (0)