Skip to content

Commit aa398c8

Browse files
authored
Merge pull request #20 from opensolvers/site-updates
Add DGEMM and NumPy lib pages
2 parents 5e6a2a4 + dad1741 commit aa398c8

8 files changed

Lines changed: 123 additions & 50 deletions

File tree

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,11 @@ Recent highlights on the Orange Pi RV2 (SpaceMiT X60, RVV): fixing an OpenBLAS `
1010

1111
Library-level probes — performance *and* numerical correctness:
1212

13-
- **[BLAS](scientific-libs/blas.html)** — OpenBLAS improvements (U74 tuned kernel, X60 RVV `gemv_n` fix) and `difftest` / `bench_dgemm` verification
14-
- **[LAPACK](scientific-libs/lapack.html)** — NumPy `dgemm` and `eigvalsh` as a real SciPy-stack probe
15-
- **[ELPA](scientific-libs/elpa.html)** — dense eigensolver mixing BLAS-2 and BLAS-3 (CP2K / VASP class workloads)
13+
- **[BLAS](scientific-libs/blas.html)** — OpenBLAS improvements (U74 kernel, X60 `gemv_n` fix)
14+
- **[DGEMM](scientific-libs/dgemm.html)**`bench_dgemm` + `difftest` performance and correctness probes
15+
- **[NumPy](scientific-libs/numpy.html)**`bench_blas.py` DGEMM and `eigvalsh` through the SciPy stack
16+
- **[LAPACK](scientific-libs/lapack.html)** — LAPACK path via NumPy `eigvalsh`
17+
- **[ELPA](scientific-libs/elpa.html)** — dense eigensolver (CP2K / VASP class workloads)
1618

1719
## Apps
1820

_config.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,10 @@ navigation_apps:
2121
navigation_scientific_libs:
2222
- title: BLAS
2323
url: /scientific-libs/blas.html
24+
- title: DGEMM
25+
url: /scientific-libs/dgemm.html
26+
- title: NumPy
27+
url: /scientific-libs/numpy.html
2428
- title: LAPACK
2529
url: /scientific-libs/lapack.html
2630
- title: ELPA

boards/F3.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,10 @@ See also the [HPL app overview](../apps/hpl.html).
2222

2323
| Probe | Scalar | Patched RVV | Speedup |
2424
| ----- | ------ | ----------- | ------- |
25-
| `bench_dgemm` N=2048, 1 core | 1.26 GFLOP/s | 2.96 GFLOP/s | 2.35× |
26-
| `bench_dgemm` N=4096, 8 threads || 17.71 GFLOP/s ||
27-
| NumPy DGEMM N=4096 | 4.91 GFLOP/s | **17.51 GFLOP/s** | **3.6×** |
28-
| NumPy `eigvalsh` N=2048 | 9.59 s | 5.94 s | 1.6× |
25+
| [DGEMM](../scientific-libs/dgemm.html) N=2048, 1 core | 1.26 GFLOP/s | 2.96 GFLOP/s | 2.35× |
26+
| [DGEMM](../scientific-libs/dgemm.html) N=4096, 8 threads || 17.71 GFLOP/s ||
27+
| [NumPy](../scientific-libs/numpy.html) DGEMM N=4096 | 4.91 GFLOP/s | 17.51 GFLOP/s | 3.6× |
28+
| [NumPy](../scientific-libs/numpy.html) `eigvalsh` N=2048 | 9.59 s | 5.94 s | 1.6× |
2929
| ELPA `na=3000` | 50.42 s | **34.83 s** | 1.45× |
3030

3131
Stock unpatched RVV: HPL and ELPA fail (`nan`); NumPy `eigvalsh` raises `LinAlgError: Eigenvalues did not converge`.

scientific-libs/blas.md

Lines changed: 5 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# BLAS (OpenBLAS)
22

3-
Improvements to **OpenBLAS 0.3.30** on RISC-V boards — built via EasyBuild, deployed through [EESSI](https://www.eessi.io/) with **FlexiBLAS** runtime swapping. Verification tools and A/B scripts live in the [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks) repo (`dgemm/`, `numpy/`).
3+
Improvements to **OpenBLAS 0.3.30** on RISC-V boards — built via EasyBuild, deployed through [EESSI](https://www.eessi.io/) with **FlexiBLAS** runtime swapping.
4+
5+
Verification and A/B probes: [DGEMM](dgemm.html) (`bench_dgemm`, `difftest`), [NumPy](numpy.html) (`bench_blas.py`). Source repo: [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks).
46

57
**Base stack:** GCC 14.3.0, OpenBLAS 0.3.30, EESSI `2025.06-001` ([`dev.eessi.io/riscv`](https://www.eessi.io/docs/repositories/dev.eessi.io-riscv/)).
68

@@ -9,34 +11,8 @@ Improvements to **OpenBLAS 0.3.30** on RISC-V boards — built via EasyBuild, de
911
| Board / CPU | Problem (stock 0.3.30) | Fix | Result |
1012
| ----------- | ---------------------- | --- | ------ |
1113
| [VisionFive 2](../boards/VisionFive2.html) — SiFive **U74** (scalar) | No U74 kernel; falls back to generic `RISCV64_GENERIC` C `2×2` GEMM | New **4×4 DGEMM micro-kernel** in RV64 assembly, `TARGET=U74` | Single-core DGEMM **~1.4 → 1.77 GFLOP/s**; 4-core DGEMM **6.31 GFLOP/s**; [HPL **1.69×**](../apps/hpl.html) |
12-
| [Orange Pi RV2](../boards/RV2.html) — SpacemiT **X60** (RVV VLEN=256) | RVV `gemv_n` zeroes an **uninitialized** vector register → `dgemv` returns NaN | Backport upstream `gemv_n` fix; `TARGET=RISCV64_ZVL256B` | Correct `dgemv`; see [HPL](../apps/hpl.html) and A/B tables below |
13-
| [Banana Pi F3](../boards/F3.html) — same K1 / X60 SoC | Same RVV `gemv_n` bug as RV2 | Same [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) fix | HPL **11.52 GFLOP/s** PASSED; `difftest` bit-identical to RV2; threaded DGEMM **17.71 GFLOP/s** |
14-
15-
## X60 A/B — patched RVV vs scalar (Orange Pi RV2)
16-
17-
All runs use a **fixed** RVV OpenBLAS (`gemv_n` patch) swapped in via FlexiBLAS. Source: [benchmarks/dgemm](https://github.com/opensolvers/benchmarks/tree/main/dgemm), [benchmarks/numpy](https://github.com/opensolvers/benchmarks/tree/main/numpy).
18-
19-
| Probe | Scalar (`RISCV64_GENERIC`) | Patched RVV (`ZVL256B`) | Speedup |
20-
| ----- | -------------------------- | ----------------------- | ------- |
21-
| `bench_dgemm` N=2048, 1 core | 1.16 GFLOP/s | 2.62 GFLOP/s | **2.3×** |
22-
| NumPy `A @ B` N=4096, 8 threads | 4.77 GFLOP/s | 11.52 GFLOP/s | **2.4×** |
23-
| NumPy `eigvalsh` N=2048, 8 threads | 10.54 s | 6.72 s | **1.6×** |
24-
25-
## X60 — isolating the `gemv_n` bug (`difftest`)
26-
27-
The [benchmarks `difftest`](https://github.com/opensolvers/benchmarks/blob/main/dgemm/difftest.c) tool `dlopen`s a BLAS `.so` and reports NaN counts per routine:
28-
29-
| Backend | `dgemv` NaN | `dgemm` NaN | `dgemv` sum |
30-
| ------- | ----------- | ----------- | ----------- |
31-
| Stock EESSI, default RVV | **192** | 0 | 198.94 (wrong) |
32-
| Stock EESSI, forced scalar | 0 | 0 | 42.06549 (reference) |
33-
| Patched RVV (`gemv_n` fix) | 0 | 0 | 42.06549 (matches) |
34-
35-
`dgemm` looks fine on the broken build — which is why a raw GEMM benchmark can pass while [HPL](../apps/hpl.html) and [ELPA](elpa.html) fail.
36-
37-
### Banana Pi BPI-F3 (cross-board confirmation)
38-
39-
Bit-identical `difftest` to RV2. `bench_dgemm` (1 core, N=2048): **1.26 → 2.96 GFLOP/s** (2.35×). Threaded (8 cores, N=4096): **17.71 GFLOP/s**.
14+
| [Orange Pi RV2](../boards/RV2.html) — SpacemiT **X60** (RVV VLEN=256) | RVV `gemv_n` zeroes an **uninitialized** vector register → `dgemv` returns NaN | Backport upstream `gemv_n` fix; `TARGET=RISCV64_ZVL256B` | [DGEMM](dgemm.html) 2.3×; [NumPy](numpy.html) DGEMM 2.4×; [HPL](../apps/hpl.html) **10.53 GFLOP/s** |
15+
| [Banana Pi F3](../boards/F3.html) — same K1 / X60 SoC | Same RVV `gemv_n` bug as RV2 | Same [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) fix | [DGEMM](dgemm.html) 2.35×; [NumPy](numpy.html) DGEMM **3.6×**; HPL **11.52 GFLOP/s** |
4016

4117
## Packages
4218

scientific-libs/dgemm.md

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# DGEMM
2+
3+
Micro-benchmarks for **BLAS level-3 performance** and **per-routine correctness** — from [opensolvers/benchmarks/dgemm](https://github.com/opensolvers/benchmarks/tree/main/dgemm). Used to isolate the OpenBLAS 0.3.30 RVV `gemv_n` NaN bug before fixing downstream apps.
4+
5+
See also the [BLAS overview](blas.html) for OpenBLAS packages and board context.
6+
7+
## Tools
8+
9+
| File | Purpose |
10+
| ---- | ------- |
11+
| [`bench_dgemm.c`](https://github.com/opensolvers/benchmarks/blob/main/dgemm/bench_dgemm.c) | Times square `C = A×B` (3 reps), reports GFLOP/s, prints `C[0]` |
12+
| [`difftest.c`](https://github.com/opensolvers/benchmarks/blob/main/dgemm/difftest.c) | `dlopen`s a BLAS `.so`, runs level-1/2/3 routines, reports `sum` / NaN counts |
13+
14+
Both switch backends at runtime via FlexiBLAS or `OPENBLAS_CORETYPE` — no recompile.
15+
16+
```bash
17+
gcc -O2 bench_dgemm.c -o bench_dgemm -lflexiblas
18+
gcc -O2 difftest.c -o difftest -ldl -lm
19+
20+
OPENBLAS_NUM_THREADS=8 ./bench_dgemm 4096
21+
./difftest /path/to/libopenblas.so
22+
```
23+
24+
## Correctness — `difftest` (X60, RV2 & F3)
25+
26+
Bit-identical on [Orange Pi RV2](../boards/RV2.html) and [Banana Pi F3](../boards/F3.html):
27+
28+
| Backend | `dgemv` NaN | `dgemm` NaN | `dgemv` sum |
29+
| ------- | ----------- | ----------- | ----------- |
30+
| Stock EESSI, default RVV | **192** | 0 | 198.94 (wrong) |
31+
| Forced scalar | 0 | 0 | 42.06549 (reference) |
32+
| Patched RVV (`gemv_n` fix) | 0 | 0 | 42.06549 (matches) |
33+
34+
Fault is in **`dgemv` only** — plain `dgemm` looks fine on the broken build, which is why [HPL](../apps/hpl.html) can NaN while a GEMM micro-benchmark passes.
35+
36+
## Performance — `bench_dgemm`
37+
38+
### Orange Pi RV2 (1 core, N=2048)
39+
40+
| Backend | GFLOP/s | `C[0]` |
41+
| ------- | ------- | ------ |
42+
| Scalar | 1.16 | 245.24 |
43+
| Patched RVV | 2.62 | 245.24 |
44+
45+
**2.3×** faster, numerically identical.
46+
47+
### Banana Pi F3 (cross-board)
48+
49+
| Backend | GFLOP/s | `C[0]` |
50+
| ------- | ------- | ------ |
51+
| Scalar | 1.26 | 245.24 |
52+
| Patched RVV | 2.96 | 245.24 |
53+
54+
**2.35×** at 1 core. Threaded (8 cores, N=4096): **17.71 GFLOP/s** on patched RVV.

scientific-libs/lapack.md

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,25 @@
11
# LAPACK
22

3-
LAPACK correctness and performance here are probed through **NumPy** (`eigvalsh``dsyevd`) and pure BLAS (`A @ B``dgemm`) — a real-application path for SciPy/NumPy stacks on EESSI.
3+
LAPACK performance and correctness probed via NumPy **`eigvalsh`** (`dsyevd`) — the LAPACK-heavy half of the [NumPy benchmark](numpy.html).
44

5-
Benchmark source: [opensolvers/benchmarks/numpy](https://github.com/opensolvers/benchmarks/tree/main/numpy).
5+
Benchmark source: [opensolvers/benchmarks/numpy](https://github.com/opensolvers/benchmarks/tree/main/numpy) (`bench_blas.py`). For DGEMM (`A @ B`) see the [NumPy](numpy.html) or [DGEMM](dgemm.html) pages.
66

7-
## Orange Pi RV2 (SpaceMiT X60, 8 threads)
7+
## Orange Pi RV2 (8 threads)
88

9-
Backends swapped via FlexiBLAS on a NumPy linked against the FlexiBLAS hub (`SciPy-bundle` module). Patched RVV OpenBLAS uses the `gemv_n` fix from [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444).
9+
Patched RVV OpenBLAS via FlexiBLAS (`SciPy-bundle` module, [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444)).
1010

11-
| Kernel | Scalar (`RISCV64_GENERIC`) | Patched RVV (`ZVL256B`) | Speedup |
12-
| ------ | -------------------------- | ----------------------- | ------- |
13-
| DGEMM (`A @ B`, N=4096) | 4.77 GFLOP/s | 11.52 GFLOP/s | **2.4×** |
11+
| Kernel | Scalar | Patched RVV | Speedup |
12+
| ------ | ------ | ----------- | ------- |
1413
| EIGH (`eigvalsh`, N=2048) | 10.54 s | 6.72 s | **1.6×** |
1514

16-
Both patched runs are **finite** (correct through real LAPACK). The eigensolver speedup is lower than DGEMM because `eigvalsh` mixes BLAS-3 with BLAS-2 tridiagonalization — the same pattern seen in [ELPA](elpa.html).
15+
Stock EESSI default RVV: `eigvalsh` returns NaN / fails to converge. See [DGEMM `difftest`](dgemm.html).
1716

18-
Stock EESSI default RVV would return NaN on the eigensolver path; see the [BLAS `difftest` table](blas.html).
19-
20-
### Banana Pi BPI-F3 (cross-board confirmation)
17+
## Banana Pi F3 (cross-board)
2118

2219
| Kernel | Scalar | Patched RVV | Speedup |
2320
| ------ | ------ | ----------- | ------- |
24-
| DGEMM N=4096 | 4.91 GFLOP/s | **17.51 GFLOP/s** | **3.6×** |
2521
| EIGH N=2048 | 9.59 s | 5.94 s | **1.6×** |
2622

27-
Stock unpatched RVV: `eigvalsh` aborts with `LinAlgError: Eigenvalues did not converge` (DGEMM alone stays finite at 10.94 GFLOP/s).
23+
Stock unpatched RVV: `LinAlgError: Eigenvalues did not converge`.
24+
25+
For the full NumPy benchmark (DGEMM + EIGH), see [NumPy](numpy.html). For a heavier eigensolver probe, see [ELPA](elpa.html).

scientific-libs/numpy.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# NumPy
2+
3+
A **SciPy-stack probe** for BLAS and LAPACK through NumPy — from [opensolvers/benchmarks/numpy](https://github.com/opensolvers/benchmarks/tree/main/numpy) (`bench_blas.py`).
4+
5+
| Kernel | NumPy call | Backend | Metric |
6+
| ------ | ---------- | ------- | ------ |
7+
| DGEMM | `A @ B` | `dgemm` | GFLOP/s |
8+
| EIGH | `np.linalg.eigvalsh(S)` | `dsyevd` | seconds |
9+
10+
Unlike raw [DGEMM](dgemm.html), `eigvalsh` exercises BLAS-2 tridiagonalization — a correctness gate for the same `gemv_n` bug that breaks [HPL](../apps/hpl.html) and [ELPA](elpa.html).
11+
12+
```bash
13+
module load SciPy-bundle/2025.07-gfbf-2025b
14+
OMP_NUM_THREADS=8 OPENBLAS_NUM_THREADS=8 python3 bench_blas.py [dgemm_N] [eigh_N]
15+
# defaults: dgemm_N=4096 eigh_N=2048
16+
```
17+
18+
Swap backends per run: `OPENBLAS_CORETYPE=RISCV64_GENERIC` or `FLEXIBLAS=/path/to/libopenblas.so`.
19+
20+
## Orange Pi RV2 (8 threads, patched RVV)
21+
22+
| Kernel | Scalar | Patched RVV | Speedup |
23+
| ------ | ------ | ----------- | ------- |
24+
| DGEMM N=4096 | 4.77 GFLOP/s | 11.52 GFLOP/s | **2.4×** |
25+
| EIGH N=2048 | 10.54 s | 6.72 s | **1.6×** |
26+
27+
Both patched results finite. See [LAPACK](lapack.html) for the LAPACK angle.
28+
29+
## Banana Pi F3 (cross-board, 8 threads)
30+
31+
| Kernel | Scalar | Patched RVV | Speedup |
32+
| ------ | ------ | ----------- | ------- |
33+
| DGEMM N=4096 | 4.91 GFLOP/s | **17.51 GFLOP/s** | **3.6×** |
34+
| EIGH N=2048 | 9.59 s | 5.94 s | **1.6×** |
35+
36+
**Stock unpatched RVV:** `eigvalsh` raises `LinAlgError: Eigenvalues did not converge`; DGEMM alone stays finite at 10.94 GFLOP/s (only `gemv` is broken — see [DGEMM difftest](dgemm.html)).
37+
38+
Eigensolver speedup (~1.6×) is lower than DGEMM (~2.4–3.6×) because `eigvalsh` mixes BLAS-3 with latency-bound BLAS-2 — same pattern as [ELPA](elpa.html).

website.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Context file for structural and content decisions on [opensolvers.com](https://w
2020
1. **Home**
2121
2. **Boards** — VisionFive 2, OrangePi RV2, BananaPi F3
2222
3. **Apps** — HPL
23-
4. **Scientific libs** — BLAS, LAPACK, ELPA
23+
4. **Scientific libs** — BLAS, DGEMM, NumPy, LAPACK, ELPA
2424

2525
Nav config: `_config.yml` (`navigation`, `navigation_boards`, `navigation_apps`, `navigation_scientific_libs`). Rendered in `_includes/header.html`. Cayman theme requires `_layouts/default.html` override to include the header.
2626

@@ -44,6 +44,7 @@ Nav config: `_config.yml` (`navigation`, `navigation_boards`, `navigation_apps`,
4444

4545
| Date | Decision |
4646
|------|----------|
47+
| 2026-07-14 | Add dedicated DGEMM and NumPy scientific-lib pages from benchmarks repo |
4748
| 2026-07-14 | BPI-F3 results from `opensolvers/benchmarks`: HPL 11.52 GFLOP/s, 3.7 GB RAM limit |
4849
| 2026-07-12 | Homepage scope widened: scientific libs + apps, not HPL-only |
4950
| 2026-07-12 | Add `website.md`; sync HPL/BLAS from `opensolvers/benchmarks`; add scientific libs LAPACK + ELPA |

0 commit comments

Comments
 (0)