Skip to content

Commit d8e4c11

Browse files
Sync HPL and BLAS from benchmarks repo; add LAPACK and ELPA.
Add scientific-libs pages from opensolvers/benchmarks, expand HPL A/B tables for Orange Pi RV2, and add website.md to track site decisions. Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 2c940f0 commit d8e4c11

7 files changed

Lines changed: 162 additions & 13 deletions

File tree

_config.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,7 @@ navigation_apps:
2121
navigation_scientific_libs:
2222
- title: BLAS
2323
url: /scientific-libs/blas.html
24+
- title: LAPACK
25+
url: /scientific-libs/lapack.html
26+
- title: ELPA
27+
url: /scientific-libs/elpa.html

apps/hpl.md

Lines changed: 38 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,50 @@
11
# HPL results overview
22

3-
Cross-board summary of **High Performance Linpack (HPL)** runs on consumer RISC-V hardware through the EESSI stack.
3+
Cross-board summary of **High Performance Linpack (HPL)** on consumer RISC-V hardware through the EESSI stack. Configs and an A/B runner (`run-hpl-ab.sh`) are in [opensolvers/benchmarks/hpl](https://github.com/opensolvers/benchmarks/tree/main/hpl).
44

5-
**Toolchain:** GCC 14.3.0, OpenBLAS 0.3.30, HPL 2.3.0. EESSI runs use `2025.06-001` on [`dev.eessi.io/riscv`](https://www.eessi.io/docs/repositories/dev.eessi.io-riscv/).
5+
**Toolchain:** GCC 14.3.0, OpenBLAS 0.3.30, HPL 2.3.0. EESSI `2025.06-001` on [`dev.eessi.io/riscv`](https://www.eessi.io/docs/repositories/dev.eessi.io-riscv/).
6+
7+
## Cross-board summary
68

79
| Board | Cores | Before | After |
810
| ----- | ----- | ------ | ----- |
9-
| [StarFive VisionFive 2](../boards/VisionFive2.html) | 4× SiFive U74 | 3.13 GFLOP/s | **5.28 GFLOP/s** |
11+
| [VisionFive 2](../boards/VisionFive2.html) | 4× SiFive U74 | 3.13 GFLOP/s | **5.28 GFLOP/s** |
1012
| [Orange Pi RV2](../boards/RV2.html) | 8× SpacemiT X60 | FAILED (`nan`) | **10.53 GFLOP/s** |
1113
| [Banana Pi F3](../boards/F3.html) | 8× SpacemiT X60 | FAILED (`nan`) ||
1214

13-
**Before** — stock EESSI OpenBLAS 0.3.30 on each board.
15+
**Before** — stock EESSI OpenBLAS 0.3.30 (same `xhpl` binary throughout).
16+
17+
**After** — fixed OpenBLAS via EasyBuild + FlexiBLAS backend swap (no HPL rebuild). See [BLAS overview](../scientific-libs/blas.html).
18+
19+
## Orange Pi RV2 — detailed results
20+
21+
Stock EESSI dispatches RVV `ZVL256B` on the X60, but the unpatched `gemv_n` kernel makes HPL report ~8.5 GFLOP/s while **failing** the residual check (`nan`). With the [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) fix, all runs below **PASSED**.
22+
23+
### EESSI walkthrough (fixed backend)
24+
25+
| Config | Grid | N | Result |
26+
| ------ | ---- | - | ------ |
27+
| Stock EESSI, default RVV | 1×8 | 8000 | ~8.5 GFLOP/s, **FAILED** (`nan`) |
28+
| Fixed RVV, peak | 2×4 | 20000 | **10.53 GFLOP/s**, PASSED |
29+
30+
Walkthrough: [EESSI/docs#819](https://github.com/EESSI/docs/pull/819).
31+
32+
### A/B — scalar vs patched RVV (`run-hpl-ab.sh`)
33+
34+
Same `xhpl`, backend swapped via FlexiBLAS. All **PASSED** with the fixed vector library:
35+
36+
| Config | Scalar (`RISCV64_GENERIC`) | Patched RVV (`ZVL256B`) | Speedup |
37+
| ------ | -------------------------- | ----------------------- | ------- |
38+
| `HPL.dat` (N=8000, 1×8) | 6.41 GFLOP/s | 11.55 GFLOP/s | **1.80×** |
39+
| `HPL_big.dat` (N=28672, 1×8) | 7.38 GFLOP/s | 13.41 GFLOP/s | **1.82×** |
40+
| `HPL-sweep.dat` (N=20000, 2×4) || ~10.5 GFLOP/s ||
41+
42+
A squarer **2×4** grid beats 1×8 for peak throughput. `HPL_big.dat` needs ~6.6 GB RAM — tight on 8 GB boards.
43+
44+
## VisionFive 2
1445

15-
**After** — board-specific fixed OpenBLAS via EasyBuild, swapped in at runtime with FlexiBLAS (no HPL rebuild). Details in the [BLAS overview](../scientific-libs/blas.html).
46+
Scalar U74 — stock OpenBLAS uses a generic kernel; the U74-tuned build lifts HPL **3.13 → 5.28 GFLOP/s** (**1.69×**). Walkthrough: [EESSI/docs#818](https://github.com/EESSI/docs/pull/818).
1647

17-
On **VisionFive 2** (scalar U74), stock OpenBLAS uses a generic kernel; the fix adds a tuned U74 micro-kernel ([easyconfigs#26436](https://github.com/easybuilders/easybuild-easyconfigs/pull/26436), [EESSI/docs#818](https://github.com/EESSI/docs/pull/818)).
48+
## Banana Pi F3
1849

19-
On **X60 boards** (RVV 1.0, VLEN=256), stock OpenBLAS already dispatches RVV kernels but **0.3.30's `gemv_n` produces NaN**, so HPL fails its residual check despite reporting plausible GFLOP/s. The fix backports the upstream `gemv_n` correction ([easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444), [OpenBLAS#5408](https://github.com/OpenMathLib/OpenBLAS/pull/5408), [EESSI/docs#819](https://github.com/EESSI/docs/pull/819)). Banana Pi F3 uses the same K1/X60 SoC — the same fix applies; peak result not yet recorded.
50+
Same K1 / X60 SoC as the Orange Pi RV2 — same stock failure and same [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) fix applies. Peak HPL not yet recorded.

boards/RV2.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ End-to-end on real Orange Pi RV2 hardware using [EESSI](https://www.eessi.io/) `
1313
| | Before | After |
1414
| --- | ------ | ----- |
1515
| HPL (8 cores, N=20000, 2×4) | ~8.5 GFLOP/s, **FAILED** (`nan`) | **10.53 GFLOP/s**, **PASSED** |
16+
17+
With the fixed backend, scalar-vs-RVV A/B ([benchmarks/hpl](https://github.com/opensolvers/benchmarks/tree/main/hpl)): **6.41 → 11.55 GFLOP/s** (N=8000, 1×8) and **7.38 → 13.41 GFLOP/s** (N=28672, 1×8).
1618
| Residual (N=8000, 1×8) | `nan` | 4.04e-03 |
1719

1820
**Before** — stock EESSI OpenBLAS 0.3.30 (RVV `gemv_n` bug). **After** — fixed OpenBLAS built with `TARGET=RISCV64_ZVL256B` and a backported `gemv_n` patch ([easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444)), swapped in via FlexiBLAS — no HPL rebuild.

scientific-libs/blas.md

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# BLAS (OpenBLAS)
22

3-
Improvements to **OpenBLAS 0.3.30** on RISC-V boards in this project — built via EasyBuild, deployed through [EESSI](https://www.eessi.io/) with **FlexiBLAS** runtime swapping. See also the [HPL app](../apps/hpl.html) for end-to-end benchmark results.
3+
Improvements to **OpenBLAS 0.3.30** on RISC-V boards — built via EasyBuild, deployed through [EESSI](https://www.eessi.io/) with **FlexiBLAS** runtime swapping. Verification tools and A/B scripts live in the [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks) repo (`dgemm/`, `numpy/`).
44

55
**Base stack:** GCC 14.3.0, OpenBLAS 0.3.30, EESSI `2025.06-001` ([`dev.eessi.io/riscv`](https://www.eessi.io/docs/repositories/dev.eessi.io-riscv/)).
66

@@ -9,8 +9,30 @@ Improvements to **OpenBLAS 0.3.30** on RISC-V boards in this project — built v
99
| Board / CPU | Problem (stock 0.3.30) | Fix | Result |
1010
| ----------- | ---------------------- | --- | ------ |
1111
| [VisionFive 2](../boards/VisionFive2.html) — SiFive **U74** (scalar) | No U74 kernel; falls back to generic `RISCV64_GENERIC` C `2×2` GEMM | New **4×4 DGEMM micro-kernel** in RV64 assembly, `TARGET=U74` | Single-core DGEMM **~1.4 → 1.77 GFLOP/s**; 4-core DGEMM **6.31 GFLOP/s**; [HPL **1.69×**](../apps/hpl.html) |
12-
| [Orange Pi RV2](../boards/RV2.html) — SpacemiT **X60** (RVV VLEN=256) | RVV `gemv_n` zeroes an **uninitialized** vector register → `dgemv` returns NaN; HPL **fails** residual despite ~8.5 GFLOP/s | Backport upstream `gemv_n` fix; `TARGET=RISCV64_ZVL256B` | HPL **PASSED** (residual ~4e-03); peak **10.53 GFLOP/s** |
13-
| [Banana Pi F3](../boards/F3.html) — same K1 / X60 SoC | Same RVV `gemv_n` bug as RV2 | Same [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) fix | Fix applies; peak HPL not yet recorded |
12+
| [Orange Pi RV2](../boards/RV2.html) — SpacemiT **X60** (RVV VLEN=256) | RVV `gemv_n` zeroes an **uninitialized** vector register → `dgemv` returns NaN | Backport upstream `gemv_n` fix; `TARGET=RISCV64_ZVL256B` | Correct `dgemv`; see [HPL](../apps/hpl.html) and A/B tables below |
13+
| [Banana Pi F3](../boards/F3.html) — same K1 / X60 SoC | Same RVV `gemv_n` bug as RV2 | Same [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) fix | Fix applies; peak results not yet recorded |
14+
15+
## X60 A/B — patched RVV vs scalar (Orange Pi RV2)
16+
17+
All runs use a **fixed** RVV OpenBLAS (`gemv_n` patch) swapped in via FlexiBLAS. Source: [benchmarks/dgemm](https://github.com/opensolvers/benchmarks/tree/main/dgemm), [benchmarks/numpy](https://github.com/opensolvers/benchmarks/tree/main/numpy).
18+
19+
| Probe | Scalar (`RISCV64_GENERIC`) | Patched RVV (`ZVL256B`) | Speedup |
20+
| ----- | -------------------------- | ----------------------- | ------- |
21+
| `bench_dgemm` N=2048, 1 core | 1.16 GFLOP/s | 2.62 GFLOP/s | **2.3×** |
22+
| NumPy `A @ B` N=4096, 8 threads | 4.77 GFLOP/s | 11.52 GFLOP/s | **2.4×** |
23+
| NumPy `eigvalsh` N=2048, 8 threads | 10.54 s | 6.72 s | **1.6×** |
24+
25+
## X60 — isolating the `gemv_n` bug (`difftest`)
26+
27+
The [benchmarks `difftest`](https://github.com/opensolvers/benchmarks/blob/main/dgemm/difftest.c) tool `dlopen`s a BLAS `.so` and reports NaN counts per routine:
28+
29+
| Backend | `dgemv` NaN | `dgemm` NaN | `dgemv` sum |
30+
| ------- | ----------- | ----------- | ----------- |
31+
| Stock EESSI, default RVV | **192** | 0 | 198.94 (wrong) |
32+
| Stock EESSI, forced scalar | 0 | 0 | 42.06549 (reference) |
33+
| Patched RVV (`gemv_n` fix) | 0 | 0 | 42.06549 (matches) |
34+
35+
`dgemm` looks fine on the broken build — which is why a raw GEMM benchmark can pass while [HPL](../apps/hpl.html) and [ELPA](elpa.html) fail.
1436

1537
## Packages
1638

@@ -19,10 +41,10 @@ Improvements to **OpenBLAS 0.3.30** on RISC-V boards in this project — built v
1941
| SiFive U74 | [easyconfigs#26436](https://github.com/easybuilders/easybuild-easyconfigs/pull/26436) | [OpenBLAS#5903](https://github.com/OpenMathLib/OpenBLAS/pull/5903) | [EESSI/docs#818](https://github.com/EESSI/docs/pull/818) |
2042
| SpacemiT X60 | [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) | [OpenBLAS#5408](https://github.com/OpenMathLib/OpenBLAS/pull/5408), [#5476](https://github.com/OpenMathLib/OpenBLAS/pull/5476) | [EESSI/docs#819](https://github.com/EESSI/docs/pull/819) |
2143

22-
Build from a PR with `eb --from-pr <num> --robot` into an **EESSI-extend** user install, then register with `flexiblas add` / `flexiblas default` — no need to rebuild downstream apps like HPL.
44+
Build with `eb --from-pr <num> --robot` into an **EESSI-extend** user install, then `flexiblas add` / `flexiblas default` — no downstream rebuild.
2345

2446
## Notes
2547

2648
- **U74** — performance kernel; stock OpenBLAS works but leaves FP throughput on the table.
27-
- **X60** — correctness fix first: the stock EESSI `DYNAMIC_ARCH` build *does* dispatch RVV `ZVL256B` kernels, but 0.3.30's broken `gemv_n` makes HPL silently wrong. OpenBLAS **≥ 0.3.34** should carry the fix natively and improve RVV performance further.
28-
- On the experimental `dev.eessi.io/riscv` toolchain, pin a valid `-march` via `EASYBUILD_OPTARCH` when building (see board walkthroughs).
49+
- **X60** — correctness fix first. Stock EESSI `DYNAMIC_ARCH` *does* dispatch RVV, but 0.3.30's broken `gemv_n` corrupts BLAS-2 paths. OpenBLAS **≥ 0.3.34** should fix this natively.
50+
- Pin a valid `-march` via `EASYBUILD_OPTARCH` on the experimental `dev.eessi.io/riscv` toolchain (see board walkthroughs).

scientific-libs/elpa.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# ELPA
2+
3+
[ELPA](https://elpa.mpcdf.mpg.de/) is a dense real-symmetric eigensolver used in DFT codes (CP2K, VASP, ELSI). It exercises a **mix of BLAS-2 and BLAS-3** (`dsymv`, `dsyr2k`, `dgemm`, `dtrmm`) — a more representative BLAS probe than raw `dgemm`.
4+
5+
Benchmark source: [opensolvers/benchmarks/elpa](https://github.com/opensolvers/benchmarks/tree/main/elpa).
6+
7+
## Orange Pi RV2 (SpaceMiT X60, 8 cores)
8+
9+
`na=3000`, 1-stage solver, 8 threads, single MPI rank. Backends swapped via FlexiBLAS — no rebuild.
10+
11+
| Backend | Time | Correctness |
12+
| ------- | ---- | ----------- |
13+
| Stock EESSI OpenBLAS 0.3.30, default RVV | 42.17 s | **`ev0=nan`, finite=0 — FAIL** |
14+
| Scalar (`RISCV64_GENERIC`) | 54.92 s | finite=1 |
15+
| Patched RVV (`gemv_n` fix) | **34.81 s** | finite=1 |
16+
17+
The stock vector backend is faster than scalar but **wrong** — the same OpenBLAS 0.3.30 `gemv_n` NaN bug that breaks [HPL](../apps/hpl.html). With the patch, RVV is **1.58×** faster than scalar (54.92 / 34.81) and numerically correct.
18+
19+
Speedup is smaller than pure `dgemm` (~2.3×) because tridiagonalization is latency-bound BLAS-2 — by design.
20+
21+
See also the [BLAS overview](blas.html) and [LAPACK](lapack.html) NumPy probe.

scientific-libs/lapack.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# LAPACK
2+
3+
LAPACK correctness and performance here are probed through **NumPy** (`eigvalsh``dsyevd`) and pure BLAS (`A @ B``dgemm`) — a real-application path for SciPy/NumPy stacks on EESSI.
4+
5+
Benchmark source: [opensolvers/benchmarks/numpy](https://github.com/opensolvers/benchmarks/tree/main/numpy).
6+
7+
## Orange Pi RV2 (SpaceMiT X60, 8 threads)
8+
9+
Backends swapped via FlexiBLAS on a NumPy linked against the FlexiBLAS hub (`SciPy-bundle` module). Patched RVV OpenBLAS uses the `gemv_n` fix from [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444).
10+
11+
| Kernel | Scalar (`RISCV64_GENERIC`) | Patched RVV (`ZVL256B`) | Speedup |
12+
| ------ | -------------------------- | ----------------------- | ------- |
13+
| DGEMM (`A @ B`, N=4096) | 4.77 GFLOP/s | 11.52 GFLOP/s | **2.4×** |
14+
| EIGH (`eigvalsh`, N=2048) | 10.54 s | 6.72 s | **1.6×** |
15+
16+
Both patched runs are **finite** (correct through real LAPACK). The eigensolver speedup is lower than DGEMM because `eigvalsh` mixes BLAS-3 with BLAS-2 tridiagonalization — the same pattern seen in [ELPA](elpa.html).
17+
18+
Stock EESSI default RVV would return NaN on the eigensolver path; see the [BLAS `difftest` table](blas.html) for how the fault isolates to `dgemv`.

website.md

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# OpenSolvers website — decisions log
2+
3+
Context file for structural and content decisions on [opensolvers.com](https://www.opensolvers.com) (`opensolvers/opensolvers.github.io`).
4+
5+
**Workflow:** commit and deploy directly to `main` via PR (no need to ask first).
6+
7+
---
8+
9+
## Site structure
10+
11+
| Area | Path | Purpose |
12+
|------|------|---------|
13+
| Homepage | `README.md``/` | Intro, board summaries |
14+
| Boards | `boards/` | Per-board hardware + benchmark notes |
15+
| Apps | `apps/` | End-to-end application benchmarks (e.g. HPL) |
16+
| Scientific libs | `scientific-libs/` | Library-level probes (BLAS, LAPACK, ELPA) |
17+
18+
## Navigation groups
19+
20+
1. **Home**
21+
2. **Boards** — VisionFive 2, OrangePi RV2, BananaPi F3
22+
3. **Apps** — HPL
23+
4. **Scientific libs** — BLAS, LAPACK, ELPA
24+
25+
Nav config: `_config.yml` (`navigation`, `navigation_boards`, `navigation_apps`, `navigation_scientific_libs`). Rendered in `_includes/header.html`. Cayman theme requires `_layouts/default.html` override to include the header.
26+
27+
## Content sources
28+
29+
| Source | Used for |
30+
|--------|----------|
31+
| [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks) | BLAS verification (`dgemm/`), HPL configs (`hpl/`), ELPA (`elpa/`), NumPy/LAPACK (`numpy/`) |
32+
| [EESSI/docs#818](https://github.com/EESSI/docs/pull/818) | VisionFive 2 / U74 OpenBLAS + HPL |
33+
| [EESSI/docs#819](https://github.com/EESSI/docs/pull/819) | Orange Pi RV2 / X60 RVV `gemv_n` fix + HPL |
34+
| [easyconfigs#26436](https://github.com/easybuilders/easybuild-easyconfigs/pull/26436) | U74 OpenBLAS package |
35+
| [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) | X60 OpenBLAS package |
36+
37+
## Key numbering conventions
38+
39+
- **Before / After** tables — stock EESSI OpenBLAS 0.3.30 vs fixed EasyBuild module (FlexiBLAS swap, no app rebuild).
40+
- **Orange Pi RV2 HPL:** stock EESSI = FAILED (`nan`) ~8.5 GFLOP/s; fixed peak = **10.53 GFLOP/s** (N=20000, 2×4). Scalar A/B baselines from `run-hpl-ab.sh`: **6.41** and **7.38 GFLOP/s** — these are *scalar* backends, not “after” results.
41+
- **X60 DNF/FAIL** — OpenBLAS 0.3.30 RVV `gemv_n` bug; fixed in ≥ 0.3.31, EESSI bump ≥ 0.3.34 preferred.
42+
43+
## Changelog
44+
45+
| Date | Decision |
46+
|------|----------|
47+
| 2026-07-12 | Add `website.md`; sync HPL/BLAS from `opensolvers/benchmarks`; add scientific libs LAPACK + ELPA |
48+
| 2026-07-12 | Split nav: Boards / Apps / Scientific libs; move HPL to `apps/` |
49+
| 2026-07-12 | Fix homepage RV2 numbers (FAILED → 10.53 GFLOP/s, not 7.38 native) |
50+
| 2026-07-11 | Override Cayman `_layouts/default.html` so `_includes/header.html` renders |
51+
| 2026-07-11 | Site focus: RISC-V learnings; tagline “RISC-V learnings and fun” |

0 commit comments

Comments
 (0)