You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sync HPL-on-BLIS and IME1 scale-build benchmarks to the site.
Pull in benchmarks PR #21 (Linpack on BLIS) and #22 (+4.3% IME1 pp512) across HPL, BLIS, RV2, and homepage.
Co-authored-by: Cursor <cursoragent@cursor.com>
Copy file name to clipboardExpand all lines: README.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -30,14 +30,14 @@ Each RISC-V board exposes several compute paths. We benchmark and tune them inde
30
30
|**Custom**| Custom ISA extensions beyond standard RVV | X60 **IME** / **XsmtVdot** (`smt.vmadot`) int8 on [RV2](boards/RV2.html) / [F3](boards/F3.html); [ONNX Runtime](apps/onnx.html) int4 via MLAS |
31
31
|**GPU**| Integrated Imagination GPUs (OpenCL / Vulkan) |**IMG BXE-4-32 MC1** on [VisionFive 2](boards/VisionFive2.html) (JH7110); **IMG BXE-2-32** on [RV2](boards/RV2.html) / [F3](boards/F3.html) (K1) — present on silicon, **GPGPU benchmarks not yet**|
32
32
33
-
Recent highlights on the Orange Pi RV2 (SpaceMiT X60, RVV): fixing an OpenBLAS `gemv_n` bug restores correctness across [BLAS](scientific-libs/blas.html), [LAPACK](scientific-libs/lapack.html), [ELPA](scientific-libs/elpa.html), [ScaLAPACK](scientific-libs/scalapack.html), [HPL](apps/hpl.html), and [Quantum ESPRESSO](apps/qe.html). [BLIS](scientific-libs/blis.html) RVV assembly beats patched OpenBLAS **~1.29×** on single-thread DGEMM (N=4096). [FFTW `r5v`](scientific-libs/fftw.html) wins **1.06–1.60×** in isolation but **~0%** inside a real QE SCF; [GROMACS](apps/gromacs.html) sees **1.23×** on isolated `PME 3D-FFT` and **3.31×** whole-app with a hand-written RVV `Force` backend. ONNX `accuracy_level=4` unlocks **9–10×** int4 decode — [ONNX Runtime](apps/onnx.html) / [MLAS](scientific-libs/mlas.html).
33
+
Recent highlights on the Orange Pi RV2 (SpaceMiT X60, RVV): fixing an OpenBLAS `gemv_n` bug restores correctness across [BLAS](scientific-libs/blas.html), [LAPACK](scientific-libs/lapack.html), [ELPA](scientific-libs/elpa.html), [ScaLAPACK](scientific-libs/scalapack.html), [HPL](apps/hpl.html), and [Quantum ESPRESSO](apps/qe.html). [BLIS](scientific-libs/blis.html) RVV assembly beats patched OpenBLAS **~1.29×** on single-thread DGEMM (N=4096), but HPL linked to BLIS is correct yet only **0.35–0.53×** OpenBLAS-RVV. [FFTW `r5v`](scientific-libs/fftw.html) wins **1.06–1.60×** in isolation but **~0%** inside a real QE SCF; [GROMACS](apps/gromacs.html) sees **1.23×** on isolated `PME 3D-FFT` and **3.31×** whole-app with a hand-written RVV `Force` backend. ONNX `accuracy_level=4` unlocks **9–10×** int4 decode — [ONNX Runtime](apps/onnx.html) / [MLAS](scientific-libs/mlas.html). llama.cpp IME1 scale-build opt: **+4.3%** pp512 microkernel, bit-exact ([RV2 IME](boards/RV2.html#ime-integer-matrix-extension)).
End-to-end application benchmarks on the same boards and EESSI toolchain:
51
51
52
-
-**[HPL](apps/hpl.html)** — High Performance Linpack; cross-board summary and A/B configs from [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks)
52
+
-**[HPL](apps/hpl.html)** — High Performance Linpack; OpenBLAS A/B + BLIS-linked validation from [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks)
Copy file name to clipboardExpand all lines: apps/hpl.md
+24Lines changed: 24 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -48,6 +48,30 @@ Same `xhpl`, backend swapped via FlexiBLAS. All **PASSED** with the fixed vector
48
48
49
49
A squarer **2×4** grid beats 1×8 for peak throughput. `HPL_big.dat` needs ~6.6 GB RAM — tight on 8 GB boards.
50
50
51
+
### HPL on BLIS — end-to-end validation
52
+
53
+
Unlike the FlexiBLAS OpenBLAS A/B above, BLIS cannot be swapped at runtime on the RV2 (no FlexiBLAS). A dedicated `xhpl` is linked statically against RVV `libblis.a` (`Make.rv64_blis`, `build-hpl-blis.sh`). Source: [benchmarks/hpl](https://github.com/opensolvers/benchmarks/tree/main/hpl).
54
+
55
+
This validates that RVV BLIS drives a real Linpack solve — not just square DGEMM — because HPL's panel path leans on `dgemv`/`dtrsv` (the same reason stock OpenBLAS NaN'd).
Largest square problem that fits (~5.2 GB). N=28672 OOMs on this 8 GB no-swap board (rank 7 signal 9).
65
+
66
+
| Grid | GFLOP/s | Time | Residual |
67
+
| ---- | ------: | ---: | -------- |
68
+
|**2×4**|**5.87**| 1904.5 s | 3.37e-03 PASSED |
69
+
| 1×8 | 5.84 | 1916.0 s | 2.84e-03 PASSED |
70
+
| 4×2 | 5.23 | 2140.5 s | 4.07e-03 PASSED |
71
+
| 8×1 | 4.31 | 2592.9 s | 5.75e-03 PASSED |
72
+
73
+
**Takeaway:** correctness holds (no NaN, residual on par with OpenBLAS). Throughput trails patched-RVV OpenBLAS — HPL is panel-heavy (thin k=256, `dtrsm`, level-2), not the large square DGEMM where BLIS wins ~1.2–1.3× single-thread. Wide grids beat tall (8×1 −26% vs 2×4). See [BLIS](../scientific-libs/blis.html).
Library-level kernel numbers: [MLAS](../scientific-libs/mlas.html). Raw IME microbenchmarks: [Orange Pi RV2](../boards/RV2.html#ime-integer-matrix-extension).
7
+
Library-level kernel numbers: [MLAS](../scientific-libs/mlas.html). Raw IME microbenchmarks: [Orange Pi RV2](../boards/RV2.html#ime-integer-matrix-extension). Related (llama.cpp, not ORT): IME1 scale-build prefill patch **+4.3%** on the isolated Q4_0 GEMM — see [RV2 IME](../boards/RV2.html#ime1-scale-build-prefill-optimization-llamacpp).
Copy file name to clipboardExpand all lines: boards/F3.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,7 +14,7 @@ Cross-board confirmation from [opensolvers/benchmarks](https://github.com/openso
14
14
15
15
Same K1 / X60 silicon as the Orange Pi RV2 — each chip has SpaceMiT's **IME** int8 matrix unit (`smt.vmadot`) on **cluster 0** (cores 0–3, **512 KB shared L2**). The IME is separate from RVV: one instruction accumulates a **4×4 int32** tile from **4×8 int8** operands.
16
16
17
-
Detailed `ime-bench` numbers were measured on the [RV2](RV2.html) (peak **~42 GOP/s** int8 GEMM vs ~5 GOP/s RVV int8, **~7–8×**). The BPI-F3 has the same IME hardware; pin to core 0 when running [benchmarks/ime](https://github.com/opensolvers/benchmarks/tree/main/ime). End-to-end int4 inference: [ONNX Runtime](../apps/onnx.html) and [MLAS](../scientific-libs/mlas.html).
17
+
Detailed `ime-bench` numbers were measured on the [RV2](RV2.html) (peak **~42 GOP/s** int8 GEMM vs ~5 GOP/s RVV int8, **~7–8×**; IME1 scale-build **+4.3%** on llama.cpp Q4_0). The BPI-F3 has the same IME hardware; pin to core 0 when running [benchmarks/ime](https://github.com/opensolvers/benchmarks/tree/main/ime). End-to-end int4 inference: [ONNX Runtime](../apps/onnx.html) and [MLAS](../scientific-libs/mlas.html).
Copy file name to clipboardExpand all lines: boards/RV2.md
+17Lines changed: 17 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,6 +28,21 @@ Microbenchmarks in [opensolvers/benchmarks/ime](https://github.com/opensolvers/b
28
28
29
29
Peak **~42 GOP/s** single-core — vs ~5 GOP/s for a straightforward RVV int8 path. End-to-end int4 LLM decode through [ONNX Runtime](../apps/onnx.html) and isolated [MLAS](../scientific-libs/mlas.html) kernel rates use the same IME hardware; see also [papers/x60-ime-block-scale-optimization](https://github.com/opensolvers/benchmarks/blob/main/papers/x60-ime-block-scale-optimization.md) in the benchmarks repo.
llama.cpp's block-scaled Q4_0 kernel (`gemm_kernel_i8i4`) pays a per-block FP scale tax (~31–37% vs raw `s8s8s32`). Patch [`llama-ime1-scalebuild-opt.patch`](https://github.com/opensolvers/benchmarks/blob/main/ime/llama-ime1-scalebuild-opt.patch) rebuilds `As×Bs` scales with `vfmul.vv` (`LOAD_SCALE_4x16_FP16_OPT`) instead of the masked `vfmul.vf` chain.
34
+
35
+
Isolated interleaved A/B on this board (30 rounds, cluster-0 pinned, performance governor, pp512 = 512³) — [benchmarks/ime](https://github.com/opensolvers/benchmarks/tree/main/ime):
36
+
37
+
|| stock | scaleopt |
38
+
| --- | ---: | ---: |
39
+
| median GOP/s | 22.50 |**23.47**|
40
+
| range | 22.4–22.7 | 23.2–23.6 |
41
+
42
+
**+4.3%** (+0.97 GOP/s); scaleopt faster in **30/30** rounds; zero overlap; paired t = 50.3. Bit-exact (`sum`/`sumsq`/`max` identical across three shapes).
43
+
44
+
The kernel win is real; end-to-end `llama-bench` pp512 on this multi-tenant part is **not resolvable above ±15–20% noise** (see the paper linked above). Decode (`tg`) uses the untouched M1/GEMV path.
45
+
31
46
## HPL via EESSI
32
47
33
48
See also the [HPL app overview](../apps/hpl.html).
@@ -60,6 +75,8 @@ Full walkthrough: [EESSI/docs#819](https://github.com/EESSI/docs/pull/819) — *
60
75
61
76
See [BLIS](../scientific-libs/blis.html) — FLAME BLIS `rv64iv` with hand-written RVV assembly vs patched OpenBLAS on the same `bench_dgemm.c`. Single-thread DGEMM at N=4096: **2.95 vs 2.28 GFLOP/s** (**1.29×** BLIS); 8-thread OpenBLAS still leads at large N. TRSM: **2400 cases, 0 fails** on BLIS.
62
77
78
+
End-to-end [HPL on BLIS](../apps/hpl.html#hpl-on-blis--end-to-end-validation): all **PASSED**, but **0.35–0.53×** OpenBLAS-RVV (best full-memory **5.87 GFLOP/s** at N=25600, 2×4) — the DGEMM win does not carry to Linpack.
79
+
63
80
## FFTW RVV
64
81
65
82
See [FFTW](../scientific-libs/fftw.html) — r5v wins **1.06–1.60×** in `tests/bench`, but **~0%** end-to-end in [Quantum ESPRESSO](../apps/qe.html) (`FFTW_ESTIMATE`).
Copy file name to clipboardExpand all lines: scientific-libs/blis.md
+7-1Lines changed: 7 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ See also [BLAS (OpenBLAS)](blas.html) for the OpenBLAS fixes and verification su
10
10
11
11
## Why link, not FlexiBLAS
12
12
13
-
Other BLAS-axis benchmarks ([HPL](../apps/hpl.html), [NumPy](numpy.html), [QE](../apps/qe.html)) swap backends at runtime via FlexiBLAS. **FlexiBLAS is not installed on the RV2**, so this A/B links the same unchanged `bench_dgemm.c` against each library in turn — still one variable (the BLAS implementation), identical `-O3 -march=rv64imafdcv_zvl256b`.
13
+
Other BLAS-axis benchmarks ([HPL](../apps/hpl.html), [NumPy](numpy.html), [QE](../apps/qe.html)) swap backends at runtime via FlexiBLAS. **FlexiBLAS is not installed on the RV2**, so this A/B links the same unchanged `bench_dgemm.c` against each library in turn — still one variable (the BLAS implementation), identical `-O3 -march=rv64imafdcv_zvl256b`. The same constraint applies end-to-end: [HPL on BLIS](../apps/hpl.html#hpl-on-blis--end-to-end-validation) builds a dedicated `xhpl` against static `libblis.a`.
-**8 threads:** OpenBLAS scales slightly better (BLIS **0.80–0.89×**). Both get ~3.5–5× from 8 cores; BLIS's OpenMP path leaves headroom vs OpenBLAS threading.
54
54
-**Correctness first:** both backends numerically identical on DGEMM; BLIS TRSM verified independently.
55
55
56
+
## End-to-end: HPL on BLIS
57
+
58
+
Linking HPL against the same RVV `libblis.a` ([HPL app](../apps/hpl.html#hpl-on-blis--end-to-end-validation)): all configs **PASSED**, but BLIS is **0.35–0.53×** patched OpenBLAS-RVV (4.02 / 5.57 GFLOP/s; full-memory best **5.87** at N=25600, 2×4).
59
+
60
+
The 1T square-DGEMM advantage does **not** predict Linpack: HPL hits skinny rank-k / `dtrsm` / `dgemv` shapes where BLIS RVV trails. Details and grid sweep: [benchmarks/hpl](https://github.com/opensolvers/benchmarks/tree/main/hpl).
Copy file name to clipboardExpand all lines: scientific-libs/mlas.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@
4
4
5
5
Benchmark source: [opensolvers/benchmarks/onnx](https://github.com/opensolvers/benchmarks/tree/main/onnx) — [`bench_qnbit_mlas.cpp`](https://github.com/opensolvers/benchmarks/blob/main/onnx/bench_qnbit_mlas.cpp) links directly against `libonnxruntime_mlas.a` and calls the same entry points as ORT's `matmul_nbits.cc`.
6
6
7
-
See also the [ONNX Runtime app](../apps/onnx.html) for end-to-end int4 LLM-FFN inference and the [IME microbenchmarks](../boards/RV2.html#ime-integer-matrix-extension) on the Orange Pi RV2.
7
+
See also the [ONNX Runtime app](../apps/onnx.html) for end-to-end int4 LLM-FFN inference and the [IME microbenchmarks](../boards/RV2.html#ime-integer-matrix-extension) on the Orange Pi RV2. Related (llama.cpp path): [IME1 scale-build +4.3%](../boards/RV2.html#ime1-scale-build-prefill-optimization-llamacpp).
0 commit comments