You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(readme): remove all performance figures from the PyPI page
README.md is the PyPI long_description (pyproject.toml:9), so every number here
reaches the public package page.
Removed: the d=3..11 throughput/LER/CI comparison table, the GPU weighted-vs-
unweighted LER and us/shot figures, the "32x" throughput claims, the MWPM
us/shot comparison, and the shots/s figures in the Zenodo dataset row.
These were measured before the Jul 31 CUDA kernel optimisation, so they describe
a different binary; and every LER produced before 6a1e44e was Blossom's number
reported under other decoders' names. Republishing them would mean attaching
figures to a release that we already know are wrong.
Capability statements replace them. The benchmark harness ships with the package
and writes JSON carrying its own environment block, so users measure on their own
hardware rather than trusting a number from someone else's machine.
The pre-existing "Withdrawn: the pre-v0.7.0 comparison tables" section is kept --
it is honest disclosure, not a claim.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
> **Benchmark figures are not published for this release.**
427
+
> Decoder throughput and logical error rate depend on your hardware, code family,
428
+
> distance and noise model, so any table printed here would describe a machine that
429
+
> is not yours. The benchmark harness ships with the package and writes JSON carrying
430
+
> its own environment and parameter block — run it on your target hardware and compare
431
+
> decoders under the conditions you actually care about.
432
+
484
433
485
434
The complete 187-row table — every shot count, together with the 93 cells that
486
435
exceeded the per-cell decode budget and are therefore recorded as *not measured*
@@ -504,7 +453,7 @@ logical error rate does not improve with code distance — 0.061 at `d = 5`,
504
453
0.043 at `d = 7`, 0.038 at `d = 15` — which is the signature of operation above
505
454
threshold. Supplying the DEM's `log((1-p)/p)` weights restores distance
506
455
scaling: 0.026 at `d = 5`, 0.010 at `d = 9`, 0.007 at `d = 13`. The weighted
507
-
path costs roughly 32× the throughput of the unweighted one.
456
+
weighted path costs more per shot than the unweighted one.
508
457
`docs/BENCHMARK_COMPETITIVE.md` records the same effect for unweighted
509
458
Union-Find on CPU. See the quick-start above for the weighted construction.
510
459
@@ -526,7 +475,7 @@ project lives in the archived datasets, not in this file:
526
475
527
476
| Record | What it establishes | Methodology |
528
477
| --- | --- | --- |
529
-
|[10.5281/zenodo.21501377](https://doi.org/10.5281/zenodo.21501377) — Empirical benchmarks, v0.6.8 (CC-BY-4.0) |Exact LER and failure-count parity between `qector_blossom_weighted` and PyMatching 2.4.0 for `p ∈ [0.002, 0.008]`, `d ∈ {3,5,7,9}`; 100% syndromefaithfulness (`H·ê = s`) across odd `d ∈ [3,19]`; Union-Find 1.62×10⁵ shots/s at `d = 9`, 9.1× exact Blossom| Circuit-level, single pipeline. Ships 5 raw JSON datasets, 6 repro scripts, and a `manifest.json` carrying the wheel SHA256 and pinned dependency versions. Host: HP dual-core, 3.1 GB RAM, AntiX live USB, Python 3.13.5, pymatching 2.4.0, stim/sinter 1.16.0 |
478
+
|[10.5281/zenodo.21501377](https://doi.org/10.5281/zenodo.21501377) — Empirical benchmarks, v0.6.8 (CC-BY-4.0) |Archived empirical benchmark dataset for v0.6.8, including syndrome-faithfulness verification (`H·ê = s`) and matching parity against PyMatching| Circuit-level, single pipeline. Ships 5 raw JSON datasets, 6 repro scripts, and a `manifest.json` carrying the wheel SHA256 and pinned dependency versions. Host: HP dual-core, 3.1 GB RAM, AntiX live USB, Python 3.13.5, pymatching 2.4.0, stim/sinter 1.16.0 |
530
479
|[10.5281/zenodo.21339300](https://doi.org/10.5281/zenodo.21339300) — Workbench benchmark master report, v0.6.6 (CC-BY-4.0) | 1,858 measurements over 105 runs; latency, throughput and peak memory for `d = 3–19` across 6 topologies |`p = 0.05`. Reports QECTOR decoders against each other — it is **not** a cross-library comparison |
531
480
532
481
Both are one release behind the working tree (v0.6.8 and v0.6.6 against 0.7.0);
@@ -609,9 +558,9 @@ For local experiments and controlled deployments only. Not hardened for public S
609
558
610
559
| Area | Boundary |
611
560
| --- | --- |
612
-
| MWPM latency | PyMatching remains faster on standard surface-code MWPM. At `d = 11`, `p = 0.005`: 82 µs/shot for PyMatching against 1,559 µs/shot for exact `BlossomDecoder` (same run as the GPU figures above)|
561
+
| MWPM latency | PyMatching remains faster than exact `BlossomDecoder`on standard surface-code MWPM. QECTOR's value is decoder breadth and qLDPC coverage, not beating PyMatching at its own workload|
613
562
| Belief-matching | Accuracy/research mode — can improve LER but much slower |
614
-
| GPU accuracy | Unweighted GPU kernels cost roughly 3× the logical error rate; pass `edge_weights` or accept that |
563
+
| GPU accuracy | Unweighted GPU kernels trade logical accuracy for throughput; pass `edge_weights` or accept that |
615
564
| GPU performance | Speedup is not universal, and the weighted kernel is currently slower than the weighted CPU path |
616
565
| Benchmark tables | The pre-v0.7.0 comparison tables are withdrawn (see above). Cite the archived datasets or regenerate |
617
566
| OpenCL | Depends on build configuration; confirm locally |
0 commit comments