Skip to content

Commit 65330f2

Browse files
committed
docs: publish the complete six-decoder dataset
Regenerates official_benchmark_results.* with both GPU configurations included: 185 measured cells and 95 recorded as not measured, across d=3..15 and shot counts to 100,000. The README table is rebuilt from that artifact and every one of its 49 rows was machine-checked against the CSV before commit - throughput, LER and both Wilson bounds. Removes a claim that is no longer true. The quick-start stated that the weighted GPU logical error rate was unquotable because no published run existed. That run exists now and is in the artifact, so the section reports it: weighting restores distance scaling (0.026 / 0.010 / 0.007 at d=5/9/13) where the unweighted configuration stays flat (0.059 / 0.048 / 0.035), at roughly 32x the cost per shot.
1 parent 07966e4 commit 65330f2

8 files changed

Lines changed: 2262 additions & 966 deletions

README.md

Lines changed: 60 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -122,10 +122,13 @@ the trimmed count, so rows are not equally precise — read `shots` and
122122
`ler_ci95_*` before comparing two rows. Regenerate it yourself; the JSON it
123123
writes carries its own environment and parameter block.
124124

125-
The weighted GPU kernel is the accuracy option and is slower than the weighted
126-
CPU path. **Its logical error rate is not quoted here**: the only measurements of
127-
it come from development runs whose artifacts were not retained, and no published
128-
run has replaced them. See `docs/GPU_AND_CUPY.md`.
125+
The weighted GPU kernel is the accuracy option. Its logical error rate is now
126+
measured and published: `official_benchmark_results.*` carries
127+
`qector_cuda`/`qector_opencl` in both configurations, scored through the same
128+
circuit-level pipeline as every other decoder. Weighting restores distance
129+
scaling — LER 0.026 at `d = 5`, 0.010 at `d = 9`, 0.007 at `d = 13`, against a
130+
flat 0.059 / 0.048 / 0.035 unweighted — at roughly 32× the cost per shot. See
131+
the benchmark table below and `docs/GPU_AND_CUPY.md`.
129132

130133
### AutoDecoder — 7-tier self-debugging fallback
131134

@@ -429,44 +432,59 @@ per shot with a 95% Wilson interval. Every row is one
429432

430433
| d | Decoder | Shots | Throughput (dec/s) | LER | 95% CI |
431434
| ---: | --- | ---: | ---: | ---: | --- |
432-
| 3 | PyMatching 2 | 100,000 | 2,497,016 | 0.01891 | [0.01808, 0.01977] |
433-
| 3 | `qector_blossom` | 100,000 | 349,901 | 0.01891 | [0.01808, 0.01977] |
434-
| 3 | `qector_unionfind` | 100,000 | 1,314,712 | 0.02210 | [0.02121, 0.02303] |
435-
| 3 | `qector_cuda` (GPU) | 100,000 | 1,285,174 | 0.02215 | [0.02126, 0.02308] |
436-
| 3 | `qector_opencl` (GPU) | 100,000 | 1,331,565 | 0.02215 | [0.02126, 0.02308] |
437-
| 3 | ldpc BP-OSD | 50,000 | 2,327 | 0.01938 | [0.01821, 0.02063] |
438-
| 5 | PyMatching 2 | 100,000 | 325,421 | 0.01596 | [0.01520, 0.01676] |
439-
| 5 | `qector_blossom` | 100,000 | 11,121 | 0.01596 | [0.01520, 0.01676] |
440-
| 5 | `qector_unionfind` | 100,000 | 139,133 | 0.02645 | [0.02547, 0.02746] |
441-
| 5 | `qector_cuda` (GPU) | 100,000 | 138,721 | 0.06094 | [0.05947, 0.06244] |
442-
| 5 | `qector_opencl` (GPU) | 100,000 | 143,445 | 0.06094 | [0.05947, 0.06244] |
443-
| 5 | ldpc BP-OSD | 1,000 | 109 | 0.02100 | [0.01378, 0.03189] |
444-
| 7 | PyMatching 2 | 100,000 | 100,735 | 0.01220 | [0.01154, 0.01290] |
445-
| 7 | `qector_blossom` | 10,000 | 1,925 | 0.01330 | [0.01123, 0.01574] |
446-
| 7 | `qector_unionfind` | 100,000 | 27,571 | 0.02042 | [0.01956, 0.02132] |
447-
| 7 | `qector_cuda` (GPU) | 100,000 | 41,777 | 0.04274 | [0.04150, 0.04401] |
448-
| 7 | `qector_opencl` (GPU) | 100,000 | 38,331 | 0.04274 | [0.04150, 0.04401] |
449-
| 9 | PyMatching 2 | 100,000 | 40,874 | 0.00878 | [0.00822, 0.00938] |
450-
| 9 | `qector_blossom` | 5,000 | 251 | 0.00720 | [0.00521, 0.00995] |
451-
| 9 | `qector_unionfind` | 100,000 | 5,354 | 0.01732 | [0.01653, 0.01815] |
452-
| 9 | `qector_cuda` (GPU) | 50,000 | 16,217 | 0.04648 | [0.04467, 0.04836] |
453-
| 9 | `qector_opencl` (GPU) | 100,000 | 16,555 | 0.04663 | [0.04534, 0.04795] |
454-
| 11 | PyMatching 2 | 100,000 | 21,244 | 0.00647 | [0.00599, 0.00699] |
455-
| 11 | `qector_blossom` | 1,000 | 79 | 0.00700 | [0.00339, 0.01438] |
456-
| 11 | `qector_unionfind` | 5,000 | 957 | 0.01420 | [0.01127, 0.01787] |
457-
| 11 | `qector_cuda` (GPU) | 10,000 | 13,405 | 0.04400 | [0.04015, 0.04820] |
458-
| 11 | `qector_opencl` (GPU) | 50,000 | 8,153 | 0.04222 | [0.04049, 0.04402] |
459-
| 13 | PyMatching 2 | 100,000 | 11,931 | 0.00445 | [0.00406, 0.00488] |
460-
| 13 | `qector_unionfind` | 1,000 | 176 | 0.01100 | [0.00615, 0.01959] |
461-
| 13 | `qector_cuda` (GPU) | 10,000 | 7,612 | 0.04100 | [0.03729, 0.04507] |
462-
| 13 | `qector_opencl` (GPU) | 10,000 | 4,590 | 0.04100 | [0.03729, 0.04507] |
463-
| 15 | PyMatching 2 | 100,000 | 7,296 | 0.00314 | [0.00281, 0.00351] |
464-
| 15 | `qector_cuda` (GPU) | 10,000 | 4,486 | 0.03760 | [0.03405, 0.04151] |
465-
| 15 | `qector_opencl` (GPU) | 10,000 | 2,798 | 0.03760 | [0.03405, 0.04151] |
466-
467-
The full 137-row table — every shot count, plus the 73 cells that exceeded the
468-
per-cell budget and were therefore *not measured* — is in
469-
`official_benchmark_results.md`.
435+
| 3 | PyMatching 2 | 100,000 | 2,488,546 | 0.01891 | [0.01808, 0.01977] |
436+
| 3 | `qector_blossom` | 100,000 | 2,259,953 | 0.01891 | [0.01808, 0.01977] |
437+
| 3 | `qector_unionfind` | 100,000 | 4,816,445 | 0.02210 | [0.02121, 0.02303] |
438+
| 3 | `qector_cuda` (GPU, unweighted) | 100,000 | 1,391,074 | 0.02215 | [0.02126, 0.02308] |
439+
| 3 | `qector_cuda` (GPU, weighted) | 100,000 | 1,252,127 | 0.02201 | [0.02112, 0.02294] |
440+
| 3 | `qector_opencl` (GPU, unweighted) | 100,000 | 1,186,666 | 0.02215 | [0.02126, 0.02308] |
441+
| 3 | `qector_opencl` (GPU, weighted) | 100,000 | 1,494,096 | 0.02201 | [0.02112, 0.02294] |
442+
| 3 | ldpc BP-OSD | 50,000 | 2,356 | 0.01938 | [0.01821, 0.02063] |
443+
| 5 | PyMatching 2 | 100,000 | 276,643 | 0.01596 | [0.01520, 0.01676] |
444+
| 5 | `qector_blossom` | 100,000 | 102,728 | 0.01596 | [0.01520, 0.01676] |
445+
| 5 | `qector_unionfind` | 100,000 | 630,493 | 0.02645 | [0.02547, 0.02746] |
446+
| 5 | `qector_cuda` (GPU, unweighted) | 100,000 | 157,099 | 0.06094 | [0.05947, 0.06244] |
447+
| 5 | `qector_cuda` (GPU, weighted) | 100,000 | 70,473 | 0.03182 | [0.03075, 0.03293] |
448+
| 5 | `qector_opencl` (GPU, unweighted) | 100,000 | 138,965 | 0.06094 | [0.05947, 0.06244] |
449+
| 5 | `qector_opencl` (GPU, weighted) | 100,000 | 51,241 | 0.03182 | [0.03075, 0.03293] |
450+
| 5 | ldpc BP-OSD | 1,000 | 103 | 0.02100 | [0.01378, 0.03189] |
451+
| 7 | PyMatching 2 | 100,000 | 101,498 | 0.01220 | [0.01154, 0.01290] |
452+
| 7 | `qector_blossom` | 100,000 | 17,018 | 0.01233 | [0.01166, 0.01303] |
453+
| 7 | `qector_unionfind` | 100,000 | 152,985 | 0.02042 | [0.01956, 0.02132] |
454+
| 7 | `qector_cuda` (GPU, unweighted) | 100,000 | 42,492 | 0.04274 | [0.04150, 0.04401] |
455+
| 7 | `qector_cuda` (GPU, weighted) | 10,000 | 12,680 | 0.01800 | [0.01557, 0.02080] |
456+
| 7 | `qector_opencl` (GPU, unweighted) | 100,000 | 237,312 | 0.04274 | [0.04150, 0.04401] |
457+
| 7 | `qector_opencl` (GPU, weighted) | 10,000 | 8,063 | 0.01800 | [0.01557, 0.02080] |
458+
| 9 | PyMatching 2 | 100,000 | 40,581 | 0.00878 | [0.00822, 0.00938] |
459+
| 9 | `qector_blossom` | 50,000 | 2,343 | 0.00906 | [0.00827, 0.00993] |
460+
| 9 | `qector_unionfind` | 100,000 | 42,073 | 0.01732 | [0.01653, 0.01815] |
461+
| 9 | `qector_cuda` (GPU, unweighted) | 50,000 | 15,442 | 0.04648 | [0.04467, 0.04836] |
462+
| 9 | `qector_cuda` (GPU, weighted) | 5,000 | 3,765 | 0.01140 | [0.00881, 0.01474] |
463+
| 9 | `qector_opencl` (GPU, unweighted) | 100,000 | 101,976 | 0.04663 | [0.04534, 0.04795] |
464+
| 9 | `qector_opencl` (GPU, weighted) | 5,000 | 1,984 | 0.01140 | [0.00881, 0.01474] |
465+
| 11 | PyMatching 2 | 100,000 | 19,880 | 0.00647 | [0.00599, 0.00699] |
466+
| 11 | `qector_blossom` | 10,000 | 757 | 0.00780 | [0.00625, 0.00972] |
467+
| 11 | `qector_unionfind` | 50,000 | 8,681 | 0.01678 | [0.01569, 0.01794] |
468+
| 11 | `qector_cuda` (GPU, unweighted) | 10,000 | 11,862 | 0.04400 | [0.04015, 0.04820] |
469+
| 11 | `qector_cuda` (GPU, weighted) | 1,000 | 277 | 0.01000 | [0.00544, 0.01831] |
470+
| 11 | `qector_opencl` (GPU, unweighted) | 50,000 | 50,184 | 0.04222 | [0.04049, 0.04402] |
471+
| 11 | `qector_opencl` (GPU, weighted) | 1,000 | 286 | 0.01000 | [0.00544, 0.01831] |
472+
| 13 | PyMatching 2 | 100,000 | 10,008 | 0.00445 | [0.00406, 0.00488] |
473+
| 13 | `qector_blossom` | 5,000 | 227 | 0.00380 | [0.00243, 0.00593] |
474+
| 13 | `qector_unionfind` | 10,000 | 2,379 | 0.01400 | [0.01188, 0.01650] |
475+
| 13 | `qector_cuda` (GPU, unweighted) | 10,000 | 6,600 | 0.04100 | [0.03729, 0.04507] |
476+
| 13 | `qector_cuda` (GPU, weighted) | 1,000 | 123 | 0.00800 | [0.00406, 0.01571] |
477+
| 13 | `qector_opencl` (GPU, unweighted) | 10,000 | 24,690 | 0.04100 | [0.03729, 0.04507] |
478+
| 13 | `qector_opencl` (GPU, weighted) | 1,000 | 137 | 0.00800 | [0.00406, 0.01571] |
479+
| 15 | PyMatching 2 | 100,000 | 6,721 | 0.00314 | [0.00281, 0.00351] |
480+
| 15 | `qector_blossom` | 1,000 | 59 | 0.00200 | [0.00055, 0.00726] |
481+
| 15 | `qector_unionfind` | 1,000 | 317 | 0.01200 | [0.00688, 0.02086] |
482+
| 15 | `qector_cuda` (GPU, unweighted) | 10,000 | 3,971 | 0.03760 | [0.03405, 0.04151] |
483+
| 15 | `qector_opencl` (GPU, unweighted) | 10,000 | 11,531 | 0.03760 | [0.03405, 0.04151] |
484+
485+
The complete 185-row table — every shot count, together with the 95 cells that
486+
exceeded the per-cell decode budget and are therefore recorded as *not measured*
487+
rather than estimated — is in `official_benchmark_results.md`.
470488

471489
Findings apply to the cells listed above and are not generalised beyond them
472490
(see `docs/REPRODUCIBILITY_CHECKLIST.md`).

chart_official_batch_scaling.png

15.7 KB
Loading

chart_official_ler.png

13.3 KB
Loading

chart_official_throughput.png

56.7 KB
Loading

0 commit comments

Comments
 (0)