Skip to content

Commit 6a1e44e

Browse files
Guillaume Lessardclaude
andcommitted
fix(workbench): LER was Blossom's, reported under every decoder's name
_ler() accepted `kind` and ignored it. The body built pymatching_compat.Matching.from_detector_error_model(sdem) and decoded with that, and Matching is hard-wired to BlossomDecoder -- its __init__ declares `self._decoder: BlossomDecoder` and from_detector_error_model takes no decoder argument. Every row therefore carried Blossom's logical error rate while being labelled with whatever decoder had just been benchmarked. That is how the workbench export could report two_stage at a 5.0% logical error rate while 42.5% of two_stage's own corrections failed to reproduce their syndrome, and colour_code at 29% with a 1% syndrome match rate. The two columns were measuring different decoders. Now only kinds whose LER can actually be attributed produce a figure; the rest return None. A missing number, never a wrong one. syndrome_match_rate remains per-decoder and is the honest accuracy signal for the others. Also corrects docs/RELEASING.md: the pre-flight said "expect 221 passed" when the suite is 303, and pytest must run through dev.bat or 58 CUDA tests fail on licensing alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 59d68ef commit 6a1e44e

2 files changed

Lines changed: 39 additions & 5 deletions

File tree

docs/RELEASING.md

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,13 +72,21 @@ get **both** CUDA and OpenCL — which is why a locally built extension can expo
7272
## Pre-flight checklist
7373

7474
```bash
75-
cargo test --no-default-features # expect 221 passed, 0 failed
75+
cargo test --no-default-features # expect 303 passed, 0 failed
7676
cargo clippy --no-default-features --all-targets # expect 0 warnings
7777
ruff check python/ # expect 0 errors
7878
ruff format --check python/ # expect no reformatting needed
79-
python -m pytest python/tests -q # see test-results/pytest.txt
79+
dev.bat python -m pytest python/tests -q # see test-results/pytest.txt
8080
```
8181

82+
**Run pytest through `dev.bat`, never bare.** `dev.bat` exports the Enterprise token that
83+
unlocks the GPU paths. A bare `python -m pytest python/tests` fails 58 CUDA tests
84+
(`test_max_capacity.py::TestGPUCapacity::test_cuda_large_batch[*]` and
85+
`test_syndrome_faithfulness.py::test_cuda_bit_identical_and_faithful[*]`) purely on licensing,
86+
which looks like a broken tree and is not one. Confirm the unlock first with
87+
`dev.bat python scripts\_verify_enterprise_unlock.py` — expect
88+
`PASS: Enterprise/GPU unlock is confirmed working.`
89+
8290
Then, from the repo that owns the fulfilment worker:
8391

8492
```bash

python/qector_decoder_v3/workbench.py

Lines changed: 29 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -328,11 +328,36 @@ def _resolve_problems(self, spec, source):
328328
dists = spec.get("distances", [spec.get("distance", 5)])
329329
return [(_CODE_FAMILIES[fam](int(d)), None) for d in dists]
330330

331+
#: Decoder kinds whose LER this method can honestly measure. ``Matching``
332+
#: in ``pymatching_compat`` is hard-wired to ``BlossomDecoder``
333+
#: (``self._decoder: BlossomDecoder``; ``from_detector_error_model`` takes no
334+
#: decoder argument), so it is the only decoder actually exercised here.
335+
_LER_MEASURABLE_KINDS = frozenset({"blossom"})
336+
331337
def _ler(self, kind, code, ler_ctx, shots) -> float | None:
332-
"""Real LER from Stim shots for a stim-loaded problem (graphlike only)."""
338+
"""Real LER from Stim shots, for the decoder named by ``kind``.
339+
340+
Returns ``None`` when the LER cannot be attributed to ``kind`` — a
341+
missing number, never a wrong one.
342+
343+
This previously accepted ``kind`` and then ignored it: the body built
344+
``pymatching_compat.Matching.from_detector_error_model(sdem)`` and
345+
decoded with that, and ``Matching`` only ever drives ``BlossomDecoder``.
346+
Every row therefore carried Blossom's LER while being labelled with
347+
whatever decoder had just been benchmarked. That is how a row could
348+
report a 5.0% logical error rate while 42.5% of that decoder's own
349+
corrections failed to reproduce their syndrome — the two columns were
350+
measuring different decoders.
351+
352+
Until the Stim-shot path can drive an arbitrary decoder, only the kinds
353+
in :attr:`_LER_MEASURABLE_KINDS` yield a figure. Read
354+
``syndrome_match_rate`` for the others: it *is* measured per decoder,
355+
and a correction that does not reproduce its syndrome is not a valid
356+
correction regardless of any logical-error figure.
357+
"""
358+
if kind not in self._LER_MEASURABLE_KINDS:
359+
return None
333360
try:
334-
import stim
335-
336361
from . import pymatching_compat
337362

338363
circuit = ler_ctx["circuit"]
@@ -344,6 +369,7 @@ def _ler(self, kind, code, ler_ctx, shots) -> float | None:
344369
pred = np.asarray(m.decode_batch(det), np.uint8).reshape(shots, -1)
345370
return float(np.any(pred != obs, axis=1).mean())
346371
except (ImportError, AttributeError, TypeError, ValueError, RuntimeError):
372+
# Unsupported problem shape (e.g. a non-graphlike DEM) or Stim absent.
347373
return None
348374

349375
# ------------------------------------------------------------- job queue

0 commit comments

Comments
 (0)