Skip to content

Commit bdf1021

Browse files
authored
Rslib direct deps (#280)
* Restructure pecos-rslib: direct deps, split crates, faster builds Replace pecos metacrate dependency with direct crate deps, split decoders into pecos-rslib-qec and LLVM IR into pecos-rslib-llvm. Fix build pipeline to call maturin directly instead of through uv run (which caused full rebuilds every time). - pecos-rslib now depends on individual crates instead of pecos metacrate - New pecos-rslib-qec crate for QEC decoder bindings - New pecos-rslib-llvm crate for LLVM IR generation bindings - pecos python build now builds all rslib crates - just build no longer triggers uv sync forced rebuilds - Add skip-if-no-cuda-rust marker for doc tests - Remove dead qir_bindings.rs and llvm_context_bindings.rs * Optimize just build: skip selene when unchanged, simplify dev recipes * Cache LLVM in CI, simplify workflows, add new rslib crates to paths - Cache LLVM installation across CI runs (fixed version, keyed by OS+arch) - Skip LLVM download when cache hit (saves ~5 min per job) - Install PECOS CLI before LLVM setup in python-test (avoid cargo run overhead) - Update rust-cache workspaces to include pecos-rslib-qec and pecos-rslib-llvm - Add new rslib crate paths to rust-test trigger paths - Reduce python-test timeout from 240 to 120 minutes - Simplify Windows/macOS setup steps - Remove duplicated LLVM install/find/check cargo run invocations * Fix panic on zero-sized array construction (e.g., shape (0, 5)) * lint * Merge pecos-rslib-qec back into pecos-rslib, keep only pecos-rslib-llvm split * Fix CI: add just to docs workflow, fix Windows venv path * Fix docs CI: skip selene build, call pecos python build directly * Fix CI: install LLVM before CLI (inkwell needs LLVM_SYS_140_PREFIX) * Fix CI: docs missing deps, engine_type cfg gate bug - Docs workflow: use --all-packages to install guppylang etc. - Fix engine_type.rs cfg gate: was feature="llvm" but code uses pecos_qis (needs feature="qis") * Fix unified_sim.rs cfg gates: use feature=qis not feature=llvm for fallbacks * Fix remaining cfg gate bugs: engine_setup.rs and engine_type.rs qis/llvm mismatch * Fix macOS build (@rpath), make pecos-rslib-llvm optional - macOS: unset Homebrew library paths inline before build (GITHUB_ENV can't unset) - Move pecos-rslib-llvm from required to optional dep of quantum-pecos - Guard pecos_rslib_llvm imports with try/except for when LLVM not installed - Fixes release workflow test (pecos-rslib-llvm not on PyPI) * Add pecos-rslib-llvm wheel builds to release workflow - Build pecos-rslib-llvm wheels alongside pecos-rslib in cibuildwheel - Download and install llvm wheels in quantum-pecos test jobs - Keep pecos-rslib-llvm as required dep (not optional) - Revert try/except import guard in gen_qir.py - Organize llvm wheel artifacts separately in collect step * Clean up PR review issues: remove hardcoded paths, deduplicate unified_sim, optimize CI * Fix release: add libffi-devel to pecos-rslib-llvm Linux cibuildwheel * Move Go, Julia, Python check/test from CLI to Justfile recipes * Fix macOS: clear Homebrew library paths in maturin builds to prevent @rpath errors * Fix macOS @rpath/libc++ and Windows PowerShell in just build - Add -rpath /usr/lib to RUSTFLAGS on macOS so cdylibs can find system libc++ - Use just's - prefix for julia/go builds (cross-platform error ignoring) * Split CLI into pecos-cli crate, remove thin wrapper commands, move dev workflows to Justfile * Fix stale cfg feature references in pecos-cli (qasm/phir/llvm/quest -> runtime) * Simplify pecos doctor and info: remove always-true capability checks * Rewrite pecos doctor to check things that actually break * Add just doctor for fast dev environment checks without compilation * Update docs: doctor usage, CLI install instructions, Justfile recipe names * Merge install-cli and reinstall-cli into one recipe * Fix go-lint/go-test bugs, remove stale recipes, merge test/test-all * Rename clean-everything to clean-all * Fix clean-all comment to describe what it actually cleans * Remove legacy clean paths (--llvm, --cuda), simplify clean.py and Justfile * Remove clean-selene recipe (just clean already includes it) * Remove dead private recipes and unnecessary aliases * Collapse clean recipes into one: just clean [cache|deps|all|dry-run] * Add just doctor to quick-start banner * Remove devc/devc-full (Rust build auto-detects CUDA, Python extras are a separate step) * Remove fmt-fix (lint-fix already handles it) * Merge lint/lint-fix into one recipe: just lint (fixes) or just lint check * Rename dev-full to premerge * Rename premerge to check-all * lint * Remove decoder recipes (straight cargo commands, not worth wrapping) * Rename updatereqs to updatelocks * Make updatelocks nuke and regenerate both uv.lock and Cargo.lock * update lock * Remove clean from dev group (it belongs in clean group only) * Make just pytest run all Python tests by default, remove pytest-all * Fix cuQuantum version parsing (substring match hit wrong #define, .last() grabbed comment) * Remove pytest-selene (just pytest already runs selene tests) * Remove pytest-numpy (just pytest already runs numpy tests) * Fix CI: pytest-all -> pytest, lint -> lint check * Make check-cli auto-install if missing or outdated (fixes check-all after clean) * Describe parameters in recipe comments for discoverability * Update workspace keywords for better crates.io discoverability * Clean up workspace Cargo.toml: fix author spacing, simplify dev/test profiles * Add strip=true to release profile for smaller binaries * Remove commented-out zstd-sys workaround * Remove clean from check-all (incremental build is sufficient, clean + rebuild is fragile) * Fix build-selene SIGPIPE crash (find|head pipeline fails with set -eo pipefail) * Reorganize workspace deps by purpose, alphabetize internal crates * Fix flaky LLVM tests: don't assert on log message content * Remove stale test file accidentally restored in pecos crate * Fix stray closing paren in llvm test * Fix double-free in pecos-neo: Box the recursive Conditional variant in AdaptedOp * lint * Remove: wrapper functions, banner comments, duplicate detect functions, useless docs * Trim metacrate lib.rs: remove tutorial docs, banner comments, duplicate examples (832 -> 221 lines) * Fix CI: add --force to cargo install pecos-cli (binary name conflicts with old pecos package) * lint * Clean up clippy allows: remove 4 unnecessary, mark remaining 7 as TODO to localize * Localize too-many-arguments: remove workspace allow, add per-function allows with justification * Refactor QEC: DecoderAccumulator and PropagationBuffers structs replace long arg lists * Refactor pecos_array: FlatBuffers struct replaces 6 separate buffer args in 3 functions * Remove unused-self workspace allow: convert 38 methods to associated functions, 7 local PyO3 allows * Remove items-after-statements workspace allow: move 50 items before let bindings * Update allow comments with accurate counts, keep needless-pass-by-value and too-many-lines * Remove missing-errors-doc and missing-panics-doc workspace allows: add meaningful docs to 66 functions * Remove needless-pass-by-value workspace allow: fix 29 Rust functions, crate-level allow for PyO3 * Remove cast-sign-loss workspace allow: 92 casts given explicit per-line allows with safety comments * Remove cast-possible-wrap workspace allow: 52 casts given explicit per-line allows with safety comments * Keep cast-precision-loss and cast-possible-truncation as workspace allows (465 + 250 hits, pervasive) * Remove cast-possible-truncation workspace allow: 250 casts given explicit per-line allows * lint * Remove cast-precision-loss workspace allow: localize to functions/modules/files * Add --all-features to clippy commands to match CI * Fix clippy warnings with --all-features: benchmark cast allows and items-after-statements * Make dev use debug tests, check-all use release tests and release build * Fix doctests broken by needless-pass-by-value refactor (add & to doc examples) * Remove module-name-repetitions allow (zero warnings), keep physics naming and too-many-lines * Fix unnecessary_wraps in CLI: remove Result from functions that never error, remove stale allows * Remove stale allows, fix dem_sampler doc/ref/if warnings, use iterator in pcg_test * lint * Fix redundant_closure with method reference, fix DiEdgeBuilder allow comment * Fix method reference that takes &self but map_or passes owned value * Make just dev accept language argument: just dev rust, just dev python, etc. * Make clippy CUDA-aware: only use --all-features when CUDA is available
1 parent e96ef09 commit bdf1021

296 files changed

Lines changed: 4293 additions & 6039 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/julia-release.yml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -107,34 +107,34 @@ jobs:
107107
if: runner.os != 'Windows'
108108
run: |
109109
echo "Installing LLVM using pecos..."
110-
cargo run -p pecos --features cli --release -- install llvm
110+
cargo run -p pecos-cli --release -- install llvm
111111
112112
echo "Setting LLVM environment variables..."
113-
export PECOS_LLVM=$(cargo run -p pecos --features cli --release -- llvm find 2>/dev/null)
113+
export PECOS_LLVM=$(cargo run -p pecos-cli --release -- llvm find 2>/dev/null)
114114
export LLVM_SYS_140_PREFIX="$PECOS_LLVM"
115115
116116
echo "PECOS_LLVM=$PECOS_LLVM" >> $GITHUB_ENV
117117
echo "LLVM_SYS_140_PREFIX=$LLVM_SYS_140_PREFIX" >> $GITHUB_ENV
118118
119119
echo "Verifying LLVM installation..."
120-
cargo run -p pecos --features cli --release -- llvm check
120+
cargo run -p pecos-cli --release -- llvm check
121121
122122
- name: Install LLVM 14.0.6 using pecos-llvm (Windows)
123123
if: runner.os == 'Windows'
124124
shell: pwsh
125125
run: |
126126
Write-Host "Installing LLVM using pecos..."
127-
cargo run -p pecos --features cli --release -- install llvm
127+
cargo run -p pecos-cli --release -- install llvm
128128
129129
Write-Host "Setting LLVM environment variables..."
130-
$env:PECOS_LLVM = (cargo run -p pecos --features cli --release -- llvm find 2>$null)
130+
$env:PECOS_LLVM = (cargo run -p pecos-cli --release -- llvm find 2>$null)
131131
$env:LLVM_SYS_140_PREFIX = $env:PECOS_LLVM
132132
133133
"PECOS_LLVM=$env:PECOS_LLVM" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
134134
"LLVM_SYS_140_PREFIX=$env:LLVM_SYS_140_PREFIX" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
135135
136136
Write-Host "Verifying LLVM installation..."
137-
cargo run -p pecos --features cli --release -- llvm check
137+
cargo run -p pecos-cli --release -- llvm check
138138
139139
- name: Install Rust target
140140
run: |

.github/workflows/julia-test.yml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -58,34 +58,34 @@ jobs:
5858
# if: runner.os != 'Windows'
5959
# run: |
6060
# echo "Installing LLVM using pecos..."
61-
# cargo run -p pecos --features cli --release -- install llvm
61+
# cargo run -p pecos-cli --release -- install llvm
6262
#
6363
# echo "Setting LLVM environment variables..."
64-
# export PECOS_LLVM=$(cargo run -p pecos --features cli --release -- llvm find 2>/dev/null)
64+
# export PECOS_LLVM=$(cargo run -p pecos-cli --release -- llvm find 2>/dev/null)
6565
# export LLVM_SYS_140_PREFIX="$PECOS_LLVM"
6666
#
6767
# echo "PECOS_LLVM=$PECOS_LLVM" >> $GITHUB_ENV
6868
# echo "LLVM_SYS_140_PREFIX=$LLVM_SYS_140_PREFIX" >> $GITHUB_ENV
6969
#
7070
# echo "Verifying LLVM installation..."
71-
# cargo run -p pecos --features cli --release -- llvm check
71+
# cargo run -p pecos-cli --release -- llvm check
7272
#
7373
# - name: Install LLVM 14.0.6 using pecos (Windows)
7474
# if: runner.os == 'Windows'
7575
# shell: pwsh
7676
# run: |
7777
# Write-Host "Installing LLVM using pecos..."
78-
# cargo run -p pecos --features cli --release -- install llvm
78+
# cargo run -p pecos-cli --release -- install llvm
7979
#
8080
# Write-Host "Setting LLVM environment variables..."
81-
# $env:PECOS_LLVM = (cargo run -p pecos --features cli --release -- llvm find 2>$null)
81+
# $env:PECOS_LLVM = (cargo run -p pecos-cli --release -- llvm find 2>$null)
8282
# $env:LLVM_SYS_140_PREFIX = $env:PECOS_LLVM
8383
#
8484
# "PECOS_LLVM=$env:PECOS_LLVM" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
8585
# "LLVM_SYS_140_PREFIX=$env:LLVM_SYS_140_PREFIX" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
8686
#
8787
# Write-Host "Verifying LLVM installation..."
88-
# cargo run -p pecos --features cli --release -- llvm check
88+
# cargo run -p pecos-cli --release -- llvm check
8989

9090
- name: Cache Rust
9191
uses: Swatinem/rust-cache@v2

.github/workflows/python-release.yml

Lines changed: 92 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -217,7 +217,7 @@ jobs:
217217
else
218218
echo "Skipping CUDA installation (GPU support not enabled for this build)"
219219
fi
220-
cargo run --release -p pecos --features cli -- install llvm --force
220+
cargo run --release -p pecos-cli -- install llvm --force
221221
CIBW_REPAIR_WHEEL_COMMAND_LINUX: >
222222
auditwheel repair -w {dest_dir} {wheel} &&
223223
pipx run abi3audit --strict --report {wheel}
@@ -230,7 +230,7 @@ jobs:
230230
curl -sSf https://sh.rustup.rs | sh -s -- -y
231231
source $HOME/.cargo/env
232232
rustup update
233-
cargo run --release -p pecos --features cli -- install llvm --force
233+
cargo run --release -p pecos-cli -- install llvm --force
234234
# Create a codesign wrapper that strips DYLD_LIBRARY_PATH to prevent
235235
# crashes on macOS 15 when bundled libc++ conflicts with system libc++
236236
mkdir -p $HOME/.pecos/bin
@@ -248,7 +248,7 @@ jobs:
248248
echo "=== Installing LLVM using pecos ===" &&
249249
rustup update &&
250250
echo "=== Running pecos install llvm ===" &&
251-
cargo run --release -p pecos --features cli -- install llvm --force &&
251+
cargo run --release -p pecos-cli -- install llvm --force &&
252252
echo "=== Checking LLVM installation ===" &&
253253
(test -d "C:\\Users\\runneradmin\\.pecos\\deps\\llvm" && echo "LLVM directory exists" && ls -la "C:\\Users\\runneradmin\\.pecos\\deps\\llvm" && (ls -la "C:\\Users\\runneradmin\\.pecos\\deps\\llvm\\bin" || echo "bin directory not found")) || (echo "ERROR: LLVM directory not found!" && exit 1) &&
254254
echo "=== Verifying LLVM_SYS_140_PREFIX ===" &&
@@ -264,12 +264,72 @@ jobs:
264264
delvewheel repair -v --add-path "C:\\Users\\runneradmin\\.pecos\\deps\\llvm\\bin" --no-dll "combase.dll;rmclient.dll" -w {dest_dir} {wheel} &&
265265
pipx run abi3audit --strict --report {wheel}
266266
267-
- name: Upload wheels
267+
- name: Upload pecos-rslib wheels
268268
uses: actions/upload-artifact@v7
269269
with:
270270
name: wheel-pecos-rslib-${{ matrix.os }}-${{ matrix.architecture }}
271271
path: ./wheelhouse/*.whl
272272

273+
# Build pecos-rslib-llvm wheel (reuses LLVM already installed by pecos-rslib build)
274+
- name: Build pecos-rslib-llvm wheels
275+
uses: pypa/cibuildwheel@v3.3.1
276+
with:
277+
package-dir: python/pecos-rslib-llvm
278+
output-dir: wheelhouse-llvm
279+
env:
280+
CIBW_BUILD: "cp310-*"
281+
CIBW_SKIP: "*-win32 *-manylinux_i686 *-musllinux*"
282+
CIBW_ARCHS_LINUX: ${{ matrix.cibw_archs }}
283+
CIBW_MANYLINUX_X86_64_IMAGE: "manylinux_2_28"
284+
CIBW_MANYLINUX_AARCH64_IMAGE: "manylinux_2_28"
285+
CIBW_ENVIRONMENT_LINUX: >
286+
PATH=$HOME/.cargo/bin:$HOME/.pecos/deps/llvm/bin:$PATH
287+
LLVM_SYS_140_PREFIX=$HOME/.pecos/deps/llvm
288+
CIBW_BEFORE_ALL_LINUX: |
289+
curl -sSf https://sh.rustup.rs | sh -s -- -y
290+
source $HOME/.cargo/env
291+
dnf install libffi-devel -y
292+
cargo run --release -p pecos-cli -- install llvm --force
293+
CIBW_REPAIR_WHEEL_COMMAND_LINUX: >
294+
auditwheel repair -w {dest_dir} {wheel} &&
295+
pipx run abi3audit --strict --report {wheel}
296+
CIBW_ENVIRONMENT_MACOS: >
297+
PATH=$HOME/.cargo/bin:$HOME/.pecos/deps/llvm/bin:$PATH
298+
LLVM_SYS_140_PREFIX=$HOME/.pecos/deps/llvm
299+
MACOSX_DEPLOYMENT_TARGET=13.2
300+
CIBW_BEFORE_ALL_MACOS: |
301+
source $HOME/.cargo/env 2>/dev/null || { curl -sSf https://sh.rustup.rs | sh -s -- -y && source $HOME/.cargo/env; }
302+
if [ ! -d "$HOME/.pecos/deps/llvm/bin" ]; then
303+
rustup update
304+
cargo run --release -p pecos-cli -- install llvm --force
305+
else
306+
echo "LLVM already installed from pecos-rslib build, skipping"
307+
fi
308+
mkdir -p $HOME/.pecos/bin
309+
printf '#!/bin/bash\nunset DYLD_LIBRARY_PATH\nexec /usr/bin/codesign "$@"\n' > $HOME/.pecos/bin/codesign
310+
chmod +x $HOME/.pecos/bin/codesign
311+
CIBW_REPAIR_WHEEL_COMMAND_MACOS: >
312+
PATH=$HOME/.pecos/bin:$PATH DYLD_LIBRARY_PATH=$HOME/.pecos/deps/llvm/lib delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel} &&
313+
pipx run abi3audit --strict --report {wheel}
314+
CIBW_ENVIRONMENT_WINDOWS: >
315+
PATH="C:\\Users\\runneradmin\\.pecos\\deps\\llvm\\bin;$PATH"
316+
LLVM_SYS_140_PREFIX="C:\\Users\\runneradmin\\.pecos\\deps\\llvm"
317+
CIBW_BEFORE_ALL_WINDOWS: >
318+
rustup update &&
319+
if not exist "C:\Users\runneradmin\.pecos\deps\llvm\bin" (cargo run --release -p pecos-cli -- install llvm --force) else (echo LLVM already installed from pecos-rslib build)
320+
CIBW_BEFORE_BUILD_WINDOWS: >
321+
pip install delvewheel &&
322+
python -c "import delvewheel._dll_list as d,inspect,re as r;p=inspect.getfile(d);c=open(p).read();n=chr(10);open(p,'w').write(c.replace(r\"re.compile('api-.*'),\",r\"re.compile('api-.*'),\"+n+r\" re.compile('ext-.*'),\")) if 'ext-.*' not in c else None"
323+
CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: >
324+
delvewheel repair -v --add-path "C:\\Users\\runneradmin\\.pecos\\deps\\llvm\\bin" --no-dll "combase.dll;rmclient.dll" -w {dest_dir} {wheel} &&
325+
pipx run abi3audit --strict --report {wheel}
326+
327+
- name: Upload pecos-rslib-llvm wheels
328+
uses: actions/upload-artifact@v7
329+
with:
330+
name: wheel-pecos-rslib-llvm-${{ matrix.os }}-${{ matrix.architecture }}
331+
path: ./wheelhouse-llvm/*.whl
332+
273333
test_abi3_wheels:
274334
needs: build_wheelspecos_rslib
275335
if: |
@@ -371,8 +431,16 @@ jobs:
371431
name: wheel-pecos-rslib-ubuntu-latest-x86_64
372432
path: ./pecos-rslib-wheel
373433

374-
- name: Install pecos-rslib
375-
run: pip install ./pecos-rslib-wheel/*.whl
434+
- name: Download pecos-rslib-llvm wheel
435+
uses: actions/download-artifact@v7
436+
with:
437+
name: wheel-pecos-rslib-llvm-ubuntu-latest-x86_64
438+
path: ./pecos-rslib-llvm-wheel
439+
440+
- name: Install pecos-rslib and pecos-rslib-llvm
441+
run: |
442+
pip install ./pecos-rslib-wheel/*.whl
443+
pip install ./pecos-rslib-llvm-wheel/*.whl
376444
377445
- name: Install build dependencies
378446
run: pip install build
@@ -416,8 +484,16 @@ jobs:
416484
name: wheel-pecos-rslib-ubuntu-latest-x86_64
417485
path: ./pecos-rslib-wheel
418486

419-
- name: Install pecos-rslib
420-
run: pip install ./pecos-rslib-wheel/*.whl
487+
- name: Download pecos-rslib-llvm wheel
488+
uses: actions/download-artifact@v7
489+
with:
490+
name: wheel-pecos-rslib-llvm-ubuntu-latest-x86_64
491+
path: ./pecos-rslib-llvm-wheel
492+
493+
- name: Install pecos-rslib and pecos-rslib-llvm
494+
run: |
495+
pip install ./pecos-rslib-wheel/*.whl
496+
pip install ./pecos-rslib-llvm-wheel/*.whl
421497
422498
- name: Install build dependencies
423499
run: pip install build
@@ -464,7 +540,14 @@ jobs:
464540
echo "=== Downloaded artifacts structure ==="
465541
ls -la temp-artifacts/
466542
467-
# Move pecos-rslib wheels to distribution directory
543+
# Move pecos-rslib and pecos-rslib-llvm wheels to distribution directory
544+
mkdir -p dist/pecos-rslib-llvm
545+
for artifact in temp-artifacts/wheel-pecos-rslib-llvm-*/; do
546+
if [ -d "$artifact" ]; then
547+
echo "Processing $artifact"
548+
mv "$artifact"*.whl dist/pecos-rslib-llvm/ 2>/dev/null || true
549+
fi
550+
done
468551
for artifact in temp-artifacts/wheel-pecos-rslib-*/; do
469552
if [ -d "$artifact" ]; then
470553
echo "Processing $artifact"

0 commit comments

Comments
 (0)