Skip to content

[codex] Add optional Rust retrieval backend and release path#19

Draft
brodieacc wants to merge 4 commits into
codex/rust-retrieval-prototypefrom
codex/rust-retrieval-backend
Draft

[codex] Add optional Rust retrieval backend and release path#19
brodieacc wants to merge 4 commits into
codex/rust-retrieval-prototypefrom
codex/rust-retrieval-backend

Conversation

@brodieacc
Copy link
Copy Markdown
Contributor

What changed

  • add opt-in search_backend="rust" support for ItemStore and AssocStore on the retrieval modes that benchmarked well
  • add a public comparison example and performance docs for the native backend
  • vendor the native library into wheels and prefer the packaged _native path at runtime
  • add wheel packaging via setup.py/MANIFEST.in
  • switch the release path to cibuildwheel, including a real manylinux build
  • add a reusable installed-wheel smoke test and a manual TestPyPI publish-and-verify workflow

Why

The prototype showed a clear win for discrete and sparse retrieval kernels. This PR turns that into a shippable, opt-in backend with an actual distribution path, while keeping NumPy as the default and fallback behavior.

Validation

  • uv run --extra dev pytest --no-cov
  • uv run --extra dev mypy holovec
  • uv run --extra dev ruff check holovec tests benchmarks setup.py scripts
  • CIBW_PLATFORM=linux uvx cibuildwheel --output-dir wheelhouse
  • uvx twine check wheelhouse/*
  • HOLOVEC_BUILD_RUST=1 uv build --wheel
  • HOLOVEC_SKIP_RUST_BUILD=1 uv build --sdist
  • clean wheel-install smoke via python scripts/release_smoke_rust_backend.py

Stack

This PR is stacked on codex/rust-retrieval-prototype.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant