feat: embed Rust backend via maturin build system (#17)

* refactor: move gamecov-core Rust crate into monorepo

* refactor: merge gamecov-core into single maturin-built package

Replace hatchling with maturin as the build backend, embedding the Rust
extension (gamecov-core) directly into the gamecov package. The compiled
module is now installed as gamecov._gamecov_core.

- Move Rust sources from gamecov-core/src/ to src/
- Add Cargo.toml at repo root with module-name config
- Add type stub _gamecov_core.pyi (eliminates type: ignore)
- Update CI to install Rust toolchain before build
- Delete gamecov-core/ subdirectory

Users can now install the full package with: pip install .

* doc: rust code

* feat: add pytest-benchmark suite for Python vs Rust monitor comparison

* add python v.s. rust performance benchmark

* ci: add Rust fmt, clippy, and test to CI and pre-commit hooks

* doc: update for rust dev

* doc: add Rust vs Python benchmark results to README

* chore: bump version to 0.2.0

Author: EYH0602

.github/workflows/pytest.yml

@@ -3,19 +3,22 @@ name: Unit Testing
 on: [pull_request]
 
 jobs:
-  unittest:
-    runs-on: ubuntu-latest
+    unittest:
+        runs-on: ubuntu-latest
 
-    steps:
-      - uses: actions/checkout@v2
-        with:
-          fetch-depth: 1
+        steps:
+            - uses: actions/checkout@v4
+              with:
+                  fetch-depth: 1
 
-      - name: Install uv
-        uses: astral-sh/setup-uv@v5
+            - name: Install Rust
+              uses: dtolnay/rust-toolchain@stable
 
-      - name: Set up Python
-        run: uv sync
+            - name: Install uv
+              uses: astral-sh/setup-uv@v5
 
-      - name: Run Unit Tests
-        run: uv run pytest -n auto
+            - name: Set up Python
+              run: uv sync
+
+            - name: Run Unit Tests
+              run: uv run pytest -n auto

.github/workflows/rust.yml

@@ -0,0 +1,29 @@
+name: Rust CI
+
+on: [pull_request]
+
+jobs:
+    rust-ci:
+        runs-on: ubuntu-latest
+
+        steps:
+            - uses: actions/checkout@v4
+              with:
+                  fetch-depth: 1
+
+            - name: Install Rust toolchain
+              uses: dtolnay/rust-toolchain@stable
+              with:
+                  components: rustfmt, clippy
+
+            - name: Rust cache
+              uses: Swatinem/rust-cache@v2
+
+            - name: Check formatting
+              run: cargo fmt --all -- --check
+
+            - name: Clippy
+              run: cargo clippy --all-targets --all-features -- -D warnings
+
+            - name: Tests
+              run: cargo test --all-features

.gitignore

@@ -15,4 +15,8 @@ assets/
 .vscode/
 .hypothesis/
 prof/
-.coverage
+.coverage
+
+# Rust build artifacts
+target/
+Cargo.lock

.pre-commit-config.yaml

@@ -19,3 +19,25 @@ repos:
           - id: ruff
             args: [--fix]
           - id: ruff-format
+
+    - repo: local
+      hooks:
+          - id: cargo-fmt
+            name: cargo fmt
+            entry: cargo fmt --all
+            language: system
+            types: [rust]
+            pass_filenames: false
+          - id: cargo-clippy
+            name: cargo clippy
+            entry: cargo clippy --all-targets --all-features -- -D warnings
+            language: system
+            types: [rust]
+            pass_filenames: false
+          - id: cargo-test
+            name: cargo test
+            entry: cargo test --all-features
+            language: system
+            types: [rust]
+            pass_filenames: false
+            stages: [pre-push]

AGENTS.md

@@ -13,20 +13,29 @@ Future metrics (e.g., audio coverage, state-graph coverage) will follow the same
 
 ```
 .
+├── Cargo.toml                   # Rust manifest (maturin builds the extension)
+├── pyproject.toml               # Project metadata, maturin build backend
 ├── src/
 │   ├── main.py                  # CLI entry point (Typer)
+│   ├── lib.rs                   # PyO3 module entry point (Rust)
+│   ├── bktree.rs                # BK-tree<u64> with POPCNT Hamming distance
+│   ├── unionfind.rs             # Flat Vec-based union-find
+│   ├── monitor.rs               # CoverageTracker (BK-tree + UnionFind combined)
 │   └── gamecov/
 │       ├── __init__.py          # Public API re-exports
+│       ├── _gamecov_core.pyi    # Type stub for Rust extension
 │       ├── cov_base.py          # Abstract protocols: CoverageItem, Coverage, CoverageMonitor
 │       ├── frame.py             # Frame dataclass (PIL Image wrapper with average-hash)
 │       ├── dedup.py             # Deduplication algorithms (pHash, SSIM [deprecated])
-│       ├── frame_cov.py         # FrameCoverage, FrameMonitor, BKFrameMonitor, BK-tree, UnionFind
+│       ├── frame_cov.py         # FrameCoverage, FrameMonitor, BKFrameMonitor, RustBKFrameMonitor, BK-tree, UnionFind
 │       ├── loader.py            # MP4 loading: bulk, lazy (generator), last-n
 │       ├── writer.py            # MP4 writing: imageio and OpenCV backends
 │       ├── stitch.py            # Panorama stitching of unique frames
 │       ├── generator.py         # Hypothesis strategies for property-based testing
 │       ├── env.py               # Runtime config (RADIUS env var)
 │       └── py.typed             # PEP 561 marker
+├── rust-tests/
+│   └── prop_tests.rs            # Rust proptest property-based tests
 ├── tests/
 │   ├── test_generators.py       # Frame/FrameList generation strategies
 │   ├── test_dedup.py            # Dedup monotonicity properties
@@ -35,19 +44,34 @@ Future metrics (e.g., audio coverage, state-graph coverage) will follow the same
 │   ├── test_load_write_assets.py# Differential tests across loaders on real videos
 │   ├── test_monotone.py         # Coverage monotonicity (FrameMonitor & BKFrameMonitor)
 │   ├── test_BK_frame_monitor.py # Differential: FrameMonitor vs BKFrameMonitor
+│   ├── test_rust_frame_monitor.py # Differential & monotonicity: BKFrameMonitor vs RustBKFrameMonitor
 │   └── test_monotone_smb.py     # Real-world monotonicity on SMB dataset
+├── benchmarks/
+│   ├── conftest.py              # Session-scoped fixtures (pre-generated FrameCoverage)
+│   └── test_bench_monitor.py    # Python vs Rust monitor throughput benchmarks
 ├── assets/
 │   ├── videos/                  # Small sample MP4s for integration tests
 │   └── smb/                     # Super Smash Bros recordings for stress tests
 ├── docs/
 │   └── design.md                # Architecture and design documentation
-├── pyproject.toml               # Project metadata, dependencies, tool configs
-├── .pre-commit-config.yaml      # Pre-commit hooks
-├── .github/workflows/           # CI: pytest, mypy, ruff, pylint
+├── rustfmt.toml                 # Rust formatting config
+├── .pre-commit-config.yaml      # Pre-commit hooks (Python + Rust)
+├── .github/workflows/           # CI: pytest, mypy, ruff, pylint, rust (fmt/clippy/test)
 ├── AGENTS.md                    # This file
 └── README.md                    # Human-facing documentation
 ```
 
+### Rust extension (gamecov-core)
+
+The Rust extension is built as part of the package via maturin. The compiled
+module is installed as `gamecov._gamecov_core` and provides high-performance
+replacements for the BK-tree, union-find, and coverage tracker.
+
+Build the package (includes Rust compilation): `uv sync` or `pip install .`
+Run Rust tests independently: `cargo test`
+Check Rust formatting: `cargo fmt --all -- --check`
+Run Rust linting: `cargo clippy --all-targets --all-features -- -D warnings`
+
 ## Design
 
 See [docs/design.md](docs/design.md) for the coverage framework architecture, frame coverage pipeline, BK-tree optimization, and loading strategies.
@@ -59,7 +83,7 @@ See [docs/design.md](docs/design.md) for the coverage framework architecture, fr
 | `cov_base.py` | `CoverageItem`, `Coverage[T]`, `CoverageMonitor[T]` protocols/ABC |
 | `frame.py` | `Frame` dataclass (PIL Image + average-hash) |
 | `dedup.py` | `is_dup()`, `dedup_unique_frames()`, `dedup_unique_hashes()`, `ssim_dedup()` [deprecated] |
-| `frame_cov.py` | `FrameCoverage`, `FrameMonitor`, `BKFrameMonitor`, `get_frame_cov()`, `_UnionFind`, `_BKTree` |
+| `frame_cov.py` | `FrameCoverage`, `FrameMonitor`, `BKFrameMonitor`, `RustBKFrameMonitor`, `get_frame_cov()`, `_UnionFind`, `_BKTree` |
 | `loader.py` | `load_mp4()`, `load_mp4_lazy()`, `load_mp4_last_n()` |
 | `writer.py` | `write_mp4()`, `write_mp4_cv2()` |
 | `stitch.py` | `stitch_images()` (panorama via AffineStitcher) |
@@ -80,7 +104,8 @@ See [docs/design.md](docs/design.md) for the coverage framework architecture, fr
 | `opencv-python` | Color conversion, video writing, image processing |
 | `scikit-image` | SSIM metric (deprecated path) |
 | `stitching` | Panorama stitching via OpenCV features |
-| `numpy`, `numba` | Numerical arrays, optional JIT acceleration |
+| `numpy` | Numerical arrays |
+| `gamecov._gamecov_core` | Built-in Rust extension: BK-tree, union-find, coverage tracker (PyO3/maturin) |
 | `returns` | Functional `Result` type for error handling |
 | `deprecated` | `@deprecated` decorator |
 | `typer-slim` | CLI framework |
@@ -93,6 +118,7 @@ See [docs/design.md](docs/design.md) for the coverage framework architecture, fr
 | `mypy` | Static type checking (strict mode, returns plugin) |
 | `ruff` | Linting and formatting |
 | `pre-commit` | Pre-commit hook runner |
+| `pytest-benchmark` | Performance benchmarking (Python vs Rust) |
 | `pytest-xdist` | Parallel test execution (`-n auto`) |
 | `pytest-cov` | Coverage reporting |
 | `pytest-profiling` | Performance profiling |
@@ -110,6 +136,7 @@ uv run pytest -n auto
 - **Integration** (real assets): `test_load_n.py`, `test_load_write_assets.py`
 - **Monotonicity**: `test_monotone.py` (random data), `test_monotone_smb.py` (real SMB recordings)
 - **Differential**: `test_BK_frame_monitor.py` (FrameMonitor vs BKFrameMonitor produce identical results)
+- **Rust backend**: `test_rust_frame_monitor.py` (differential Python vs Rust, order-independence, monotonicity)
 
 Some tests require assets in `assets/videos/` or `assets/smb/` and will skip if missing.
 
@@ -118,6 +145,24 @@ Some tests require assets in `assets/videos/` or `assets/smb/` and will skip if
 - `RADIUS` — Hamming distance threshold (default `5`).
 - `N_MAX` — Maximum number of recordings to process in monotonicity tests (default `100`).
 
+## Benchmarks
+
+```bash
+# Run benchmarks (disabled by default during normal test runs)
+uv run pytest benchmarks/ --benchmark-enable
+
+# Group output by backend for side-by-side comparison
+uv run pytest benchmarks/ --benchmark-enable --benchmark-group-by=param:backend
+
+# Save results for later comparison
+uv run pytest benchmarks/ --benchmark-enable --benchmark-save=baseline
+uv run pytest benchmarks/ --benchmark-enable --benchmark-compare=baseline
+```
+
+Benchmarks live in `benchmarks/` and are excluded from the normal test suite.
+They compare `BKFrameMonitor` (Python) vs `RustBKFrameMonitor` (Rust) throughput
+at the monitor level (`add_cov`/`is_seen` operations).
+
 ## Development
 
 - Before start working, refresh your knowledge from contents in `.agents` first.
@@ -129,7 +174,8 @@ Some tests require assets in `assets/videos/` or `assets/smb/` and will skip if
   Local variables' types are optional as long as the types can be easily inferred.
 - Use f-strings for string interpolation.
 - Use `TypedDict`, `Literal`, `Protocol`, and `TypeVar` from `typing` module when appropriate.
-- Always run `mypy`, and `ruff` to ensure code quality after updating code in `src/`.
+- Always run `mypy` and `ruff` to ensure code quality after updating Python code in `src/`.
+- Always run `cargo fmt`, `cargo clippy -- -D warnings`, and `cargo test` after updating Rust code in `src/`.
 - Never commit changes or create PRs. Suggest commit messages to the human developer for review after your changes to the codebase.
 - Always use `typer` to handle CLI commands.
 

Cargo.toml

@@ -0,0 +1,23 @@
+[package]
+name = "gamecov-core"
+version = "0.2.0"
+edition = "2021"
+description = "Rust-accelerated core for gamecov frame coverage monitoring"
+
+[lib]
+name = "gamecov_core"
+crate-type = ["cdylib", "rlib"]
+
+[dependencies]
+pyo3 = { version = "0.23", features = ["extension-module"] }
+
+[dev-dependencies]
+proptest = "1"
+
+[[test]]
+name = "prop_tests"
+path = "rust-tests/prop_tests.rs"
+
+[profile.release]
+lto = "fat"
+codegen-units = 1

README.md

@@ -25,7 +25,7 @@ Two frames are considered duplicates if the Hamming distance between their perce
 
 ## Installation
 
-Requires Python >= 3.11.
+Requires Python >= 3.11 and a [Rust toolchain](https://rustup.rs/) (for building from source).
 
 ### As a package
 
@@ -41,6 +41,8 @@ Or with pip:
 pip install git+https://github.com/SecurityLab-UCD/gamecov.git
 ```
 
+This builds both the Python package and the embedded Rust extension (`gamecov._gamecov_core`) in a single step.
+
 ### For development
 
 Clone the repo and sync dependencies:
@@ -79,6 +81,24 @@ print(f"Total unique frames: {len(monitor.item_seen)}")
 print(f"Unique paths: {len(monitor.path_seen)}")
 ```
 
+### Rust-accelerated monitor
+
+`RustBKFrameMonitor` provides the same interface as `BKFrameMonitor`,
+backed by an embedded Rust extension for significantly higher throughput:
+
+```python
+from gamecov import FrameCoverage, RustBKFrameMonitor
+
+monitor = RustBKFrameMonitor()  # same API as BKFrameMonitor
+
+for recording in recordings:
+    cov = FrameCoverage(recording)
+    if not monitor.is_seen(cov):
+        monitor.add_cov(cov)
+
+print(f"Coverage components: {monitor.coverage_count}")
+```
+
 ### CLI
 
 ```bash
@@ -90,8 +110,12 @@ uv run python src/main.py --input-mp4-path path/to/video.mp4 --confidence-thresh
 
 ### Prerequisites
 
+- Python >= 3.11
+- [Rust toolchain](https://rustup.rs/) (stable)
+- [uv](https://docs.astral.sh/uv/)
+
 ```bash
-# Install dependencies
+# Install dependencies (builds the Rust extension automatically)
 uv sync
 
 # Install pre-commit hooks
@@ -101,11 +125,24 @@ uv run pre-commit install
 ### Running Tests
 
 ```bash
-# Run all tests in parallel
+# Run all Python tests in parallel
 uv run pytest -n auto
 
 # Run with coverage
 uv run pytest -n auto --cov=gamecov
+
+# Run Rust unit and property tests
+cargo test
+```
+
+### Benchmarks
+
+```bash
+# Compare Python vs Rust monitor throughput
+uv run pytest benchmarks/ --benchmark-enable
+
+# Side-by-side grouped by backend
+uv run pytest benchmarks/ --benchmark-enable --benchmark-group-by=param:backend
 ```
 
 ### Code Quality
@@ -121,3 +158,29 @@ uv run ruff check src/
 ### CI
 
 GitHub Actions runs four checks on every PR: `pytest`, `mypy`, `ruff`, and `pylint`.
+The CI workflow installs the Rust toolchain before building.
+
+## Performance: Rust vs Python Backend
+
+The embedded Rust extension (`RustBKFrameMonitor`) provides significant speedups
+over the pure-Python `BKFrameMonitor` for the core `add_cov`/`is_seen` monitor operations.
+The advantage grows with workload size as the BK-tree and union-find structures scale.
+
+Benchmark results (mean time per iteration, lower is better):
+
+| Recordings | Python (ms) | Rust (ms) | Speedup |
+| ---------- | ----------- | --------- | ------- |
+| 10         | 4.04        | 2.31      | 1.75x   |
+| 50         | 42.95       | 15.00     | 2.86x   |
+| 200        | 424.36      | 111.03    | 3.82x   |
+| 500        | 2,349.74    | 549.40    | 4.28x   |
+
+The Rust backend achieves **1.8x -- 4.3x** speedup,
+with larger gains at higher workloads where BK-tree traversal and union-find operations dominate.
+Each recording contains randomly generated `FrameCoverage` objects with perceptual hashes.
+
+Reproduce these results with:
+
+```bash
+uv run pytest benchmarks/ --benchmark-enable --benchmark-group-by=param:backend
+```