Skip to content

Commit 171401d

Browse files
sidmohan0claude
andcommitted
Fix CI failures and clarify README install instructions
- Python CI: use `pip install ".[dev]"` instead of `maturin develop` (which requires a virtualenv that CI doesn't create) - WASM: add `getrandom` with `js` feature to fix wasm32 build - Benchmarks: remove invalid `--output-format bencher` flag - README: add explicit `pip install .` install step, clarify that maturin compiles Rust behind the scenes, add dev setup section Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 479fedd commit 171401d

3 files changed

Lines changed: 24 additions & 29 deletions

File tree

.github/workflows/ci.yml

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ jobs:
3636
name: Python Build & Test
3737
runs-on: ${{ matrix.os }}
3838
strategy:
39+
fail-fast: false
3940
matrix:
4041
os: [ubuntu-latest, macos-latest]
4142
python-version: ["3.10", "3.11", "3.12", "3.13"]
@@ -46,11 +47,8 @@ jobs:
4647
with:
4748
python-version: ${{ matrix.python-version }}
4849

49-
- name: Install dependencies
50-
run: pip install maturin pytest
51-
52-
- name: Build and install (editable)
53-
run: maturin develop
50+
- name: Install package
51+
run: pip install ".[dev]"
5452

5553
- name: Run Python tests
5654
run: pytest -v
@@ -79,4 +77,4 @@ jobs:
7977
- uses: Swatinem/rust-cache@v2
8078

8179
- name: Run benchmarks
82-
run: cargo bench --package datafog-core -- --output-format bencher
80+
run: cargo bench --package datafog-core

README.md

Lines changed: 19 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -4,27 +4,23 @@ Fast PII detection and anonymization, built in Rust with Python and WASM binding
44

55
DataFog detects structured PII (emails, phone numbers, SSNs, credit cards, IPs, dates of birth, ZIP codes) using compiled regex patterns, and optionally detects soft PII (names, organizations, addresses) using a GLiNER ONNX model via NER. The regex engine runs in microseconds per kilobyte with zero external dependencies.
66

7-
## Prerequisites
7+
## Install (Python)
88

9-
- **Rust** (stable) &mdash; install via [rustup](https://rustup.rs/)
10-
- **Python 3.9+** &mdash; for the Python bindings
11-
- **maturin** &mdash; for building the Rust-to-Python bridge
9+
DataFog is a Rust library with Python bindings built by [maturin](https://www.maturin.rs/). You need a Rust toolchain to install from source.
1210

13-
## Quick start (Python)
11+
**Prerequisites:** [Rust](https://rustup.rs/) (stable) and Python 3.9+.
1412

1513
```bash
14+
# From PyPI (once published)
15+
pip install datafog
16+
17+
# From source
1618
git clone https://github.com/DataFog/datafog.git
1719
cd datafog
18-
19-
python3 -m venv .venv
20-
source .venv/bin/activate # Windows: .venv\Scripts\activate
21-
pip install maturin pytest
22-
23-
maturin develop # builds Rust, installs editable Python package
24-
pytest -v # runs the test suite
20+
pip install .
2521
```
2622

27-
Then in Python:
23+
That's it. `pip install .` compiles the Rust code and installs DataFog as a Python package. No separate Rust build step needed — maturin handles it behind the scenes.
2824

2925
```python
3026
from datafog import DataFog, detect, anonymize_text
@@ -43,7 +39,7 @@ fog = DataFog()
4339
results = fog.detect_batch(["john@test.com", "555-123-4567", "no pii here"])
4440
```
4541

46-
## Quick start (Rust)
42+
## Install (Rust)
4743

4844
Add to your `Cargo.toml`:
4945

@@ -71,8 +67,8 @@ The NER engine uses a [GLiNER](https://github.com/urchade/GLiNER) ONNX model to
7167
Building with NER pulls in `gline-rs` + ONNX Runtime (~50 MB binary size increase).
7268

7369
```bash
74-
# Build the Python package with NER + model auto-download
75-
maturin develop --features full
70+
# Install from source with NER + model auto-download
71+
pip install . --config-settings="build-args=--features full"
7672
```
7773

7874
```python
@@ -152,19 +148,19 @@ datafog/
152148
## Development
153149

154150
```bash
155-
# Run Rust tests
156-
cargo test --workspace
151+
git clone https://github.com/DataFog/datafog.git
152+
cd datafog
157153

158-
# Run clippy and check formatting
154+
# Rust
155+
cargo test --workspace
159156
cargo clippy --workspace -- -D warnings
160157
cargo fmt --all -- --check
161-
162-
# Run benchmarks
163158
cargo bench --package datafog-core
164159

165-
# Build and test Python
160+
# Python
161+
python3 -m venv .venv
166162
source .venv/bin/activate
167-
maturin develop
163+
pip install ".[dev]" # installs datafog + maturin + pytest
168164
pytest -v
169165
```
170166

crates/datafog-wasm/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ wasm-bindgen = "0.2"
1616
serde = { workspace = true }
1717
serde_json = { workspace = true }
1818
serde-wasm-bindgen = "0.6"
19+
getrandom = { version = "0.2", features = ["js"] }
1920

2021
[dev-dependencies]
2122
wasm-bindgen-test = "0.3"

0 commit comments

Comments
 (0)