Skip to content

Commit 718f33c

Browse files
claudeclaude
committed
feat: Add comprehensive Python bindings for terraphim_automata
This commit introduces a complete Python binding for the terraphim_automata Rust crate using PyO3, enabling fast autocomplete and text processing capabilities for Python applications. Key Features: - Full PyO3 bindings exposing core automata functionality - Autocomplete with FST-based prefix search - Fuzzy search using Jaro-Winkler and Levenshtein algorithms - Text matching and replacement with multiple link formats - Paragraph extraction based on term matches Components Added: 1. Rust Bindings (src/lib.rs): - PyAutocompleteIndex wrapper class - PyAutocompleteResult and PyMatched data classes - Functions: load_thesaurus, build_index, find_all_matches, replace_with_links, extract_paragraphs 2. Python Package (python/): - __init__.py: Package entry point with clean API exports - __init__.pyi: Complete type stubs for IDE support - Pythonic API with default arguments and clear docstrings 3. Comprehensive Tests (python/tests/): - test_autocomplete.py: 15 test cases for autocomplete functionality - test_matcher.py: 12 test cases for text matching/replacement - test_thesaurus.py: 11 test cases for thesaurus loading - conftest.py: Shared fixtures and test data 4. Performance Benchmarks (python/benchmarks/): - benchmark_autocomplete.py: Index building and search benchmarks - benchmark_matcher.py: Text processing benchmarks - Tests with small (100), medium (1K), large (10K) datasets 5. Documentation and Examples: - README.md: Comprehensive documentation with API reference - examples/basic_autocomplete.py: Basic usage patterns - examples/fuzzy_search.py: Fuzzy search demonstrations - examples/text_processing.py: Text replacement examples 6. Build Infrastructure: - pyproject.toml: Package metadata with maturin build system - Cargo.toml: PyO3 dependencies with abi3-py39 compatibility - scripts/build_and_test.sh: Automated build and test workflow 7. CI/CD Pipeline (.github/workflows/python-bindings.yml): - Multi-platform testing (Ubuntu, macOS, Windows) - Python versions: 3.9, 3.10, 3.11, 3.12 - Code quality checks: black, ruff, mypy - Automated wheel building for releases - PyPI publishing workflow with trusted publishing Technical Details: - Uses PyO3 0.23 with stable ABI (abi3-py39) for Python 3.9+ - Fully qualified imports to avoid naming conflicts - Type hints and stubs for full IDE integration - Comprehensive error handling with Python exceptions - Zero-copy operations where possible for performance Testing Coverage: - 38 unit tests covering all API functions - Multiple test scenarios including edge cases - Benchmark suite for performance validation - Fixtures for reusable test data Package Management: - Configured for uv package manager - Maturin for Rust-Python integration - Development dependencies isolated in optional group This implementation provides a production-ready Python interface to terraphim_automata with excellent performance, comprehensive testing, and professional documentation.
1 parent e1e5d61 commit 718f33c

19 files changed

Lines changed: 3180 additions & 0 deletions

‎.github/workflows/python-bindings.yml‎

Lines changed: 239 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
name: Python Bindings CI/CD
2+
3+
on:
4+
push:
5+
branches: [main, develop, "claude/**"]
6+
paths:
7+
- "crates/terraphim_automata_py/**"
8+
- "crates/terraphim_automata/**"
9+
- "crates/terraphim_types/**"
10+
- ".github/workflows/python-bindings.yml"
11+
pull_request:
12+
branches: [main, develop]
13+
paths:
14+
- "crates/terraphim_automata_py/**"
15+
- "crates/terraphim_automata/**"
16+
- "crates/terraphim_types/**"
17+
- ".github/workflows/python-bindings.yml"
18+
release:
19+
types: [published]
20+
21+
env:
22+
CARGO_TERM_COLOR: always
23+
RUST_BACKTRACE: 1
24+
25+
jobs:
26+
lint:
27+
name: Lint Python Code
28+
runs-on: ubuntu-latest
29+
steps:
30+
- uses: actions/checkout@v4
31+
32+
- name: Install uv
33+
uses: astral-sh/setup-uv@v5
34+
with:
35+
enable-cache: true
36+
37+
- name: Set up Python
38+
uses: actions/setup-python@v5
39+
with:
40+
python-version: "3.12"
41+
42+
- name: Install dependencies
43+
working-directory: crates/terraphim_automata_py
44+
run: |
45+
uv venv
46+
uv pip install black ruff mypy
47+
48+
- name: Check formatting with Black
49+
working-directory: crates/terraphim_automata_py
50+
run: uv run black --check python/
51+
52+
- name: Lint with Ruff
53+
working-directory: crates/terraphim_automata_py
54+
run: uv run ruff check python/
55+
56+
- name: Type check with mypy
57+
working-directory: crates/terraphim_automata_py
58+
run: uv run mypy python/terraphim_automata/ --ignore-missing-imports
59+
continue-on-error: true
60+
61+
test:
62+
name: Test on ${{ matrix.os }} - Python ${{ matrix.python-version }}
63+
runs-on: ${{ matrix.os }}
64+
strategy:
65+
fail-fast: false
66+
matrix:
67+
os: [ubuntu-latest, macos-latest, windows-latest]
68+
python-version: ["3.9", "3.10", "3.11", "3.12"]
69+
70+
steps:
71+
- uses: actions/checkout@v4
72+
73+
- name: Set up Rust
74+
uses: dtolnay/rust-toolchain@stable
75+
76+
- name: Cache Rust dependencies
77+
uses: Swatinem/rust-cache@v2
78+
with:
79+
workspaces: "crates/terraphim_automata_py -> target"
80+
81+
- name: Install uv
82+
uses: astral-sh/setup-uv@v5
83+
with:
84+
enable-cache: true
85+
86+
- name: Set up Python ${{ matrix.python-version }}
87+
uses: actions/setup-python@v5
88+
with:
89+
python-version: ${{ matrix.python-version }}
90+
91+
- name: Install maturin
92+
run: uv pip install --system maturin
93+
94+
- name: Build Python package
95+
working-directory: crates/terraphim_automata_py
96+
run: maturin develop
97+
98+
- name: Install test dependencies
99+
working-directory: crates/terraphim_automata_py
100+
run: uv pip install --system pytest pytest-cov
101+
102+
- name: Run tests
103+
working-directory: crates/terraphim_automata_py
104+
run: pytest python/tests/ -v --cov=terraphim_automata --cov-report=xml --cov-report=term
105+
106+
- name: Upload coverage to Codecov
107+
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12'
108+
uses: codecov/codecov-action@v4
109+
with:
110+
files: ./crates/terraphim_automata_py/coverage.xml
111+
flags: python-bindings
112+
name: python-${{ matrix.python-version }}
113+
114+
benchmark:
115+
name: Benchmark Performance
116+
runs-on: ubuntu-latest
117+
steps:
118+
- uses: actions/checkout@v4
119+
120+
- name: Set up Rust
121+
uses: dtolnay/rust-toolchain@stable
122+
123+
- name: Cache Rust dependencies
124+
uses: Swatinem/rust-cache@v2
125+
126+
- name: Install uv
127+
uses: astral-sh/setup-uv@v5
128+
with:
129+
enable-cache: true
130+
131+
- name: Set up Python
132+
uses: actions/setup-python@v5
133+
with:
134+
python-version: "3.12"
135+
136+
- name: Install maturin
137+
run: uv pip install --system maturin
138+
139+
- name: Build Python package (release mode)
140+
working-directory: crates/terraphim_automata_py
141+
run: maturin develop --release
142+
143+
- name: Install benchmark dependencies
144+
working-directory: crates/terraphim_automata_py
145+
run: uv pip install --system pytest pytest-benchmark
146+
147+
- name: Run benchmarks
148+
working-directory: crates/terraphim_automata_py
149+
run: |
150+
pytest python/benchmarks/ -v --benchmark-only \
151+
--benchmark-json=benchmark-results.json \
152+
--benchmark-columns=min,max,mean,stddev,median,ops
153+
154+
- name: Store benchmark results
155+
uses: actions/upload-artifact@v4
156+
with:
157+
name: benchmark-results
158+
path: crates/terraphim_automata_py/benchmark-results.json
159+
160+
build-wheels:
161+
name: Build wheels on ${{ matrix.os }}
162+
runs-on: ${{ matrix.os }}
163+
if: github.event_name == 'release'
164+
strategy:
165+
fail-fast: false
166+
matrix:
167+
os: [ubuntu-latest, macos-latest, windows-latest]
168+
169+
steps:
170+
- uses: actions/checkout@v4
171+
172+
- name: Set up Rust
173+
uses: dtolnay/rust-toolchain@stable
174+
175+
- name: Install uv
176+
uses: astral-sh/setup-uv@v5
177+
178+
- name: Build wheels
179+
uses: PyO3/maturin-action@v1
180+
with:
181+
working-directory: crates/terraphim_automata_py
182+
args: --release --out dist --interpreter 3.9 3.10 3.11 3.12
183+
sccache: "true"
184+
manylinux: auto
185+
186+
- name: Upload wheels
187+
uses: actions/upload-artifact@v4
188+
with:
189+
name: wheels-${{ matrix.os }}
190+
path: crates/terraphim_automata_py/dist
191+
192+
build-sdist:
193+
name: Build source distribution
194+
runs-on: ubuntu-latest
195+
if: github.event_name == 'release'
196+
steps:
197+
- uses: actions/checkout@v4
198+
199+
- name: Build sdist
200+
uses: PyO3/maturin-action@v1
201+
with:
202+
working-directory: crates/terraphim_automata_py
203+
command: sdist
204+
args: --out dist
205+
206+
- name: Upload sdist
207+
uses: actions/upload-artifact@v4
208+
with:
209+
name: sdist
210+
path: crates/terraphim_automata_py/dist
211+
212+
publish:
213+
name: Publish to PyPI
214+
runs-on: ubuntu-latest
215+
if: github.event_name == 'release'
216+
needs: [lint, test, build-wheels, build-sdist]
217+
environment:
218+
name: pypi
219+
url: https://pypi.org/p/terraphim-automata
220+
permissions:
221+
id-token: write
222+
223+
steps:
224+
- uses: actions/download-artifact@v4
225+
with:
226+
pattern: wheels-*
227+
path: dist
228+
merge-multiple: true
229+
230+
- uses: actions/download-artifact@v4
231+
with:
232+
name: sdist
233+
path: dist
234+
235+
- name: Publish to PyPI
236+
uses: PyO3/maturin-action@v1
237+
with:
238+
command: upload
239+
args: --non-interactive --skip-existing dist/*

‎Cargo.lock‎

Lines changed: 80 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎crates/terraphim_automata_py/.gitignore‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Python
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
*.so
6+
.Python
7+
build/
8+
develop-eggs/
9+
dist/
10+
downloads/
11+
eggs/
12+
.eggs/
13+
lib/
14+
lib64/
15+
parts/
16+
sdist/
17+
var/
18+
wheels/
19+
*.egg-info/
20+
.installed.cfg
21+
*.egg
22+
23+
# Virtual environments
24+
venv/
25+
env/
26+
ENV/
27+
.venv
28+
29+
# Testing
30+
.pytest_cache/
31+
.coverage
32+
coverage.xml
33+
htmlcov/
34+
.tox/
35+
.hypothesis/
36+
benchmark-results.json
37+
38+
# IDEs
39+
.vscode/
40+
.idea/
41+
*.swp
42+
*.swo
43+
*~
44+
.DS_Store
45+
46+
# Rust
47+
target/
48+
Cargo.lock
49+
50+
# Maturin
51+
*.whl

0 commit comments

Comments
 (0)