Skip to content

Commit 7ae646b

Browse files
keithamuskeithamus
committed
implement preds.py as a Rust wheel with PyO3 & Maturin
This change is a proof of concept for how possible it would be to take individual python modules and re-implement them in Rust, in a "Ship of Theseus" style Rust rewrite. This takes some of the simplest parts of the codebase - a list of predicate free functions - and turns them into Rust checks. These functions become quite a bit more trivial in Rust as Rust implements many of these functions on `char`, so we simply delegate to those methods. These functions where chosen not because they're a slow path, but more because they are straightforward functions that return booleans and show demonstrate a good proof of concept. They certainly don't make things _slower_, however. The next steps would be to take some of the bigger functions in preds.py that do full string comparison (such as isXMLishTagname), and port those. This was avoided for now as these might be slightly more controversial, as we might want to include dependencies for fast string matching. I'm not very well versed in Python, this was mostly cobbled together using https://medium.com/@MatthieuL49/a-mixed-rust-python-project-24491e2af424 and https://colliery.io/blog/rust-python-pattern/ as guides. Some completely non empirical evaluation of timings, I ran the tests with/without Rust bindings: ```sh $ time BIKESHED_USE_RUST=0 ./bikeshed.py test --no-update Running tests |████████████████████| 502/502 [100%] in 3:04.5 (2.72/s) ✔ All tests passed. ________________________________________________________ Executed in 184.82 secs fish external usr time 176.82 secs 717.00 micros 176.82 secs sys time 1.99 secs 0.00 micros 1.99 secs $ time BIKESHED_USE_RUST=1 ./bikeshed.py test --no-update Running tests [R] |████████████████████| 502/502 [100%] in 3:04.4 (2.72/s) ✔ All tests passed. ________________________________________________________ Executed in 184.72 secs fish external usr time 177.16 secs 481.00 micros 177.16 secs sys time 1.96 secs 172.00 micros 1.96 secs ``` As expected both take essentially the same time. This may prove 1 of 2 things: - I'm an idiot and haven't done this right. - The FFI boundary between Python/Rust isn't costing us much (at least for simple checks like these).
1 parent 3a2640e commit 7ae646b

12 files changed

Lines changed: 493 additions & 5 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,12 +36,25 @@ jobs:
3636
uses: actions/setup-python@v5
3737
with:
3838
python-version: ${{ matrix.python-version }}
39+
- name: Set up Rust
40+
uses: dtolnay/rust-toolchain@stable
3941
- name: Install dependencies
4042
run: |
4143
pip install --upgrade pip wheel
4244
pip install --editable .
43-
- name: Test with bikeshed
45+
- name: Build Rust extension
46+
run: |
47+
pip install maturin
48+
maturin build --release
49+
pip install --force-reinstall --find-links rust/target/wheels bikeshed_rust
50+
- name: Test with bikeshed (Python mode)
51+
run: bikeshed --no-update test
52+
env:
53+
BIKESHED_USE_RUST: '0'
54+
- name: Test with bikeshed (Rust mode)
4455
run: bikeshed --no-update test
56+
env:
57+
BIKESHED_USE_RUST: '1'
4558

4659
lint:
4760

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,5 @@ node_modules/
1717
/playwright-report/
1818
/playwright/.cache/
1919
/env
20-
/docs/*.html
20+
/docs/*.html
21+
/rust/target/

‎README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,13 @@ though most such specs have switched their source file extensions to `.bs` now.
6767
Using `.src.html` in most text editors will display the file with HTML source formatting,
6868
which isn't generally what you want.)
6969

70+
Rust
71+
-----------
72+
73+
Bikeshed includes optional Rust extensions, in an effort to port some or all of the code into Rust.
74+
75+
To enable: `export BIKESHED_USE_RUST=1`
76+
7077
License
7178
-------
7279

‎bikeshed/h/parser/parser.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
from ... import config, constants, t
88
from ... import messages as m
9-
from . import preds
9+
from . import preds_wrapper as preds
1010
from .nodes import (
1111
Comment,
1212
Doctype,

‎bikeshed/h/parser/preds_wrapper.py‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
from __future__ import annotations
2+
3+
import os
4+
5+
_USE_RUST = os.environ.get("BIKESHED_USE_RUST", "").lower() in ("1", "true")
6+
7+
if _USE_RUST:
8+
try:
9+
import bikeshed_rust
10+
from . import preds as _preds
11+
12+
isASCII = bikeshed_rust.is_ascii
13+
isASCIIAlpha = bikeshed_rust.is_ascii_alpha
14+
isASCIIAlphanum = bikeshed_rust.is_ascii_alphanum
15+
isASCIILowerAlpha = bikeshed_rust.is_ascii_lower_alpha
16+
isASCIIUpperAlpha = bikeshed_rust.is_ascii_upper_alpha
17+
isAttrNameChar = bikeshed_rust.is_attr_name_char
18+
isControl = bikeshed_rust.is_control
19+
isDigit = bikeshed_rust.is_digit
20+
isHexDigit = bikeshed_rust.is_hex_digit
21+
isNoncharacter = bikeshed_rust.is_noncharacter
22+
isTagnameChar = bikeshed_rust.is_tagname_char
23+
isWhitespace = bikeshed_rust.is_whitespace
24+
25+
charRefs = _preds.charRefs
26+
xmlishTagnames = _preds.xmlishTagnames
27+
isXMLishTagname = _preds.isXMLishTagname
28+
except ImportError:
29+
from . import preds as _preds
30+
31+
charRefs = _preds.charRefs
32+
xmlishTagnames = _preds.xmlishTagnames
33+
isASCII = _preds.isASCII
34+
isASCIIAlpha = _preds.isASCIIAlpha
35+
isASCIIAlphanum = _preds.isASCIIAlphanum
36+
isASCIILowerAlpha = _preds.isASCIILowerAlpha
37+
isASCIIUpperAlpha = _preds.isASCIIUpperAlpha
38+
isAttrNameChar = _preds.isAttrNameChar
39+
isControl = _preds.isControl
40+
isDigit = _preds.isDigit
41+
isHexDigit = _preds.isHexDigit
42+
isNoncharacter = _preds.isNoncharacter
43+
isTagnameChar = _preds.isTagnameChar
44+
isWhitespace = _preds.isWhitespace
45+
isXMLishTagname = _preds.isXMLishTagname
46+
47+
else:
48+
from . import preds as _preds
49+
50+
charRefs = _preds.charRefs
51+
xmlishTagnames = _preds.xmlishTagnames
52+
isASCII = _preds.isASCII
53+
isASCIIAlpha = _preds.isASCIIAlpha
54+
isASCIIAlphanum = _preds.isASCIIAlphanum
55+
isASCIILowerAlpha = _preds.isASCIILowerAlpha
56+
isASCIIUpperAlpha = _preds.isASCIIUpperAlpha
57+
isAttrNameChar = _preds.isAttrNameChar
58+
isControl = _preds.isControl
59+
isDigit = _preds.isDigit
60+
isHexDigit = _preds.isHexDigit
61+
isNoncharacter = _preds.isNoncharacter
62+
isTagnameChar = _preds.isTagnameChar
63+
isWhitespace = _preds.isWhitespace
64+
isXMLishTagname = _preds.isXMLishTagname
65+
66+
__all__ = [
67+
"charRefs",
68+
"xmlishTagnames",
69+
"isASCII",
70+
"isASCIIAlpha",
71+
"isASCIIAlphanum",
72+
"isASCIILowerAlpha",
73+
"isASCIIUpperAlpha",
74+
"isAttrNameChar",
75+
"isControl",
76+
"isDigit",
77+
"isHexDigit",
78+
"isNoncharacter",
79+
"isTagnameChar",
80+
"isWhitespace",
81+
"isXMLishTagname",
82+
]

‎bikeshed/test.py‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,19 @@
1212
from . import messages as m
1313
from .Spec import Spec
1414

15+
16+
def _getTestTitle() -> str:
17+
try:
18+
from .h.parser import preds_wrapper
19+
# Check if we're using the Rust implementation
20+
if hasattr(preds_wrapper, 'isASCII'):
21+
module = getattr(preds_wrapper.isASCII, '__module__', '')
22+
if 'bikeshed_rust' in module:
23+
return "Running tests [R]"
24+
except Exception:
25+
pass
26+
return "Running tests"
27+
1528
if t.TYPE_CHECKING:
1629
import argparse
1730

@@ -101,7 +114,7 @@ def run(
101114
numPassed = 0
102115
total = 0
103116
fails = []
104-
pathProgress = alive_it(paths, dual_line=True, length=20)
117+
pathProgress = alive_it(paths, dual_line=True, length=20, title=_getTestTitle())
105118
try:
106119
for path in pathProgress:
107120
testName = testNameForPath(path)
@@ -149,7 +162,7 @@ def rebase(
149162
if len(paths) == 0:
150163
m.p("No tests were found.")
151164
return True
152-
pathProgress = alive_it(paths, dual_line=True, length=20)
165+
pathProgress = alive_it(paths, dual_line=True, length=20, title=_getTestTitle())
153166
try:
154167
for path in pathProgress:
155168
testName = testNameForPath(path)

‎pyproject.toml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,17 @@
1+
[build-system]
2+
requires = ["maturin>=1.0,<2.0"]
3+
build-backend = "maturin"
4+
5+
[project]
6+
name = "bikeshed"
7+
version = "0.1.0"
8+
requires-python = ">=3.9"
9+
10+
[tool.maturin]
11+
manifest-path = "rust/Cargo.toml"
12+
module-name = "bikeshed_rust"
13+
python-source = "bikeshed"
14+
115
[tool.black]
216
line-length = 120
317

‎rust/Cargo.lock‎

Lines changed: 163 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎rust/Cargo.toml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
[package]
2+
name = "bikeshed_rust"
3+
version = "0.1.0"
4+
edition = "2024"
5+
6+
[lib]
7+
name = "bikeshed_rust"
8+
crate-type = ["cdylib"]
9+
10+
[dependencies]
11+
pyo3 = { version = "0.26", features = ["extension-module"] }
12+
13+
[profile.release]
14+
lto = true
15+
codegen-units = 1
16+
strip = true

0 commit comments

Comments
 (0)