Skip to content

Commit 3ef42b9

Browse files
aclark4lifeaclark4life
committed
Add comprehensive Rust extension testing to Evergreen CI
- Add create_rust_extension_tasks() and create_rust_extension_variants() to generate_config.py - Test Rust extension on Linux (RHEL8), macOS ARM64, and Windows - Test with Python 3.10, 3.12, and 3.14 across different topologies - Add PR task for Python 3.10 on standalone to run on every PR - Create install-rust.sh script to install Rust toolchain and maturin - Update setup-tests.sh to install Rust when PYMONGO_BUILD_RUST is set - Add RUST_TESTING.md documentation explaining the test setup - Generated tasks run full BSON test suite with PYMONGO_USE_RUST=1 This ensures the Rust BSON extension maintains 100% compatibility with the C extension across all supported platforms and Python versions.
1 parent c6860b8 commit 3ef42b9

6 files changed

Lines changed: 409 additions & 0 deletions

File tree

‎.evergreen/RUST_TESTING.md‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# Rust Extension Testing in Evergreen CI
2+
3+
This document describes the Rust BSON extension testing setup in Evergreen CI.
4+
5+
## Overview
6+
7+
The Rust BSON extension (`bson/_rbson`) is an alternative implementation of the BSON encoding/decoding functionality, implemented in Rust using PyO3 instead of C. This provides memory safety and easier maintenance while maintaining 100% compatibility with the C extension.
8+
9+
## Test Configuration
10+
11+
### Automated Tests
12+
13+
Rust extension tests are automatically generated by `.evergreen/scripts/generate_config.py` and run on multiple platforms:
14+
15+
1. **Linux (RHEL 8)** - Primary platform, runs on every PR
16+
- Python 3.10 on standalone (PR task)
17+
- Python 3.12 on replica_set
18+
- Python 3.14 on sharded_cluster
19+
20+
2. **macOS ARM64** - Weekly batch
21+
- Same Python versions and topologies as Linux
22+
- Important for M1/M2 Mac compatibility
23+
24+
3. **Windows** - Weekly batch
25+
- Same Python versions and topologies as Linux
26+
- Important for cross-platform compatibility
27+
28+
### Environment Variables
29+
30+
The following environment variables control Rust extension building and testing:
31+
32+
- `PYMONGO_BUILD_RUST=1` - Build the Rust extension during installation
33+
- `PYMONGO_USE_RUST=1` - Use the Rust extension for BSON operations
34+
35+
These are automatically set by the Evergreen configuration for Rust test tasks.
36+
37+
## Test Execution Flow
38+
39+
1. **Setup Phase** (`.evergreen/scripts/setup-tests.sh`)
40+
- Detects `PYMONGO_BUILD_RUST` environment variable
41+
- Installs Rust toolchain via `.evergreen/scripts/install-rust.sh`
42+
- Installs maturin (Rust-Python build tool)
43+
44+
2. **Build Phase**
45+
- PyMongo is installed with `PYMONGO_BUILD_RUST=1`
46+
- This triggers the Rust extension build via `build.py`
47+
- The Rust extension is compiled and installed alongside the C extension
48+
49+
3. **Test Phase** (`.evergreen/scripts/run-tests.sh`)
50+
- Tests run with `PYMONGO_USE_RUST=1`
51+
- This causes PyMongo to use the Rust extension instead of C
52+
- Full BSON test suite runs to ensure 100% compatibility
53+
54+
## Manual Testing
55+
56+
### Local Testing with Just
57+
58+
```bash
59+
# Build the Rust extension
60+
just rust-build
61+
62+
# Install PyMongo with Rust extension
63+
just rust-install
64+
65+
# Run BSON tests with Rust extension
66+
just rust-test
67+
68+
# Verify Rust extension is loaded
69+
just rust-check
70+
```
71+
72+
### Local Testing with Evergreen Scripts
73+
74+
```bash
75+
# Set environment variables
76+
export PYMONGO_BUILD_RUST=1
77+
export PYMONGO_USE_RUST=1
78+
79+
# Run setup
80+
.evergreen/scripts/setup-tests.sh default_sync
81+
82+
# Run tests
83+
.evergreen/scripts/run-tests.sh
84+
```
85+
86+
## Test Coverage
87+
88+
The Rust extension tests run the full BSON test suite (`test/test_bson.py`) which includes:
89+
90+
- All BSON types (int, string, document, array, binary, ObjectId, datetime, etc.)
91+
- Edge cases and error handling
92+
- Encoding/decoding round-trips
93+
- Codec options and type preservation
94+
- Performance characteristics
95+
96+
## Performance Monitoring
97+
98+
While the Rust extension is currently ~5x slower than the C extension (see README.md for analysis), the tests ensure:
99+
100+
1. **Correctness** - 100% test pass rate
101+
2. **Compatibility** - Identical behavior to C extension
102+
3. **Stability** - No regressions in functionality
103+
104+
## Adding New Rust Tests
105+
106+
To add new test configurations:
107+
108+
1. Edit `.evergreen/scripts/generate_config.py`
109+
2. Modify `create_rust_extension_tasks()` to add new Python versions or topologies
110+
3. Modify `create_rust_extension_variants()` to add new platforms
111+
4. Run `python .evergreen/scripts/generate_config.py` to regenerate configs
112+
5. Commit the changes to `.evergreen/generated_configs/`
113+
114+
## Troubleshooting
115+
116+
### Rust Not Found
117+
118+
If tests fail with "Rust not found":
119+
- Check that `.evergreen/scripts/install-rust.sh` is executable
120+
- Verify the script runs successfully in the setup phase
121+
- Check Evergreen logs for Rust installation errors
122+
123+
### Build Failures
124+
125+
If the Rust extension fails to build:
126+
- Check that maturin is installed (`maturin --version`)
127+
- Verify Rust toolchain is up to date (`rustc --version`)
128+
- Check for compilation errors in the build logs
129+
- Ensure PyO3 dependencies are compatible
130+
131+
### Test Failures
132+
133+
If tests fail with Rust extension:
134+
- Verify `PYMONGO_USE_RUST=1` is set
135+
- Check that the Rust extension is actually loaded (run `just rust-check`)
136+
- Compare behavior with C extension (run tests without `PYMONGO_USE_RUST`)
137+
- Check for platform-specific issues (endianness, pointer size, etc.)
138+
139+
## References
140+
141+
- **Implementation**: `bson/_rbson/src/lib.rs`
142+
- **Build Script**: `bson/_rbson/build.sh`
143+
- **PR #2695**: Complete implementation history
144+
- **PYTHON-5683**: Investigation ticket
145+
- **Performance Analysis**: See README.md "Rust BSON Extension Performance Analysis" section

‎.evergreen/generated_configs/tasks.yml‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2581,6 +2581,84 @@ tasks:
25812581
- func: send dashboard data
25822582
tags: [perf]
25832583

2584+
# Rust extension tests
2585+
- name: test-rust-latest-python3.10-noauth-nossl-standalone
2586+
commands:
2587+
- func: run server
2588+
vars:
2589+
AUTH: noauth
2590+
SSL: nossl
2591+
TOPOLOGY: standalone
2592+
VERSION: latest
2593+
PYMONGO_BUILD_RUST: "1"
2594+
PYMONGO_USE_RUST: "1"
2595+
- func: run tests
2596+
vars:
2597+
AUTH: noauth
2598+
SSL: nossl
2599+
TOPOLOGY: standalone
2600+
VERSION: latest
2601+
PYMONGO_BUILD_RUST: "1"
2602+
PYMONGO_USE_RUST: "1"
2603+
TOOLCHAIN_VERSION: "3.10"
2604+
TEST_NAME: default_sync
2605+
tags:
2606+
- test-rust
2607+
- python-3.10
2608+
- standalone-noauth-nossl
2609+
- rust
2610+
- pr
2611+
- name: test-rust-latest-python3.12-noauth-ssl-replica-set
2612+
commands:
2613+
- func: run server
2614+
vars:
2615+
AUTH: noauth
2616+
SSL: ssl
2617+
TOPOLOGY: replica_set
2618+
VERSION: latest
2619+
PYMONGO_BUILD_RUST: "1"
2620+
PYMONGO_USE_RUST: "1"
2621+
- func: run tests
2622+
vars:
2623+
AUTH: noauth
2624+
SSL: ssl
2625+
TOPOLOGY: replica_set
2626+
VERSION: latest
2627+
PYMONGO_BUILD_RUST: "1"
2628+
PYMONGO_USE_RUST: "1"
2629+
TOOLCHAIN_VERSION: "3.12"
2630+
TEST_NAME: default_sync
2631+
tags:
2632+
- test-rust
2633+
- python-3.12
2634+
- replica_set-noauth-ssl
2635+
- rust
2636+
- name: test-rust-latest-python3.14-auth-ssl-sharded-cluster
2637+
commands:
2638+
- func: run server
2639+
vars:
2640+
AUTH: auth
2641+
SSL: ssl
2642+
TOPOLOGY: sharded_cluster
2643+
VERSION: latest
2644+
PYMONGO_BUILD_RUST: "1"
2645+
PYMONGO_USE_RUST: "1"
2646+
- func: run tests
2647+
vars:
2648+
AUTH: auth
2649+
SSL: ssl
2650+
TOPOLOGY: sharded_cluster
2651+
VERSION: latest
2652+
PYMONGO_BUILD_RUST: "1"
2653+
PYMONGO_USE_RUST: "1"
2654+
TOOLCHAIN_VERSION: "3.14"
2655+
TEST_NAME: default_sync
2656+
tags:
2657+
- test-rust
2658+
- python-3.14
2659+
- sharded_cluster-auth-ssl
2660+
- rust
2661+
25842662
# Search index tests
25852663
- name: test-search-index-helpers
25862664
commands:

‎.evergreen/generated_configs/variants.yml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -477,6 +477,40 @@ buildvariants:
477477
expansions:
478478
SUB_TEST_NAME: pyopenssl
479479

480+
# Rust extension tests
481+
- name: test-rust-extension
482+
tasks:
483+
- name: .test-rust
484+
display_name: Test Rust Extension
485+
run_on:
486+
- rhel87-small
487+
expansions:
488+
PYMONGO_BUILD_RUST: "1"
489+
PYMONGO_USE_RUST: "1"
490+
tags: [rust, pr]
491+
- name: test-rust-extension---macos-arm64
492+
tasks:
493+
- name: .test-rust !.pr
494+
display_name: Test Rust Extension - macOS ARM64
495+
run_on:
496+
- macos-14-arm64
497+
batchtime: 10080
498+
expansions:
499+
PYMONGO_BUILD_RUST: "1"
500+
PYMONGO_USE_RUST: "1"
501+
tags: [rust]
502+
- name: test-rust-extension---windows
503+
tasks:
504+
- name: .test-rust !.pr
505+
display_name: Test Rust Extension - Windows
506+
run_on:
507+
- windows-64-vsMulti-small
508+
batchtime: 10080
509+
expansions:
510+
PYMONGO_BUILD_RUST: "1"
511+
PYMONGO_USE_RUST: "1"
512+
tags: [rust]
513+
480514
# Search index tests
481515
- name: search-index-helpers-rhel8
482516
tasks:

‎.evergreen/scripts/generate_config.py‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1283,6 +1283,100 @@ def create_send_dashboard_data_func():
12831283
return "send dashboard data", cmds
12841284

12851285

1286+
def create_rust_extension_tasks():
1287+
"""Create tasks for testing the Rust BSON extension."""
1288+
tasks = []
1289+
# Test Rust extension on a subset of Python versions and topologies
1290+
# Focus on min, max, and latest stable Python versions
1291+
test_pythons = [MIN_MAX_PYTHON[0], "3.12", MIN_MAX_PYTHON[-1]]
1292+
1293+
for python, topology in zip_cycle(
1294+
test_pythons, ["standalone", "replica_set", "sharded_cluster"]
1295+
):
1296+
if "t" in python:
1297+
# Skip free-threaded Python for now (Rust extension may need updates)
1298+
continue
1299+
1300+
auth, ssl = get_standard_auth_ssl(topology)
1301+
tags = [
1302+
"test-rust",
1303+
f"python-{python}",
1304+
f"{topology}-{auth}-{ssl}",
1305+
"rust",
1306+
]
1307+
1308+
# Add PR tag for one configuration to run on every PR (Python 3.10 on standalone)
1309+
if python == MIN_MAX_PYTHON[0] and topology == "standalone":
1310+
tags.append("pr")
1311+
1312+
expansions = dict(
1313+
AUTH=auth,
1314+
SSL=ssl,
1315+
TOPOLOGY=topology,
1316+
VERSION="latest",
1317+
PYMONGO_BUILD_RUST="1",
1318+
PYMONGO_USE_RUST="1",
1319+
)
1320+
1321+
name = get_task_name("test-rust", python=python, **expansions)
1322+
server_func = FunctionCall(func="run server", vars=expansions)
1323+
test_vars = expansions.copy()
1324+
test_vars["TOOLCHAIN_VERSION"] = python
1325+
test_vars["TEST_NAME"] = "default_sync"
1326+
test_func = FunctionCall(func="run tests", vars=test_vars)
1327+
tasks.append(EvgTask(name=name, tags=tags, commands=[server_func, test_func]))
1328+
1329+
return tasks
1330+
1331+
1332+
def create_rust_extension_variants():
1333+
"""Create build variants for testing the Rust BSON extension."""
1334+
variants = []
1335+
1336+
# Test on Linux (primary platform) - runs on PRs
1337+
variant = create_variant(
1338+
[".test-rust"],
1339+
"Test Rust Extension",
1340+
host=DEFAULT_HOST,
1341+
tags=["rust", "pr"],
1342+
expansions=dict(
1343+
PYMONGO_BUILD_RUST="1",
1344+
PYMONGO_USE_RUST="1",
1345+
),
1346+
)
1347+
variants.append(variant)
1348+
1349+
# Test on macOS ARM64 (important for M1/M2 Macs)
1350+
variant = create_variant(
1351+
[".test-rust !.pr"],
1352+
"Test Rust Extension - macOS ARM64",
1353+
host=HOSTS["macos-arm64"],
1354+
tags=["rust"],
1355+
batchtime=BATCHTIME_WEEK,
1356+
expansions=dict(
1357+
PYMONGO_BUILD_RUST="1",
1358+
PYMONGO_USE_RUST="1",
1359+
),
1360+
)
1361+
variants.append(variant)
1362+
1363+
# Test on Windows (important for cross-platform compatibility)
1364+
variant = create_variant(
1365+
[".test-rust !.pr"],
1366+
"Test Rust Extension - Windows",
1367+
host=HOSTS["win64"],
1368+
tags=["rust"],
1369+
batchtime=BATCHTIME_WEEK,
1370+
expansions=dict(
1371+
PYMONGO_BUILD_RUST="1",
1372+
PYMONGO_USE_RUST="1",
1373+
),
1374+
)
1375+
variants.append(variant)
1376+
1377+
return variants
1378+
1379+
12861380
mod = sys.modules[__name__]
12871381
write_variants_to_file(mod)
12881382
write_tasks_to_file(mod)

0 commit comments

Comments
 (0)