Commit 3ebed59
feat(python): add WASM evaluator and 3-way comparison benchmarks (#73)
## Summary
- Add `flagd_evaluator_wasm` package: a Python WASM-based flag evaluator
using wasmtime-py that mirrors the Go (wazero) and Java (Chicory)
implementations
- Host-side context filtering, pre-evaluated caching, and index-based
evaluation — same optimization pipeline as Go/Java
- 19 tests mirroring the existing PyO3 test suite
- 3-way head-to-head benchmarks: PyO3 (Rust native) vs WASM (wasmtime)
vs pure Python (panzi-json-logic)
## Benchmark Results (8 scenarios)
| Scenario | PyO3 | WASM | Python | Winner |
|----------|------|------|--------|--------|
| E1: Static, empty ctx | 140 ns | 350 ns | 250 ns | PyO3 |
| E2: Static, small ctx | 240 ns | 440 ns | 360 ns | PyO3 |
| E3: Static, large ctx | 490 ns | 450 ns | 370 ns | Python |
| E4: Targeting, small ctx | 140 ns | 440 ns | 360 ns | PyO3 |
| E5: Targeting, large ctx (110) | 1.5 µs | 103 µs | 3.4 µs | PyO3 |
| E5b: Targeting, xlarge (1003) | 1.6 µs | 102 µs | 3.8 µs | PyO3 |
| E6: Complex targeting, small | 1.5 µs | 102 µs | 7.4 µs | PyO3 |
| E10: Disabled (cache) | 2.2 µs | 106 µs | 11 µs | PyO3 |
Key insight: wasmtime-py's per-call overhead (~100 µs) dominates, making
it slower than both PyO3 and pure Python for all scenarios. The WASM
approach is better suited to Go/Java where the WASM runtimes
(wazero/Chicory) have lower per-call overhead.
## Test plan
- [x] 19 WASM evaluator tests pass (`pytest tests/test_wasm_evaluator.py
-v`)
- [x] 24 comparison benchmarks pass (`pytest
benchmarks/test_wasm_comparison.py --benchmark-only -v`)
- [x] Existing 15 PyO3 tests unaffected
- [ ] WASM binary must be built before running tests: `cargo build
--target wasm32-unknown-unknown --no-default-features --release --lib &&
cp target/wasm32-unknown-unknown/release/flagd_evaluator.wasm
python/flagd_evaluator_wasm/`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>1 parent 9e85110 commit 3ebed59
7 files changed
Lines changed: 1355 additions & 2 deletions
File tree
- .github/workflows
- python
- benchmarks
- flagd_evaluator_wasm
- src
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
141 | 141 | | |
142 | 142 | | |
143 | 143 | | |
| 144 | + | |
| 145 | + | |
144 | 146 | | |
145 | 147 | | |
146 | 148 | | |
| |||
151 | 153 | | |
152 | 154 | | |
153 | 155 | | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
154 | 161 | | |
155 | 162 | | |
156 | 163 | | |
| |||
0 commit comments