feat(python): add WASM evaluator and 3-way comparison benchmarks#73
Merged
Conversation
aepfli
force-pushed
the
feat/python-wasm-evaluator
branch
from
4188854 to
93bf09f
Compare
8 tasks
Add a Python WASM-based flag evaluator (wasmtime-py) that mirrors the Go (wazero) and Java (Chicory) implementations with host-side context filtering, pre-evaluated caching, and index-based evaluation. - flagd_evaluator_wasm package: WasmFlagEvaluator class with same API as the PyO3 FlagEvaluator (evaluate, evaluate_bool/string/int/float) - 9 host functions matching Go's host.go (time, random, wbindgen stubs) - Pre-allocated WASM buffers (256B flag key, 1MB context) - Context key filtering: serialize only needed keys before WASM call - Thread-safe via threading.Lock - 19 tests mirroring test_basic.py + test_flag_evaluation.py - 3-way benchmarks (PyO3 vs WASM vs pure Python json-logic) across 8 scenarios: static/targeting/disabled with empty/small/large/xlarge contexts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ndings Add host-side optimizations to the PyO3 FlagEvaluator matching the WASM and Go implementations: - Pre-evaluated cache for static/disabled flags (skip Rust evaluation) - Context key filtering: extract only required keys from PyDict before converting to serde_json::Value, avoiding full dict serialization - evaluate_flag_pre_enriched path for pre-enriched filtered contexts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The flagd_evaluator_wasm package requires the WASM binary at runtime. Build it from Rust source and copy to the package directory before running tests. Also add wasm32-unknown-unknown target to Rust toolchain. Refs: #83 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The flagd_evaluator_wasm package lives in the python/ directory as a plain Python package. Without pythonpath configured, pytest in CI cannot find it when running inside a uv-managed venv. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
aepfli
force-pushed
the
feat/python-wasm-evaluator
branch
from
52889f2 to
9991941
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
flagd_evaluator_wasmpackage: a Python WASM-based flag evaluator using wasmtime-py that mirrors the Go (wazero) and Java (Chicory) implementationsBenchmark Results (8 scenarios)
Key insight: wasmtime-py's per-call overhead (~100 µs) dominates, making it slower than both PyO3 and pure Python for all scenarios. The WASM approach is better suited to Go/Java where the WASM runtimes (wazero/Chicory) have lower per-call overhead.
Test plan
pytest tests/test_wasm_evaluator.py -v)pytest benchmarks/test_wasm_comparison.py --benchmark-only -v)cargo build --target wasm32-unknown-unknown --no-default-features --release --lib && cp target/wasm32-unknown-unknown/release/flagd_evaluator.wasm python/flagd_evaluator_wasm/🤖 Generated with Claude Code