Skip to content

Commit a8acf4e

Browse files
Jonathan D.A. Jewellclaude
andcommitted
feat: add comprehensive performance benchmark suite
Implement Enhancement #3: Performance benchmarking infrastructure comparing Ephapax to Rust and AssemblyScript across compilation speed, binary size, and runtime performance. Components: - Criterion benchmarks for compilation time (parse, typecheck, codegen) - Binary size measurement tool (raw + gzipped WASM) - Runtime performance benchmarks using wasmtime - Reference implementations (Fibonacci, Factorial) in all 3 languages - Comparison scripts (Bash runner + Python report generator) - GitHub Actions CI integration with weekly runs Files added (16 total): - benches/*.rs (3 Criterion benchmarks) - benches/comparison/* (reference implementations) - benches/scripts/* (automation) - .github/workflows/benchmark.yml (CI) - docs/BENCHMARKS.md (complete guide) Usage: ./benches/scripts/run_all_benchmarks.sh cargo bench --bench compile_time cargo run --release --bin binary_size CI Features: - Runs on push, PR, and weekly schedule - Generates markdown comparison reports - Posts results as PR comments - Archives artifacts for historical tracking Status: Enhancement #3 complete (100%) Next: Enhancement #4 (VS Code Extension) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent fadeed8 commit a8acf4e

16 files changed

Lines changed: 1093 additions & 525 deletions

.github/workflows/benchmark.yml

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# SPDX-License-Identifier: PMPL-1.0-or-later
2+
name: Performance Benchmarks
3+
4+
on:
5+
push:
6+
branches: [main]
7+
pull_request:
8+
branches: [main]
9+
schedule:
10+
# Run benchmarks weekly on Monday at 00:00 UTC
11+
- cron: '0 0 * * 1'
12+
13+
permissions:
14+
contents: write # For pushing benchmark results
15+
16+
jobs:
17+
benchmark:
18+
runs-on: ubuntu-latest
19+
20+
steps:
21+
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
22+
23+
- name: Install Rust
24+
uses: dtolnay/rust-toolchain@4be9e76fd7c4901c61fb841f559994984270fce7 # stable
25+
with:
26+
toolchain: stable
27+
targets: wasm32-unknown-unknown
28+
29+
- name: Install Node.js
30+
uses: actions/setup-node@v4
31+
with:
32+
node-version: '20'
33+
34+
- name: Cache Cargo
35+
uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
36+
37+
- name: Install AssemblyScript dependencies
38+
run: |
39+
cd benches/comparison/assemblyscript
40+
npm install
41+
42+
- name: Build Ephapax compiler
43+
run: cargo build --release
44+
45+
- name: Run compilation time benchmarks
46+
run: cargo bench --bench compile_time -- --output-format bencher | tee compile_output.txt
47+
48+
- name: Run binary size measurements
49+
run: cargo run --release --bin binary_size | tee size_output.txt
50+
51+
- name: Run runtime performance benchmarks
52+
run: cargo bench --bench runtime_perf -- --output-format bencher | tee runtime_output.txt
53+
54+
- name: Generate comparison report
55+
run: |
56+
python3 benches/scripts/compare_results.py
57+
58+
- name: Upload benchmark results
59+
uses: actions/upload-artifact@v4
60+
with:
61+
name: benchmark-results
62+
path: |
63+
target/criterion/
64+
benches/BENCHMARK-RESULTS.md
65+
compile_output.txt
66+
size_output.txt
67+
runtime_output.txt
68+
69+
- name: Comment PR with results
70+
if: github.event_name == 'pull_request'
71+
uses: actions/github-script@v7
72+
with:
73+
script: |
74+
const fs = require('fs');
75+
const report = fs.readFileSync('benches/BENCHMARK-RESULTS.md', 'utf8');
76+
77+
github.rest.issues.createComment({
78+
issue_number: context.issue.number,
79+
owner: context.repo.owner,
80+
repo: context.repo.repo,
81+
body: `## 📊 Benchmark Results\n\n${report}`
82+
});

Cargo.toml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,3 +56,28 @@ ariadne = "0.4"
5656
clap = { version = "4", features = ["derive"] }
5757
colored = "2"
5858
rustyline = "14"
59+
60+
# Benchmarking dependencies
61+
criterion = { version = "0.5", features = ["html_reports"] }
62+
wasmtime = "27"
63+
64+
# Benchmark binaries
65+
[[bin]]
66+
name = "binary_size"
67+
path = "benches/binary_size.rs"
68+
required-features = []
69+
70+
[dev-dependencies]
71+
criterion.workspace = true
72+
wasmtime.workspace = true
73+
ephapax-syntax.workspace = true
74+
ephapax-typing.workspace = true
75+
ephapax-wasm.workspace = true
76+
77+
[[bench]]
78+
name = "compile_time"
79+
harness = false
80+
81+
[[bench]]
82+
name = "runtime_perf"
83+
harness = false

0 commit comments

Comments
 (0)