Skip to content

Commit ef596f5

Browse files
committed
ci: add benchmark workflow
We add a custom benchmark workflow with an initial simple througput scenario that can give a quick feedback about the performance impact of each PR, compared to the base branch.
1 parent 66584dc commit ef596f5

3 files changed

Lines changed: 496 additions & 0 deletions

File tree

.github/workflows/benchmarks.yml

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
name: Benchmarks
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- .github/workflows/benchmarks.yml
7+
- scripts/benchmark.py
8+
- bytecode/**
9+
- src/**
10+
11+
jobs:
12+
benchmark:
13+
runs-on: ubuntu-latest
14+
15+
steps:
16+
- name: Check out PR
17+
uses: actions/checkout@v4
18+
with:
19+
path: bytecode-dev
20+
21+
- name: Check out base branch
22+
uses: actions/checkout@v4
23+
with:
24+
ref: ${{ github.event.pull_request.base.sha }}
25+
path: bytecode-base
26+
27+
- name: Set up Python
28+
uses: actions/setup-python@v5
29+
with:
30+
python-version: "3.13"
31+
32+
- name: Install base version
33+
run: |
34+
python -m venv .venv-base
35+
.venv-base/bin/pip install --upgrade pip
36+
.venv-base/bin/pip install -r bytecode-dev/scripts/requirements-bm.txt
37+
.venv-base/bin/pip install ./bytecode-base
38+
39+
- name: Install dev version
40+
run: |
41+
python -m venv .venv-dev
42+
.venv-dev/bin/pip install --upgrade pip
43+
.venv-dev/bin/pip install -r bytecode-dev/scripts/requirements-bm.txt
44+
.venv-dev/bin/pip install ./bytecode-dev
45+
46+
- name: Run benchmark (base)
47+
run: |
48+
.venv-base/bin/python bytecode-dev/scripts/benchmark.py \
49+
--version base \
50+
--output results-base.json
51+
52+
- name: Run benchmark (dev)
53+
run: |
54+
.venv-dev/bin/python bytecode-dev/scripts/benchmark.py \
55+
--version dev \
56+
--output results-dev.json
57+
58+
- name: Render report
59+
run: |
60+
.venv-dev/bin/python bytecode-dev/scripts/benchmark.py \
61+
--merge results-base.json results-dev.json \
62+
--format markdown \
63+
> comment.txt
64+
cat comment.txt
65+
66+
- name: Post results on PR
67+
uses: marocchino/sticky-pull-request-comment@v2
68+
with:
69+
header: benchmarks
70+
path: comment.txt

0 commit comments

Comments
 (0)