Skip to content

Commit 43f1484

Browse files
committed
ci: add benchmark workflow
We add a custom benchmark workflow with an initial simple througput scenario that can give a quick feedback about the performance impact of each PR, compared to the base branch.
1 parent 66584dc commit 43f1484

3 files changed

Lines changed: 500 additions & 0 deletions

File tree

.github/workflows/benchmarks.yml

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
name: Benchmarks
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- .github/workflows/benchmarks.yml
7+
- scripts/benchmark.py
8+
- bytecode/**
9+
- src/**
10+
11+
jobs:
12+
benchmark:
13+
runs-on: ubuntu-latest
14+
15+
steps:
16+
- name: Check out PR
17+
uses: actions/checkout@v4
18+
with:
19+
path: bytecode-dev
20+
21+
- name: Check out base branch
22+
uses: actions/checkout@v4
23+
with:
24+
ref: ${{ github.event.pull_request.base.sha }}
25+
path: bytecode-base
26+
27+
- name: Set up Python
28+
uses: actions/setup-python@v5
29+
with:
30+
python-version: "3.13"
31+
32+
- name: Install base version
33+
env:
34+
SETUPTOOLS_SCM_PRETEND_VERSION: "0.0.0"
35+
run: |
36+
python -m venv .venv-base
37+
.venv-base/bin/pip install --upgrade pip
38+
.venv-base/bin/pip install -r bytecode-dev/scripts/requirements-bm.txt
39+
.venv-base/bin/pip install ./bytecode-base
40+
41+
- name: Install dev version
42+
env:
43+
SETUPTOOLS_SCM_PRETEND_VERSION: "0.0.0"
44+
run: |
45+
python -m venv .venv-dev
46+
.venv-dev/bin/pip install --upgrade pip
47+
.venv-dev/bin/pip install -r bytecode-dev/scripts/requirements-bm.txt
48+
.venv-dev/bin/pip install ./bytecode-dev
49+
50+
- name: Run benchmark (base)
51+
run: |
52+
.venv-base/bin/python bytecode-dev/scripts/benchmark.py \
53+
--version base \
54+
--output results-base.json
55+
56+
- name: Run benchmark (dev)
57+
run: |
58+
.venv-dev/bin/python bytecode-dev/scripts/benchmark.py \
59+
--version dev \
60+
--output results-dev.json
61+
62+
- name: Render report
63+
run: |
64+
.venv-dev/bin/python bytecode-dev/scripts/benchmark.py \
65+
--merge results-base.json results-dev.json \
66+
--format markdown \
67+
> comment.txt
68+
cat comment.txt
69+
70+
- name: Post results on PR
71+
uses: marocchino/sticky-pull-request-comment@v2
72+
with:
73+
header: benchmarks
74+
path: comment.txt

0 commit comments

Comments
 (0)