Skip to content

perf: improve general performance#172

Merged
MatthieuDartiailh merged 3 commits into
MatthieuDartiailh:mainfrom
P403n1x87:perf/improve-performance
Nov 20, 2025
Merged

perf: improve general performance#172
MatthieuDartiailh merged 3 commits into
MatthieuDartiailh:mainfrom
P403n1x87:perf/improve-performance

Conversation

@P403n1x87
Copy link
Copy Markdown
Contributor

@P403n1x87 P403n1x87 commented Oct 17, 2025

We try to use sets for quick lookups and cache the result of some repetitive operations to speed up some operations. We also remove a good deal of asserts embedded within the code that adde extra overhead. Where appropriate, these should be replaced with actual tests.

The current change squeezes about 3/4% performance from a decompile/recompile loop.

Benchmarking

The following round-trip script has been used to benchmark the performance of the proposed changes

from time import monotonic

from bytecode import Bytecode

n = []
for _ in range(30):
    start = monotonic()
    for _ in range(2000):
        Bytecode.from_code(Bytecode.from_code.__code__).to_code()
    n.append(monotonic() - start)


avg = sum(n) / len(n)
sdev = (sum(_**2 for _ in n) / len(n) - avg**2) ** 0.5
print(f"{avg:0.2f} +/- {sdev:0.2f}")

Baseline: 1.10 +/- 0.02
This PR: 1.03 +/- 0.02

@P403n1x87 P403n1x87 force-pushed the perf/improve-performance branch from d1eec34 to abd3d1d Compare October 17, 2025 11:30
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Oct 17, 2025

Codecov Report

❌ Patch coverage is 90.90909% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.37%. Comparing base (7b6825f) to head (425a9bf).

Files with missing lines Patch % Lines
src/bytecode/concrete.py 87.50% 2 Missing and 1 partial ⚠️
src/bytecode/cfg.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #172      +/-   ##
==========================================
- Coverage   95.43%   95.37%   -0.06%     
==========================================
  Files           7        7              
  Lines        2147     2142       -5     
  Branches      481      483       +2     
==========================================
- Hits         2049     2043       -6     
  Misses         56       56              
- Partials       42       43       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Owner

@MatthieuDartiailh MatthieuDartiailh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for starting this, just one small question. Also you may need some convincing for Mypy to be happy with you without all the assertions.

Comment thread src/bytecode/instr.py
We try to use sets for quick lookups and cache the result of some
repetitive operations to speed up some operations. We also remove
a good deal of asserts embedded within the code that adde extra
overhead.
@P403n1x87 P403n1x87 force-pushed the perf/improve-performance branch from abd3d1d to 7ac2792 Compare October 17, 2025 14:15
@P403n1x87 P403n1x87 force-pushed the perf/improve-performance branch from d5a0515 to 7ac2792 Compare November 20, 2025 11:53
@P403n1x87 P403n1x87 marked this pull request as ready for review November 20, 2025 12:09
@MatthieuDartiailh
Copy link
Copy Markdown
Owner

This looks good to me even though the removal of some assertion looks slightly scary to me but I think it is fine. I will merge shortly.

@MatthieuDartiailh
Copy link
Copy Markdown
Owner

Regarding version checks I wonder if we could try to cythonize the code and use tempita to actually remove the runtime version checks. Is this something you would be interested in ? H5py uses tempita with their cython code h5py/h5py#2637

@MatthieuDartiailh MatthieuDartiailh self-requested a review November 20, 2025 12:58
@MatthieuDartiailh MatthieuDartiailh merged commit 317dd0f into MatthieuDartiailh:main Nov 20, 2025
14 checks passed
@P403n1x87
Copy link
Copy Markdown
Contributor Author

P403n1x87 commented Nov 20, 2025

Regarding version checks I wonder if we could try to cythonize the code and use tempita to actually remove the runtime version checks. Is this something you would be interested in ? H5py uses tempita with their cython code h5py/h5py#2637

I have Cythonised this library in the past as an experiment and observed a general 2x speedup. Whilst not massive, it still a good improvement. So I would definitely be interested in this project moving to native. What I would ask is that wheels are built for as many platforms as possible, including e.g. alpine/musl.

@P403n1x87 P403n1x87 deleted the perf/improve-performance branch November 20, 2025 14:58
@MatthieuDartiailh
Copy link
Copy Markdown
Owner

cibuildwheel make the process rather straightforward so I have no problem on this front.
Could you update your branch and make a PR so that I get an idea of what it would look like.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants