Skip to content

Commit f1bcc88

Browse files
Fidget-SpinnerFidget-Spinner
authored
Reflections on CPython's JIT compiler (#1)
1 parent cb692b8 commit f1bcc88

2 files changed

Lines changed: 136 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# My Personal Blog
22

3+
* [Reflections on 2 years of CPython's JIT Compiler: The good, the bad, the ugly](./posts/jit-reflections.md)
34
* [My Final Year Project / Bachelors' Theses](./posts/fyp.md).
45
* [My apology for Python's tail-calling interpreter's results](./posts/apology-tail-call.md).
56

‎posts/jit-reflections.md‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Reflections on 2 years of CPython's JIT Compiler: The good, the bad, the ugly
2+
3+
5 July 2025
4+
5+
This blog post includes by honest opinions on the CPython JIT. What I think we did well,
6+
what I think we could have done better. I'll also do some brief qualititative
7+
analysis.
8+
9+
I've been working on CPython's JIT compiler since before the very start.
10+
I don't know how long that is at this point ... 2.5, maybe almost 3 years?
11+
Anyways, I'm primarly responsible for Python's JIT compiler's
12+
[optimizer](https://docs.python.org/3.13/whatsnew/3.13.html#an-experimental-just-in-time-jit-compiler).
13+
14+
15+
## Here's a short summary:
16+
17+
The good:
18+
1. I think we're starting to build a community around the JIT, which is great.
19+
2. The JIT is also **teachable**. We have newcomers coming in and contributing.
20+
21+
The bad (could use improvement):
22+
1. Performance
23+
2. Inaccurate coverage of the JIT
24+
25+
26+
## Good: A community-driven JIT
27+
28+
CPython's JIT is community-driven at this point. You may have heard
29+
of the layoffs at Microsoft affecting the Faster CPython team. However,
30+
my underestanding is that from the very start, the JIT was meant to be
31+
a community project.
32+
33+
This wasn't always the case. When the JIT started out, it was practically
34+
only Brandt working on the machine code generator. I had help from Mark (Shannon)
35+
and Guido in landing the initial optimizer, but after that it was mostly me.
36+
Later I got busier with school work and Brandt became the sole contributor to
37+
the optimizer for a a few months or so. Those were dark times.
38+
39+
I'm really happy to say that we have more contributors today though:
40+
* Savannah works on the machine code generator, reproducible JIT stencils, and
41+
sometimes the optimizer.
42+
* Tomáš works on the optimizer and is a codeowner of it!
43+
* Diego works on the machine code generator to improve it on ARM, and sometimes
44+
the optimizer.
45+
* We also have various non core-dev drive-by contributors. Zheaoli and Noam are names
46+
that I remember. Though I'm definitely missing a few names here.
47+
48+
This community building was somewhat organic, but also very intentional. We
49+
actively tried to make the JIT easier to work on. If you dig up past discussions,
50+
one of Mark's arguments for a tracing JIT was easier static analysis. This easiness
51+
isn't just that it doesn't require a meet or join in general or that it requires
52+
only a single pass, but more that static anlaysis of a single basic block is
53+
easier to teach than a whole control-flow graph.
54+
55+
We also actively welcome people to work on the JIT with us. CPython doesn't have much
56+
optimizing compiler expertise interested in working on the JIT. We have some compiler
57+
people, but the subset of those interested in working on the JIT is even smaller. So
58+
we aim to train up people even if they don't have any background in compilers.
59+
60+
## Good: A teachable JIT
61+
62+
As I mentioned earlier, tracing was one decision to make the JIT easier to teach.
63+
There are a few other design decisions too, but those will be their own blog post.
64+
So I'm not talking about them here.
65+
66+
67+
## Bad: Performance
68+
69+
CPython 3.13's JIT ranges from slower to the interpreter
70+
to roughly equivalent to the interpreter.
71+
Calling a spade a spade: CPython 3.13's JIT is slow. It hurts me to say this considering
72+
I work on it, but I don't want to sugarcoat my words here.
73+
74+
The argument at the time was that it was a new feature and we needed to lay the foundations
75+
and test the waters. You might think that surely, CPython 3.14's JIT is a lot faster right? Nope.
76+
The answer is again... complicated. When using a modern compiler like Clang 20
77+
to build CPython 3.14, I often found the interpreter outperforms the JIT. The JIT only really starts reaching
78+
parity or outperforming the interpreter if we use an old compiler like GCC 11 to build the interpreter.
79+
However, IMO that's not entirely fair to the interpreter, as we're purposely limiting it by using a compiler
80+
we _know_ is worse for it. You can see this effect very clearly on Thomas Wouter's analysis
81+
[here](https://github.com/Yhg1s/python-benchmarking-public). In short, the JIT is almost always slower
82+
than the interpreter if you use a modern compiler. This also assumes the interpreter doesn't get hit
83+
by random performance bugs on the side (which have happened many times now).
84+
85+
You might ask: why is the 3.14 JIT not much faster? The real answer, which
86+
again hurts me to say is that the 3.14 JIT has almost no major performance
87+
features over 3.13. In 3.14, we were mostly expanding the existing types
88+
analysis pass to cover more bytecodes. We were also using that as a way to
89+
teach new contributors about the JIT and encourage contribution. In short, we
90+
were building up new talent. I personally think we were quite low on
91+
contributors at the start. I also had other commitments which made features
92+
that were supposed to go into the JIT not go in, which I'm sorry for.
93+
Personally, I think building up more talent over prioritizing immediate
94+
performance is the right choice for long-term sustainability of the JIT.
95+
96+
## Bad: Inaccurate coverage
97+
98+
The initial coverage of the JIT got the numbers wrong. There was this number
99+
of "2-9%" faster than the interpreter being spread around. I think the first
100+
major blog post that covered this was
101+
[this one](https://tonybaloney.github.io/posts/python-gets-a-jit.html#is-it-faster). Note that I'm friends with the
102+
author of that post and I'm not trying to say that they did a bad job.
103+
Conveying performance is a really hard job. One that I'm still struggling with
104+
[myself](./apology-tail-call.md). However, in good conscience, and as an
105+
aspiring scientist, I can't stand by and watch people say the JIT is "2-9%"
106+
faster than the intepreter. It's really more nuanced than that (see section
107+
above). Often times, the CPython 3.13 JIT is a lot slower than the interpreter.
108+
Furthermore, the linked comment is that the JIT is 2-9% faster than the
109+
_tier 2_ interpreter. That's the interpreter that executes our JIT
110+
intermediate representation by interpreting it, which is super slow. It's not
111+
copmaring to the actual CPython interpreter.
112+
113+
I've seen other sources repeat this number too. It frustrates me a lot. The
114+
problem with saying the 3.13 JIT is faster is that it sets the wrong
115+
expectations. Again, users on the Python Discourse forum and privately have
116+
shared performance numbers where the JIT is a significant regression for them.
117+
This goes against the grain of what's reported online. We do not have control over the numbers, but I still would like to clear the air on what the real expectation should be.
118+
119+
## Conclusion and looking forward
120+
121+
I'm still hopeful for the JIT. As I mentioned above, we've built a significant
122+
community around it. We're now starting to pick up momentum on issues and new
123+
optimizations that could bring single-digit percentage speedups to the JIT in
124+
3.14 (note: this is the geometric mean of our benchmarks, so real speedups
125+
might be greater or lesser). Brandt has already merged some
126+
[optimizations](https://github.com/python/cpython/pull/135905)
127+
for the JIT's machine code. I
128+
don't want to bring unwanted attention to the other efforts for the moment.
129+
Just know this: there are multiple parallel efforts to improve the JIT now
130+
that we have a bigger community around it that can enable such work.
131+
132+
133+
134+
135+

0 commit comments

Comments
 (0)