You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/future-python-type-checkers.md
+20-20Lines changed: 20 additions & 20 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
+++
2
-
title = "Ty versus Pyrefly versus Zuban: Reviewing the future of Python type checkers (Draft)"
2
+
title = "How Well Do New Python Type Checkers Conform? A Deep Dive into Ty, Pyrefly, and Zuban"
3
3
date = 2025-08-29
4
-
description = "A comparison of three new Rust-based Python type checkers: Astral's ty, Meta's pyrefly, and David Halter's zuban"
4
+
description = "A comparison of three new Rust-based Python type checkers through the lens of typing spec conformance: Astral's ty, Meta's pyrefly, and David Halter's zuban"
@@ -39,7 +39,7 @@ Before examining these new Rust-based tools, it's worth understanding the curren
39
39
40
40
**pyre** - A type checker from Meta, written in a mix of OCaml and Python. Designed to handle Meta's massive Python codebase, pyre introduced several performance optimizations and incremental checking capabilities. However, Meta is now developing pyrefly as pyre's successor.
41
41
42
-
**pytype** - Google's type checker that takes a unique approach by performing type inference on unannotated Python code. Unlike other checkers that primarily validate existing type annotations, pytype can infer types from runtime behavior and generate stub files for gradual typing adoption. Written in Python, it's particularly useful for analyzing legacy codebases without type hints. However, Google announced in 2024 that pytype is being deprecated and Python 3.12 will be the last supported version of Python.
42
+
**pytype** - Google's type checker that takes a unique approach by performing type inference on unannotated Python code. Unlike other checkers that primarily validate existing type annotations, pytype can infer types from runtime behaviour and generate stub files for gradual typing adoption. Written in Python, it's particularly useful for analysing legacy codebases without type hints. However, Google announced in 2024 that pytype is being deprecated and Python 3.12 will be the last supported version of Python.
43
43
44
44
## The newcomers
45
45
@@ -49,6 +49,7 @@ Before examining these new Rust-based tools, it's worth understanding the curren
- Created by the author of the popular Python LSP tool `jedi`
78
81
- Aims for high-degree of compatibility with `mypy` to make adoption in large existing codebases seamless.
79
82
- Not FOSS[^2], will require a license for codebases above 1.5 MB (~50,000 lines of code)[^3].
80
83
- Currently maintained by a single author seems a potential risk to long-term sustainability as Python typing does not stand still.
81
84
82
85
**Philosophy:** Zuban aims to provide the smoothest possible migration path from existing type checkers, particularly `mypy`, making it attractive for organizations with substantial existing typed codebases.
83
86
84
-
85
87
# Typing Conformance Suite Analysis
86
88
87
-
The Python Typing Council maintains a [Conformance test suite](https://github.com/python/typing/tree/main/conformance) which validates the behavior of static type checkers against the expectations defined in the [Python typing specification](https://typing.python.org/en/latest/spec/index.html).
89
+
The Python Typing Council maintains a [Conformance test suite](https://github.com/python/typing/tree/main/conformance) which validates the behaviour of static type checkers against the expectations defined in the [Python typing specification](https://typing.python.org/en/latest/spec/index.html).
88
90
89
91
`ty` and `pyrefly` have not yet been added to the conformance suite, so it's harder to establish a baseline for their progress on this front.
90
92
91
93
To help resolve this gap I have [expanded the current test harness to support both of them](https://github.com/sinon/typing/pull/1)[^4], this also adds some additional debugging information to the html report to show the split between false negatives and false positives that the suite has detected in each test case.
92
94
93
95
I've also included a local build of `ty` for two main reasons:
94
96
95
-
* I've wanted to investigate contributing to `ty` and/or `ruff` for a while, so this was good impetus to get things set up.
96
-
*`ty` releases are cut relatively infrequently and I am impatient.
97
+
- I've wanted to investigate contributing to `ty` and/or `ruff` for a while, so this was good impetus to get things set up.
98
+
-`ty` releases are cut relatively infrequently and I am impatient.
97
99
98
100
## Summary
99
101
@@ -105,12 +107,12 @@ __Generated 29/08/2025__
105
107
>
106
108
> That being said even though `ty` is lagging on this metric at the moment it is still the type checker that I am most excited to use long-term because of the quality of the tooling Astral has built so far.
107
109
108
-
| Type Checker | Total Test Case Passes | Total Test Case Partial | Total False Positives | Total False Negatives | False Negatives |
@@ -132,6 +134,7 @@ The conformance test suite focuses heavily on advanced typing features that, whi
132
134
- Intricate interactions between multiple typing features
133
135
134
136
In contrast, the majority of Python codebases primarily use:
137
+
135
138
- Basic type annotations (`str`, `int`, `List[str]`, etc.)
136
139
- Simple class hierarchies
137
140
- Optional types and Union types
@@ -151,14 +154,12 @@ For teams evaluating these type checkers, the conformance scores provide valuabl
151
154
-**For large, complex codebases or libraries leaning on more esoteric generic patterns**: Higher conformance scores may indicate better handling of advanced patterns you might encounter
152
155
-**For teams new to typing**: The difference in conformance scores may be less relevant than IDE integration, error message quality, and performance
153
156
154
-
155
157
# Other resources to learn more
156
158
157
-
*[Edward Li's excellent blog post comparing `ty` and `pyrefly`][1] which also contains the videos recorded at the PyCon typing summit which both `ty` and `pyrefly` gave presentations at.
158
-
*[Happy Path Programming - 114 ty: Fast Python Type Checking with Carl Meyer](https://www.youtube.com/watch?v=V1OmqEYoSz4)
159
-
*[Happy Path Programming - 115 More Python Type Checking! Pyrefly with Aaron Pollack & Steven Troxler](https://www.youtube.com/watch?v=huHF0Rv8L14)
160
-
*[Talk Python - ty: Astral's New Type Checker (Formerly Red-Knot)](https://talkpython.fm/episodes/show/506/ty-astrals-new-type-checker-formerly-red-knot)
161
-
159
+
-[Edward Li's excellent blog post comparing `ty` and `pyrefly`][1] which also contains the videos recorded at the PyCon typing summit which both `ty` and `pyrefly` gave presentations at.
160
+
-[Happy Path Programming - 114 ty: Fast Python Type Checking with Carl Meyer](https://www.youtube.com/watch?v=V1OmqEYoSz4)
161
+
-[Happy Path Programming - 115 More Python Type Checking! Pyrefly with Aaron Pollack & Steven Troxler](https://www.youtube.com/watch?v=huHF0Rv8L14)
162
+
-[Talk Python - ty: Astral's New Type Checker (Formerly Red-Knot)](https://talkpython.fm/episodes/show/506/ty-astrals-new-type-checker-formerly-red-knot)
162
163
163
164
# Footnotes
164
165
@@ -167,6 +168,5 @@ For teams evaluating these type checkers, the conformance scores provide valuabl
167
168
[^3]: Full pricing information at: <https://zubanls.com/pricing/>
168
169
[^4]: This is just for this blog post, no plans to seek merging this.
0 commit comments