|
| 1 | ++++ |
| 2 | +title = "Ty versus Pyrefly versus Zuban: Reviewing the future of Python type checkers (Draft)" |
| 3 | +date = 2025-08-29 |
| 4 | +description = "A comparison of three new Rust-based Python type checkers: Astral's ty, Meta's pyrefly, and David Halter's zuban" |
| 5 | + |
| 6 | +[taxonomies] |
| 7 | +tags = ["rust", "python", "type-checkers", "typing"] |
| 8 | ++++ |
| 9 | + |
| 10 | +# Table of Contents |
| 11 | + |
| 12 | +- [Introduction](#introduction) |
| 13 | + - [`ty` from Astral](#ty-from-astral) |
| 14 | + - [`pyrefly` from Meta](#pyrefly-from-meta) |
| 15 | + - [`zuban` from David Halter](#zuban-from-david-halter) |
| 16 | +- [Typing Conformance Suite Analysis](#typing-conformance-suite-analysis) |
| 17 | + - [Summary](#summary) |
| 18 | + - [Review of progress](#review-of-progress) |
| 19 | +- [Other resources to learn more](#other-resources-to-learn-more) |
| 20 | +- [Footnotes](#footnotes) |
| 21 | + |
| 22 | +# Introduction |
| 23 | + |
| 24 | +The Python type checking landscape is experiencing a particularly active phase of innovation in 2025. This year has witnessed the emergence of not one, not two, but **three** new Python type checking tools, each backed by significant pedigree and resources. While these tools have slightly different goals and philosophies, they share a common foundation: all are built in Rust with performance as a core design principle. |
| 25 | + |
| 26 | +This continued shift toward Rust-based implementations represents a significant evolution in the Python tooling ecosystem, promising faster type checking, better IDE integration, and improved developer experience for large codebases. |
| 27 | + |
| 28 | + |
| 29 | +## `ty` from Astral |
| 30 | + |
| 31 | +**Repository:** <https://github.com/astral-sh/ty/> |
| 32 | +**Development Repository:** <https://github.com/astral-sh/ruff/> |
| 33 | + |
| 34 | +**Key Highlights:** |
| 35 | +- Strong focus on gradual guarantee principles |
| 36 | +- Planned tight integration with Astral's existing linting tool `ruff`, with the aim to support type based linting rules.[^1] |
| 37 | +- Backing from a team combining Python core developers and very experienced Rust tooling developers |
| 38 | +- Incremental computation at the function level leveraging the [salsa](https://github.com/salsa-rs/salsa) library used by [rust-analyzer](https://github.com/rust-lang/rust-analyzer/). This could be particularly important on the LSP that `ty` provides to give very quick/responsive feedback loops in code editors. |
| 39 | + |
| 40 | +**Philosophy:** Astral's approach emphasizes reliability and gradual adoption, making it easier for teams to incrementally add type checking to existing codebases without overwhelming developers with false positives. |
| 41 | + |
| 42 | +## `pyrefly` from Meta |
| 43 | + |
| 44 | +**Repository:** <https://github.com/facebook/pyrefly> |
| 45 | + |
| 46 | +**Key Highlights:** |
| 47 | +- Successor to Meta's existing [pyre](https://github.com/facebook/pyre-check) type checker, designed to eventually replace it |
| 48 | +- Enhanced type inference capabilities by default |
| 49 | +- Potentially higher upfront adoption cost due to aggressive inference which might flag issues with correct code |
| 50 | +- Backed by Meta's substantial engineering resources and real-world usage at scale |
| 51 | + |
| 52 | +**Philosophy:** Meta prioritizes powerful inference and catching more potential issues out of the box, even if this means a steeper learning curve for teams new to type checking. |
| 53 | + |
| 54 | +## `zuban` from David Halter |
| 55 | + |
| 56 | +**Homepage:** <https://zubanls.com/> |
| 57 | +**Documentation:** <https://docs.zubanls.com/en/latest/> |
| 58 | + |
| 59 | +**Key Highlights:** |
| 60 | +- Created by the author of the popular Python LSP tool `jedi` |
| 61 | +- Aims for high-degree of compatibility with `mypy` to make adoption in large existing codebases seamless. |
| 62 | +- Not FOSS[^2], will require a license for codebases above 1.5 MB (~50,000 lines of code)[^3]. |
| 63 | +- Currently maintained by a single author seems a potential risk to long-term sustainability as Python typing does not stand still. |
| 64 | + |
| 65 | +**Philosophy:** Zuban aims to provide the smoothest possible migration path from existing type checkers, particularly `mypy`, making it attractive for organizations with substantial existing typed codebases. |
| 66 | + |
| 67 | + |
| 68 | +# Typing Conformance Suite Analysis |
| 69 | + |
| 70 | +The Python Typing Council maintains a [Conformance test suite](https://github.com/python/typing/tree/main/conformance) which validates the behavior of static type checkers against the expectations defined in the [Python typing specification](https://typing.python.org/en/latest/spec/index.html). |
| 71 | + |
| 72 | +`ty` and `pyrefly` have not yet been added to the conformance suite, so it's harder to establish a baseline for their progress on this front. |
| 73 | + |
| 74 | +To help resolve this gap I have [expanded the current test harness to support both of them](https://github.com/sinon/typing/pull/1)[^4], this also adds some additional debugging information to the html report to show the split between false negatives and false positives that the suite has detected in each test case. |
| 75 | + |
| 76 | +I've also included a local build of `ty` for two main reasons: |
| 77 | + |
| 78 | +* I've wanted to investigate contributing to `ty` and/or `ruff` for a while, so this was good impetus to get things set up. |
| 79 | +* `ty` releases are cut relatively infrequently and I am impatient. |
| 80 | + |
| 81 | +## Summary |
| 82 | + |
| 83 | +__Generated 29/08/2025__ |
| 84 | + |
| 85 | +> NOTE |
| 86 | +> |
| 87 | +> The following section is slightly unfair, all of these tools are in alpha and only one of these tools (Zuban) has opted into the Conformance suite. |
| 88 | +> |
| 89 | +> That being said even though `ty` is lagging on this metric at the moment it is still the type checker that I am most excited to use long-term because of the quality of the tooling Astral has built so far. |
| 90 | +
|
| 91 | +| Type Checker | Total Test Case Passes | Total Test Case Partial | Total False Positives | Total False Negatives | False Negatives | |
| 92 | +|:-----------------------------------------------:|:----------------------:|:-----------------------:|:---------------------:|:---------------------:|-----------------| |
| 93 | +| zuban 0.0.20 | 97 | 42 | 152 | 89 | 92 | |
| 94 | +| ty 0.0.1-alpha.19 (e9cb838b3 2025-08-19) | 20 | 119 | 371 | 603 | 603 | |
| 95 | +| Local:ty ruff/0.12.11+27 (0bf5d2a20 2025-08-29) | 20 | 119 | 370 | 590 | 590 | |
| 96 | +| pyrefly 0.30.0 | 81 | 58 | 100 | 187 | 187 | |
| 97 | + |
| 98 | +## Review of progress |
| 99 | + |
| 100 | +`zuban` has a lead, having full `Pass` on ~69% of test cases, compared with ~15% for `ty` and ~58% for `pyrefly`. Which makes sense as though it's released in a similar time period to `ty` and `pyrefly` it has been in active development in private for several years. |
| 101 | + |
| 102 | +The thing that surprised me more was how much progress `pyrefly` has made when compared to `ty`. This can maybe be partially explained from a point raised in [Edward Li's blog excellent post on the Typing Summit at PyCon 2025][1] which mentions that the `pyrefly` team devoted a lot of up front time into solving some of the hard problems, such as generics. |
| 103 | + |
| 104 | + |
| 105 | +# Other resources to learn more |
| 106 | + |
| 107 | +* [Edward Li's excellent blog post comparing `ty` and `pyrefly`][1] which also contains the videos recorded at the PyCon typing summit which both `ty` and `pyrefly` gave presentations at. |
| 108 | +* [Happy Path Programming - 114 ty: Fast Python Type Checking with Carl Meyer](https://www.youtube.com/watch?v=V1OmqEYoSz4) |
| 109 | +* [Happy Path Programming - 115 More Python Type Checking! Pyrefly with Aaron Pollack & Steven Troxler](https://www.youtube.com/watch?v=huHF0Rv8L14) |
| 110 | +* [Talk Python - ty: Astral's New Type Checker (Formerly Red-Knot)](https://talkpython.fm/episodes/show/506/ty-astrals-new-type-checker-formerly-red-knot) |
| 111 | + |
| 112 | + |
| 113 | +# Footnotes |
| 114 | + |
| 115 | +[^1]: Which can be demonstrated in the [open issues](https://github.com/astral-sh/ruff/issues?q=is%3Aissue%20state%3Aopen%20label%3Atype-inference ) on ruff tagged with `type-inference` which are bugs or new features that can only be resolved with `ruff` having access to deeper type inference data that `ty` can supply. |
| 116 | +[^2]: David has indicated a plan to make [source available in the future](https://github.com/python/typing/pull/2067#issuecomment-3177937964) when adding Zuban to the Python typing conformance suite. |
| 117 | +[^3]: Full pricing information at: <https://zubanls.com/pricing/> |
| 118 | +[^4]: This is just for this blog post, no plans to seek merging this. |
| 119 | + |
| 120 | + |
| 121 | +<!-- Reference links ---> |
| 122 | +[1]: https://blog.edward-li.com/tech/comparing-pyrefly-vs-ty/ |
0 commit comments