Discussion about supported platforms for wheels

[lesteve]

It would be nice to have a discussion at the ecosystem level on supported platforms for wheels.

The idea was floated in the past in an numpy context, e.g. this comment by @rgommers numpy/numpy#22318 (comment)

There is also the pypackaging-native page: Expectations that projects provide ever more wheels.

The following questions seems to be relevant:

• does it make sense to have this decision at the Scientific Python level?
• can we think of a lightweight decision process that involves a few "core" Scientific Python projects?
• should we punt the can down the road towards root dependencies e.g. numpy?
• is there some simple rule of thumb that we could come up with that would help making a decision to add or remove a platform?
• are we fine removing wheels for some platforms? There is a feeling that once you add a wheel for a platform it's hard to remove it, i.e. "wheels is a larger commitment" or even "wheels are forever", but we did historically remove wheels e.g. for PyPy or 32bit.
• when do we decide to push back and tell people to do their own PyPI index like https://piwheels.org (for Rasperry Pi), where the burden is on the community that care about the platform rather than on the project maintainers?

About some rule of thumb:

• it feels like a supported platform would need 2 active core devs (maybe 1 is enough for small projects) that are willing to debug issues on this platform
• one rule could be based on PyPI download stats (I am not convinced that these stats can be used in a reasonable manner see my recent talk) but I am also opened to suggestions on this.

I am opening this following two recent organized and funded effort to push for wheels for Windows arm and ppc64le (IBM-specific wheels) across the ecosystem. I felt like each project was a bit on its own to decide and push back and that a more coordinated approach could have helped.

[rgommers]

Thanks for the summary @lesteve, that all sounds about right. It does feel particularly timely given that the number of supported platforms on PyPI, and the asks/pushes to support them, are growing a lot at the moment. Here is a quick list:

• PowerPC (ppc64le): supported for a long time, but a funded (IBM) push over the past 6 months
• Windows on Arm64: new-ish and GitHub Actions support quite new, with funded (Microsoft) push
• RISC-V: coming to PyPI probably, has funding - see https://riseproject.dev/2025/05/14/easy-installation-of-binary-python-packages-on-riscv64-devices and https://discuss.python.org/t/packaging-support-for-riscv64/58475/13
• iOS: new (PEP 730), see also https://www.anaconda.com/blog/beeware-mobile-python about both iOS and Android
• Android: new (PEP 738)
• Emscripten: in the pipeline, see PEP 783

Platforms for which we've mostly dropped support by now:

• PyPy: it's essentially end-of-life, won't go beyond support Python 3.11
• 32-bit x86 Linux (i686): still testing in CI, but there's no real demand, wheels were dropped
• 32-bit x86 Windows: numpy still publishes wheels, most other projects do not.

Platforms which have had support for a long time but most projects never supported wheels for and without a push for change:

• IBM-Z (s390x): well maybe IBM would like a push if their ppc64le push succeeds, but s390x is really too niche
• 32-bit Arm on Linux (armv7l): has wheels on https://piwheels.org/, no real discussion on moving that upstream.

Other relevant changes in wheel build load:

• Free-threaded Python (temporarily) added extra load, need separate cp313t/cp314t wheels. Note that cp313t can likely be dropped in the near future
• Stable ABI has matured: for most projects it will be possible to support this on Python >=3.12, see Tracking issue: Stable ABI support scipy/scipy#23791 for an overview. Pybind11 is still the last obvious blocker for a subset of projects. I'd expect all-but-numpy to be unblocked soon.
  • Note that for Python 3.15 we'll very likely have Stable ABI support for both with-GIL and free-threaded CPython with a single ABI. There are two competing PEPs for that, PEP 803 and PEP 809.
  • When projects move to the Stable ABI, the number of wheels that needs building really drops by a lot.
• For numpy and scipy specifically, we added a full extra set of macOS wheels built against Accelerate
• "wheel variants", if they make it, will allow the ability to provide general support for build variants, so for example projects could decide to publish optimized wheels for x86-64 Linux with psABI v4 (using AVX2/AVX-512).
• To give an impression of the current load, compare the latest SciPy release (1.16.3, from a few days ago, see here) with 60 wheels (10 platforms x 6 Python interpreters) to a three year old release (1.9.3, Oct 2022, see here) which has 20 wheels (5 platforms x 4 Python interpreters).
  • A 3x growth in 3 years is probably on the more extreme end, it'll be less on average. But still, I think most projects have seen significant growth and are feeling the pain already.

[rgommers]

@lesteve was there any discussion on this during the event?

[lesteve]

I am afraid there was only a few scattered discussions so not much progress to report ...

As my own quick summary:

• some projects did not really feel the pain, maybe this is a concern only for the projects who are at the bottom of the stack?
• people seemed fine to let numpy write his own rule as the main filter, and then each project gets to decide its own rule on top.

Something I thought about to help us coming with a quick rule is to look at the Google BigQuery PyPI download stats to get some kind of idea how much a sdist is downloaded for the more exotic platforms.

For example here is a query for scikit-learn:

SELECT COUNT(*) as count, details.cpu, details.system.name -- , details.distro.name
  FROM `bigquery-public-data.pypi.file_downloads`
  WHERE DATE(timestamp) BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY) AND CURRENT_DATE()
    AND file.project = 'scikit-learn'
    -- only select latest version, for some reason there are plenty of sdist downloads for very old versions
    AND file.version = '1.7.2'
    AND file.type = 'sdist'
  GROUP BY details.cpu, file.type, details.system.name -- , details.distro.name
  ORDER BY count desc

count	cpu	name
5048	x86_64	Linux
3413	NULL	NULL
2625	aarch64	Linux
512	AMD64	Windows
407	NULL	Linux
236	s390x	Linux
211	arm64	Darwin
209	ppc64le	Linux
106	riscv64	Linux
94	ARM64	Windows
60	armv7l	Linux
36	armv8l	Linux
15	x86_64	Darwin
10	amd64	FreeBSD
10	aarch64	Android
9	x86	Windows
6	iPad13,18	iPadOS
6	3931	OS/390
6	iPad14,8	iPadOS
5	iPad8,11	iPadOS
4	iPad13,16	iPadOS
3	iPhone15,4	iOS
3	NULL	Windows
2	8561	OS/390
2	iPad13,4	iPadOS
2	iPhone15,3	iOS
2	iPhone17,2	iOS
2	x86_64	MSYS_NT-10.0-26200
2	iPhone14,2	iOS
2	iPad15,7	iPadOS
2	iPhone15,5	iOS
2	iPad15,3	iPadOS
2	iPad12,1	iPadOS
1	i686	Linux
1	iPad16,3	iPadOS
1	iPhone13,3	iOS
1	iPhone14,5	iOS
1	iPad8,12	iOS
1	iPad7,1	iPadOS
1	armv6l	Linux
1	iPhone16,2	iOS
1	iPhone17,1	iOS
1	iPad11,6	iPadOS
1	iPad8,7	iPadOS
1	iPad8,10	Darwin
1	iPhone11,8	iOS
1	iPad13,2	iPadOS
1	iPhone13,4	iOS

Comments:

• I filter on the latest scikit-learn release, for some reason there are a number of sdist downloads for old scikit-learn versions. I don't think they should be taken into account.
• NULL is for other installer types than pip and uv. For example, poetry reports distribution but not CPU, requests and others don't report either

[betatim]

Off topic: I'm surprised how many i{Phone,Pads} there are in the list. How do people use Python on an iPad/Phone?

[lesteve]

Off topic: I'm surprised how many i{Phone,Pads} there are in the list. How do people use Python on an iPad/Phone?

Maybe BeeWare? See https://www.anaconda.com/blog/beeware-mobile-python mentioned by Ralf above or https://beeware.org/.

[bsipocz]

look at the Google BigQuery PyPI download stats to get some kind of idea how much a sdist is downloaded for the more exotic platforms.

I always wonder how much of these numbers are for CI/packagers/other automation rather a proxy for actual end users?

[lesteve]

I always wonder how much of these numbers are for CI/packagers/other automation rather a proxy for actual end users?

Same feeling, as I mentioned in the (long) issue description 😉.

one rule could be based on PyPI download stats (I am not convinced that these stats can be used in a reasonable manner see my recent talk) but I am also opened to suggestions on this.

[bsipocz]

Same feeling, as I mentioned in the (long) issue description 😉.

oops, sorry, there is certainly a downside of not paying full attention to the whole issue (or forgetting what the top had) 😊

[lesteve]

oops, sorry, there is certainly a downside of not paying full attention to the whole issue (or forgetting what the top had) 😊

No worries, I wrote the long description to have it in one place, I don't expect anyone to read the whole thing (and even less remember it 😉).

[da-woods]

One thing Cython has done is handled the less popular platforms via Stable ABI (but kept the others built fully). So currently Windows ARM, anything 32-bit, x64 OSX, and musllinux are Stable ABI.

Some of the logic behind that is pretty specific to Cython:

• our own code very "pythony" and so the sort of code that tends to be a bit slower in the Stable ABI while a lot of packages using Cython would be less different. So we've got a fairly clear, measurable hierarchy of "per-version wheels" (fastest) > "Stable ABI wheels" > "not compiled" (slowest). Many of you might not and might just find it's better to switch everything or nothing.
• It was partly an exercise in testing our Stable ABI support a bit.

So it was worthwhile for us - we increased our coverage of platforms while decreasing the number of wheels. However some of the reasoning might not make sense for most you you.