@@ -23,18 +23,15 @@ Abstract
2323========
2424
2525This PEP defines using ABI features as environment markers for project
26- dependencies, through a new ``sys_abi_features `` environment marker and
27- :data: `!sys.abi_features ` attribute in the :mod: `sys ` module.
26+ dependencies, through a new ``sys_abi_features `` environment marker.
2827:pep: `508 ` (later moved to :ref: `packaging:dependency-specifiers `) introduced
2928environment markers to specify dependencies based on rules that describe
3029when the dependency should be used.
3130This PEP extends the environment markers to allow specifying dependencies
3231based on specific ABI features of the Python interpreter.
3332For this, it defines a set of `ABI Features `_ and specifies how they are made
34- available via an `addition to the Python Standard Library <pep-780-sys.abi_features _>`_
35- in the form of a new attribute :data: `!sys.abi_features `, as well as for
36- `environment markers <pep-780-sys_abi_features _>`_ as a new marker variable,
37- ``sys_abi_features ``.
33+ available for `environment markers <pep-780-sys_abi_features _>`_ as a new
34+ marker variable, ``sys_abi_features ``.
3835
3936Motivation
4037==========
@@ -44,26 +41,24 @@ based on environment conditions.
4441The development of free-threaded CPython [#python-free-threading ]_
4542has underlined the need for an environment marker to discriminate between
4643different ABI features that the interpreter was built with.
47- For example, currently there is
48- no way to distinguish between a GIL-enabled and a free-threaded CPython
49- interpreter with an environment marker. This leads to real world issues for the
50- adoption of free-threading and its incremental rollout. When a Python package
51- is being made compatible with free-threaded CPython, it also needs all its
52- build and runtime dependencies to be compatible. Capturing the first version of
53- a dependency that is compatible precisely in metadata is currently not
54- possible, and increasing the minimum version of a dependency also for the
55- GIL-enabled build is usually undesirable since it unnecessarily limits
56- compatibility between packages.
44+ For example, currently there is no way to distinguish between a GIL-enabled and
45+ a free-threaded CPython interpreter with an environment marker. This leads to
46+ real world issues for the adoption of free-threading and its incremental
47+ rollout. When a Python package is being made compatible with free-threaded
48+ CPython, it also needs all its build and runtime dependencies to be compatible.
49+ Capturing the first version of a dependency that is compatible precisely in
50+ metadata is currently not possible, and increasing the minimum version of a
51+ dependency also for the GIL-enabled build is usually undesirable since it
52+ unnecessarily limits compatibility between packages.
5753
5854Some concrete examples of such issues have been discussed in the `Environment
5955marker for free-threading `_ Discourse thread:
6056
6157- Cython has (experimental) support for free-threading only in its master
6258 branch, and is used by a lot of projects that already publish ``cp313t ``
6359 wheels. Picking up the wrong Cython version is causing a lot of obscure build
64- failures and runtime crashes. It would be beneficial if the metadata
65- could express that (c.f. `Require Cython Pre-release for Free-Threaded
66- Python `_).
60+ failures and runtime crashes. It would be beneficial if the metadata could
61+ express that (c.f. `Require Cython Pre-release for Free-Threaded Python `_).
6762- CFFI has no support for free-threading yet, and `Armin Rigo, one of the
6863 maintainers, has stated
6964 <https://github.com/python-cffi/cffi/pull/143#issuecomment-2580781899> `__
@@ -79,11 +74,12 @@ marker for free-threading`_ Discourse thread:
7974While these concrete examples may be addressed later this year by Cython and
8075CFFI making compatible releases, the same issue is going to repeat further up
8176the stack. The free-threading rollout is expected to take several years, and an
82- environment marker for free-threading will make that rollout significantly easier.
77+ environment marker for free-threading will make that rollout significantly
78+ easier.
8379
8480Another important ABI feature that is not yet covered by environment markers is
8581the bitness of the interpreter. In most cases, the ``sys_platform `` or
86- ``platform_system `` markers are enough, because there's only a single bitness
82+ ``platform_system `` markers are enough, because there is only a single bitness
8783in use per platform. This is not the case on Windows however: both 32-bit and
888464-bit Python interpreters are widely used on x86-64 Windows. Not being able to
8985distinguish between the two may be relevant for packages that provide compiled
@@ -104,17 +100,6 @@ on the existing ecosystem.
104100The existing grammar proposed in :pep: `508 ` lends itself to a straightforward
105101extension to include the new environment marker.
106102
107- Adding ``sys.abi_features `` to the Python Standard Library
108- ----------------------------------------------------------
109-
110- The Python standard library already has mechanisms to provide information about
111- the ABI of the interpreter, like the :data: `!sys.abiflags ` attribute and the
112- :mod: `sysconfig ` module, but these do not provide all the `ABI Features `_ that
113- are relevant for environment markers.
114- The new attribute :data: `!sys.abi_features ` is a natural extension to the
115- existing mechanisms, and it is a simple and intuitive way to provide this
116- information, including the information from :data: `!sys.abiflags `.
117-
118103A Forward Looking View on Free-Threaded Python
119104----------------------------------------------
120105
@@ -169,11 +154,11 @@ ABI Features
169154------------
170155
171156ABI features are intrinsic properties of the Python interpreter, expressed as
172- simple, understandable strings.
173- However, not all features are equally applicable to all Python interpreters or
174- Python versions. For example, the distinction between free-threaded and
175- GIL-enabled interpreters is only relevant for CPython 3.13 onwards, but the bitness
176- of the interpreter is relevant for all interpreters.
157+ simple, understandable strings. However, not all features are equally
158+ applicable to all Python interpreters or Python versions. For example, the
159+ distinction between free-threaded and GIL-enabled interpreters is only relevant
160+ for CPython 3.13 onwards, but the bitness of the interpreter is relevant for
161+ all interpreters.
177162
178163All interpreters MUST handle the following ABI features as stated.
179164ABI features that are restricted to particular interpreters MUST NOT be
@@ -199,43 +184,14 @@ which case there MUST be at most one feature present.
199184 build [#bitness ]_. If the bitness is unknown or neither 32-bit nor 64-bit,
200185 this feature MUST NOT be present.
201186
202- .. _pep-780-sys.abi_features :
203-
204- The :data: `!sys.abi_features ` Attribute
205- ---------------------------------------
206-
207- Making the ABI features available in an easily accessible, expressive,
208- standardized way is useful beyond the scope of environment markers.
209- For example, ``"32-bit" in sys.abi_features `` is much more expressive than the
210- current standard test of comparing ``sys.maxsize `` with ``2**32 ``, which can
211- be found more than ten thousand times on GitHub.
212- If one wants to determine whether the interpreter is a debug build, there is
213- currently no standardized, cross platform way to do so, and ``sys.abiflags `` is
214- not available on Windows.
215- Hence, the `ABI Features `_ listed above are added to the Python standard
216- library.
217-
218- Since they are all the result of compile time choices describing basic features
219- of the interpreter, the most intuitive place to put them is in ``sys ``.
220- Since there is no intrinsic order, nor a possibility for duplication, they are
221- added as a ``frozenset `` of strings.
222-
223- All Python interpreters MUST provide the ``sys.abi_features `` attribute as a
224- ``frozenset `` of strings, which MUST contain only the `ABI Features `_ that are
225- defined in this PEP or in a subsequent PEP.
226-
227- An example value would be ``sys.abi_features == frozenset({"free-threading",
228- "debug", "32-bit"}) `` on a free-threaded debug build for win32.
229-
230187.. _pep-780-sys_abi_features :
231188
232189The ``sys_abi_features `` Environment Marker
233190-------------------------------------------
234191
235192To make ABI features available in dependency specifications, a new environment
236193marker variable, ``sys_abi_features ``, is added to the format of dependency
237- specifiers with the same semantics as the ``sys.abi_features `` attribute
238- proposed above.
194+ specifiers.
239195
240196To do this, we need to extend the grammar laid out in :pep: `508 ` and maintained
241197in the :ref: `packaging:dependency-specifiers ` and document the possible values.
@@ -262,19 +218,14 @@ Like the grammar, also the overview table of environment markers in
262218 - Python equivalent
263219 - Sample values
264220 * - ``sys_abi_features ``
265- - `` sys.abi_features `` [ #sys-abi-features ]_
266- - ``frozenset() ``, `` frozenset( {'free-threading', '64-bit'}) ``,
267- ``frozenset( {'gil-enabled', 'debug', '32-bit'}) ``
221+ - no direct equivalent available
222+ - ``{'free-threading', '64-bit'} ``,
223+ ``{'gil-enabled', 'debug', '32-bit'} ``
268224
269225With these additions, ABI features can be used in dependency specifications via
270226the ``in `` operator to test for the presence of a feature, or the ``not in ``
271227operator to test for the absence of a feature.
272228
273- Note that the presence of ``sys.abi_features `` in the Python standard library
274- makes implementation particularly easy for new Python versions, but its absence
275- in older versions does not prevent the implementation of the new environment
276- markers, as demonstrated in the `Reference Implementation `_.
277-
278229Examples
279230========
280231
@@ -348,17 +299,10 @@ in :ref:`packaging:dependency-specifiers`.
348299The new environment marker can be introduced in the same document.
349300Additionally, both for package authors and users, free-threading specific
350301guidance can be provided at the `Python free-threading guide `_.
351- The new ``sys.abi_features `` attribute will be documented in the Python
352- standard library documentation.
353-
354302
355303Reference Implementation
356304========================
357305
358- The reference implementation for the ``sys.abi_features `` attribute can be
359- found in `Add abi_features to sys
360- <https://github.com/zklaus/cpython/pull/1> `__.
361-
362306The reference implementation for the environment markers is available in a fork
363307of the ``packaging `` library at `Environment markers for ABI features
364308<https://github.com/zklaus/packaging/pull/1> `__.
@@ -382,6 +326,7 @@ In an early discussion of the topic (`Environment marker for free-threading`_),
382326the idea of a general extension mechanism for environment markers was brought
383327up. While it is appealing to forego a whole PEP process should the need for
384328new environment markers arise in the future, there are two main challenges.
329+
385330First, a completely dynamic mechanism would present difficulties for tools that
386331rely on static analysis of dependency specifications.
387332
@@ -392,11 +337,6 @@ Second, the introduction of a dynamic mechanism would require a more complex
392337implementation in the packaging library, which would be a significant departure
393338from the current approach.
394339
395- Having said that, the new ``sys.abi_features `` attribute provides a natural
396- extension point for any new ABI features, even if specific to a subset of
397- interpreters, should the need arise to add such new features with a subsequent
398- PEP.
399-
400340Open Issues
401341===========
402342
@@ -426,11 +366,6 @@ Footnotes
426366 not sufficient to reliably determine the bitness of the interpreter,
427367 particularly on platforms that allow the execution of either kind of binary.
428368
429- .. [#sys-abi-features ] This is contingent on the availability of the
430- ``sys.abi_features `` attribute in the Python standard library. In
431- versions that don't provide this attribute, the marker will be constructed
432- from other available information, such as the ``sys.abiflags `` string.
433-
434369
435370 .. _Python experimental support for free threading : https://docs.python.org/3/howto/free-threading-python.html
436371.. _Python free-threading guide : https://py-free-threading.github.io/
0 commit comments