PEP: Project status markers in the simple index

Signed-off-by: William Woodruff <william@yossarian.net>

Author: woodruffw

peps/pep-9999.rst

@@ -0,0 +1,371 @@
+PEP: 9999
+Title: Project status markers in the simple index
+Author: William Woodruff <william@yossarian.net>,
+        Facundo Tuesca <facundo.tuesca@trailofbits.com>
+Sponsor: Donald Stufft <donald@stufft.io>
+PEP-Delegate: Donald Stufft <donald@stufft.io>
+Discussions-To: https://discuss.python.org/t/79356
+Status: Draft
+Type: Standards Track
+Topic: Packaging
+Created: 08-Apr-2025
+Post-History: `03-Feb-2025 <https://discuss.python.org/t/79356/>`__
+
+Abstract
+========
+
+This PEP proposes a standardized set of project status markers, as well
+as a mechanism for communicating those markers in the HTML and JSON
+simple indices.
+
+Rationale and Motivation
+========================
+
+The "status" of a project is an important piece of metadata, made more important
+by growth in both the breadth and depth of the Python packaging ecosystem.
+Both humans and tools use project statuses (or proxies for status,
+such as recent activity) to determine whether the project is maintained or
+otherwise suitable for consumption.
+
+Python packaging has at least three different mechanisms for communicating
+the "status" of a project:
+
+1. Distributions can include *trove classifiers* in their metadata, as
+   originally specified in :pep:`301`. The list of supported classifiers is
+   `maintained by the PyPA <https://github.com/pypa/trove-classifiers>`_,
+   and includes the ``Development Status`` hierarchy. For example, a
+   distribution can include the ``Development Status :: 7 - Inactive``
+   classifier to indicate that the distribution's project is inactive.
+
+   Trove classifiers are flexible, but also come with significant limitations:
+   they're machine-readable and are rendered on indices like PyPI, but
+   they also require the maintainer to push one or more *new* distributions
+   each time they wish to update their project's development status.
+   Furthermore, because distributions are *de facto* immutable in the Python
+   packaging ecosystem, older distributions can't have their classifiers
+   updated to reflect the current status of the project.
+
+2. Indices can mark distributions and releases as "yanked", as originally
+   specified in :pep:`592`. Yanked distributions are not considered
+   eligible for dependency resolution.
+
+   When a distribution has been yanked, it is marked with ``data-yanked``
+   in the HTML index and with ``yanked: bool | str`` in the JSON index.
+   Additionally, indices like PyPI will hide yanked distributions by default
+   and will render them with a notice when the user navigates directly to them.
+
+   Yanking is machine-readable like trove classifiers, but is single-purpose
+   rather than general-purpose: users can specify a free-form "yank reason,"
+   but the semantics of yanking are fixed.
+
+3. PyPI itself has *project statuses*, which apply to the entire project
+   (i.e., all releases and distributions). Project statuses have both
+   maintainer- and index-admin-controllable states:
+
+   * PyPI administrators can "quarantine" a project. Quarantine behaves like
+     a strengthened yank: the project remains entirely uninstallable while
+     quarantined.
+
+   * Project maintainers can "archive" a project. Archiving a project
+     disables new release and distribution uploads to that project,
+     but otherwise has no effect on the project's downloadability.
+
+   Project statuses are machine-readable *in principle*, but are not currently
+   exposed via any of PyPI's APIs. Instead, PyPI renders project statuses on
+   each project's user-facing (i.e. non-index) HTML.
+
+In summary: there are multiple ways to communicate the "status" of a project in
+Python packaging, but none of these mechanisms are *simultaneously* (1)
+machine-readable, (2) general, (3) index-agnostic, and (4) updatable across
+the entire project (instead of per-release or per-distribution).
+
+.. csv-table::
+    :header: "Mechanism", "Machine-readable?", "General?", "Index-agnostic?", "Project updatable?"
+
+    "Trove classifiers", "✅", "✅", "✅", "❌"
+    "Yanking", "✅", "❌", "✅", "✅"
+    "Project statuses", "✅", "✅", "❌", "✅"
+
+This PEP proposes adopting PyPI's project statuses as an index-agnostic
+mechanism, satisfying all four conditions.
+
+Specification
+=============
+
+This PEP specifies two aspects: a set of project status markers,
+as well as their presentation in the standard HTML and JSON indices.
+
+Project status markers
+----------------------
+
+This PEP proposes the following project status markers.
+
+A project **MUST** have exactly one project status marker at any given time.
+
+Indices **MAY** implement any subset of the status markers specified in this
+PEP, as applicable.
+
+``active``
+~~~~~~~~~~
+
+Description: The project is active. This is the default status for a project.
+
+Index semantics: none.
+
+Installer semantics: none.
+
+``archived``
+~~~~~~~~~~~~
+
+Description: The project does not expect to be updated in the future.
+
+Index semantics:
+
+* The index hosting the project **MUST NOT** allow uploads of new distributions to
+  the project.
+* The index **MUST** offer existing distributions of the project for download.
+
+Installer semantics:
+
+* Installers **MAY** produce warnings about a project's archival.
+
+``quarantined``
+~~~~~~~~~~~~~~~
+
+Description: The project is considered generally unsafe for use, e.g. due to
+malware.
+
+Index semantics:
+
+* The index hosting the project **MUST NOT** allow uploads of new distributions to
+  the project.
+* The index **MUST NOT** offer any distributions of the project for download.
+
+Installer semantics:
+
+* Installers **MAY** produce warnings about a project's quarantine, although
+  doing so is effectively moot (as the index will not offer any distributions
+  for installation).
+
+``deprecated``
+~~~~~~~~~~~~~~
+
+Description: The project is considered obsolete, and may have been superseded
+by another project.
+
+Index semantics:
+
+* The index hosting the project **MUST** allow uploads of new distributions to
+  the project.
+* The index **MUST** offer existing distributions of the project for download.
+
+Installer semantics:
+
+* Installers **MAY** produce warnings about a project's deprecation.
+
+Status markers in the index APIs
+--------------------------------
+
+This PEP defines version 1.4 of the index APIs.
+
+HTML index
+~~~~~~~~~~
+
+The following changes are made to the
+:ref:`simple repository API <packaging:simple-repository-api-base>`:
+
+* The index **SHALL** define the ``pypi:repository-version`` as ``1.4``.
+* The index **SHOULD** add an appropriate ``pypi:project-status`` meta tag, with
+  a ``content`` of the project's status marker. The index **MAY** choose to omit
+  the ``pypi:project-status`` meta tag if the project is marked as ``active``.
+
+For example, the following would be a valid HTML index response for
+``sampleproject`` after is has been marked as ``quarantined``:
+
+.. code-block:: html
+    :emphasize-lines: 5
+
+    <!DOCTYPE html>
+    <html>
+      <head>
+        <meta name="pypi:repository-version" content="1.4">
+        <meta name="pypi:project-status" content="quarantined">
+        <title>Links for sampleproject</title>
+      </head>
+      <body>
+        <h1>Links for sampleproject</h1>
+      </body>
+    </html>
+
+Observe that, per the ``quarantined`` semantics above, the index response
+contains no distribution links for the project.
+
+JSON index
+~~~~~~~~~~
+
+The following changes are made to the
+:ref:`JSON simple index <packaging:simple-repository-api-json>`:
+
+* The index **SHALL** define the ``meta.api-version`` as ``1.4``.
+* The index **SHOULD** include a ``project-status`` key in the JSON response,
+  with a value of the project's status marker. The index **MAY** choose to omit
+  the ``project-status`` key if the project is marked as ``active``.
+
+For example, the following would be a valid JSON index response for
+``sampleproject`` after is has been marked as ``quarantined``:
+
+.. code-block:: json
+    :emphasize-lines: 5
+
+    {
+      "meta": {
+        "api-version": "1.4"
+      },
+      "project-status": "quarantined",
+      "alternate-locations": [],
+      "files": [],
+      "name": "sampleproject",
+      "versions": [
+        "1.2.0",
+        "1.3.0",
+        "1.3.1",
+        "2.0.0",
+        "3.0.0",
+        "4.0.0"
+      ]
+    }
+
+Observe that, like with the HTML index, the JSON response contains no
+distribution links for the ``quarantined`` project.
+
+Future Considerations
+=====================
+
+This PEP defines only four project status markers: ``active``, ``archived``,
+``quarantined``, and ``deprecated``.
+
+Future PEPs (or PyPA standards processes) may define additional project
+status markers, as needed.
+
+As specified in this PEP, project status markers are "bare," i.e. they
+convey no additional user-controlled metadata (such as an explanation
+for a project's archival).
+
+A future PEP may choose to extend the project
+status mechanism to include user-controlled metadata, in a manner similar
+to the free-form text allowed during release yanking.
+
+Security Implications
+=====================
+
+This PEP does not identify any positive or negative security implications
+associated with adding project status markers.
+
+How to Teach This
+=================
+
+Educating the Python community about this PEP has two aspects:
+
+* Ordinary package maintainers will need to be informed of their ability to
+  set project status markers, e.g. to inform their downstreams that
+  a project has been archived or deprecated.
+
+  If this PEP is accepted, the authors of this PEP will coordinate with
+  PyPI on appropriate maintainer-oriented documentation and communication,
+  including feature announcement blog posts and updates to
+  `PyPI's user documentation <https://docs.pypi.org>`_.
+
+* Installer and index maintainers will need to be informed of the new project
+  status markers, and how to interpret them.
+
+  If this PEP is accepted, the authors of this PEP will perform its
+  implementation on PyPI, serving as a reference implementation for other
+  indices.
+
+  This PEP does not *mandate* any changes in installer behavior. However,
+  if this PEP is accepted, the authors of this PEP will coordinate with
+  the maintainers of popular installers (e.g. ``pip``) to help each determine
+  the extent to which they wish to surface project statuses.
+
+Rejected Ideas
+==============
+
+Using "reserved" keys
+---------------------
+
+One alternative to this PEP is to avoid standardizing project status
+markers directly, but instead leverage existing mechanisms within the standards
+to communicate them in a non-standard fashion.
+
+For example, the `JSON simple index <packaging:simple-repository-api-json>`_
+says the following:
+
+    Keys (at any level) with a leading underscore are reserved as private for
+    index server use. No future standard will assign a meaning to any such key.
+
+In effect, this means that the following would be standards-compliant:
+
+.. code-block:: json
+    :emphasize-lines: 5
+
+    {
+      "meta": {
+        "api-version": "1.4"
+      },
+      "_project-status": "quarantined",
+      "alternate-locations": [],
+      "files": [],
+      "name": "sampleproject",
+      "versions": [
+        "1.2.0",
+        "1.3.0",
+        "1.3.1",
+        "2.0.0",
+        "3.0.0",
+        "4.0.0"
+      ]
+    }
+
+However, this approach has several drawbacks:
+
+* Standards-aligned tools (such as ``pip``, ``pip-audit``, and ``uv``)
+  may find it unacceptable to use a "reserved" key, since that key will
+  have no standard semantics or compatibility properties.
+* The "reserved" approach is only suitable for the JSON simple index;
+  no equivalent mechanism exists for the HTML simple index.
+  This would disadvantage consumers of the HTML simple index, as well as
+  mirror implementations that may consume the JSON index but only expose
+  an HTML index.
+
+Project markers in PyPI's non-standard JSON API
+-----------------------------------------------
+
+Another standardization-avoidance alternative is to expose project status
+markers, but only in PyPI's
+`non-standard JSON API <https://docs.pypi.org/api/json/>`_. PyPI has full
+control over the layout of this API, and could include a ``project-status``
+or similar key without needing a PEP or underscore prefix.
+
+This has similar drawbacks as the "reserved" keys approach above,
+and more generally deepens the differences between the standard
+and non-standard APIs.
+
+Multiple project status markers at once
+---------------------------------------
+
+An earlier version of this PEP considered proposing support for
+multiple project markers at once. For example, a project could be marked
+as both ``archived`` and ``quarantined``.
+
+After consideration, this was rejected for complexity reasons: having multiple
+project status markers requires the PEP to specify a conflict resolution
+mechanism when merging their semantics, as well as as state machine for which
+markers are exclusive (for example, ``active`` is conceptually exclusive with
+all other markers, while ``archived`` and ``quarantined`` are conceptually
+compatible with each other).
+
+Copyright
+=========
+
+This document is placed in the public domain or under the CC0-1.0-Universal
+license, whichever is more permissive.