PEP 770: Add the dist-info.files table and reserve dist-info dirs

peps/pep-0770.rst

@@ -139,6 +139,29 @@ These decisions mean this PEP is capable of supporting any SBOM standard
 and does not favor one over the other, instead deferring the decision to
 producing projects and tools and consuming user tooling.
 
+Adding data to Python packages without new metadata versions
+------------------------------------------------------------
+
+The rollout of a new metadata version and field requires that many different
+projects and teams need to adopt the metadata version in sequence to avoid
+widespread breakage. This effect usually means a substantial delay in how
+quickly users and tools can start using new packaging features.
+
+For example, a single metadata version bump requires
+updates to PyPI, various ``pyproject.toml`` parsing and schema projects,
+the ``packaging`` library, wait for releases, then ``pip`` and other installers
+need to bundle the changes to ``packaging`` and release, then build backends can
+begin emitting the new metadata version, again wait for releases, and only then
+can projects begin using the new features. Even with this careful approach it's
+not guaranteed that tools won't break on new metadata versions and fields.
+
+To avoid this delay, this PEP proposes a new mechanism ``[additional-files]``
+to safely add data to a Python package through a registry of reserved names that
+avoids the need for new metadata fields and versions.  This mechanism allows
+build backends and tools to begin using the features described in this PEP
+immediately after acceptance without the head-of-line blocking on other
+projects adopting the PEP.
+
 What are the differences between PEP 770 and PEP 725?
 -----------------------------------------------------
 
@@ -149,8 +172,10 @@ software within Python packaging metadata. This section aims to show how these
 two PEPs are tracking different information and serving different use-cases:
 
 * PEP 725 describes **abstract dependencies**, such as requiring "a C compiler"
-  as a build-time dependency (``virtual:compiler/c``). PEP 770 describes
-  **concrete dependencies**, such as an exact name, version, architecture, and
+  as a build-time dependency (``virtual:compiler/c``) or an "OpenSSL-like
+  library" (``pkg:generic/openssl``). PEP 770 describes
+  **concrete dependencies**, more akin to dependencies in a "lock file", such as
+  an exact name, version, architecture, and
   hash of a software library distributed through AlmaLinux distribution
   (``pkg:rpm/almalinux/libssl3@3.2.0``). For cases like build dependencies this
   might result in a dependency being requested via PEP 725 and then recorded
@@ -208,12 +233,16 @@ will be made through the PEP process. The initial values in this registry are:
     ================= ==========
     ``licenses``      :pep:`639`
     ``license_files`` N/A (See :ref:`770-backwards-compat`)
+    ``LICENSES``      N/A (See :ref:`770-backwards-compat`)
     ``sboms``         :pep:`770`
     ================= ==========
 
 Build backends MUST NOT create subdirectories in the ``.dist-info`` directory
 beyond the names in the registry to avoid collisions with future reserved names.
 
+Build frontends and publishing tools MAY warn users or raise a validation error
+if any ``.dist-info`` subdirectories aren't in the registry.
+
 .. _770-spec-project-source-metadata:
 
 Project source metadata
@@ -406,13 +435,17 @@ Not shown above are around ~50 other subdirectory names that are used in a
 single project. From these results we can see:
 
 * Most subdirectories under ``.dist-info`` are to do with licensing,
-  one of which (``licenses``) is specified by :pep:`639` and other
-  (``license_files``) which is being used by the Hatch build backend.
+  one of which (``licenses``) is specified by :pep:`639` and others
+  (``license_files``, ``LICENSES``) are from draft implementations
+  of :pep:`639`.
 * The ``sboms`` subdirectory doesn't collide with existing use.
 * Other subdirectory names under ``.dist-info`` appear to be either not
   widespread or accidental.
 
-As a result of this query
+As a result of this query we can see there are already some projects placing
+directories under ``.dist-info``, so we can't require that build frontends
+raise errors for unregistered subdirectories. Instead the recommendation is
+that build frontends MAY warn the user or raise an error in this scenario.
 
 Security Implications
 =====================
@@ -509,6 +542,12 @@ documented and recommended to users. For projects which require
 additional manual SBOM annotation: tips for contributing this data and tools for
 maintaining the data can be recommended.
 
+Note that SBOM documents can vary across different Python package archives
+due to variance in dependencies, Python version, platform, architecture, etc.
+For this reason users SHOULD only use the SBOM documents contained within
+the actual downloaded and installed Python package archive and not assume that
+the SBOM documents are the same for all archives in a given package release.
+
 Reference Implementation
 ========================
 
@@ -613,8 +652,7 @@ Acknowledgements
 ================
 
 Thanks to Karolina Surma for authoring and leading :pep:`639` to acceptance.
-This PEP copies the specification from :pep:`639` for specifying files in
-project source metadata, Core Metadata, and project formats is based on.
+This PEP's initial design was heavily based on 639
 
 Copyright
 =========