Merge branch 'main' into c-thread-pep

Author: ZeroIntensity

peps/pep-0728.rst

@@ -187,9 +187,10 @@ determining `supported and unsupported operations
 The ``extra_items`` Class Parameter
 -----------------------------------
 
-For a TypedDict type that specifies ``extra_items``, during construction, the
-value type of each unknown item is expected to be non-required and assignable
-to the ``extra_items`` argument. For example::
+By default ``extra_items`` is unset.  For a TypedDict type that specifies
+``extra_items``, during construction, the value type of each unknown item
+is expected to be non-required and assignable to the ``extra_items`` argument.
+For example::
 
     class Movie(TypedDict, extra_items=bool):
         name: str
@@ -233,15 +234,19 @@ Here, ``'year'`` in ``a`` is an extra key defined on ``Movie`` whose value type
 is ``int``. ``'other_extra_key'`` in ``b`` is another extra key whose value type
 must be assignable to the value of ``extra_items`` defined on ``MovieBase``.
 
-``extra_items`` is also supported with the functional syntax::
-
-    Movie = TypedDict("Movie", {"name": str}, extra_items=int | None)
-
 .. _typed-dict-closed:
 
 The ``closed`` Class Parameter
 ------------------------------
 
+When neither ``extra_items`` nor ``closed=True`` is specified, ``closed=False``
+is assumed. The TypedDict should allow non-required extra items of value type
+``ReadOnly[object]`` during inheritance or assignability checks, to
+preserve the default TypedDict behavior. Extra keys included in TypedDict
+object construction should still be caught, as mentioned in TypedDict's
+`typing spec
+<https://typing.python.org/en/latest/spec/typeddict.html#supported-and-unsupported-operations.>`__.
+
 When ``closed=True`` is set, no extra items are allowed. This is equivalent to
 ``extra_items=Never``, because there can't be a value type that is assignable to
 :class:`~typing.Never`. It is a runtime error to use the ``closed`` and
@@ -275,8 +280,11 @@ child class is also closed::
 
 As a consequence of ``closed=True`` being equivalent to ``extra_items=Never``,
 the same rules that apply to ``extra_items=Never`` also apply to
-``closed=True``. It is possible to use ``closed=True`` when subclassing if the
-``extra_items`` argument is a read-only type::
+``closed=True``. While they both have the same effect, ``closed=True`` is
+preferred over ``extra_items=Never``.
+
+It is possible to use ``closed=True`` when subclassing if the ``extra_items``
+argument is a read-only type::
 
     class Movie(TypedDict, extra_items=ReadOnly[str]):
         pass
@@ -290,11 +298,6 @@ the same rules that apply to ``extra_items=Never`` also apply to
 This will be further discussed in
 :ref:`a later section <pep728-inheritance-read-only>`.
 
-When neither ``extra_items`` nor ``closed=True`` is specified, the TypedDict
-is assumed to allow non-required extra items of value type ``ReadOnly[object]``
-during inheritance or assignability checks. This preserves the existing behavior
-of TypedDict.
-
 ``closed`` is also supported with the functional syntax::
 
     Movie = TypedDict("Movie", {"name": str}, closed=True)
@@ -326,13 +329,21 @@ For type checking purposes, ``Unpack[SomeTypedDict]`` with extra items should be
 treated as its equivalent in regular parameters, and the existing rules for
 function parameters still apply::
 
-    class Movie(TypedDict, extra_items=int):
+    class MovieNoExtra(TypedDict):
+        name: str
+
+    class MovieExtra(TypedDict, extra_items=int):
         name: str
 
-    def f(**kwargs: Unpack[Movie]) -> None: ...
+    def f(**kwargs: Unpack[MovieNoExtra]) -> None: ...
+    def g(**kwargs: Unpack[MovieExtra]) -> None: ...
 
     # Should be equivalent to:
-    def f(*, name: str, **kwargs: int) -> None: ...
+    def f(*, name: str) -> None: ...
+    def g(*, name: str, **kwargs: int) -> None: ...
+
+    f(name="No Country for Old Men", year=2007) # Not OK. Unrecognized item
+    g(name="No Country for Old Men", year=2007) # OK
 
 Interaction with Read-only Items
 --------------------------------
@@ -585,8 +596,33 @@ arguments of this type when constructed by calling the class object::
         year=2007,
     )  # Not OK. Extra items not allowed
 
-Interaction with Mapping[KT, VT]
---------------------------------
+Supported and Unsupported Operations
+------------------------------------
+
+This statement from the `typing spec
+<https://typing.python.org/en/latest/spec/typeddict.html#supported-and-unsupported-operations>`__
+still holds true.
+
+    Operations with arbitrary str keys (instead of string literals or other
+    expressions with known string values) should generally be rejected.
+
+This means that indexed accesses and assignments with arbitrary keys can still
+be rejected even when ``extra_items`` is specified.
+
+Operations that already apply to ``NotRequired`` items should generally also
+apply to extra items, following the same rationale from the `typing spec
+<https://typing.python.org/en/latest/spec/typeddict.html#supported-and-unsupported-operations>`__:
+
+    The exact type checking rules are up to each type checker to decide. In some
+    cases potentially unsafe operations may be accepted if the alternative is to
+    generate false positive errors for idiomatic code.
+
+Some operations are allowed due to the TypedDict being
+:term:`typing:assignable` to ``Mapping[str, VT]`` or ``dict[str, VT]``.
+The two following sections will expand on that.
+
+Interaction with Mapping[str, VT]
+---------------------------------
 
 A TypedDict type is :term:`typing:assignable` to a type of the form ``Mapping[str, VT]``
 when all value types of the items in the TypedDict
@@ -618,12 +654,12 @@ and ``items()`` on such TypedDict types::
         reveal_type(movie.items())  # Revealed type is 'dict_items[str, str]'
         reveal_type(movie.values())  # Revealed type is 'dict_values[str, str]'
 
-Interaction with dict[KT, VT]
------------------------------
+Interaction with dict[str, VT]
+------------------------------
 
 Because the presence of ``extra_items`` on a closed TypedDict type
 prohibits additional required keys in its :term:`typing:structural`
-:term:`typing:subtypes <subtype>`, we can determine if the TypedDict type and
+:term:`subtypes <subtype>`, we can determine if the TypedDict type and
 its structural subtypes will ever have any required key during static analysis.
 
 The TypedDict type is :term:`typing:assignable` to ``dict[str, VT]`` if all
@@ -708,8 +744,25 @@ been removed in Python 3.13.
 Because this is a type-checking feature, it can be made available to older
 versions as long as the type checker supports it.
 
-Open Issues
-===========
+Rejected Ideas
+==============
+
+Use ``@final`` instead of  ``closed`` Class Parameter
+-----------------------------------------------------
+
+This was discussed `here <https://github.com/python/mypy/issues/7981>`__.
+
+Quoting a relevant `comment
+<https://github.com/python/mypy/issues/7981#issuecomment-2080161813>`__
+from Eric Traut:
+
+    The @final class decorator indicates that a class cannot be subclassed. This
+    makes sense for classes that define nominal types. However, TypedDict is a
+    structural type, similar to a Protocol. That means two TypedDict classes
+    with different names but the same field definitions are equivalent types.
+    Their names and hierarchies don't matter for determining type consistency.
+    For that reason, @final has no impact on a TypedDict type consistency rules,
+    nor should it change the behavior of items or values.
 
 Use a Special ``__extra_items__`` Key with the ``closed`` Class Parameter
 -------------------------------------------------------------------------
@@ -725,7 +778,7 @@ where ``closed=True`` is required for ``__extra_items__`` to be treated
 specially, to avoid key collision.
 
 Some members of the community concern about the elegance of the syntax.
-Practiaclly, the key collision with a regular key can be mitigated with
+Practically, the key collision with a regular key can be mitigated with
 workarounds, but since using a reserved key is central to this proposal,
 there are limited ways forward to address the concerns.
 
@@ -767,9 +820,6 @@ types altogether, but there are some disadvantages. `For example
 - The types don't appear in an annotation context, so their evaluation will
   not be deferred.
 
-Rejected Ideas
-==============
-
 Allowing Extra Items without Specifying the Type
 ------------------------------------------------
 
@@ -827,19 +877,24 @@ For example:
         [index: string]: number | string
     }
 
-This is a known limitation discussed in `TypeScript's issue tracker
-<https://github.com/microsoft/TypeScript/issues/17867>`__,
-where it is suggested that there should be a way to exclude the defined keys
-from the index signature so that it is possible to define a type like
-``MovieWithExtraNumber``.
+While this restriction allows for sound indexed accesses with arbitrary keys,
+it comes with usability limitations discussed in `TypeScript's issue tracker
+<https://github.com/microsoft/TypeScript/issues/17867>`__.
+A suggestion was to allow excluding the defined keys from the index signature,
+to define a type like ``MovieWithExtraNumber``. This probably involves
+subtraction types, which is beyond the scope of this PEP.
 
 Reference Implementation
 ========================
 
-An earlier revision of proposal is supported in `pyright 1.1.352
-<https://github.com/microsoft/pyright/releases/tag/1.1.352>`_, and `pyanalyze
+This is supported in `pyright 1.1.386
+<https://github.com/microsoft/pyright/releases/tag/1.1.386>`_, and an earlier
+revision is supported in `pyanalyze
 0.12.0 <https://github.com/quora/pyanalyze/releases/tag/v0.12.0>`_.
 
+This is also supported in `typing-extensions 4.13.0
+<https://pypi.org/project/typing-extensions/4.13.0/>`_.
+
 Acknowledgments
 ===============