Initial revision.

Author: ZeroIntensity

peps/pep-9999.rst

@@ -0,0 +1,327 @@
+PEP: 9999
+Title: Supporting 'yield from' in asynchronous generators
+Author: Peter Bierma <peter@python.org>
+Discussions-To: Pending
+Status: Draft
+Type: Standards Track
+Created: 07-Mar-2026
+Python-Version: 3.15
+Post-History: Pending
+
+
+Abstract
+========
+
+This PEP introduces support for :keyword:`yield from <yield>` in an
+:ref:`asynchronous generator function <asynchronous-generator-functions>`.
+
+For example, the following code is valid under this PEP:
+
+.. code-block:: python
+
+    def generator():
+        yield 1
+        yield 2
+
+    async def main():
+        yield from generator()
+
+
+
+In addition, this PEP introduces a new ``async yield from`` syntax to use
+existing ``yield from`` semantics on an asynchronous generator:
+
+.. code-block:: python
+
+    async def agenerator():
+        yield 1
+        yield 2
+
+    async def main():
+        async yield from agenerator()
+
+
+In order to allow use of ``async yield from`` as an expression, this PEP
+removes the existing limitation that asynchronous generators may not return
+a non-``None`` value. For example, the following code is valid under this
+proposal:
+
+.. code-block:: python
+
+    async def agenerator():
+        yield 1
+        return 2
+
+    async def main():
+        result = (async yield from agenerator())
+        assert result == 2
+
+
+Terminology
+===========
+
+This PEP refers to an ``async def`` function that contains a ``yield``
+as an :term:`asynchronous generator`, sometimes suffixed with "function".
+
+In contrast, the object returned by an asynchronous generator is referred to
+as an :term:`asynchronous generator iterator` in this PEP.
+
+
+Motivation
+==========
+
+
+Implementation complexity has gone down
+---------------------------------------
+
+Historically, ``yield from`` was not added to asynchronous generators due to
+concerns about the complexity of the implementation. To quote :pep:`525`:
+
+    While it is theoretically possible to implement yield from support for
+    asynchronous generators, it would require a serious redesign of the
+    generators implementation.
+
+As of March 2026, the author of this proposal does not believe this to be true
+given the current state of CPython's asynchronous generator implementation.
+This proposal comes with a reference implementation to argue this point, but
+it is acknowledged that complexity is often subjective.
+
+
+Symmetry with synchronous generators
+------------------------------------
+
+``yield from`` was added to synchronous generators in :pep:`380` because
+delegation to another generator is a useful thing to do. Due to the
+aforementioned complexity in CPython's generator implementation, PEP 525
+omitted support for ``yield from`` in asynchronous generators, but this has
+left a gap in the language.
+
+This gap has not gone unnoticed by users. There have been three separate
+requests for ``yield from`` or ``return`` behavior (which are closely related)
+in asynchronous generators:
+
+1. https://discuss.python.org/t/8897
+2. https://discuss.python.org/t/47050
+3. https://discuss.python.org/t/66886
+
+Additionally, this design flaw has `come up
+<https://stackoverflow.com/questions/47376408>`_ on StackOverflow.
+
+
+Subgenerator delegation is useful for asynchronous generators
+-------------------------------------------------------------
+
+The current workaround for the lack of ``yield from`` support in asynchronous
+generators is to use a ``for``/``async for`` loop that manually yields each
+item. This comes with a few drawbacks:
+
+1. It obscures the intent of the code and increases the amount of effort
+   necessary to work with asynchronous generators, because each delegation
+   point becomes a loop. This damages the power of asynchronous generators.
+2. :meth:`~agend.asend`, :meth:`~agen.athrow`, and :meth:`~agen.aclose`,
+   do not interact properly with the caller. This is the primary reason that
+   ``yield from`` was added in the first place.
+3. Return values are not natively supported with asynchronous generators. The
+   workaround for this it to raise an exception, which increases boilerplate.
+
+
+Specification
+=============
+
+Syntax
+------
+
+
+Compiler changes
+^^^^^^^^^^^^^^^^
+
+The compiler will no longer emit a :exc:`SyntaxError` for
+:keyword:`yield from <yield>` and :keyword:`return` statements inside
+asynchronous generators.
+
+
+Grammar changes
+^^^^^^^^^^^^^^^
+
+The ``yield_expr`` and ``simple_stmt`` rules need to be updated for the new
+``async yield from`` syntax:
+
+.. code-block::
+
+    yield_expr[expr_ty]:
+        | 'async' 'yield' 'from' a=expression
+
+    simple_stmt[stmt_ty] (memo):
+        | &('yield' | 'async') yield_stmt
+
+
+``yield from`` semantics
+------------------------
+
+This PEP retains all existing ``yield from`` semantics; the only detail is
+that asynchronous generators may now use it.
+
+Because the existing ``yield from`` behavior may only yield from a synchronous
+generator, this is true for asynchronous generators as well.
+
+For example:
+
+.. code-block:: python
+
+    def generator():
+        yield 1
+        yield 2
+        yield 3
+
+    async def main():
+        yield from generator()
+        yield 4
+
+In the above code, ``main`` will yield ``1``, ``2``, ``3``, ``4``.
+All subgenerator delegation semantics are retained.
+
+
+``async yield from`` base semantics
+-----------------------------------
+
+``async yield from`` is equivalent to ``yield from``, with the exception that:
+
+1. :meth:`~object.__aiter__` is called to retrieve the asynchronous
+   generator iterator.
+2. :meth:`~object.__asend__` is called to advance the asynchronous generator.
+
+``async yield from`` is only allowed in an asynchronous generator function;
+using it elsewhere will raise a :exc:`SyntaxError`.
+
+In an asynchronous generator, ``async yield from`` is conceptually equivalent to:
+
+.. code-block:: python
+
+    async for item in agenerator():
+        yield item
+
+``async yield from`` retains all the subgenerator delegation behavior present
+in standard ``yield from`` expressions. This behavior is outlined in :pep:`380`
+and :ref:`the documentation <yieldexpr>`. In short, values passed with
+:meth:`~generator.send` and exceptions supplied with :meth:`~generator.throw`
+are also passed to the target generator.
+
+
+``async yield from`` as an expression
+-------------------------------------
+
+``async yield from`` may also be used as an expression. For reference,
+the result of a ``yield from`` expression is the object returned by the
+synchronous generator. ``async yield from`` does the same; the expression
+value is the value returned by the executed asynchronous generator.
+
+However, Python currently prevents asynchronous generators from returning
+any non-``None`` value. This limitation is removed by this PEP.
+
+When an asynchronous generator iterator is exhausted, it will raise a
+:exc:`StopAsyncIteration` exception with a ``value`` attribute, similar
+to the existing :exc:`StopIteration` behavior with synchronous generators.
+To visualize:
+
+.. code-block:: python
+
+    async def agenerator():
+        yield 1
+        return 2
+
+    async def main():
+        gen = agenerator()
+        print(await gen.asend(None))  # 1
+        try:
+            await gen.asend(None)
+        except StopAsyncIteration as result:
+            print(result.value)  # 2
+
+
+The contents of the ``value`` attribute will be the result of the ``async
+yield from`` expression.
+
+For example:
+
+.. code-block:: python
+
+    async def agenerator():
+        yield 1
+        return 2
+
+    async def main():
+        result = (async yield from agenerator())
+        print(result)  # 2
+
+
+Rationale
+=========
+
+The distinction between ``yield from`` and ``async yield from`` in this proposal
+is consistent with existing asynchronous syntax constructs in Python.
+For example, there are two constructs for context managers: ``with`` and
+``async with``.
+
+This PEP follows this pattern; ``yield from`` continues to be synchronous, even
+in asynchronous generators, and ``async yield from`` is the asynchronous
+variation.
+
+
+Backwards Compatibility
+=======================
+
+This PEP introduces a backwards-compatible syntax change.
+
+
+Security Implications
+=====================
+
+This PEP has no known security implications.
+
+
+How to Teach This
+=================
+
+The details of this proposal will be located in Python's canonical
+documentation, as with all other language constructs. However, this PEP
+intends to be very intuitive; users should be able to deduce the behavior of
+``yield from`` in an asynchronous generator based on their own background
+knowledge of ``yield from`` in synchronous generators.
+
+
+Reference Implementation
+========================
+
+A reference implementation of this branch can be found
+`here <https://github.com/python/cpython/compare/main...zerointensity:cpython:async-yield-from>`_.
+
+
+Rejected Ideas
+==============
+
+TBD.
+
+
+Acknowledgements
+================
+
+Thanks to Bartosz Sławecki for aiding in the development of the reference
+implementation of this PEP.
+
+In the reference implementation of this proposal, the :exc:`StopAsyncIteration`
+changes in addition to the support for non-``None`` return values inside
+asynchronous generators were largely based on Alex Dixon's design from
+`python/cpython#125401 <https://github.com/python/cpython/pull/125401>`_
+
+
+Change History
+==============
+
+TBD.
+
+
+Copyright
+=========
+
+This document is placed in the public domain or under the
+CC0-1.0-Universal license, whichever is more permissive.