PEP 828: Supporting 'yield from' in asynchronous generators (#4854)

Co-authored-by: Brian Schubert <brianm.schubert@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

Author: 4 people

.github/CODEOWNERS

@@ -701,6 +701,7 @@ peps/pep-0822.rst  @methane
 peps/pep-0825.rst  @warsaw @dstufft
 peps/pep-0826.rst  @savannahostrowski
 peps/pep-0827.rst  @1st1
+peps/pep-0828.rst  @ZeroIntensity
 # ...
 peps/pep-2026.rst  @hugovk
 # ...

peps/pep-0828.rst

@@ -0,0 +1,361 @@
+PEP: 828
+Title: Supporting 'yield from' in asynchronous generators
+Author: Peter Bierma <peter@python.org>
+Discussions-To: Pending
+Status: Draft
+Type: Standards Track
+Created: 07-Mar-2026
+Python-Version: 3.15
+Post-History: `07-Mar-2026 <https://discuss.python.org/t/106430>`__
+
+
+Abstract
+========
+
+This PEP introduces support for :keyword:`yield from <yield>` in an
+:ref:`asynchronous generator function <asynchronous-generator-functions>`.
+
+For example, the following code is valid under this PEP:
+
+.. code-block:: python
+
+    def generator():
+        yield 1
+        yield 2
+
+    async def main():
+        yield from generator()
+
+
+
+In addition, this PEP introduces a new ``async yield from`` syntax to use
+existing ``yield from`` semantics on an asynchronous generator:
+
+.. code-block:: python
+
+    async def agenerator():
+        yield 1
+        yield 2
+
+    async def main():
+        async yield from agenerator()
+
+
+In order to allow use of ``async yield from`` as an expression, this PEP
+removes the existing limitation that asynchronous generators may not return
+a non-``None`` value. For example, the following code is valid under this
+proposal:
+
+.. code-block:: python
+
+    async def agenerator():
+        yield 1
+        return 2
+
+    async def main():
+        result = async yield from agenerator()
+        assert result == 2
+
+
+Terminology
+===========
+
+This PEP refers to an ``async def`` function that contains a ``yield``
+as an :term:`asynchronous generator`, sometimes suffixed with "function".
+
+In contrast, the object returned by an asynchronous generator is referred to
+as an :term:`asynchronous generator iterator` in this PEP.
+
+
+Motivation
+==========
+
+
+Implementation complexity has gone down
+---------------------------------------
+
+Historically, ``yield from`` was not added to asynchronous generators due to
+concerns about the complexity of the implementation. To quote :pep:`525`:
+
+    While it is theoretically possible to implement ``yield from`` support for
+    asynchronous generators, it would require a serious redesign of the
+    generators implementation.
+
+As of March 2026, the author of this proposal does not believe this to be true
+given the current state of CPython's asynchronous generator implementation.
+This proposal comes with a reference implementation to argue this point, but
+it is acknowledged that complexity is often subjective.
+
+
+Symmetry with synchronous generators
+------------------------------------
+
+``yield from`` was added to synchronous generators in :pep:`380` because
+delegation to another generator is a useful thing to do. Due to the
+aforementioned complexity in CPython's generator implementation, PEP 525
+omitted support for ``yield from`` in asynchronous generators, but this has
+left a gap in the language.
+
+This gap has not gone unnoticed by users. There have been three separate
+requests for ``yield from`` or ``return`` behavior (which are closely related)
+in asynchronous generators:
+
+1. https://discuss.python.org/t/8897
+2. https://discuss.python.org/t/47050
+3. https://discuss.python.org/t/66886
+
+Additionally, this design decision has `come up
+<https://stackoverflow.com/questions/47376408>`__ on Stack Overflow.
+
+
+Subgenerator delegation is useful for asynchronous generators
+-------------------------------------------------------------
+
+The current workaround for the lack of ``yield from`` support in asynchronous
+generators is to use a ``for``/``async for`` loop that manually yields each
+item. This comes with a few drawbacks:
+
+1. It obscures the intent of the code and increases the amount of effort
+   necessary to work with asynchronous generators, because each delegation
+   point becomes a loop. This damages the power of asynchronous generators.
+2. :meth:`~agen.asend`, :meth:`~agen.athrow`, and :meth:`~agen.aclose`,
+   do not interact properly with the caller. This is the primary reason that
+   ``yield from`` was added in the first place.
+3. Return values are not natively supported with asynchronous generators. The
+   workaround for this it to raise an exception, which increases boilerplate.
+
+
+Specification
+=============
+
+Syntax
+------
+
+
+Compiler changes
+^^^^^^^^^^^^^^^^
+
+The compiler will no longer emit a :exc:`SyntaxError` for
+:keyword:`yield from <yield>` and :keyword:`return` statements inside
+asynchronous generators.
+
+
+Grammar changes
+^^^^^^^^^^^^^^^
+
+The ``yield_expr`` and ``simple_stmt`` rules need to be updated for the new
+``async yield from`` syntax:
+
+.. code-block:: peg
+
+    yield_expr[expr_ty]:
+        | 'async' 'yield' 'from' a=expression
+
+    simple_stmt[stmt_ty] (memo):
+        | &('yield' | 'async') yield_stmt
+
+
+``yield from`` behavior in asynchronous generators
+--------------------------------------------------
+
+This PEP retains all existing ``yield from`` semantics; the only detail is
+that asynchronous generators may now use it.
+
+Because the existing ``yield from`` behavior may only yield from a synchronous
+generator, this is true for asynchronous generators as well.
+
+For example:
+
+.. code-block:: python
+
+    def generator():
+        yield 1
+        yield 2
+        yield 3
+
+    async def main():
+        yield from generator()
+        yield 4
+
+In the above code, ``main`` will yield ``1``, ``2``, ``3``, ``4``.
+All subgenerator delegation semantics are retained.
+
+
+``async yield from`` as a statement
+-----------------------------------
+
+``async yield from`` is equivalent to ``yield from``, with the exception that:
+
+1. :meth:`~object.__aiter__` is called to retrieve the asynchronous
+   generator iterator.
+2. :meth:`~agen.asend` is called to advance the asynchronous generator
+   iterator.
+
+``async yield from`` is only allowed in an asynchronous generator function;
+using it elsewhere will raise a :exc:`SyntaxError`.
+
+In an asynchronous generator, ``async yield from`` is conceptually equivalent to:
+
+.. code-block:: python
+
+    async for item in agenerator():
+        yield item
+
+``async yield from`` retains all the subgenerator delegation behavior present
+in standard ``yield from`` expressions. This behavior is outlined in :pep:`380`
+and :ref:`the documentation <yieldexpr>`. In short, values passed with
+:meth:`~agen.asend` and exceptions supplied with :meth:`~agen.athrow`
+are also passed to the target generator.
+
+
+``async yield from`` as an expression
+-------------------------------------
+
+``async yield from`` may also be used as an expression. For reference,
+the result of a ``yield from`` expression is the object returned by the
+synchronous generator. ``async yield from`` does the same; the expression
+value is the value returned by the executed asynchronous generator.
+
+However, Python currently prevents asynchronous generators from returning
+any non-``None`` value. This limitation is removed by this PEP.
+
+When an asynchronous generator iterator is exhausted, it will raise a
+:exc:`StopAsyncIteration` exception with a ``value`` attribute, similar
+to the existing :exc:`StopIteration` behavior with synchronous generators.
+To visualize:
+
+.. code-block:: python
+
+    async def agenerator():
+        yield 1
+        return 2
+
+    async def main():
+        gen = agenerator()
+        print(await gen.asend(None))  # 1
+        try:
+            await gen.asend(None)
+        except StopAsyncIteration as result:
+            print(result.value)  # 2
+
+
+The contents of the ``value`` attribute will be the result of the ``async
+yield from`` expression.
+
+For example:
+
+.. code-block:: python
+
+    async def agenerator():
+        yield 1
+        return 2
+
+    async def main():
+        result = async yield from agenerator()
+        print(result)  # 2
+
+
+Rationale
+=========
+
+The distinction between ``yield from`` and ``async yield from`` in this proposal
+is consistent with existing asynchronous syntax constructs in Python.
+For example, there are two constructs for context managers: ``with`` and
+``async with``.
+
+This PEP follows this pattern; ``yield from`` continues to be synchronous, even
+in asynchronous generators, and ``async yield from`` is the asynchronous
+variation.
+
+
+Backwards Compatibility
+=======================
+
+This PEP introduces a backwards-compatible syntax change.
+
+
+Security Implications
+=====================
+
+This PEP has no known security implications.
+
+
+How to Teach This
+=================
+
+The details of this proposal will be located in Python's canonical
+documentation, as with all other language constructs. However, this PEP
+intends to be very intuitive; users should be able to deduce the behavior of
+``yield from`` in an asynchronous generator based on their own background
+knowledge of ``yield from`` in synchronous generators.
+
+
+Potential footguns
+------------------
+
+Forgetting to ``await`` a future
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In :mod:`asyncio`, a :ref:`future <asyncio-future-obj>` object is natively
+iterable. This means that if one were trying to iterate over the result of a
+future, forgetting to :keyword:`await` the future may accidentally await the
+future itself, leading to a spurious error.
+
+For example:
+
+.. code-block:: python
+
+    import asyncio
+
+    async def steps():
+        await asyncio.sleep(0.25)
+        await asyncio.sleep(0.25)
+        await asyncio.sleep(0.25)
+        return [1, 2, 3]
+
+    async def generator():
+        # Forgot to await!
+        yield from asyncio.ensure_future(steps())
+
+    async def run():
+        total = 0
+        async for i in generator():
+            # TypeError?!
+            total += i
+        print(total)
+
+
+Reference Implementation
+========================
+
+A reference implementation of this PEP can be found
+`here <https://github.com/python/cpython/compare/main...zerointensity:cpython:async-yield-from>`__.
+
+
+Rejected Ideas
+==============
+
+TBD.
+
+
+Acknowledgements
+================
+
+Thanks to Bartosz Sławecki for aiding in the development of the reference
+implementation of this PEP. In addition, the :exc:`StopAsyncIteration`
+changes in addition to the support for non-``None`` return values inside
+asynchronous generators were largely based on Alex Dixon's design from
+`python/cpython#125401 <https://github.com/python/cpython/pull/125401>`__
+
+
+Change History
+==============
+
+TBD.
+
+
+Copyright
+=========
+
+This document is placed in the public domain or under the
+CC0-1.0-Universal license, whichever is more permissive.