Skip to content

Make __exit__ return types of generator context managers specifiable #14439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
johnslavik wants to merge 10 commits into from
+20 −12
Closed

Make __exit__ return types of generator context managers specifiable #14439

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
527f396
Implement `_SuppressedExcReturnT`
johnslavik Jul 22, 2025
4e72222
Increase LoB and improve comments
johnslavik Jul 22, 2025
8e0aea7
Remove `_F` and `_AF` type var
johnslavik Jul 22, 2025
085da7f
Nit: remove commas where incorrect
johnslavik Jul 22, 2025
136beb4
Improve the comments
johnslavik Jul 22, 2025
d179fa9
Delegate `_ExcReturnT` to `_GeneratorContextManager` optional type arg
johnslavik Jul 22, 2025
c6534ee
Add note on type argument inference
johnslavik Jul 22, 2025
bac955b
Add `_ExitT_co` to generator context manager types
johnslavik Sep 7, 2025
d565db1
Is this correct?
johnslavik Sep 7, 2025
7740f2a
Merge branch 'main' into contextdecorator-call-ret-type
srittau Oct 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 20 additions & 12 deletions stdlib/contextlib.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ from abc import ABC, abstractmethod
from collections.abc import AsyncGenerator, AsyncIterator, Awaitable, Callable, Generator, Iterator
from types import TracebackType
from typing import IO, Any, Generic, Protocol, TypeVar, overload, runtime_checkable, type_check_only
from typing_extensions import ParamSpec, Self, TypeAlias
from typing_extensions import Never, ParamSpec, Self, TypeAlias

__all__ = [
"contextmanager",
Expand All @@ -32,9 +32,9 @@ _T = TypeVar("_T")
_T_co = TypeVar("_T_co", covariant=True)
_T_io = TypeVar("_T_io", bound=IO[str] | None)
_ExitT_co = TypeVar("_ExitT_co", covariant=True, bound=bool | None, default=bool | None)
_F = TypeVar("_F", bound=Callable[..., Any])
_G_co = TypeVar("_G_co", bound=Generator[Any, Any, Any] | AsyncGenerator[Any, Any], covariant=True)
_P = ParamSpec("_P")
_R = TypeVar("_R")

_SendT_contra = TypeVar("_SendT_contra", contravariant=True, default=None)
_ReturnT_co = TypeVar("_ReturnT_co", covariant=True, default=None)
Expand Down Expand Up @@ -66,9 +66,16 @@ class AbstractAsyncContextManager(ABC, Protocol[_T_co, _ExitT_co]): # type: ign
self, exc_type: type[BaseException] | None, exc_value: BaseException | None, traceback: TracebackType | None, /
) -> _ExitT_co: ...

class ContextDecorator:
_ExcReturnT = TypeVar("_ExcReturnT", Never, None, default=Never)

# __exit__ can suppress exceptions by returning a true value.
# _ExcReturnT describes function's return type after an exception occurs:
# - Never (default, the context manager never suppresses exceptions)
Copy link
Copy Markdown
Collaborator

@srittau srittau Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we can really use Never to say "may raise an exception". Every function can potentially raise an exception – for example KeyboardInterrupt. As Never is the bottom type, X | Never is equivalent to just X:

from typing import Never

def foo() -> Never | int:
    return 3

reveal_type(foo())  # Revealed type is "builtins.int"
foo() + 5  # legal

Copy link
Copy Markdown
Member Author

@johnslavik johnslavik Oct 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

X | Never is equivalent to just X

Yes, that's the entire point of this solution!
We don't use Never to signal something may raise an exception, but to not extend the return type with None when no suppression happens. This way we leave the return type unchanged, because we union it with Never.

It's an intentional design that's exactly based on what stems from your explanation of Never :-)

Copy link
Copy Markdown
Collaborator

@srittau srittau Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, makes sense.

Copy link
Copy Markdown
Member Author

@johnslavik johnslavik Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can make this comment a little clearer, FWIW. Would you like me to do that? @srittau

# - None (the context manager may suppress exceptions)
# See #13512.
class ContextDecorator(Generic[_ExcReturnT]):
def _recreate_cm(self) -> Self: ...
def __call__(self, func: _F) -> _F: ...
def __call__(self, func: Callable[_P, _R]) -> Callable[_P, _R | _ExcReturnT]: ...

class _GeneratorContextManagerBase(Generic[_G_co]):
# Ideally this would use ParamSpec, but that requires (*args, **kwargs), which this isn't. see #6676
Expand All @@ -81,29 +88,30 @@ class _GeneratorContextManagerBase(Generic[_G_co]):
class _GeneratorContextManager(
_GeneratorContextManagerBase[Generator[_T_co, _SendT_contra, _ReturnT_co]],
AbstractContextManager[_T_co, bool | None],
ContextDecorator,
ContextDecorator[_ExcReturnT], # _ExcReturnT is inferred by the type checker
Generic[_T_co, _SendT_contra, _ReturnT_co, _ExcReturnT, _ExitT_co],
):
def __exit__(
self, typ: type[BaseException] | None, value: BaseException | None, traceback: TracebackType | None
) -> bool | None: ...
) -> _ExitT_co: ...

def contextmanager(func: Callable[_P, Iterator[_T_co]]) -> Callable[_P, _GeneratorContextManager[_T_co]]: ...

if sys.version_info >= (3, 10):
_AF = TypeVar("_AF", bound=Callable[..., Awaitable[Any]])

class AsyncContextDecorator:
# _ExcReturnT: see ContextDecorator.
class AsyncContextDecorator(Generic[_ExcReturnT]):
def _recreate_cm(self) -> Self: ...
def __call__(self, func: _AF) -> _AF: ...
def __call__(self, func: Callable[_P, _R]) -> Callable[_P, _R | _ExcReturnT]: ...

class _AsyncGeneratorContextManager(
_GeneratorContextManagerBase[AsyncGenerator[_T_co, _SendT_contra]],
AbstractAsyncContextManager[_T_co, bool | None],
AsyncContextDecorator,
AsyncContextDecorator[_ExcReturnT], # _ExcReturnT is inferred by the type checker
Generic[_T_co, _SendT_contra, _ReturnT_co, _ExcReturnT, _ExitT_co],
):
async def __aexit__(
self, typ: type[BaseException] | None, value: BaseException | None, traceback: TracebackType | None
) -> bool | None: ...
) -> _ExitT_co: ...

else:
class _AsyncGeneratorContextManager(
Expand Down
Loading