Skip to content

Commit c4569a9

Browse files
Editorial pass
1 parent 16feb31 commit c4569a9

File tree

1 file changed

+44
-42
lines changed

1 file changed

+44
-42
lines changed

peps/pep-0788.rst

Lines changed: 44 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,10 @@ interpreter by preventing finalization. In particular:
2222
1. :c:type:`PyInterpreterGuard`, which prevents an interpreter
2323
from finalizing.
2424
2. :c:type:`PyInterpreterView`, which provides a thread-safe way to get an
25-
interpreter guard for an interpreter without holding an
26-
:term:`attached thread state`.
25+
interpreter without holding an :term:`attached thread state`.
2726
3. :c:func:`PyThreadState_Ensure`, :c:func:`PyThreadState_EnsureFromView`,
2827
and :c:func:`PyThreadState_Release`, which are high-level APIs for
29-
getting an attached thread state whilst in arbitrary native code (similar
28+
getting an attached thread state while in arbitrary native code (similar
3029
to :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release`).
3130

3231

@@ -75,8 +74,8 @@ Foreign threads hang during interpreter finalization
7574
----------------------------------------------------
7675

7776
Many large libraries might need to call Python code in highly asynchronous
78-
situations where the desired interpreter may be finalizing or deleted, but
79-
want to continue running code after invoking the interpreter. This desire has been
77+
situations where the desired interpreter may be finalizing or has already been finalized,
78+
but want to continue running code after invoking the interpreter. This desire has been
8079
`brought up by users <https://discuss.python.org/t/78850/>`_.
8180
For example, a callback that wants to call Python code might be invoked when:
8281

@@ -122,7 +121,7 @@ Locks in native extensions can be unusable during finalization
122121

123122
When acquiring locks in a native API, it's common (and often necessary)
124123
to release the GIL (or critical sections on the free-threaded build) to
125-
allow other code to execute during lock-aquisition.
124+
allow other code to execute during aquisition of the lock.
126125
This can be problematic during finalization, because threads holding locks might
127126
be hung. For example:
128127

@@ -200,9 +199,8 @@ Interpreter guards
200199
function. The guard must eventually be closed with
201200
:c:func:`PyInterpreterGuard_Close`.
202201
203-
If the interpreter no longer exists or cannot safely run Python code,
204-
or if the process is out of memory, this function returns ``NULL`` without
205-
setting an exception.
202+
If the interpreter no longer exists, is already finalizing, or out of memory,
203+
then this function returns ``NULL`` without setting an exception.
206204
207205
The caller does not need to hold an attached thread state.
208206
@@ -215,7 +213,8 @@ Interpreter guards
215213
to enter finalization.
216214
217215
After an interpreter guard is closed, it may not be used in
218-
:c:func:`PyThreadState_Ensure`. Doing so is undefined behavior.
216+
:c:func:`PyThreadState_Ensure`. Doing so will result in undefined
217+
behavior.
219218
220219
This function cannot fail, and the caller doesn't need to hold an
221220
attached thread state.
@@ -228,38 +227,41 @@ Interpreter views
228227
229228
An opaque view of an interpreter.
230229
231-
This is a thread-safe way to access an interpreter that may have been
232-
deleted or otherwise cannot execute in another thread.
230+
This is a thread-safe way to access an interpreter that may have be
231+
finalizing or already destroyed.
233232
234233
235234
.. c:function:: PyInterpreterView *PyInterpreterView_FromCurrent(void)
236235
237236
Create a view to the current interpreter.
238237
239-
This function is generally meant to be used in tandem with
240-
:c:func:`PyInterpreterGuard_FromView`.
238+
This function is generally meant to be used alongside
239+
:c:func:`PyInterpreterGuard_FromView` or :c:func:`PyThreadState_EnsureFromView`.
241240
242241
On success, this function returns a view to the current interpreter; on
243-
failure, it returns ``0`` with an exception set.
242+
failure, it returns ``NULL`` with an exception set.
244243
245244
The caller must hold an attached thread state.
246245
247246
248247
.. c:function:: void PyInterpreterView_Close(PyInterpreterView *view)
249248
250249
Delete an interpreter view. If an interpreter view is never closed, the
251-
view's memory will never be freed.
250+
view's memory will never be freed, but there are no other consequences.
251+
(In contrast, forgetting to close a guard will infinitely hang the main
252+
thread during finalization.)
252253
253254
This function cannot fail, and the caller doesn't need to hold an
254255
attached thread state.
255256
256257
257258
.. c:function:: PyInterpreterView *PyInterpreterView_FromMain()
258259
259-
Create a view for the "main" interpreter.
260+
Create a view for the main interpreter (the first and default
261+
interpreter in a Python process).
260262
261263
On success, this function returns a view to the main
262-
interpreter; on failure, it returns ``0`` without an exception set.
264+
interpreter; on failure, it returns ``NULL`` without an exception set.
263265
Failure indicates that the process is out of memory.
264266
265267
The caller does not need to hold an attached thread state.
@@ -301,9 +303,10 @@ replace :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release`.
301303
for *guard*. It is then attached and marked as owned by ``PyThreadState_Ensure``.
302304
303305
This function will return ``NULL`` to indicate a memory allocation failure, and
304-
otherwise return an opaque view to the thread state that was previously attached
306+
otherwise return a pointer to the thread state that was previously attached
305307
(which might have been ``NULL``, in which case an non-``NULL`` sentinel value is
306-
returned instead to differentiate between failure).
308+
returned instead to differentiate between failure -- this means that this function
309+
will sometimes return an invalid ``PyThreadState`` pointer).
307310
308311
To visualize, this function is roughly equivalent to the following:
309312
@@ -347,8 +350,8 @@ replace :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release`.
347350
348351
*view* must not be ``NULL``. If the interpreter referenced by *view* has been
349352
finalized or is currently finalizing, then this function returns ``NULL`` without
350-
setting. This function may also return ``NULL`` to indicate that the process is
351-
out of memory.
353+
setting an exception. This function may also return ``NULL`` to indicate that the
354+
process is out of memory.
352355
353356
The interpreter referenced by *view* will be implicitly guarded. The
354357
guard will be released upon the corresponding :c:func:`PyThreadState_Release`
@@ -388,7 +391,8 @@ replace :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release`.
388391
for each call to ``PyThreadState_Ensure``.
389392
390393
This function will decrement an internal counter on the attached thread state. If
391-
this counter ever reaches below zero, this function emits a fatal error.
394+
this counter ever reaches below zero, this function emits a fatal error (via
395+
:c:func:`Py_FatalError`).
392396
393397
If the attached thread state is owned by ``PyThreadState_Ensure``, then the
394398
attached thread state will be deallocated and deleted upon the internal counter
@@ -483,12 +487,13 @@ The term "GIL" in ``PyGILState`` is confusing for free-threading
483487
484488
This PEP uses the prefix ``PyThreadState`` instead of ``PyGILState``
485489
is because the term "GIL" in the C API is semantically misleading.
486-
Again, in modern Python versions, :c:func:`PyGILState_Ensure` is about
490+
In modern Python versions, :c:func:`PyGILState_Ensure` is about
487491
attaching a thread state, which only incidentally acquires the GIL.
488492
489493
An attached thread state is still required to invoke the C API on the free-threaded
490-
build, but with a name that contains "GIL", it is often confused to why it is
491-
still needed.
494+
build, but with a name that contains "GIL", it is often confusing to why calls to
495+
``PyGILState_Ensure`` and ``PyGILState_Release`` are still needed from foreign
496+
threads.
492497
493498
494499
Finalization behavior for ``PyGILState_Ensure`` cannot change
@@ -499,7 +504,7 @@ There will always have to be a point in a Python program where
499504
If the interpreter is long dead, then Python obviously can't give a
500505
thread a way to invoke it. Unfortunately, :c:func:`PyGILState_Ensure`
501506
doesn't have any meaningful way to return a failure, so it has no choice
502-
but to terminate the thread or emit a fatal error. For example, this
507+
but to terminate (or hang) the thread or emit a fatal error. For example, this
503508
was discussed in
504509
`python/cpython#124622 <https://github.com/python/cpython/issues/124622>`_:
505510
@@ -538,10 +543,10 @@ For example:
538543
6. ``PyMem_Realloc`` doesn't own the buffer in the list; crash!
539544
540545
The author of this PEP acknowledges that subinterpreters are not
541-
currently a popular use-case, but strongly believes that a proposal
542-
introducing a new C API for attaching and detaching thread states
543-
should also fix this issue; otherwise, the new API may have to be
544-
deprecated in the future in order to support subinterpreters.
546+
currently a popular use-case, but believes that it would be
547+
difficult to design a new API that does not also improve the situtation
548+
for subinterpreters. Opting out of subinterpreter is support is available
549+
through :c:func:`PyThreadView_FromMain`.
545550

546551

547552
Backwards Compatibility
@@ -551,8 +556,8 @@ This PEP specifies no breaking changes.
551556

552557
Existing code **does not** have to be rewritten to use the new APIs from
553558
this PEP, and all ``PyGILState`` APIs will continue to work. Use of ``PyGILState``
554-
APIs will not emit any form of warning during compilation or at runtime. The APIs
555-
will merely not be developed further.
559+
APIs will not emit any form of warning during compilation or at runtime. There
560+
will merely not be any *new* ``PyGILState`` APIs in future versions of Python.
556561

557562

558563
Security Implications
@@ -565,9 +570,7 @@ How to Teach This
565570
=================
566571

567572
As with all C API functions, all the new APIs in this PEP will be documented
568-
in the C API documentation, ideally under the ":ref:`python:gilstate`" section.
569-
The existing ``PyGILState`` documentation should be updated accordingly to point
570-
to the new APIs.
573+
in the C API documentation.
571574

572575

573576
Examples
@@ -623,11 +626,11 @@ This example shows how to acquire a C lock in a Python method defined from C.
623626

624627
If this were called from a daemon thread, the interpreter could hang the
625628
thread while reattaching its thread state, leaving us with the lock held,
626-
in which case, any future finalizer that attempts to acquire the lock would
627-
be deadlocked.
629+
in which case any future finalizer that attempts to acquire the lock would
630+
deadlock.
628631

629632
By guarding the interpreter while the lock is held, we can be sure that the
630-
thread won't be clobbered.
633+
thread won't be clobbered or hung:
631634

632635
.. code-block:: c
633636
@@ -885,7 +888,7 @@ Hard deprecating ``PyGILState``
885888

886889
This PEP used to specify a "hard" deprecation of all APIs in the
887890
``PyGILState`` family, with a planned removal in Python 3.20 (five years)
888-
of Python 3.25 (ten years).
891+
or Python 3.25 (ten years).
889892

890893
This was eventually decided against, because while it is acknowledged that
891894
``PyGILState_Ensure`` does have some fundamental flaws, it has worked for over
@@ -894,8 +897,7 @@ for Python's ecosystem.
894897

895898
Even with the finalization issues addressed by this PEP, a large majority of
896899
existing code that uses ``PyGILState_Ensure`` currently works, and will continue
897-
to work regardless of whether new APIs exist. To quote :pep:`20`: "practicality
898-
beats purity".
900+
to work regardless of whether new APIs exist.
899901

900902

901903
Interpreter reference counting

0 commit comments

Comments
 (0)