@@ -22,11 +22,10 @@ interpreter by preventing finalization. In particular:
22221. :c:type: `PyInterpreterGuard `, which prevents an interpreter
2323 from finalizing.
24242. :c:type: `PyInterpreterView `, which provides a thread-safe way to get an
25- interpreter guard for an interpreter without holding an
26- :term: `attached thread state `.
25+ interpreter without holding an :term: `attached thread state `.
27263. :c:func: `PyThreadState_Ensure `, :c:func: `PyThreadState_EnsureFromView `,
2827 and :c:func: `PyThreadState_Release `, which are high-level APIs for
29- getting an attached thread state whilst in arbitrary native code (similar
28+ getting an attached thread state while in arbitrary native code (similar
3029 to :c:func: `PyGILState_Ensure ` and :c:func: `PyGILState_Release `).
3130
3231
@@ -75,8 +74,8 @@ Foreign threads hang during interpreter finalization
7574----------------------------------------------------
7675
7776Many large libraries might need to call Python code in highly asynchronous
78- situations where the desired interpreter may be finalizing or deleted, but
79- want to continue running code after invoking the interpreter. This desire has been
77+ situations where the desired interpreter may be finalizing or has already been finalized,
78+ but want to continue running code after invoking the interpreter. This desire has been
8079`brought up by users <https://discuss.python.org/t/78850/ >`_.
8180For example, a callback that wants to call Python code might be invoked when:
8281
@@ -122,7 +121,7 @@ Locks in native extensions can be unusable during finalization
122121
123122When acquiring locks in a native API, it's common (and often necessary)
124123to release the GIL (or critical sections on the free-threaded build) to
125- allow other code to execute during lock- aquisition.
124+ allow other code to execute during aquisition of the lock .
126125This can be problematic during finalization, because threads holding locks might
127126be hung. For example:
128127
@@ -200,9 +199,8 @@ Interpreter guards
200199 function. The guard must eventually be closed with
201200 :c:func: `PyInterpreterGuard_Close `.
202201
203- If the interpreter no longer exists or cannot safely run Python code,
204- or if the process is out of memory, this function returns ``NULL `` without
205- setting an exception.
202+ If the interpreter no longer exists, is already finalizing, or out of memory,
203+ then this function returns ``NULL `` without setting an exception.
206204
207205 The caller does not need to hold an attached thread state.
208206
@@ -215,7 +213,8 @@ Interpreter guards
215213 to enter finalization.
216214
217215 After an interpreter guard is closed, it may not be used in
218- :c:func:`PyThreadState_Ensure`. Doing so is undefined behavior.
216+ :c:func:`PyThreadState_Ensure`. Doing so will result in undefined
217+ behavior.
219218
220219 This function cannot fail, and the caller doesn't need to hold an
221220 attached thread state.
@@ -228,38 +227,41 @@ Interpreter views
228227
229228 An opaque view of an interpreter.
230229
231- This is a thread-safe way to access an interpreter that may have been
232- deleted or otherwise cannot execute in another thread .
230+ This is a thread-safe way to access an interpreter that may have be
231+ finalizing or already destroyed .
233232
234233
235234.. c:function:: PyInterpreterView *PyInterpreterView_FromCurrent(void)
236235
237236 Create a view to the current interpreter.
238237
239- This function is generally meant to be used in tandem with
240- :c:func:`PyInterpreterGuard_FromView`.
238+ This function is generally meant to be used alongside
239+ :c:func:`PyInterpreterGuard_FromView` or :c:func:`PyThreadState_EnsureFromView` .
241240
242241 On success, this function returns a view to the current interpreter; on
243- failure, it returns ``0 `` with an exception set.
242+ failure, it returns ``NULL `` with an exception set.
244243
245244 The caller must hold an attached thread state.
246245
247246
248247.. c :function :: void PyInterpreterView_Close (PyInterpreterView *view)
249248
250249 Delete an interpreter view. If an interpreter view is never closed, the
251- view's memory will never be freed.
250+ view's memory will never be freed, but there are no other consequences.
251+ (In contrast, forgetting to close a guard will infinitely hang the main
252+ thread during finalization.)
252253
253254 This function cannot fail, and the caller doesn't need to hold an
254255 attached thread state.
255256
256257
257258.. c:function:: PyInterpreterView *PyInterpreterView_FromMain()
258259
259- Create a view for the "main" interpreter.
260+ Create a view for the main interpreter (the first and default
261+ interpreter in a Python process).
260262
261263 On success, this function returns a view to the main
262- interpreter; on failure, it returns ``0 `` without an exception set.
264+ interpreter; on failure, it returns ``NULL `` without an exception set.
263265 Failure indicates that the process is out of memory.
264266
265267 The caller does not need to hold an attached thread state.
@@ -301,9 +303,10 @@ replace :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release`.
301303 for *guard*. It is then attached and marked as owned by ``PyThreadState_Ensure``.
302304
303305 This function will return ``NULL`` to indicate a memory allocation failure, and
304- otherwise return an opaque view to the thread state that was previously attached
306+ otherwise return a pointer to the thread state that was previously attached
305307 (which might have been ``NULL ``, in which case an non-``NULL `` sentinel value is
306- returned instead to differentiate between failure).
308+ returned instead to differentiate between failure -- this means that this function
309+ will sometimes return an invalid ``PyThreadState `` pointer).
307310
308311 To visualize, this function is roughly equivalent to the following:
309312
@@ -347,8 +350,8 @@ replace :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release`.
347350
348351 *view * must not be ``NULL ``. If the interpreter referenced by *view * has been
349352 finalized or is currently finalizing, then this function returns ``NULL `` without
350- setting. This function may also return ``NULL `` to indicate that the process is
351- out of memory.
353+ setting an exception . This function may also return ``NULL `` to indicate that the
354+ process is out of memory.
352355
353356 The interpreter referenced by *view * will be implicitly guarded. The
354357 guard will be released upon the corresponding :c:func: `PyThreadState_Release `
@@ -388,7 +391,8 @@ replace :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release`.
388391 for each call to ``PyThreadState_Ensure ``.
389392
390393 This function will decrement an internal counter on the attached thread state. If
391- this counter ever reaches below zero, this function emits a fatal error.
394+ this counter ever reaches below zero, this function emits a fatal error (via
395+ :c:func: `Py_FatalError `).
392396
393397 If the attached thread state is owned by ``PyThreadState_Ensure``, then the
394398 attached thread state will be deallocated and deleted upon the internal counter
@@ -483,12 +487,13 @@ The term "GIL" in ``PyGILState`` is confusing for free-threading
483487
484488This PEP uses the prefix ``PyThreadState`` instead of ``PyGILState``
485489is because the term "GIL" in the C API is semantically misleading.
486- Again, in modern Python versions, :c:func:`PyGILState_Ensure` is about
490+ In modern Python versions, :c:func:`PyGILState_Ensure` is about
487491attaching a thread state, which only incidentally acquires the GIL.
488492
489493An attached thread state is still required to invoke the C API on the free-threaded
490- build, but with a name that contains "GIL", it is often confused to why it is
491- still needed.
494+ build, but with a name that contains "GIL", it is often confusing to why calls to
495+ ``PyGILState_Ensure`` and ``PyGILState_Release`` are still needed from foreign
496+ threads.
492497
493498
494499Finalization behavior for ``PyGILState_Ensure`` cannot change
@@ -499,7 +504,7 @@ There will always have to be a point in a Python program where
499504If the interpreter is long dead, then Python obviously can't give a
500505thread a way to invoke it. Unfortunately, :c:func:`PyGILState_Ensure`
501506doesn't have any meaningful way to return a failure, so it has no choice
502- but to terminate the thread or emit a fatal error. For example, this
507+ but to terminate (or hang) the thread or emit a fatal error. For example, this
503508was discussed in
504509`python/cpython#124622 <https:// github.com/python/cpython/issues/124622>`_:
505510
@@ -538,10 +543,10 @@ For example:
5385436. ``PyMem_Realloc`` doesn't own the buffer in the list; crash!
539544
540545 The author of this PEP acknowledges that subinterpreters are not
541- currently a popular use-case, but strongly believes that a proposal
542- introducing a new C API for attaching and detaching thread states
543- should also fix this issue; otherwise, the new API may have to be
544- deprecated in the future in order to support subinterpreters .
546+ currently a popular use-case, but believes that it would be
547+ difficult to design a new API that does not also improve the situtation
548+ for subinterpreters. Opting out of subinterpreter is support is available
549+ through :c:func: ` PyThreadView_FromMain ` .
545550
546551
547552Backwards Compatibility
@@ -551,8 +556,8 @@ This PEP specifies no breaking changes.
551556
552557Existing code **does not ** have to be rewritten to use the new APIs from
553558this PEP, and all ``PyGILState `` APIs will continue to work. Use of ``PyGILState ``
554- APIs will not emit any form of warning during compilation or at runtime. The APIs
555- will merely not be developed further .
559+ APIs will not emit any form of warning during compilation or at runtime. There
560+ will merely not be any * new * `` PyGILState `` APIs in future versions of Python .
556561
557562
558563Security Implications
@@ -565,9 +570,7 @@ How to Teach This
565570=================
566571
567572As with all C API functions, all the new APIs in this PEP will be documented
568- in the C API documentation, ideally under the ":ref: `python:gilstate `" section.
569- The existing ``PyGILState `` documentation should be updated accordingly to point
570- to the new APIs.
573+ in the C API documentation.
571574
572575
573576Examples
@@ -623,11 +626,11 @@ This example shows how to acquire a C lock in a Python method defined from C.
623626
624627If this were called from a daemon thread, the interpreter could hang the
625628thread while reattaching its thread state, leaving us with the lock held,
626- in which case, any future finalizer that attempts to acquire the lock would
627- be deadlocked .
629+ in which case any future finalizer that attempts to acquire the lock would
630+ deadlock .
628631
629632By guarding the interpreter while the lock is held, we can be sure that the
630- thread won't be clobbered.
633+ thread won't be clobbered or hung:
631634
632635.. code-block :: c
633636
@@ -885,7 +888,7 @@ Hard deprecating ``PyGILState``
885888
886889This PEP used to specify a "hard" deprecation of all APIs in the
887890``PyGILState `` family, with a planned removal in Python 3.20 (five years)
888- of Python 3.25 (ten years).
891+ or Python 3.25 (ten years).
889892
890893This was eventually decided against, because while it is acknowledged that
891894``PyGILState_Ensure `` does have some fundamental flaws, it has worked for over
@@ -894,8 +897,7 @@ for Python's ecosystem.
894897
895898Even with the finalization issues addressed by this PEP, a large majority of
896899existing code that uses ``PyGILState_Ensure `` currently works, and will continue
897- to work regardless of whether new APIs exist. To quote :pep: `20 `: "practicality
898- beats purity".
900+ to work regardless of whether new APIs exist.
899901
900902
901903Interpreter reference counting
0 commit comments