Bug description
On the free-threaded build, iterating an :func:itertools.tee iterator concurrently crashes the interpreter (a segfault from corrupted reference counts).
This is a crash-safety bug, not a request to make tee thread-safe. itertools.tee iterators are documented as not thread-safe, and concurrent use returning undefined results or raising RuntimeError is by design and is preserved. But a data race that corrupts reference counts and segfaults the interpreter is memory-unsafety, which the free-threaded build must not permit regardless of the (undefined) logical result.
It is a sibling gap in the free-threading hardening of itertools: the other iterators (chain, product, combinations, permutations, accumulate, count, zip_longest, and so on) already wrap their __next__ in Py_BEGIN_CRITICAL_SECTION and so do not crash under concurrent iteration. tee was the one that was missed.
Root cause
tee branches share a linked list of internal teedataobject cells. Three paths touch that shared state without synchronization:
- iteration reads/extends a cell (
teedataobject_getitem: numread, values[], running) and the link chain (teedataobject_jumplink: nextlink);
- each branch advances its own position (
teeobject.index, teeobject.dataobj);
- teardown clears cells and walks the chain (
teedataobject_clear / teedataobject_safe_decref).
Iterating one branch from multiple threads, or iterating sibling branches that share the same cells, races on these fields, corrupting reference counts and crashing (observed as a SEGV in teedataobject_clear).
Reproduction
On a free-threaded build with ThreadSanitizer, iterating one tee branch from several threads, or consuming each sibling branch of a tee in its own thread, reliably reports data races and then segfaults. Both patterns are covered by a regression test under Lib/test/test_free_threading/.
Fix
Lock each teedataobject while reading, extending, or clearing it, and snapshot each branch's position under the tee object's own lock, revalidating before advancing, so the two locks are never nested (a nested critical section can be suspended under contention, which would break the atomicity the fetch relies on). Concurrent iteration of one tee remains undefined and may still raise RuntimeError, as documented, but no longer crashes.
Linked PRs
Bug description
On the free-threaded build, iterating an :func:
itertools.teeiterator concurrently crashes the interpreter (a segfault from corrupted reference counts).This is a crash-safety bug, not a request to make tee thread-safe.
itertools.teeiterators are documented as not thread-safe, and concurrent use returning undefined results or raisingRuntimeErroris by design and is preserved. But a data race that corrupts reference counts and segfaults the interpreter is memory-unsafety, which the free-threaded build must not permit regardless of the (undefined) logical result.It is a sibling gap in the free-threading hardening of
itertools: the other iterators (chain,product,combinations,permutations,accumulate,count,zip_longest, and so on) already wrap their__next__inPy_BEGIN_CRITICAL_SECTIONand so do not crash under concurrent iteration.teewas the one that was missed.Root cause
teebranches share a linked list of internalteedataobjectcells. Three paths touch that shared state without synchronization:teedataobject_getitem:numread,values[],running) and the link chain (teedataobject_jumplink:nextlink);teeobject.index,teeobject.dataobj);teedataobject_clear/teedataobject_safe_decref).Iterating one branch from multiple threads, or iterating sibling branches that share the same cells, races on these fields, corrupting reference counts and crashing (observed as a SEGV in
teedataobject_clear).Reproduction
On a free-threaded build with ThreadSanitizer, iterating one tee branch from several threads, or consuming each sibling branch of a tee in its own thread, reliably reports data races and then segfaults. Both patterns are covered by a regression test under
Lib/test/test_free_threading/.Fix
Lock each
teedataobjectwhile reading, extending, or clearing it, and snapshot each branch's position under the tee object's own lock, revalidating before advancing, so the two locks are never nested (a nested critical section can be suspended under contention, which would break the atomicity the fetch relies on). Concurrent iteration of one tee remains undefined and may still raiseRuntimeError, as documented, but no longer crashes.Linked PRs