ReferencePool::drop_deferred_references is called on every attach and it checks is_empty under a process global mutex.
For multithreaded embedders it becomes a serialization point. The mutex acquisition rate scales with threads and attach rate. Contended std mutex parks threads in the kernel.
I stumbled up on this while profiling the Pants build system engine (a Rust tokio runtime driving Python 3.14t code, pyo3 0.28.3) where I can measure anti-scaling behaviour attributed to ReferencePool::drop_deferred_references parking threads.
The proposed solution is to add an atomic bool to track whether the pool is "dirty". This is not a novel idea. #1608 introduced a dirty bool which was later removed. In #4174 it was noted that a dirty flag has the same cache-coherence traffic as the mutex's own futex word in the uncontended case. That does not hold under contention, though.
#4174 also suggested replacing the mutex with a lock-free solution. The dirty flag is a more minimal fix.
ReferencePool::drop_deferred_referencesis called on every attach and it checksis_emptyunder a process global mutex.For multithreaded embedders it becomes a serialization point. The mutex acquisition rate scales with threads and attach rate. Contended
stdmutex parks threads in the kernel.I stumbled up on this while profiling the Pants build system engine (a Rust tokio runtime driving Python 3.14t code, pyo3 0.28.3) where I can measure anti-scaling behaviour attributed to
ReferencePool::drop_deferred_referencesparking threads.The proposed solution is to add an atomic bool to track whether the pool is "dirty". This is not a novel idea. #1608 introduced a dirty bool which was later removed. In #4174 it was noted that a dirty flag has the same cache-coherence traffic as the mutex's own futex word in the uncontended case. That does not hold under contention, though.
#4174 also suggested replacing the mutex with a lock-free solution. The dirty flag is a more minimal fix.