Skip to content

Commit 5501b73

Browse files
committed
Make FTP collections use adaptive_workers (not fixed num_workers)
Phase 6 step 3. With the controller state in place (commit b176751), this commit wires it into the per-collection dispatch path: _PyGC_GetParallelWorkers() now returns pool->adaptive_workers (the random-walk-controlled value, updated after each collection) instead of the configured pool->num_workers. This is the FTP analogue of the GIL build's behaviour where each dispatch site passes par_gc->adaptive_workers to _PyGC_DispatchAndWait. The per-collection mechanism on FTP (_PyGCFTParState in gc_free_threading.c::gc_collect_internal — used by update_refs, mark_heap, propagate) creates a fresh worker set per collection and sizes the page buckets accordingly. With this change, the worker count varies per collection as adaptive_workers walks. KNOWN INCOMPLETE: the persistent thread pool mechanism (_PyGCThreadPool::workers, used for *_pool_work() worker functions) still uses pool->num_workers directly via its barrier dispatch. Making those scale too requires the per-worker condvar dispatch refactor — the FTP analogue of Phase 5.3 on the GIL side. That's the next commit; this one keeps things bisect-safe by leaving the pool mechanism unchanged. Verified: FTP build, all 5 parallel-GC test files pass (177 tests). 40-iteration smoke test completes without hang.
1 parent 5ef3a61 commit 5501b73

1 file changed

Lines changed: 16 additions & 3 deletions

File tree

Include/internal/pycore_gc_ft_parallel.h

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -528,16 +528,29 @@ struct _PyGCScanHeapResult {
528528
int gc_reason; // GC reason for shutdown check
529529
};
530530

531-
// Get number of parallel GC workers based on configuration.
532-
// Returns 0 if parallel GC is disabled, otherwise the worker count.
531+
// Get number of parallel GC workers for the next collection.
532+
// Returns 0 if parallel GC is disabled, otherwise the adaptive worker count
533+
// chosen by the random-walk controller (see pycore_gc_random_walk.h).
534+
//
535+
// The thread pool persists with the maximum num_workers; this returns the
536+
// CURRENT adaptive count, which varies per collection. If the pool isn't
537+
// initialised yet (first call before enable_parallel) the configured
538+
// num_workers is used as a fall-through.
533539
static inline int
534540
_PyGC_GetParallelWorkers(PyInterpreterState *interp)
535541
{
536542
struct _gc_runtime_state *gc = &interp->gc;
537543
if (!gc->parallel_gc_enabled) {
538544
return 0;
539545
}
540-
// num_workers is required and validated by gc.enable_parallel()
546+
_PyGCThreadPool *pool = gc->thread_pool;
547+
if (pool != NULL) {
548+
// Adaptive worker count, updated by _PyGC_RandomWalkUpdate after
549+
// each collection. Guaranteed in [2, pool->num_workers].
550+
return (int)pool->adaptive_workers;
551+
}
552+
// No pool yet (e.g. during initial enable_parallel before pool init):
553+
// fall back to the configured max.
541554
return gc->parallel_gc_num_workers;
542555
}
543556

0 commit comments

Comments
 (0)