fix: optimize sandbox cleaning performance#796

Merged

aldbr merged 15 commits into

DIRACGrid:mainfrom

chrisburr:fix/sandbox-cleaning-performance

Feb 26, 2026

Commits on Feb 26, 2026

fix: optimize sandbox cleaning query performance

chrisburr
authored and
aldbr
committed
fix: split workers across both deletion query types

chrisburr
authored and
aldbr
committed
fix: use cursor-based pagination for sandbox deletion queries

chrisburr
authored and
aldbr
committed
fix: simplify to single worker with parallel S3 deletions

chrisburr
authored and
aldbr
committed
fix: separate row selection from locking to avoid mass locking

chrisburr
authored and
aldbr
committed
fix: remove row locking from sandbox cleanup to avoid lock memory bloat

chrisburr
authored and
aldbr
committed
fix: chunk DB deletes into 1000-row batches with concurrent execution

chrisburr
authored and
aldbr
committed
refactor: add per-phase timing to sandbox cleanup logging

chrisburr
authored and
aldbr
committed
fix: increase S3 connection pool to 50 for parallel bulk deletes

chrisburr
authored and
aldbr
committed
fix: make S3 connection pool size configurable

chrisburr
authored and
aldbr
committed
fix: stop iterating when batch returns fewer than batch_size candidates

chrisburr
authored and
aldbr
committed
fix: handle S3 partial delete failures and pass se_name directly

chrisburr
authored and
aldbr
committed
fix: remove idx_assigned_lastaccesstime index from schema

chrisburr
authored and
aldbr
committed
fix: make sandbox cleaning parameters configurable via settings

chrisburr
authored and
aldbr
committed
fix: remove redundant default values from settings docstrings

chrisburr
authored and
aldbr
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: optimize sandbox cleaning performance#796

fix: optimize sandbox cleaning performance#796
aldbr merged 15 commits into
DIRACGrid:mainfrom
chrisburr:fix/sandbox-cleaning-performance

Commits on Feb 26, 2026

fix: optimize sandbox cleaning query performance

fix: split workers across both deletion query types

fix: use cursor-based pagination for sandbox deletion queries

fix: simplify to single worker with parallel S3 deletions

fix: separate row selection from locking to avoid mass locking

fix: remove row locking from sandbox cleanup to avoid lock memory bloat

fix: chunk DB deletes into 1000-row batches with concurrent execution

refactor: add per-phase timing to sandbox cleanup logging

fix: increase S3 connection pool to 50 for parallel bulk deletes

fix: make S3 connection pool size configurable

fix: stop iterating when batch returns fewer than batch_size candidates

fix: handle S3 partial delete failures and pass se_name directly

fix: remove idx_assigned_lastaccesstime index from schema

fix: make sandbox cleaning parameters configurable via settings

fix: remove redundant default values from settings docstrings