Skip to content

Commit 22d6e1f

Browse files
sbryngelsonSpencer Bryngelson
andauthored
ci: enable --stepmgr for Frontier jobs to relieve slurmctld pressure (#1611)
Each CI test/bench job is a single-node allocation that runs the full regression suite in-process via ./mfc.sh test, launching one srun per target (pre_process/simulation/post_process) for ~560 cases = ~1700+ srun step-creates per job, up to 32 concurrent. This congests the Frontier Slurm controller. --stepmgr delegates step management to each job's slurmstepd instead of routing every srun through slurmctld, which is the appropriate mechanism for many-step single-allocation workloads. Added via the existing extra_sbatch hook for both frontier and frontier_amd; Phoenix unaffected. Co-authored-by: Spencer Bryngelson <sbryngelson@login10.frontier.olcf.ornl.gov>
1 parent 4cebbb0 commit 22d6e1f

1 file changed

Lines changed: 5 additions & 2 deletions

File tree

.github/scripts/submit-slurm-job.sh

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,10 @@ case "$cluster" in
5252
account="CFD154"
5353
job_prefix="MFC"
5454
qos="hackathon"
55-
extra_sbatch=""
55+
# Let each job's slurmstepd broker its own steps instead of routing
56+
# every srun through slurmctld. The in-job test suite launches ~1700+
57+
# srun steps per allocation, which congests the Frontier controller.
58+
extra_sbatch="#SBATCH --stepmgr"
5659
test_time="01:59:00"
5760
bench_time="01:59:00"
5861
gpu_partition_dynamic=false
@@ -62,7 +65,7 @@ case "$cluster" in
6265
account="CFD154"
6366
job_prefix="MFC"
6467
qos="hackathon"
65-
extra_sbatch=""
68+
extra_sbatch="#SBATCH --stepmgr"
6669
test_time="01:59:00"
6770
bench_time="01:59:00"
6871
gpu_partition_dynamic=false

0 commit comments

Comments
 (0)