Harden SLURM job monitor against transient squeue failures #4142

Job	Run time
Lint Gate	1m 19s
Detect File Changes	9s
Georgia Tech \| Phoenix (gpu-acc)	1s
Github (ubuntu, mpi, debug, true)	16m 11s
Github (ubuntu, mpi, no-debug, true)	16m 12s
Github (ubuntu, mpi, no-debug, false)	16m 11s
Github (ubuntu, mpi, debug, false)	16m 12s
Github (macos, mpi, debug, false)	16m 20s
Oak Ridge \| Frontier (AMD) (cpu)	9m 32s
Oak Ridge \| Frontier (cpu)	9m 31s
Georgia Tech \| Phoenix (gpu-omp)	1s
Github (macos, mpi, no-debug, false)	16m 19s
Oak Ridge \| Frontier (AMD) (gpu-omp)	15m 58s
Oak Ridge \| Frontier (gpu-acc)	9m 32s
Oak Ridge \| Frontier (gpu-omp)	1s
Github (ubuntu, no-mpi, single, no-debug, false)	16m 12s
Georgia Tech \| Phoenix (cpu)	1s
	2h 39m 42s

Provide feedback