Harden benchmark workflow: retry builds, proactive clean, robust monitoring #4340

Job	Run time
Detect File Changes	29s
Lint Gate	1m 22s
Georgia Tech \| Phoenix (gpu-omp)	1h 28m 2s
Georgia Tech \| Phoenix (cpu)	1h 25m 50s
Oak Ridge \| Frontier (gpu-acc)	4h 24m 45s
Oak Ridge \| Frontier (AMD) (cpu)	3h 35m 39s
Github (ubuntu, no-mpi, single, no-debug, false)	29m 1s
Github (macos, mpi, debug, false)	57m 12s
Oak Ridge \| Frontier (AMD) (gpu-omp)	3h 32m 17s
Github (ubuntu, mpi, debug, true)	2h 33m 29s
Github (ubuntu, mpi, debug, false)	3h 10m 2s
Github (ubuntu, mpi, no-debug, true)	1h 34m 0s
Oak Ridge \| Frontier (cpu)	4h 8m 44s
Github (ubuntu, mpi, no-debug, false)	2h 15m 48s
Github (macos, mpi, no-debug, false)	1h 12m 55s
Georgia Tech \| Phoenix (gpu-acc)	1h 24m 40s
Oak Ridge \| Frontier (gpu-omp)	2h 45m 52s
	1d 11h 0m 7s

Provide feedback