You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/hpc/05_submitting_jobs/01_slurm_submitting_jobs.md
+17-1Lines changed: 17 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,12 +28,28 @@ Non-stakeholders to temporarily use stakeholder resources (a stakeholder group t
28
28
:::
29
29
30
30
## Job Submission on Torch
31
-
As stated in the tuturial, always only request the compute resources (e.g., GPUs, CPUs, memory) needed for the job. Requesting too many resources can prevent your job from being scheduled within an adequate time. The `SLURM` scheduler will automatically dispatch jobs to all accessible GPU partitions that match resource requests.
31
+
As stated in the tutorial, be sure to only request the compute resources (e.g., GPUs, CPUs, memory) needed for the job. Requesting too many resources can prevent your job from being scheduled within an adequate time. The `SLURM` scheduler will automatically dispatch jobs to all accessible GPU partitions that match resource requests.
32
32
33
33
:::danger Low GPU Utilization Policy
34
34
Jobs with low GPU utilization will be automatically canceled. The exact threshold is TBD, but enforcement will be very aggressive.
35
35
:::
36
36
37
+
:::info Maximum number of jobs a user can submit
38
+
This depends on the requested wall time.<br />
39
+
You can check it with the following command:
40
+
```bash
41
+
[NetID@torch-login-b-1 ~]$ sacctmgr show qos cpu48,cpu168,gpu48,gpu168,interactive format=name%16,maxTresPU%32,MaxWall,MaxSubmitJobsPerUser
The values above are just an example and they may change without notice, so please check within your account for the most up to date results.
51
+
:::
52
+
37
53
## Preemptible jobs on Torch
38
54
On Torch, users may run "preemptible" jobs on stakeholder resources that their group does not own. This allows the stakeholder resources to be utilized by non-stakeholders which may otherwise be idle. To make the best use of these resources, you are encouraged to adopt checkpoint/restart to allow for resumption of the workload in subsequent jobs.
0 commit comments