When submitting Toil jobs to Slurm, it is possible to set Slurm qos options with slurmArgs option. However, the configured QOS is used for every job submission, regardless of whether the job requires CPU or GPU resources. This means that if a user configures a GPU-specific QOS, it will be applied to all jobs, including those that do not require GPUs. On the other hand, if a user configures a CPU-specific QOS, it will be applied to all jobs, including those that require GPUs. This is problematic in the objective of optimizing job scheduling and resource allocation based on the specific needs of each job.
In order to address this issue, it would be necessary to implement separate QOS handling for CPU and GPU jobs in the Slurm batch system integration of Toil. This would allow users to specify different QOS options for CPU and GPU jobs, ensuring that each job is submitted with the appropriate QOS based on its resource requirements.
Adding support for separate QOS options for CPU and GPU jobs would allow users to optimize jobs resources allocation.
┆Issue is synchronized with this Jira Story
┆Issue Number: TOIL-1833
When submitting Toil jobs to Slurm, it is possible to set Slurm
qosoptions withslurmArgsoption. However, the configured QOS is used for every job submission, regardless of whether the job requires CPU or GPU resources. This means that if a user configures a GPU-specific QOS, it will be applied to all jobs, including those that do not require GPUs. On the other hand, if a user configures a CPU-specific QOS, it will be applied to all jobs, including those that require GPUs. This is problematic in the objective of optimizing job scheduling and resource allocation based on the specific needs of each job.In order to address this issue, it would be necessary to implement separate QOS handling for CPU and GPU jobs in the Slurm batch system integration of Toil. This would allow users to specify different QOS options for CPU and GPU jobs, ensuring that each job is submitted with the appropriate QOS based on its resource requirements.
Adding support for separate QOS options for CPU and GPU jobs would allow users to optimize jobs resources allocation.
┆Issue is synchronized with this Jira Story
┆Issue Number: TOIL-1833