Add SLURM_MEM_PER_CPU to keep_slurm variables#51
Conversation
📝 WalkthroughWalkthroughA single environment variable, SLURM_MEM_PER_CPU, was added to the keep_slurm allowlist in Executor.run_job, so it is now preserved rather than stripped when sanitizing the environment for nested srun invocations. ChangesEnvironment allowlist update
Estimated code review effort: 1 (Trivial) | ~2 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
snakemake_executor_plugin_slurm_jobstep/__init__.py (1)
208-220: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winConsider also keeping
SLURM_MEM_PER_NODE.
SLURM_MEM_PER_CPUandSLURM_MEM_PER_NODEcorrespond to the mutually-exclusive--mem-per-cpuand--memflags respectively, and only one of them is ever set for a given job. Since this allowlist currently keeps onlySLURM_MEM_PER_CPU, jobs submitted with a plain--memallocation will still have their memory setting stripped before the nestedsrun, reproducing the same OOM regression this PR fixes for--mem-per-cpujobs.Proposed fix
"SLURM_SUBMIT_HOST", - "SLURM_MEM_PER_CPU" + "SLURM_MEM_PER_CPU", + "SLURM_MEM_PER_NODE", }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@snakemake_executor_plugin_slurm_jobstep/__init__.py` around lines 208 - 220, The allowlist in the Slurm environment preservation block currently keeps only SLURM_MEM_PER_CPU, so jobs submitted with --mem still lose their memory setting before nested srun. Update the keep_slurm set in __init__.py to also include SLURM_MEM_PER_NODE alongside the existing memory variable so both mutually exclusive Slurm memory settings are preserved.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@snakemake_executor_plugin_slurm_jobstep/__init__.py`:
- Around line 208-220: The allowlist in the Slurm environment preservation block
currently keeps only SLURM_MEM_PER_CPU, so jobs submitted with --mem still lose
their memory setting before nested srun. Update the keep_slurm set in
__init__.py to also include SLURM_MEM_PER_NODE alongside the existing memory
variable so both mutually exclusive Slurm memory settings are preserved.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ccc9a18d-6d0e-44b7-8bf7-250f16375444
📒 Files selected for processing (1)
snakemake_executor_plugin_slurm_jobstep/__init__.py
I noticed my slurm jobs started being OOM all of a sudden despite using only a fraction of the allocated memory. Eventually seems to be the changes in d0c515c, where the slurm-jobset (srun) was not inheriting the memory allocation of the slurm-executor (sbatch). I have a memory value in my config
default-resources, which was being used instead.I confirmed this change correctly forwards the memory requested to the actual jobstep via
sacct. Our HPC does not allow passing--mem=0, so this may a fragile approach to more generally passing on the mem/mem-per-cpu variable.Summary by CodeRabbit
sruncommands inherit the expected memory allocation.