[slurm-users] Preventing --exclusive on a per-partition basis

2023-03-21 Thread Russell Jones
Hi all, We are running into the documented issues of job preemption not working for jobs running in a lower priority queue, but the user used --exclusive=user in the job submission. I have found the example job_submit.lua file for preventing using this flag, but I don't want to prevent it on ever

[slurm-users] Slurm + IntelMPI

2023-03-21 Thread Hermann Schwärzler
Hi everybody, in our new cluster we have configured Slurm with SelectType=select/cons_tres SelectTypeParameters=CR_Core_Memory ProctrackType=proctrack/cgroup TaskPlugin=task/affinity,task/cgroup which I think is quite a usual setup. After installing Intel MPI (using Spack v0.19) we saw that th

Re: [slurm-users] Troubles with cgroups

2023-03-21 Thread Jason Simms
Hello Hermann, Thanks for following up about this. What you say makes sense: at Lafayette, we didn't experience the issue until upgrading to a Slurm version that supported cgroups/v2, and here at Swarthmore, we are still on a version of Slurm that doesn't and we don't have the issue (both Rocky 8)

Re: [slurm-users] Troubles with cgroups

2023-03-21 Thread Hermann Schwärzler
Hi Jason, thank you for your reply. From what I can tell your problem *is* the same as ours. BTW: we were already talking about disabling swap in our nodes as a last resort. :-) In the meantime we made some new findings: we can trigger the error when (with cgroups/v2) we set memory.high and m