[slurm-users] Hi-prio jobs are bypassed by low-prio jobs

2023-05-09 Thread MichaƂ Kadlof
Hi, A few tasks with higher priority give way to tasks with lower priority, and I don't understand why. I noticed that the hi-prio tasks require 4 or 8 x GPUs on a single node, while the bypassing tasks only use 1 x GPU, but I'm not sure if it's related. High-priority tasks have a specific v

[slurm-users] Job scheduling bug?

2023-05-09 Thread Luke Sudbery
We recently upgraded from 20.11.9 to 22.05.8 and appear to have a problem with jobs not being scheduled on nodes with free resources since then. It particularly noticeable on one particular partition with only one GPU node in it. Jobs queuing for this node are the highest priority in the queue a