Hello, I am trying to setup my SLURM cluster. One of thing I want to achieve was to schedule jobs which will be run on when there are no high priority tasks. My understanding is that this can be achieved by either configuring a partition with pre-empt mode 'Suspend/Reque' with priority for this being very low or have a QOS configured with very low priority.
Here are my questions: 1. Is there built-in 'scavenger' feature/partition or 'qos' that I can make use of? 2. When I created a 'scavenger' partition with lower priority (preempt mode=suspend), and debug partition with higher priority (preempt mode=suspend), scheduler is not pre-empting the jobs run on scavenger partition when I submit the jobs with debug partition immediately. However if I change the pre-empt mode for scavenger partition to 'requeue', pre-emptions happens immediately. Thanks