On 7/1/21 7:08 am, Brian Andrus wrote:
I have a partition where one of the nodes has a node-locked license. That license is not used by everyone that uses the partition.
This might be a case for using a reservation on that node with the MaxStartDelay flag to set the maximum amount of time (in minutes) that jobs that need to run in the reservation are willing to wait for a job on the node to clean up and exit.
The candidate jobs need to use the --signal flag with the R option to specify how many seconds of warning they would need to clean up before being preempted.
If the amount of time they say they need is less than the MaxStartDelay then they are candidates to run on those nodes _outside_ of the reservation, and when the actual work comes along they will get told to get out of the way and, if they fail to, they'll get killed.
I presume people have to request a license in Slurm to get sent to that node so you could automatically add that reservation to jobs that request the license.
All the best, Chris -- Chris Samuel : http://www.csamuel.org/ : Berkeley, CA, USA
