My cluster uses Slurm's preemption feature to allow jobs to run on otherwise idle resources but still allow owners of compute hardware to have on-demand access to their equipment. However, preemption means a cancelled job. One of our users is asking if there is a method to have his job do cleanup work when it is cancelled/preempted. Right now the user simply has cleanup code at the end of the job script but of course this won't run if the job dies or is cancelled.
Any idea what I can suggest to this person to have their "cleanup" code always run when the job ends for any reason?
-- Jeff White HPC Systems Engineer Information Technology Services - WSU
