I’m not sure if this is what you are trying to accomplish but we do something similar using features and job constraints to get jobs to run on any set of processor types that are available. We have 4 generations of processors on one of our clusters and our MPI jobs need them to run on all of one type. Each node has the appropriate processor feature assigned:
NodeName=r1i[0-3]n[0-15] Feature=wes NodeName=r2i[0-3]n[0-17] Feature=san NodeName=r3i[0-3]n[0-17] Feature=has NodeName=r4i[0-3]n[0-17] Feature=bro All of those nodes are in the same partition. Jobs that need it then request any processor type, but all require all the same type like this: #SBATCH -n 96 #SBATCH --constraint=[wes|san|has|bro] Read the sbatch man page on constraints to see the options and what those brackets mean. This works very well – users get whatever processor type is free. You might be able to adapt this to your situation. From: slurm-users <slurm-users-boun...@lists.schedmd.com> on behalf of Alexander John Mamach <alex.mam...@northwestern.edu> Reply-To: Slurm User Community List <slurm-users@lists.schedmd.com> Date: Thursday, March 22, 2018 at 4:35 PM To: Slurm Mailing List <slurm-users@lists.schedmd.com> Subject: [slurm-users] Automatically migrating jobs to different partitions? Hi all, I’ve been looking into a way to automatically migrate queued jobs from one partition to another. For example, if someone submits in partition A and must wait for resources, move their job request to partition B and try to run, and if they must still wait, then try partition C, etc? Thanks, Alex Systems Administrator Processing & Information Platform Services, Cyberinfrastructure Northwestern University Information Technology (NUIT) 2020 Ridge Ave Evanston, IL 60208-4311 O: (847) 491-2219 M: (312) 887-1881 www.it.northwestern.edu<http://www.it.northwestern.edu>
