I did some tests and this doesn't work with MPI, in the following sense: the way we were running parallel jobs was by invoking `sbatch -n4 script.sh` where the script.sh would contain an `mpirun` command without specifying the number of processors. It would then simply claim the ones assigned to it by sbatch. With one CPU as per the slurm.conf, mpirun would only ever get one processor, hence no parallelization.
The other test I did was to try OverSubscribe=Exclusive on the partition, but the slurmctld failed to start so there is some combination of config options that it is unhappy about. The workable option I can think of right now is to have everyone run their jobs with --exclusive as an extra argument. On Thu, Apr 30, 2020 at 12:22 PM Rutger Vos <rutger....@naturalis.nl> wrote: > This seems like the easiest solution, but how would that interact with MPI > jobs? Apologies if these are really basic questions, and thank you very > much for thinking along. > > Rutger > > On Wed, Apr 29, 2020 at 10:25 PM Doug Meyer <dame...@raytheon.com> wrote: > >> Change node definition in slurm.conf for that one node to 1 CPU. >> >> >> >> *Doug Meyer* >> >> >> >> *From:* slurm-users <slurm-users-boun...@lists.schedmd.com> *On Behalf >> Of *Rutger Vos >> *Sent:* Wednesday, April 29, 2020 1:20 PM >> *To:* Slurm User Community List <slurm-users@lists.schedmd.com> >> *Subject:* [External] Re: [slurm-users] one job at a time - how to set? >> >> >> >> Hi Michael, >> >> >> >> thanks very much for your swift reply. So here we would have to convince >> the users they'd have to specify this when submitting, right? I.e. 'sbatch >> --exclusive myjob.sh', if I understand correctly. Would there be a way to >> simply enforce this, i.e. at the slurm.conf level or something? >> >> >> >> Thanks again! >> >> >> >> Rutger >> >> >> >> On Wed, Apr 29, 2020 at 10:06 PM Renfro, Michael <ren...@tntech.edu> >> wrote: >> >> That’s a *really* old version, but >> https://slurm.schedmd.com/archive/slurm-15.08.13/sbatch.html indicates >> there’s an exclusive flag you can set. >> >> On Apr 29, 2020, at 1:54 PM, Rutger Vos <rutger....@naturalis.nl> wrote: >> >> *.* >> >> Hi, >> >> >> >> for a smallish machine that has been having degraded performance we want >> to implement a policy where only one job (submitted with sbatch) is allowed >> to run and any others submitted after it are supposed to wait in line. >> >> >> >> I assumed this was straightforward but I can't seem to figure it out. Can >> I set that up in slurm.conf or in some other way? Thank you very much for >> your help. BTW we are running slurm 15.08.7 if that is at all relevant. >> >> >> >> Best wishes, >> >> >> >> Dr. Rutger A. Vos >> >> Researcher / Bioinformatician >> >> >> >> >> >> >> >> +31717519600 - +31627085806 >> >> rutger....@naturalis.nl - www.naturalis.nl >> >> Darwinweg 2, 2333 CR Leiden >> >> Postbus 9517, 2300 RA Leiden >> >> >> >> <https://www.naturalis.nl/lang-leve> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> Met vriendelijke groet, >> >> >> >> Dr. Rutger A. Vos >> >> Researcher / Bioinformatician >> >> >> >> >> >> >> >> +31717519600 - +31627085806 >> >> rutger....@naturalis.nl - www.naturalis.nl >> >> Darwinweg 2, 2333 CR Leiden >> >> Postbus 9517, 2300 RA Leiden >> >> >> >> <https://www.naturalis.nl/lang-leve> >> >> >> >> >> >> >> >> >> >> >> > > > -- > > Met vriendelijke groet, > > Dr. Rutger A. Vos > Researcher / Bioinformatician > > > > > > > +31717519600 - +31627085806 > rutger....@naturalis.nl - www.naturalis.nl > Darwinweg 2, 2333 CR Leiden > Postbus 9517, 2300 RA Leiden > > <https://www.naturalis.nl/over-ons> <https://www.naturalis.nl/lang-leve> > > > > > > > > > > -- Met vriendelijke groet, Dr. Rutger A. Vos Researcher / Bioinformatician +31717519600 - +31627085806 rutger....@naturalis.nl - www.naturalis.nl Darwinweg 2, 2333 CR Leiden Postbus 9517, 2300 RA Leiden <https://www.naturalis.nl/over-ons> <https://www.naturalis.nl/lang-leve>
