I'm running an even older slurm than you (it does what I need, I am a
team of one and I have many things to take care of other than chasing
the latest version of every piece of software).
Anyway, did you try Shared=EXCLUSIVE in the partition configuration?
>From the (v14.11.7) slurm.conf man page:
Shared Controls the ability of the partition to execute more than
one job at a time on each resource [...]
EXCLUSIVE Allocates entire nodes to jobs even with select/cons_res
configured. Jobs that run in partitions with "Shared=EXCLUSIVE" will
have exclusive access to all allocated nodes. (node, socket or core
depending upon the value of SelectTypeParameters).
Allan
Rutger Vos <rutger....@naturalis.nl> writes:
> I did some tests and this doesn't work with MPI, in the following
> sense: the way we were running parallel jobs was by invoking `sbatch
> -n4 script.sh` where the script.sh would contain an `mpirun` command
> without specifying the number of processors. It would then simply claim
> the ones assigned to it by sbatch. With one CPU as per the slurm.conf,
> mpirun would only ever get one processor, hence no parallelization.
>
> The other test I did was to try OverSubscribe=Exclusive on the
> partition, but the slurmctld failed to start so there is some
> combination of config options that it is unhappy about.
>
> The workable option I can think of right now is to have everyone run
> their jobs with --exclusive as an extra argument.
>
> On Thu, Apr 30, 2020 at 12:22 PM Rutger Vos <rutger....@naturalis.nl>
> wrote:
>
> This seems like the easiest solution, but how would that interact
> with MPI jobs? Apologies if these are really basic questions, and
> thank you very much for thinking along.
>
> Rutger
>
> On Wed, Apr 29, 2020 at 10:25 PM Doug Meyer <dame...@raytheon.com>
> wrote:
>
> Change node definition in slurm.conf for that one node to 1
> CPU.
>
> Doug Meyer
>
> From: slurm-users <slurm-users-boun...@lists.schedmd.com> On
> Behalf Of Rutger Vos
> Sent: Wednesday, April 29, 2020 1:20 PM
> To: Slurm User Community List <slurm-users@lists.schedmd.com>
> Subject: [External] Re: [slurm-users] one job at a time - how
> to set?
>
> Hi Michael,
>
> thanks very much for your swift reply. So here we would have to
> convince the users they'd have to specify this when submitting,
> right? I.e. 'sbatch --exclusive myjob.sh', if I
> understand correctly. Would there be a way to simply enforce
> this, i.e. at the slurm.conf level or something?
>
> Thanks again!
>
> Rutger
>
> On Wed, Apr 29, 2020 at 10:06 PM Renfro, Michael <
> ren...@tntech.edu> wrote:
>
> That’s a *really* old version, but
> https://slurm.schedmd.com/archive/slurm-15.08.13/sbatch.html
> indicates there’s an exclusive flag you can set.
>
> On Apr 29, 2020, at 1:54 PM, Rutger Vos <
> rutger....@naturalis.nl> wrote:
>
> .
>
> Hi,
>
> for a smallish machine that has been having degraded
> performance we want to implement a policy where
> only one job (submitted with sbatch) is allowed to run
> and any others submitted after it are supposed to wait
> in line.
>
> I assumed this was straightforward but I can't seem to
> figure it out. Can I set that up in slurm.conf or in
> some other way? Thank you very much for your help. BTW
> we are running slurm 15.08.7 if that is at all
> relevant.
>
> Best wishes,
>
> Dr. Rutger A. Vos
>
> Researcher / Bioinformatician
>
> [logo-new]
>
> [+31717519600 - +31627085806 ]
>
> [rutger....@naturalis.nl - www.naturalis.nl]
>
> [Darwinweg 2, 2333 CR Leiden]
>
> [Postbus 9517, 2300 RA Leiden]
>
> [schildpad]
>
> --
>
> Met vriendelijke groet,
>
> Dr. Rutger A. Vos
>
> Researcher / Bioinformatician
>
> [logo-new]
>
> [+31717519600 - +31627085806 ]
>
> [rutger....@naturalis.nl - www.naturalis.nl]
>
> [Darwinweg 2, 2333 CR Leiden]
>
> [Postbus 9517, 2300 RA Leiden]
>
> [schildpad]
>
> --
>
> Met vriendelijke groet,
>
> Dr. Rutger A. Vos
> Researcher / Bioinformatician
> [logo-new]
>
> [+31717519600 - +31627085806 ]
> [rutger....@naturalis.nl - www.naturalis.nl]
> [Darwinweg 2, 2333 CR Leiden]
> [Postbus 9517, 2300 RA Leiden]
>
> [schildpad]
>
> --
>
> Met vriendelijke groet,
>
> Dr. Rutger A. Vos
> Researcher / Bioinformatician
> [logo-new]
>
> [+31717519600 - +31627085806 ]
> [rutger....@naturalis.nl - www.naturalis.nl]
> [Darwinweg 2, 2333 CR Leiden]
> [Postbus 9517, 2300 RA Leiden]
>
> [schildpad]
>
