On 08/29/2016 07:41 PM, Luis Torres wrote:
Hello Folks,
We have recently deployed SLURM v 15.08.7-build1 on Ubuntu 16.04 submission
and execution nodes with apt-get; we built and installed the source
packages of the same release on Ubuntu 14.04 for the controller.
Our primary issue is that we’re not able to run multiple jobs in a single
node despite closely following the configuration options suggested in the
SLURM documentation. Namely we’ve added “Shared=YES” to our NodeName
definition and have configured the scheduling Type to select/cons_res and
CR_CPU_Memory parameters.
I've included the output of scontrol show config and the slurm.conf we're
using.
Any help on the matter will be greatly appreciated,
Hello Luis,
We got into the same problem last year, when upgrading to latest version of
Slurm: only one job was allowed by Slurm to run on each node, where we
earlier shared nodes between up to 16 jobs per node.
The problem was a new default for the DefMemPerCPU configuration variable,
I think.
Our solution was to set the variables DefMemPerNode and DefMemPerCPU locally
for each partition, like this:
PartitionName=core Nodes=m[5-120,155-208] Default=YES Shared=NO
DefaultTime=00:00:01 MaxTime=14400 MaxNodes=1 DefMemPerCPU=8000 State=UP
PartitionName=node Nodes=m[5-208] Shared=EXCLUSIVE DefaultTime=00:00:01
MaxTime=14400 DefMemPerNode=0 State=UP
PartitionName=devcore Nodes=m[1-4] Shared=NO DefaultTime=00:00:01 MaxTime=60
MaxNodes=1 DefMemPerCPU=8000 State=UP
PartitionName=devel Nodes=m[1-4] Shared=EXCLUSIVE DefaultTime=00:00:01
MaxTime=60 MaxNodes=2 DefMemPerNode=0 State=UP
Our core and devcore partitions are for jobs that allocate between 1 and 16
cores on a single node. Our node and devel partitions are for jobs that
allocate one or more full nodes.
Best wishes,
-- Lennart Karlsson, UPPMAX, Uppsala University, Sweden
http://uppmax.uu.se/
