Re: [slurm-users] allocate last MPI-rank to an exclusive node?

Hi, we had the same issue and solved it by using the 'plane' distribution in combination with MPMD style srun, e.g. in your example #SBATCH -N 3 # 3 nodes with 10 cores each #SBATCH -n 21 # 21 MPI-tasks in sum #SBATCH --cpus-per-task=1 # if you do not want hyperthreading cat > mpmd.conf <<

Re: [slurm-users] allocate last MPI-rank to an exclusive node?

Hi Hendryk, Thanks for information. The 'plane' distribution works fine with me as well. Regards, Jing Gong From: slurm-users on behalf of Hendryk Bockelmann Sent: Tuesday, February 19, 2019 10:48 To: slurm-users@lists.schedmd.com Subject: Re: [slurm-

Re: [slurm-users] allocate last MPI-rank to an exclusive node?

Hi Chris, Thanks for your information. Regards, Jing Gong From: slurm-users on behalf of Chris Samuel Sent: Tuesday, February 19, 2019 03:47 To: slurm-users@lists.schedmd.com Subject: Re: [slurm-users] allocate last MPI-rank to an exclusive node? On M

Re: [slurm-users] Strange error, submission denied

--ntasks-per-node is meant to be used in conjunction with --nodes option. From https://slurm.schedmd.com/sbatch.html: *--ntasks-per-node*= Request that /ntasks/ be invoked on each node. If used with the *--ntasks* option, the *--ntasks* option will take precedence and the *--ntasks-

Re: [slurm-users] Priority access for a group of users

I just set this up a couple of weeks ago myself. Creating two partitions is definitely the way to go. I created one partition, "general" for normal, general-access jobs, and another, "interruptible" for general-access jobs that can be interrupted, and then set PriorityTier accordingly in my slu

Re: [slurm-users] Strange error, submission denied

Hi Prentice, On 2/19/19 2:58 PM, Prentice Bisbal wrote: --ntasks-per-node is meant to be used in conjunction with --nodes option. From https://slurm.schedmd.com/sbatch.html: *--ntasks-per-node*= Request that /ntasks/ be invoked on each node. If used with the *--ntasks* option, the

Re: [slurm-users] Strange error, submission denied

I just made a little bit debugging, setting the debug level to debug5 during submission. I submitted (or at least tried to) two jobs: sbatch -n 48 --wrap hostname got submitted, got jobid 199801 sbatch -N 1 --ntasks-per-node=48 --wrap hostname submission denied, got jobid 199805 The only diff

Re: [slurm-users] Strange error, submission denied

On Tuesday, 19 February 2019 10:14:21 PM PST Marcus Wagner wrote: > sbatch -N 1 --ntasks-per-node=48 --wrap hostname > submission denied, got jobid 199805 On one of our 40 core nodes with 2 hyperthreads: $ srun -C gpu -N 1 --ntasks-per-node=80 hostname | uniq -c 80 nodename02 The spec is: