Hi list, We are migrating from SGE to Slurm. I'm having hard-time to make process pinning work.
Whatever --cpu-bind is set to, I can not get the placement I want. It works as expected for cores, threads, boards and sockets, but as soon as I pass a map_cpu or mask_cpu it goes on its own. Nodes are 1 board, 2 sockets, 4 cores/socket, 1 thread per core. Tests were made with Slurm 20.11.0. I've built 20.11.1 this morning. Am I missing something ? $ srun --exclusive --cpu-bind=v,cores -N 1-1 -n 2 testmpi.sh cpu-bind=MASK - mania-8, task 0 0 [3126]: mask 0x1 set cpu-bind=MASK - mania-8, task 1 1 [3127]: mask 0x4 set $ srun --exclusive --cpu-bind=v,threads -N 1-1 -n 2 testmpi.sh cpu-bind=MASK - mania-8, task 0 0 [3261]: mask 0x1 set cpu-bind=MASK - mania-8, task 1 1 [3262]: mask 0x4 set $ srun --exclusive --cpu-bind=v,boards -N 1-1 -n 4 testmpi.sh cpu-bind=MASK - mania-8, task 0 0 [3515]: mask 0x1 set cpu-bind=MASK - mania-8, task 1 1 [3516]: mask 0x4 set cpu-bind=MASK - mania-8, task 3 3 [3518]: mask 0x40 set cpu-bind=MASK - mania-8, task 2 2 [3517]: mask 0x10 set $ srun --exclusive --cpu-bind=v,sockets -N 1-1 -n 4 testmpi.sh cpu-bind=MASK - mania-8, task 0 0 [3646]: mask 0x55 set cpu-bind=MASK - mania-8, task 1 1 [3647]: mask 0x55 set cpu-bind=MASK - mania-8, task 2 2 [3648]: mask 0x55 set cpu-bind=MASK - mania-8, task 3 3 [3649]: mask 0x55 set $ srun --exclusive --cpu-bind=v,mask_cpu:0xa -N 1-1 -n 2 testmpi.sh cpu-bind=MASK - mania-8, task 0 0 [2816]: mask 0x5 set cpu-bind=MASK - mania-8, task 1 1 [2819]: mask 0x5 set $ srun --exclusive --cpu-bind=v,mask_cpu:0x0 -N 1-1 -n 4 testmpi.sh cpu-bind=MASK - mania-8, task 0 0 [3800]: mask 0x55 set cpu-bind=MASK - mania-8, task 1 1 [3801]: mask 0x55 set cpu-bind=MASK - mania-8, task 3 3 [3803]: mask 0x55 set cpu-bind=MASK - mania-8, task 2 2 [3802]: mask 0x55 set $ srun --exclusive --cpu-bind=v,none -N 1-1 -n 4 testmpi.sh cpu-bind=MASK - mania-8, task 0 0 [5053]: mask 0x55 set cpu-bind=MASK - mania-8, task 1 1 [5054]: mask 0x55 set cpu-bind=MASK - mania-8, task 2 2 [5056]: mask 0x55 set cpu-bind=MASK - mania-8, task 3 3 [5057]: mask 0x55 set $ srun --exclusive --cpu-bind=v,none,mask_cpu:0x33,0x33,0x33,0x33 -N 1-1 -n 4 testmpi.sh cpu-bind=MASK - mania-8, task 0 0 [5191]: mask 0x55 set cpu-bind=MASK - mania-8, task 1 1 [5192]: mask 0x55 set cpu-bind=MASK - mania-8, task 2 2 [5193]: mask 0x55 set cpu-bind=MASK - mania-8, task 3 3 [5194]: mask 0x55 set Thanks, -- Ionel -- Beicip-Franlab SA - 232 av. Napoleon Bonaparte - BP 2132-92502 Rueil-Malmaison Cedex Capital: EUR 6 000 000 - TVA FR 54 679 804 047 - RCS Nanterre 679 804 047 This message and any attachments (the message) are confidential and intended solely for the addressees. Any unauthorised use, dissemination or reproduction is strictly prohibited. The sender does not accept liability for any errors or omissions in the contents of this message arising as a result of e-mail transmission.
slurm.conf
Description: slurm.conf
cgroup.conf
Description: cgroup.conf
