Ok so I ask the mpirun masters how would you do the following:
I submit a job with torque (we use --with-tm) like the following:
nodes=4:ppn=2
My desired outcome is to have 1 mpi process per 2 cpus and use
threaded blas (or my own OpenMP take your pick)
Our cluster has some 4 core machines thus the above job sometimes
ends up looking like
nodes=1:ppn=4+nodes=2:ppn=2
The mpirun -bynode command will work in the case i get 4 nodes with
only 2 cpus free. But if any machine other than the first machine is
my node with 4 cores free given to me by moab, I would end up
starting a extra process on the first node, where mpirun thinks
another cpu is free, but that cpu is really to be used by OpenMP, and
that the last process should be placed on the node that has 4 cpus free.
I hope that wasn't to confusing, Its how to i launch hybrid jobs and
make sure the process started by mpirun go where i want when my nodes
have different core counts, and I am running via torque so using -H
wont work, Also would prefer that all processes be started via TM.
Is this posable ?
Brock Palen
Center for Advanced Computing
bro...@umich.edu
(734)936-1985
