In your case, using slots or --npernode or --map-by node will result in the 
same distribution of processes because you're only launching 1 process per node 
(a.k.a. "1ppn").

They have more pronounced differences when you're launching more than 1ppn.

Let's take a step back: you should know that Open MPI uses 3 phases to plan out 
how it will launch your MPI job:

1. Mapping: where each process will go
2. Ordering: after mapping, how each process will be numbered (this translates 
to rank ordering MPI_COMM_WORLD)
3. Binding: binding processes to processors

#3 is not pertinent to this conversation, so I'll leave it out of my discussion 
below.

We're mostly talking about #1 here.  Let's look at each of the three options 
mentioned in this thread individually.  In each of the items below, I assume 
you are using *just* that option, and *neither of the other 2 options*:

1. slots: this tells Open MPI the maximum number of processes that can be 
placed on a server before it is considered to be "oversubscribed" (and Open MPI 
won't let you oversubscribe by default).

So when you say "slots=1", you're basically telling Open MPI to launch 1 
process per node and then to move on to the next node.  If you said "slots=3", 
then Open MPI would launch up to 3 processes per node before moving on to the 
next (until the total np processes were launched).

*** Be aware that we have changed the hostfile default value of slots (i.e., 
what number of slots to use if it is not specified in the hostfile) in 
different versions of Open MPI.  When using hostfiles, in most cases, you'll 
see either a default value of 1 or the total number of cores on the node.

2. --map-by node: in this case, Open MPI will map out processes round robin by 
*node* instead of its default by *core*.  Hence, even if you had "slots=3" and 
-np 9, Open MPI would first put a process on node A, then put a process on node 
B, then a process on node C, and then loop back to putting a 2nd process on 
node A, ...etc.

3. --npernode: in this case, you're telling Open MPI how many processes to put 
on each node before moving on to the next node.  E.g., if you "mpirun -np 9 
..." (and assuming you have >=3 slots per node), Open MPI will put 3 processes 
on each node before moving on to the next node.

With the default MPI_COMM_WORLD rank ordering, the practical difference in 
these three options is:

Case 1:

$ cat hostfile
a slots=3
b slots=3
c slots=3
$ mpirun --hostfile hostfile -np 9 my_mpi_executable

In this case, you'll end up with MCW ranks 0-2 on a, 3-5 on b, and 6-8 on c.

Case 2:

# Setting an arbitrarily large number of slots per host just to be explicitly 
clear for this example
$ cat hostfile
a slots=20
b slots=20
c slots=20
$ mpirun --hostfile hostfile -np 9 --map-by node my_mpi_executable

In this case, you'll end up with MCW ranks 0,3,6 on a, 1,4,7 on b, and 2,5,8 on 
c.

Case 3:

# Setting an arbitrarily large number of slots per host just to be explicitly 
clear for this example
$ cat hostfile
a slots=20
b slots=20
c slots=20
$ mpirun --hostfile hostfile -np 9 --npernode 3 my_mpi_executable

In this case, you'll end up with the same distribution / rank ordering as case 
#1, but you'll still have 17 more slots you could have used.

There are lots of variations on this, too, because these mpirun options (and 
many others) can be used in conjunction with each other.  But that gets pretty 
esoteric pretty quickly; most users don't have a need for such complexity.



> On Nov 4, 2016, at 8:57 AM, Bennet Fauber <ben...@umich.edu> wrote:
> 
> Mahesh,
> 
> Depending what you are trying to accomplish, might using the mpirun option
> 
> -pernode  -o-  --pernode
> 
> work for you?  That requests that only one process be spawned per
> available node.
> 
> We generally use this for hybrid codes, where the single process will
> spawn threads to the remaining processors.
> 
> Just a thought,   -- bennet
> 
> 
> 
> 
> 
> On Fri, Nov 4, 2016 at 8:39 AM, Mahesh Nanavalla
> <mahesh.nanavalla...@gmail.com> wrote:
>> s...........
>> 
>> Thanks for responding me.
>> i have solved that as below by limiting slots in hostfile
>> 
>> root@OpenWrt:~# cat myhostfile
>> root@10.73.145.1 slots=1
>> root@10.74.25.1  slots=1
>> root@10.74.46.1  slots=1
>> 
>> 
>> I want the difference between the slots limiting in myhostfile and runnig
>> --map-by node.
>> 
>> I am awaiting for your reply.
>> 
>> On Fri, Nov 4, 2016 at 5:25 PM, r...@open-mpi.org <r...@open-mpi.org> wrote:
>>> 
>>> My apologies - the problem is that you list the option _after_ your
>>> executable name, and so we think it is an argument for your executable. You
>>> need to list the option _before_ your executable on the cmd line
>>> 
>>> 
>>> On Nov 4, 2016, at 4:44 AM, Mahesh Nanavalla
>>> <mahesh.nanavalla...@gmail.com> wrote:
>>> 
>>> Thanks for reply,
>>> 
>>> But,with space also not running on one process one each node
>>> 
>>> root@OpenWrt:~# /usr/bin/mpirun --allow-run-as-root -np 3 --hostfile
>>> myhostfile /usr/bin/openmpiWiFiBulb --map-by node
>>> 
>>> And
>>> 
>>> If use like this it,s working fine(running one process on each node)
>>> /root@OpenWrt:~#/usr/bin/mpirun --allow-run-as-root -np 3 --host
>>> root@10.74.25.1,root@10.74.46.1,root@10.73.145.1 /usr/bin/openmpiWiFiBulb
>>> 
>>> But,i want use hostfile only..
>>> kindly help me.....
>>> 
>>> 
>>> On Fri, Nov 4, 2016 at 5:00 PM, r...@open-mpi.org <r...@open-mpi.org> wrote:
>>>> 
>>>> you mistyped the option - it is “--map-by node”. Note the space between
>>>> “by” and “node” - you had typed it with a “-“ instead of a “space”
>>>> 
>>>> 
>>>> On Nov 4, 2016, at 4:28 AM, Mahesh Nanavalla
>>>> <mahesh.nanavalla...@gmail.com> wrote:
>>>> 
>>>> Hi all,
>>>> 
>>>> I am using openmpi-1.10.3,using quad core processor(node).
>>>> 
>>>> I am running 3 processes on three nodes(provided by hostfile) each node
>>>> process is limited  by --map-by-node as below
>>>> 
>>>> root@OpenWrt:~# /usr/bin/mpirun --allow-run-as-root -np 3 --hostfile
>>>> myhostfile /usr/bin/openmpiWiFiBulb --map-by-node
>>>> 
>>>> root@OpenWrt:~# cat myhostfile
>>>> root@10.73.145.1:1
>>>> root@10.74.25.1:1
>>>> root@10.74.46.1:1
>>>> 
>>>> 
>>>> Problem is 3 process running on one node.it's not mapping one process by
>>>> node.
>>>> 
>>>> is there any library used to run like above.if yes please tell me that .
>>>> 
>>>> Kindly help me where am doing wrong...
>>>> 
>>>> Thanks&Regards,
>>>> Mahesh N
>>>> 
>>>> _______________________________________________
>>>> users mailing list
>>>> users@lists.open-mpi.org
>>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> users mailing list
>>>> users@lists.open-mpi.org
>>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>> 
>>> 
>>> _______________________________________________
>>> users mailing list
>>> users@lists.open-mpi.org
>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> users mailing list
>>> users@lists.open-mpi.org
>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>> 
>> 
>> 
>> _______________________________________________
>> users mailing list
>> users@lists.open-mpi.org
>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
> _______________________________________________
> users mailing list
> users@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/users


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

_______________________________________________
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

Reply via email to