On Dec 10, 2007, at 3:06 PM, Craig Tierney wrote:
I just built OpenMPI-1.2.4 to work on my system (IB, OFED-1.2).
When I run a job, I am getting the following message:
WARNING: There are more than one active ports on host 'w74', but the
default subnet GID prefix was detected on more than one of these
ports. If these ports are connected to different physical IB
networks, this configuration will fail in Open MPI. This version of
Open MPI requires that every physically separate IB subnet that is
used between connected MPI processes must have different subnet ID
values.
I went to the faq to read about the message. My code does complete
successfully because both nodes are connected by both meshes.
You can also assign a different subnet ID to each of the two fabrics.
OMPI will therefore be able to tell these two networks apart and you
won't get this warning message.
We only treat the default subnet ID specially because most people
don't change it, and if they have multiple fabrics, they could run
into problems because OMPI won't be able to tell them apart.
My question is, how can I tell mpirun that I only want to use of
of the ports? I specifically want to use either port 1 or port 2, but
not bond both together.
The OMPI v1.2 series has fairly lame controls for this - you can limit
how many IB ports an MPI process will use on each machine (via the
btl_openib_max_btls MCA parameter), but not which ones. OMPI will use
the first btl_openib_max_btls ports (the default is infinite).
In OMPI v1.3, there are specific MCA parameters for controlling
exactly which NICs and/or ports you want to use or not use.
Specifically:
- btl_openib_if_include: a comma-delimited list of interface names and/
or ports to use
- btl_openib_if_exclude: a comma-delimited list of interface names and/
or ports to exclude (i.e., use all others)
For example:
mpirun --mca btl_openib_if_include mthca0,mthca1:1 ...
Meaning "use all ports on mthca0" and "use port 1 on mthca1".
--
Jeff Squyres
Cisco Systems
