On May 28, 2010, at 3:29 PM, Rahul Nabar wrote: > Each of our servers has twin eth cards: 1GigE and 10GigE. How does > openmpi decide which card to use while sending messages on? One of the > cards is on a 10.0. IP address subnet whereas the other cards are on a > 192.168 adress subnet. Can I select one or the other by specifying the > --host option with the correct IP addresses?
You would want to specify the network in two places: -mca oob_tcp_if_include eth0 <== specifies which interface to use for out-of-band messages -mca btl_tcp_if_include eth1 <== specifies which interface to use for MPI messages on TCP They can be the same network - doesn't matter. Important thing is that all nodes are reachable on that network. > > How does it select the default though? Frequently I call mpirun from > within a PBS wrapper and then there is no explicit --host directive. > (I think PBS somehow communicates to mpirun what the assigned hostfile > is) In such a case though, which interface will mpirun use? For out-of-band, we default to the first one alphabetically, I believe. We don't do any connectivity testing, so the OOB interface must be common to all nodes. The TCP BTL, on the other hand, will check all available interfaces for connectivity, so the procs should "discover" a path to each other over whatever network interfaces are available. > > -- > Rahul > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
