OMPI discovers all active interfaces and automatically considers them available for its use unless instructed otherwise via the params. I’d have to look at the TCP BTL code to see the loadbalancing algo - I thought we didn’t have that “on” by default across BTLs, but I don’t know if the TCP one automatically uses all available Ethernet interfaces by default. Sounds like it must.
> On Nov 7, 2014, at 11:53 AM, Brock Palen <bro...@umich.edu> wrote: > > I was doing a test on our IB based cluster, where I was diabling IB > > --mca btl ^openib --mca mtl ^mxm > > I was sending very large messages >1GB and I was surppised by the speed. > > I noticed then that of all our ethernet interfaces > > eth0 (1gig-e) > ib0 (ip over ib, for lustre configuration at vendor request) > eoib0 (ethernet over IB interface for IB -> Ethernet gateway for some > extrnal storage support at >1Gig speed > > I saw all three were getting traffic. > > We use torque for our Resource Manager and use TM support, the hostnames > given by torque match the eth0 interfaces. > > How does OMPI figure out that it can also talk over the others? How does it > chose to load balance? > > BTW that is fine, but we will use if_exclude on one of the IB ones as ib0 and > eoib0 are the same physical device and may screw with load balancing if > anyone ver falls back to TCP. > > Brock Palen > www.umich.edu/~brockp > CAEN Advanced Computing > XSEDE Campus Champion > bro...@umich.edu > (734)936-1985 > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/11/25709.php
