Right I understand those are TCP interfaces, I was just showing that I have two TCP interfaces over one physical interface, so why I was asking how TCP interfaces were selected. It rarely if ever will mater to us.
Brock Palen www.umich.edu/~brockp CAEN Advanced Computing XSEDE Campus Champion [email protected] (734)936-1985 > On Nov 7, 2014, at 8:38 PM, Gilles Gouaillardet > <[email protected]> wrote: > > Ralph, > > IIRC there is load balancing accros all the btl, for example > between vader and scif. > So load balancing between ib0 and eoib0 is just a particular case that might > not necessarily be handled by the btl tcp. > > Cheers, > > Gilles > > Ralph Castain <[email protected]> wrote: >> OMPI discovers all active interfaces and automatically considers them >> available for its use unless instructed otherwise via the params. I’d have >> to look at the TCP BTL code to see the loadbalancing algo - I thought we >> didn’t have that “on” by default across BTLs, but I don’t know if the TCP >> one automatically uses all available Ethernet interfaces by default. Sounds >> like it must. >> >> >>> On Nov 7, 2014, at 11:53 AM, Brock Palen <[email protected]> wrote: >>> >>> I was doing a test on our IB based cluster, where I was diabling IB >>> >>> --mca btl ^openib --mca mtl ^mxm >>> >>> I was sending very large messages >1GB and I was surppised by the speed. >>> >>> I noticed then that of all our ethernet interfaces >>> >>> eth0 (1gig-e) >>> ib0 (ip over ib, for lustre configuration at vendor request) >>> eoib0 (ethernet over IB interface for IB -> Ethernet gateway for some >>> extrnal storage support at >1Gig speed >>> >>> I saw all three were getting traffic. >>> >>> We use torque for our Resource Manager and use TM support, the hostnames >>> given by torque match the eth0 interfaces. >>> >>> How does OMPI figure out that it can also talk over the others? How does >>> it chose to load balance? >>> >>> BTW that is fine, but we will use if_exclude on one of the IB ones as ib0 >>> and eoib0 are the same physical device and may screw with load balancing >>> if anyone ver falls back to TCP. >>> >>> Brock Palen >>> www.umich.edu/~brockp >>> CAEN Advanced Computing >>> XSEDE Campus Champion >>> [email protected] >>> (734)936-1985 >>> >>> >>> >>> _______________________________________________ >>> users mailing list >>> [email protected] >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>> Link to this post: >>> http://www.open-mpi.org/community/lists/users/2014/11/25709.php >> >> _______________________________________________ >> users mailing list >> [email protected] >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2014/11/25710.php > _______________________________________________ > users mailing list > [email protected] > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/11/25712.php
