Right I understand those are TCP interfaces, I was just showing that I have two TCP interfaces over one physical interface, so why I was asking how TCP interfaces were selected. It rarely if ever will mater to us.
Brock Palen www.umich.edu/~brockp CAEN Advanced Computing XSEDE Campus Champion bro...@umich.edu (734)936-1985 > On Nov 7, 2014, at 8:38 PM, Gilles Gouaillardet > <gilles.gouaillar...@gmail.com> wrote: > > Ralph, > > IIRC there is load balancing accros all the btl, for example > between vader and scif. > So load balancing between ib0 and eoib0 is just a particular case that might > not necessarily be handled by the btl tcp. > > Cheers, > > Gilles > > Ralph Castain <rhc.open...@gmail.com> wrote: >> OMPI discovers all active interfaces and automatically considers them >> available for its use unless instructed otherwise via the params. I’d have >> to look at the TCP BTL code to see the loadbalancing algo - I thought we >> didn’t have that “on” by default across BTLs, but I don’t know if the TCP >> one automatically uses all available Ethernet interfaces by default. Sounds >> like it must. >> >> >>> On Nov 7, 2014, at 11:53 AM, Brock Palen <bro...@umich.edu> wrote: >>> >>> I was doing a test on our IB based cluster, where I was diabling IB >>> >>> --mca btl ^openib --mca mtl ^mxm >>> >>> I was sending very large messages >1GB and I was surppised by the speed. >>> >>> I noticed then that of all our ethernet interfaces >>> >>> eth0 (1gig-e) >>> ib0 (ip over ib, for lustre configuration at vendor request) >>> eoib0 (ethernet over IB interface for IB -> Ethernet gateway for some >>> extrnal storage support at >1Gig speed >>> >>> I saw all three were getting traffic. >>> >>> We use torque for our Resource Manager and use TM support, the hostnames >>> given by torque match the eth0 interfaces. >>> >>> How does OMPI figure out that it can also talk over the others? How does >>> it chose to load balance? >>> >>> BTW that is fine, but we will use if_exclude on one of the IB ones as ib0 >>> and eoib0 are the same physical device and may screw with load balancing >>> if anyone ver falls back to TCP. >>> >>> Brock Palen >>> www.umich.edu/~brockp >>> CAEN Advanced Computing >>> XSEDE Campus Champion >>> bro...@umich.edu >>> (734)936-1985 >>> >>> >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>> Link to this post: >>> http://www.open-mpi.org/community/lists/users/2014/11/25709.php >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2014/11/25710.php > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/11/25712.php
