Thank you Gilles, I think that has made it clear. Regards, Rodrigo
On Mon, Mar 20, 2017 at 7:13 PM, Gilles Gouaillardet <gil...@rist.or.jp> wrote: > Rodrigo, > > > i do not understand what you mean by "deactivate my IB interfaces" > > > the hostfile is only used in the wire-up phase > > (to keep things simple, mpirun does > > ssh <hostname> orted > > under the hood, and <hostname> is coming from your hostfile. > > > so bottom line > mpirun --mca btl openib,self,sm -hostfile hosts_eth ... (With IB > interfaces down) > mpirun --mca btl openib,self,sm -hostfile hosts_ib0 ... > > are expected to have the same performance > > > since you have some Infiniband hardware, there are two options > > - you built Open MPI with MXM support, in this case you do not use the > btl/openib, but pml/cm and mtl/mxm > > if you want to force the btl/openib, you have to > > mpirun --mca pml ob1 --mca btl openib,self,sm ... > > - you did not build Open MPI with MXM support, in this case, btl/openib is > used for inter node communications, > > and btl/sm is used for intra node communications. > > > if you want the performance numbers for tcp over ethernet, your command > line is > > mpirun --mca btl tcp,self,sm --mca pml ob1 --mca btl_tcp_if_include eth0 > -hostfile hosts_eth ... > > > Cheers, > > > Gilles > > On 3/21/2017 2:07 AM, Rodrigo Escobar wrote: > > Thanks Guilles for the quick reply. I think I am confused about what the > openib BTL specifies. > What am I doing when I run with the openib BTL but specify my eth > interface (...and deactivate my IB interfaces)? > Is not openib only for IB interfaces? > Am I using RDMA here? > > These two commands give the same performance: > mpirun --mca btl openib,self,sm -hostfile hosts_eth ... (With IB > interfaces down) > mpirun --mca btl openib,self,sm -hostfile hosts_ib0 ... > > Regards, > Rodrigo > > On Mon, Mar 20, 2017 at 8:29 AM, Gilles Gouaillardet < > gilles.gouaillar...@gmail.com> wrote: > >> You will get similar results with hosts_ib and hosts_eth >> >> If you want to use tcp over ethernet, you have to >> mpirun --mca btl tcp,self,sm --mca btl_tcp_if_include eth0 ... >> If you want to use tcp over ib, then >> mpirun --mca btl tcp,self,sm --mca btl_tcp_if_include ib0 ... >> >> Keep in mind that IMB calls MPI_Init_thread(MPI_THREAD_MULTIPLE) >> this is not only unnecessary here, but it also has an impact on >> performances (with older versions, Open MPI felt back on IPoIB, >> with v2.1rc the impact should be minimal) >> >> If you simply >> mpirun --mca btl tcp,self,sm ... >> then Open MPI will multiplex messages on both ethernet and IPoIB >> >> Cheers, >> >> Gilles >> >> >> Rodrigo Escobar <rodave...@gmail.com> wrote: >> Hi, >> I have trying to run the Intel IMB benchmarks to compare the performance >> of Infiniband (IB) vs Ethernet. However, I am not seeing any difference in >> performance even for communication intensive benchmarks, such as alltoallv. >> >> Each one of my machines has one ethernet interface and an infiniband >> interface. I use the following command to run the alltoallv benchmark: >> mpirun --mca btl self,openib,sm -hostfile hosts_ib IMB-MPI1 alltoallv >> >> The hosts_ib file contains the IP addresses of the infiniband interfaces, >> but the performance is the same when I deactivate the IB interfaces and use >> my hosts_eth file which has the IP addresses of the ethernet interfaces. Am >> I missing something? What is really happening when I specify the openib btl >> if I am using the ethernet network? >> >> Thanks >> >> _______________________________________________ >> users mailing list >> users@lists.open-mpi.org >> https://rfd.newmexicoconsortium.org/mailman/listinfo/users >> > > > > _______________________________________________ > users mailing > listus...@lists.open-mpi.orghttps://rfd.newmexicoconsortium.org/mailman/listinfo/users > > > > _______________________________________________ > users mailing list > users@lists.open-mpi.org > https://rfd.newmexicoconsortium.org/mailman/listinfo/users >
_______________________________________________ users mailing list users@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/users
