Thank you, Gilles. That explains it! On Tue, Jul 19, 2016 at 1:14 AM, Gilles Gouaillardet <gil...@rist.or.jp> wrote:
> basically, there are two methods (aka pml) to send/recv messages. > > ob1 is the basic one, it works with (all ?) interconnects that can > send/recv a stream of data > > pml/ob1 uses the available btl(s) (tcp, openib, ...) > > cm is for feature rich interconnects that can send/recv messages > > pml/cm uses the available mtl (mxm, portals4, psm, ...) > > > mxm is a mtl that uses the (proprietary) mxm library from Mellanox. > > > bottom line, there are two ways to use infiniband : > > - pml/ob1 and btl/openib > > - pml/cm and mtl/mxm > > > since pml/cm has a default priority that is higher than pml/ob1, if you > want to force tcp usage, even on systems > > where mxm is available and OpenMPI has builtin support for mxm, you need to > > - force pml/ob1 (so mtl/mxm cannot be used by pml/cm) > > and > > - blacklist btl/openib > > your mpirun command line looks like this > > mpirun --mca pml ob1 --mca btl ^openib ... > > > Cheers, > > > Gilles > On 7/19/2016 1:57 PM, Saliya Ekanayake wrote: > > Thank you, but what's mxm? > > On Tue, Jul 19, 2016 at 12:52 AM, Nathan Hjelm <hje...@me.com> wrote: > >> You probably will also want to run with -mca pml ob1 to make sure mxm is >> not in use. The combination should be sufficient to force tcp usage. >> >> -Nathan >> >> > On Jul 18, 2016, at 10:50 PM, Saliya Ekanayake < <esal...@gmail.com> >> esal...@gmail.com> wrote: >> > >> > Hi, >> > >> > I read in a previous thread ( >> <https://www.open-mpi.org/community/lists/users/2014/05/24475.php> >> https://www.open-mpi.org/community/lists/users/2014/05/24475.php) that >> Jeff mentions it's possible for OpenMPI to pick up the openib transport if >> tcp is not requested explicitly. >> > >> > So, does that mean if I do, >> > >> > --mca btl ^openib >> > >> > that it's still possible for OpenMPI to use Infiniband and not TCP? >> > >> > Is there a way to guarantee that a test is using TCP, but not IB? >> > >> > Thank you, >> > saliya >> > >> > -- >> > Saliya Ekanayake >> > Ph.D. Candidate | Research Assistant >> > School of Informatics and Computing | Digital Science Center >> > Indiana University, Bloomington >> > >> > _______________________________________________ >> > users mailing list >> > us...@open-mpi.org >> > Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users >> > Link to this post: >> http://www.open-mpi.org/community/lists/users/2016/07/29682.php >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2016/07/29683.php >> > > > > -- > Saliya Ekanayake > Ph.D. Candidate | Research Assistant > School of Informatics and Computing | Digital Science Center > Indiana University, Bloomington > > > > _______________________________________________ > users mailing listus...@open-mpi.org > Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > http://www.open-mpi.org/community/lists/users/2016/07/29684.php > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2016/07/29685.php > -- Saliya Ekanayake Ph.D. Candidate | Research Assistant School of Informatics and Computing | Digital Science Center Indiana University, Bloomington
