basically, there are two methods (aka pml) to send/recv messages.
ob1 is the basic one, it works with (all ?) interconnects that can
send/recv a stream of data
pml/ob1 uses the available btl(s) (tcp, openib, ...)
cm is for feature rich interconnects that can send/recv messages
pml/cm uses the available mtl (mxm, portals4, psm, ...)
mxm is a mtl that uses the (proprietary) mxm library from Mellanox.
bottom line, there are two ways to use infiniband :
- pml/ob1 and btl/openib
- pml/cm and mtl/mxm
since pml/cm has a default priority that is higher than pml/ob1, if you
want to force tcp usage, even on systems
where mxm is available and OpenMPI has builtin support for mxm, you need to
- force pml/ob1 (so mtl/mxm cannot be used by pml/cm)
and
- blacklist btl/openib
your mpirun command line looks like this
mpirun --mca pml ob1 --mca btl ^openib ...
Cheers,
Gilles
On 7/19/2016 1:57 PM, Saliya Ekanayake wrote:
Thank you, but what's mxm?
On Tue, Jul 19, 2016 at 12:52 AM, Nathan Hjelm <[email protected]
<mailto:[email protected]>> wrote:
You probably will also want to run with -mca pml ob1 to make sure
mxm is not in use. The combination should be sufficient to force
tcp usage.
-Nathan
> On Jul 18, 2016, at 10:50 PM, Saliya Ekanayake
<[email protected] <mailto:[email protected]>> wrote:
>
> Hi,
>
> I read in a previous thread
(https://www.open-mpi.org/community/lists/users/2014/05/24475.php)
that Jeff mentions it's possible for OpenMPI to pick up the openib
transport if tcp is not requested explicitly.
>
> So, does that mean if I do,
>
> --mca btl ^openib
>
> that it's still possible for OpenMPI to use Infiniband and not TCP?
>
> Is there a way to guarantee that a test is using TCP, but not IB?
>
> Thank you,
> saliya
>
> --
> Saliya Ekanayake
> Ph.D. Candidate | Research Assistant
> School of Informatics and Computing | Digital Science Center
> Indiana University, Bloomington
>
> _______________________________________________
> users mailing list
> [email protected] <mailto:[email protected]>
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
http://www.open-mpi.org/community/lists/users/2016/07/29682.php
_______________________________________________
users mailing list
[email protected] <mailto:[email protected]>
Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/07/29683.php
--
Saliya Ekanayake
Ph.D. Candidate | Research Assistant
School of Informatics and Computing | Digital Science Center
Indiana University, Bloomington
_______________________________________________
users mailing list
[email protected]
Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/07/29684.php
